Record data

If you are using a Raspberry Pi Zero or SSH / VNC, the live preview window can drop the frame rate during recording below 30 fps. That means that less data is collected, which is a bad thing. Run this command in the terminal before entering further commands to ensure recording runs as close to 30 fps as possible:

unset DISPLAY 

If you keep the preview window open during training you will notice that the lighting and color balance fluctuates rapidly. This is intentional! The camera exposure and white balance is set randomly for each frame during training. This technique, a variant of domain randomization, shows lots of variance in something that might otherwise trick or confuse the AI (in this case, lighting and color balance). It is actually more common to artificially add this to the data at training time, but this is not covered here.

Follow these steps to record your data:

  1. Record some actions with your hands in the air. When you run the following command, it will count down from three and then record you for 15 seconds:
    python example/yeah 15 
    Suggested actions: wave your hands in the air like you are cheering. Lean left and right while doing so. Turn left and right. Lean forwards and backwards. Keep your hands pumping in the air above your head like a lunatic the whole time.
  2. Record the same behavior, but with your hands down after entering this command:
    python example/sitting 15 
    Suggested actions: Lean left and right, turn left and right, lean forwards and backwards. Keep your hands down or at keyboard height the whole time.
  3. Record some random activities that you want the network to ignore and therefore not be surprised by after entering this command:
    python example/random 15 
    Suggested actions: Cover your face, scratch your head, reach for and cover the camera, stand up and walk away, come back and sit down, get up and walk the other way.

After these steps you will have three files of similar size that contain the three types of data that you recorded. You can see the files listed by entering this command:

 $ ls -lh example/


 total 166M
 -rw-r--r-- 1 pi pi 49M Nov 27 10:59 random
 -rw-r--r-- 1 pi pi 64M Nov 27 10:59 sitting
 -rw-r--r-- 1 pi pi 54M Nov 27 10:57 yeah
Set up camera Train a network