Record data

If you are using a Raspberry Pi Zero, or SSH or VNC, the live preview window can drop the frame rate during recording below 30fps. This means that less data is collected, which is a bad thing. To ensure recording is as close as possible to 30fps, run the following command in the terminal before entering further commands:

unset DISPLAY 

If you keep the preview window open during training, notice that the lighting and color balance fluctuates rapidly. This fluctuation is intentional because the camera exposure and white balance are set randomly for each frame during training. This technique, which is a variant of domain randomization, shows lots of variances in something that might otherwise trick or confuse the AI: in this case, lighting and color balance. It is more common to artificially add this variance to the data at training time, but this is beyond the scope of this guide.

To record your data, follow these steps:

  1. Create an example directory by entering this command:
    mkdir example
  2. Record some actions with your hands in the air. When you run the following command, it will count down from three and then records you for 15 seconds:
    python example/yeah 15
    Suggested actions: waving your hands in the air like you are cheering, lean left and right while doing so. Turn left and right. Lean forwards and backwards. Keep your hands pumping in the air above your head the whole time.
  3. Enter this command, then record the same behavior, but with your hands down: 
    python3 example/sitting 15 
    Suggested actions: Lean left and right, turn left and right, lean forwards and backwards. Keep your hands down or at keyboard height the whole time.
  4. Enter this command, then record some random activities that you want the network to ignore so that these actions will not surprise the network:
    python3 example/random 15 
    Suggested actions: Cover your face, scratch your head, reach for and cover the camera, stand up and walk away, come back and sit down, get up and walk the other way.

After you have followed these steps, you will have three files of similar size that contain the three categories of gesture data that you recorded. To view the files, run the command:

 $ ls -lh example/

The three files are:

 total 166M
 -rw-r--r-- 1 pi pi 49M Nov 27 10:59 random
 -rw-r--r-- 1 pi pi 64M Nov 27 10:59 sitting
 -rw-r--r-- 1 pi pi 54M Nov 27 10:57 yeah
Previous Next