Pan Tilt #4: Face tracking
This is post #4 in a series about making a pan tilt controller for the Raspberry Pi and it's camera module. Post 1 discussed the Lego build. Post 2 mounted the camera, Post 3 used colour tracking to control the servos.
Tracking coloured balls is all well and good but what would be really neat would be tracking a face with the camera. OpenCV comes out of the box with face detection so I got to work.
The joystick and colour tracking controllers from the previous posts laid the ground work to make a pluggable architecture for my controllers. It didn't take long to add face detection capabilities. The source is on github.
After getting it working there are some problems with face tracking however. So much so I haven't bothered to record a video of it.
Problem: Raspberry Pi B+ Performance
Running on my Intel Core i7, face detection takes a few milliseconds. Running on the B+ Pi however it can take 250ms @ 128x96 per frame with the default settings on the cascade classifier. The face detection - or any object detection for that matter - must be completed as fast as possible to allow the camera to continually move and track the target. Processing times of 250ms per frame is too slow. The timing can be improved by tuning the inputs to the classifier - in this case I increased the scaling from 1.1 to 1.4 at the expense of more false negatives but even at 1.4 there isn't enough performance improvement to save it.
Below are the rough the face detection response times I measured.
|Resolution||Framerate / 10 frames (ms)|
|Raspberry Pi B+||Raspberry Pi 2||Raspberry Pi 3||Intel i7-4770|
|Scale 1.1||Scale 1.4||Scale 1.1, TBB=false||Scale 1.1, TBB=true||Scale 1.1, TBB=true||Scale 1.1, TBB=true|
|64x48||1480 (min size=5)||undetected||396||170-200||-|
|128x96||3250 (min size=20)||2100||400||330||200-220||350|
Problem: Full Frontal Views Required
Face tracking works well as long as the subject keeps their face pointed toward the camera. If they approach side on or turn their head to leave, the face detection drops out as it isn't trained for profile views. This makes the face tracking not quite as magical as originally envisaged.
Problem: Field of View
The stock lens for the Pi has a relatively narrow field of view. It doesn't take much deviation for the subject to move out of view of the camera and for it to come to a standstill. This is compounded by the face processing time required.
More work required!