EmguCV with the Raspberry Pi Camera Module

Aug 30, 2014

UPDATED: Nov 2016

In the last post, I documented how to compile EmguCV on Raspbian and get a USB camera running with the standard Emgu example code. The performance of the USB camera is unsatisfactory however and the better option is to get the Raspberry Pi Camera Module hooked into EmguCV to take advantage of the GPU.

Prerequisites

The following assumes:

  1. You have EmguCV compiled on Raspbian. Instructions below assume at ~/emgucv.
  2. You have EmguCV compiled from source on Windows. It will be simpler later if you clone into a folder named emgucv-source which is a sibling folder to PiCamCV. Compiling EmguCV from source can be tricky if you don't follow Emgu's instructions to the letter, so read their documentation again.
  3. You have a USB camera attached to your Windows host.
  4. You have a Raspberry Pi with a camera module attached.

RaspiCamCV

As noted at the end of the last post, OpenCV and hence EmguCV cannot locate the Raspberry Pi camera by default. Emil Valkov has done some work to make a C library to expose the pi camera in a manner compatible with OpenCV. We will make use of that library to make the pi camera available to EmguCV.

Follow the instructions to build a copy of this library.

PInvoke

Next, we need to create a c# assembly which PInvoke's the RaspiCamCV library. For this, see CvInvokeRaspiCamCV.cs, a snippet of which looks like this:

[DllImport(OpencvRaspiCamCVLibrary, EntryPoint=EntryPointCapture, CallingConvention = CvCallingConvention)]
public static extern IntPtr cvCreateCameraCapture(int index);

This is all fairly straight forward. Note in the full source that I added a UNIX conditional compile time constant. This allows one to develop on Windows and test OpenCV with a USB camera before switching configuration, compiling for UNIX and directing the PInvoke to the raspicamcv library for running on the Pi.

Compiling on Windows

Let's convert Emil's simple raspicamtest.c into C#. For that, see SimpleCv.cs.

We now have all the basic building blocks for a simple test. So clone the PiCamCV repository to your Windows OS.

There are two submodule solutions in the ~\picamcv\lib folder. Execute build.bat in their respective folders to ensure their packages are restored properly.

EmguCV looks for the unmanaged OpenCV dll's in x86 and x64 folders below the binary folder of the executing assembly. PiCamCV expects an environment variable to tell it where to pull the EmguCV binaries from. Execute the following powershell with the correct path to your EmguCV bin folder

[Environment]::SetEnvironmentVariable("EmguCV_Bin_Folder", "C:\Code\emgucv\bin", "User")

Open PiCamCV's solution and start the WinForms project in Debug configuration on your Windows host. With any luck you will be able to start a capture and view a video stream from the camera.

Now compile PiCamCV for UnixDebug and SCP the binary files onto your Pi at ~/picamcv. Be careful not to copy the Windows compiled emgucv binaries, these don't run on Linux. You need to use the Linux compiled versions (keep reading).

Running on the Raspberry Pi

SSH into your Pi

Copy the libraspicamcv.so that you compiled in the RaspiCamCV step above into ~/picamcv:
cp ~/robidouille/raspicam_cv/libraspicamcv.so ~/picamcv/

Copy the opencv.so and emgu.CV.* files from the emgucv bin folder to the picamcv folder: cp ~/emgucv/bin/*.* ~/picamcv

Start up X Windows if you aren't already in it.

From an X Window terminal, execute mono picam.win.exe
With any luck, your pi camera module will now be streaming into an OpenCV window!

The CPU in the video is hitting 100% because it is running in VNC. Running over HDMI, it hits around 80%.

Note: Compiling EmguCV after RaspiCamCV

If you need to compile EmguCV after you have successfully compiled RaspiCamCV, it won't work. RaspiCamCV's instructions caused apt-get to install opencv-dev libraries which interfere with the EmguCV make.

To fix this:
sudo apt-get remove libcv-dev libhighgui-dev
I also ran sudo apt-get autoremove but I think this was a mistake as I then had to repair some problems with libgeotiff-dev by apt-get again and also reinstall libgdiplus. Not doing the autoremove seems to save the trouble.

EmguCV will now compile again. To make RaspiCamCV you will need to redo it's recommended sudo apt-get lines.