ARToolKit with OpenKinect using the IR camera

More than one year ago I started adding Kinect support to ARToolKit using libfreenect. I had some results but didn’t have enough time to publish a fully tested version of it. Now I know it ins’t very useful because Ubuntu already detects Kinect as a webcam (creating a /dev/videoX device for it, or at least that happend to me with Ubuntu 12.04 UPDATE: I figured out that this happens because of the gspca_kinect module introduced in Linux 3.0 kernel), so you can use it with ARToolKit via the GStreamer video input. Still I wanted to learn more on how ARToolKit works on its inside. Now I’ve added some more features and support not only for the RGB camera but also for the IR camera, tilt and LED.

The nature of the IR camera makes detecting markers very difficult as they are very blurry when not very close to the Kinect, but hey it’s been fun hacking with libfreenect. I’ll try to publish the code ASAP. I still want to test it more.

You might also like

Install ARToolKit 2.72.1 in Ubuntu 11.10 and 12.04
The installation process is the same as in Ubuntu 10.10 but it looks like there is a problem when executing...
Install ARToolKit 2.72.1 in Ubuntu 10.10
Firsts you have to install some dependencies: sudo apt-get install freeglut3-dev libgstreamer0.10-dev...
ARSlides v1.0: Augmented Reality slide presentation application
I'm glad to inform that I've (finally) published the code for the Augmented Reality application I've...
Install osgART 2.0 RC3 with OpenSceneGraph 2.8.3 or 2.9.6 with Collada support in Ubuntu 12.04
Before installing osgART we need to install ARToolKit, collada-dom and OpenSceneGraph so look at those...
Leave a comment ?


  1. Can you publish the code or at least write about it so that we can see how you get hold of the Depth data. There are other people who do it on ROS (, I will take a look at it, but if you have also done it, it can be useful.

  2. Yes, I’ll publish it ASAP. It’s not very complicated, I based the code in one of the libfreenect basic examples. Hope I can finish it this week or at least publish a working but simple version.

  3. Be aware that it only reads the data from the camera and depth sensor, it does not do any pose estimation or anything related to that.

  4. hello, any updates on the source code?
    you did here answers my question in mind. atleast I had a proof the ARToolkit is possible on IR cam.

    thank you very much!


Leave a Comment