Using the Kinect v2 Data Feed in any C# Application

Programmed for Windows 10 with Visual C# 6.0

I recently stumbled on a bit of a blocker when my team asked me if there was a way to read a recorded Kinect v2 data stream (XEF file) into a game engine like Unity, or for any other application built on top of .NET and C#. Imagine my horror when I realized that the official Microsoft.Kinect.Tools library was not compatible with a universally accepted game engine like Unity, where the entire framework was based on .NET 3.5 whereas the Microsoft assemblies were compiled using .NET 4. The fault here mostly lies with Unity for not moving with the flow and upgrading their framework to work on newer versions of .NET.

 

Moving on, like any Software Engineer my only solution was to find a way around the roadblock, and this called for testing each and every other Kinect assembly to check which versions were supported by Unity. I found out that the Microsoft.Kinect assembly could be loaded and used inside Unity, with the exception that the library had to be referenced as using Windows.Kinect  rather than the usual using Microsoft.Kinect  method that we are used to.

 

This means that Unity could read the sensor data from a Kinect or a XEF file as long as it was connected or broadcast from the Kinect Studio. However, this meant that our program could not load and playback a Kinect data stream at will. There would be a need for human interaction to load a Xbox Event File (.xef) in order for the program to work. However, we needed a solution where any game engine (especially Unity) would be able to read in the data stream dynamically.

 

This called for a side application or service that would function as the Kinect Studio, with the added feature where it could be used programmatically to broadcast data into the application. At the moment Microsoft does provide an application called KSUtil.exe which is bundled along with the Kinect SDK 2.0, however it does not provide a huge leeway in the terms of functionality and customization that we were expecting.

 

The team specifically needed a service module that could later be modified depending on the requirement. This called for building an application from scratch.

 

In the quest for the best methods to use in order to communicate between two different C# components, I looked into pipe operations, using .NET sockets and constructing a WCF application, but in the end I settled on building a simple tray app that could also be executed from the command line using a simple Process.Start()  command from Unity.

 

The application consists of a few tweaks in order to make the developer life easier, including the option to run it both via command line as well as in the application tray. It also provides Windows 10 themed notifications to make development easier. The following code-base available in MSDN is used as the core class around which the others revolve.

 
The application can be run using two configurations from the command line. Specifying only the play argument would make the application create a playback for all the files in the XEF location given in the app.config file. On the other hand, specifying both the play command along-with an argument with a specific filename would allow you to create a playback only for the requested file as shown below.