Solving the RealSense platform conundrum

When I develop RealSense applications I invariably use Visual Studio and C# to write my applications. If you’ve done any RealSense development in .NET, you’re aware that they have a 32 bit and 64 bit version of their assemblies and you’ve probably had to deal with the conundrum about making your application support 64 bit or 32 bit assembly references. In most of the examples I have seen, the developers plump for adding references to only one platform – which is a real shame. With just a little bit of Visual Studio trickery, you can use target both platforms – in this post, I’m going to introduce you to a little command line tool that I wrote that sets your apps up to support x86 and x64.

Basically, what we’re going to do here is copy the relevant libs files from the Realsense SDK and copy them into a Libs folder in the solution folder. Obviously, we need to reference these files so the code will create x86 and x64 references inside our csproj files (and create the matching entries in the solution as well). As only one of the files is a .NET file, we need to copy the unmanaged DLL it relies on into the output folder as well. This last part is done by creating a post build event to copy the file over on successful completion of the build.

Using it is pretty straightforward – you’ll need to change the root folder for your RealSense installation inside the .config file. Look for the RSSDK key and enter your root directory – in my case, it’s C:\Intel\RSSDK, so when the code is running it uses this to build up the C:\Intel\RSSDK\bin\win32 and C:\Intel\RSSDK\bin\x64 folders. So, you need to make sure you change this value to point to the directory immediately above the bin folder.

When you run the application (it’s a console application so you’re best off running it in a command window), pass in the name of the directory that you want to update to RealSense – the code looks for all solution files and csproj files from the directory you pass in – don’t worry about the nesting level of these files, the code effectively walks the tree looking for these files. So, when I wanted to add RealSense to all of the solution and project files under C:\Dev\TestRealsenseMaker, I ran the command MakeRealsense C:\Dev\TestRealsenseMaker. And that’s it. That’s all I needed to do to make my files RealSense ready. Happy coding.

MakeRealsense.zip

Note: WordPress doesn’t like zip files, so you’ll need to rename the file from MakeRealsense.zip.doc to MakeRealsense.zip when you have downloaded it.

Sensing the future with WPF

This post is a look into a new library that I’m writing that’s intended to make life easier for WPF developers working with Intel RealSense devices. As many of you may know, I’ve been involved with the RealSense platform for a couple of years now (back from when it was called the Perceptual Computing SDK). When I develop samples with it, I tend to use WPF as my default development experience, and the idea of hooking up the Natural User Interface capabilities of RealSense devices with the NUI power of WPF in an easy to use package is just too good to resist. On top of this, I still strongly believe in WPF and it will take a lot to remove me from developing desktop applications with it because it is just so powerful.

To this end, I have started developing a library called RealSenseLight that will enable WPF developers to easily leverage the power of RealSense without having to worry about the implementation details. While it”s primarily aimed at WPF developers, the functionality available will be usable from other C# (Windows Desktop) applications, so hooking into Console applications will certainly be possible.

One of the many decisions I’ve taken is to allow configuration of features to be set via Fluent interface, so it’s possible to do things like this:

RealSenseApplication.Uses(new EmotionDetectionConfiguration())
  .Uses(SpeechRecognition.DefaultConfiguration().ChangePitch(0.8))
  .Start();

ViewModels will be able to hook into RealSense using convenient interfaces that abstract the underlying implementations. There’s no need to call Enable… to enable a RealSense capability. The simple fact of integrating a concrete implementation means that the feature is automatically available. The following example demonstrates what an IoC resolved implementation looks like:

public class EmotionViewModel : ViewModelBase
{
  private IEmotion _emotion;
  public EmotionViewModel(IEmotion emotion)
  {
    emotion.OnUserHappy(user => System.Debug.WriteLine("{0} is happy", user.DetectedUser));
  }
}

The library will provide the ability to do things such as pause/resume individual RealSense capabilities, identify and choose from the relevant RealSense compatible devices, although this does require identifying up front, what the different aspects are you’re interested in because it uses these to evaluate the devices that meet these capabilities.

I’m still fleshing out what the whole interface will look like, so all of the features haven’t been determined yet, but I will keep posting my designs and a link to the repo once I have it in a state where it’s ready for an initial commit.

Getting a RealSense of my status

Long time readers will have realised that I have been spending a lot of time with the technology that was formally known as Perceptual Computing (PerC). You may also know that this technology is now known as RealSense and that it will be rolling out to a device near you soon. What you might not know is that I’m currently writing a course on this technology for Pluralsight. As part of writing this course, I’ve been creating a few little wrapper utilities that will make your life easier when developing apps with the SDK.

In this post, I’m going to show you a handy little method for working with API methods. Pretty much every RealSense API method returns a status code to indicate whether or not it was successful. Now, it can get pretty tedious writing code that looks like this:

pxcmStatus status = Session.CreateImpl<PXCMVoiceRecognition>
  (PXCMVoiceRecognition.CUID, out voiceRecognition);
if (status < pxcmStatus.pxcmStatus.PXCM_STATUS_NO_ERROR)
{
  throw new InvalidStatusException("Could not create session");
}
status = _voiceRecognition.QueryProfile(out pInfo);
if (status < pxcmStatus.pxcmStatus.PXCM_STATUS_NO_ERROR)
{
  throw new InvalidStatusException("Could not query profile");
}

As you can imagine, the more calls you make, the more status checks you have to do. Well, I like to log information about what I’m attempting to do and what I have successfully managed to do, so this simple method really helps to write information about the methods being invoked, and to throw an exception if things go wrong.

public void PipelineInvoke(Func<pxcmStatus> pipelineMethod, string loggingInfo = "")
{
  if (!string.IsNullOrWhiteSpace(loggingInfo))
  {
    Debug.WriteLine("Start " + loggingInfo);
  }
  pxcmStatus status = pipelineMethod();
  if (status < pxcmStatus.PXCM_STATUS_NO_ERROR)
  {
    throw new InvalidStatusException(loggingInfo, status);
  }
  if (!string.IsNullOrWhiteSpace(loggingInfo))
  {
    Debug.WriteLine("Finished " + loggingInfo);
  }
}

This makes it easier to work with the API and gives us code that looks like this:

PipelineInvoke(() => 
  Session.CreateImpl<PXCMVoiceRecognition>(PXCMVoiceRecognition.CUID, 
  out _voiceRecognition), "creating voice recognition module");

And this is what InvalidStatusException looks like:

public class InvalidStatusException : Exception
{
  public InvalidStatusException(string alertMessage, pxcmStatus status)
  {
    Message = alertMessage;
    Status = status;
  }
  public string Message { get; private set; }
  public pxcmStatus Status { get; private set; }
}

Over the course of the next couple of months, I’ll share more posts with you showing the little tricks and techniques that I use to make working with the RealSense SDK a real joy.