Today I thought I'd share some of our learnings from working with the surface tables.  They're very interesting devices and while you can download the SDK and samples here, there are some things that you can't learn unless you have the hardware. Specifically I want to discuss performance, tag tracking, and object albedo.

Performance

One thing that I was surprised by was the power of the machines.  Since Bob Ballard and the Mystic Aquarium provided us with high resolution images and video we were a little concerned about performance, especially since some of our applications were going to take multiple instances of large high res imagery and throw several pixel shaders on top of it in order to produce some nice water effects.  I do most of my development on a high end laptop and it would chug, but the SUR40 handled it beautifully.  I hope that Microsoft and Samsung have plans to refresh the hardware on a more regular basis than they did v1 because I'm sure we'll be finding ways in the next year to use up every drop of power and will be wanting either more powerful tables or some way to upgrade existing devices. 

Object Albedo

The byte tag is a common way to identify and track physical objects placed on the table.

The surface guidelines give specifics on the minimum size the black area around the tag data should be in order for it to be recognizable to the device.  However one thing that we noticed is that the surface had more trouble picking up and tracking tags when the background was a lighter color.  We think that the object is reflecting the light projected from the display. This light is picked up by the PixelSense sensors, making the tag unreadable.  Giving the tag more of a black border seemed to help this.  We also noted that on dark backgrounds the touch visualizations would create bright spots on the screen that the object would reflect back so we turned those off with the following code in the constructor of our main view:

TouchVisualizer.SetShowsVisualizations(this , false);

Tag tracking

The Surface v1 was pretty good at tracking fiducial tags as they are moved around the table by a user.  The SUR40 seems to be much less reliable in this regard. Aside from the obvious issue of no longer recognizing 128 bit identity tags, the 8 bit byte tags don't track particularly well.  They're easily lost when moving and sometimes even difficult for it to recognize right off the bat.   We did note that tracking got somewhat better if the entire object was black except for the actual tag elements, but it still wasn't where we wanted to be. Ironically as soon as we finished an implementation of object tracking that worked Luis Cabrera posted this solution. His solution takes advantage of Surface v2's enhanced finger and blob tracking and foregoes the use of tags altogether.  While this solution is valid, it didn't 100% solve our design challenge as we needed to be able to tell the difference between 4 of the same object(one corresponding to each player of a game).  Instead our solution was to put the tag on a white background and then once we've found the tag and a blob that contains the point where the tag is, start tracking the blob.  We still used the tag for orientation, but it seemed to be stable enough even with that.  The code for this tracking is a bit longer than I can post to the blog, but if people want to see it I'll see if I can put a sample together.

End Note

As with all things in technology the SUR40 represents a series of tradeoffs when compared to other touch technologies.  Object recognition is unavailable when using IR, laser, or capacitive technologies.  The SUR40 performs better than most IR for certain gestures such as rotation where finger occlusion can lead to undesirable results.  And obviously a 40-inch capacitive screen is cost prohibitive.  But on the flip side sometimes you have issues such as environmental lighting, touch lag, and portability to deal with when building on Surface.  Originally I wasn't sure the trade offs would be worth it, but I've come around to believe that it's a device worth seriously considering if you can play to its strengths, just like everything else.