launched the Mystic Aquarium's "12,450 Feet Below" exhibit in
partnership with Delaney Designs and Mystic Aquarium. As part of this exhibit
we built five applications on the Microsoft Surface v2 platform using SUR40
tables. These applications included relatively simple photo and document
galleries as well as more complicated applications that utilized the tag
recognition features of the Surface platform.
thought I'd share some of our learnings from working with the surface tables.
They're very interesting devices and while you can download the SDK and
, there are some things that you
can't learn unless you have the hardware. Specifically I want to discuss
performance, tag tracking, and object albedo.
that I was surprised by was the power of the machines. Since Bob Ballard
and the Mystic Aquarium provided us with high resolution images and video we
were a little concerned about performance, especially since some of our
applications were going to take multiple instances of large high res imagery
and throw several pixel shaders on top of it in order to produce some nice
water effects. I do most of my development on a high end laptop and it
would chug, but the SUR40 handled it beautifully. I hope that Microsoft
and Samsung have plans to refresh the hardware on a more regular basis than
they did v1 because I'm sure we'll be finding ways in the next year to use up
every drop of power and will be wanting either more powerful tables or some way
to upgrade existing devices.
The byte tag is a common way to identify and track physical objects placed on the table.
guidelines give specifics on the minimum size the black area around the tag
data should be in order for it to be recognizable to the device. However
one thing that we noticed is that the surface had more trouble picking up and
tracking tags when the background was a lighter color. We think that the
object is reflecting the light projected from the display. This light is picked
up by the PixelSense sensors, making the tag unreadable. Giving the tag
more of a black border seemed to help this. We also noted that on dark
backgrounds the touch visualizations would create bright spots on the screen
that the object would reflect back so we turned those off with the following
code in the constructor of our main view:
TouchVisualizer.SetShowsVisualizations(this , false);
v1 was pretty good at tracking fiducial tags as they are moved around the table
by a user. The SUR40 seems to be much less reliable in this regard. Aside
from the obvious issue of no longer recognizing 128 bit identity tags, the 8
bit byte tags don't track particularly well. They're easily lost when moving
and sometimes even difficult for it to recognize right off the
bat. We did note that tracking got somewhat better if the
entire object was black except for the actual tag elements, but it still wasn't
where we wanted to be. Ironically as soon as we finished an implementation
of object tracking that worked Luis Cabrera posted this solution. His solution takes
advantage of Surface v2's enhanced finger and blob tracking and foregoes the
use of tags altogether. While this solution is valid, it didn't 100%
solve our design challenge as we needed to be able to tell the difference
between 4 of the same object(one corresponding to each player of a game).
Instead our solution was to put the tag on a white background and then
once we've found the tag and a blob that contains the point where the tag is,
start tracking the blob. We still used the tag for orientation, but it
seemed to be stable enough even with that. The code for this tracking is
a bit longer than I can post to the blog, but if people want to see it I'll see
if I can put a sample together.
As with all
things in technology the SUR40 represents a series of tradeoffs when compared
to other touch technologies. Object recognition is unavailable when using
IR, laser, or capacitive technologies. The SUR40 performs better than
most IR for certain gestures such as rotation where finger occlusion can lead
to undesirable results. And obviously a 40-inch capacitive screen is cost
prohibitive. But on the flip side sometimes you have issues such as
environmental lighting, touch lag, and portability to deal with when building on
Surface. Originally I wasn't sure the trade offs would be worth it, but
I've come around to believe that it's a device worth seriously considering if
you can play to its strengths, just like everything else.