A team at Tokyo University recently developed a device that translates in-air finger movement to input commands so that the device can be used without physically touching the screen. Unlike Sixth Sense, this system does not require surrounding sensors or finger markers. According to Gigazine in an article translated from the original Japanese, the device captures the finger movements with one 154 fps high-speed camera.
The development is definitely notable for not requiring additional sensors and markers, but in watching the demo it is also interesting to think about how gestures will evolve in a touchless system. The Sixth Sense demo showed Pranav Mistry tapping a phone interface projected on his hand. Even though it wasn’t technically necessary, he was still touching the physical surface of his hand. The Tokyo University demo interprets a traditional mouse click motion in air as a selection, which works for demonstration purposes but looks awkward and cumbersome in practice. As technology catches up with science fiction, it is time to figure out what gesture is a click.
One of the more interesting things to come out of the NRF convention last week was a retail digital signage proof-of-concept from Intel.
The prototype system combines a translucent holographic overlay display next to a more traditional LCD screen, allowing for different types of interactive content to be delivered on the different screens. The concept demo shows contextual information on the holographic display, overlaid onto the physical items that are visible through the glass. The LCD portion of the installation is evidently intended to have screens on both sides, allowing the unit to support up to three simultaneous users.
The interaction evidently relies heavily on gesture more than on touch. According the Intel’s documentation on the concept, the LCD screens allow customers to use a swipe gesture to scroll through the available advertisements or digital media. The holographic screen, while it behaves like a touchscreen is also evidently gestural, recognizing “pointing” gestures that function the same as touches. This informational overlay onto the physical space behind is basically augmented reality, very similar to a number of mobile experiences that have been gaining traction of late.
Finally, all of the screens use embedded cameras in order to both customize the experience and capture analytics for future analysis. Placement of the selection interface on the holographic screen is adjusted to the customer’s height, for instance. The cameras can also recognize gender and approximate age, so the content on the LCDs can be targeted to the customer. And the system can capture dwell times for the content providing marketers a feedback mechanism for how well each ad is being received by viewers, and the related demographic info. This functionality is all evidently powered by CognoVision, a Canadian company focused on that space.
Intel, while not a manufacturer of interactive signage hardware directly, of course has a vested interest in seeing more powerful computing systems installed in more places, especially if they are powered by Intel processors. This particular concept, which was also briefly presented during Paul Otellini’s keynote at CES the week before, is evidently all driven by a single dual-core i7 chip.
Intel has actually built a number of interesting proof-of-concept units to demonstrate the power of the i7. At CES this year they had a dual-screen touch wall that allowed multiple people to touch little cubes of content.
And at last year’s CES show they debuted a holographic touch screen similar to the one in the retail concept above. It’s primary function was to show off a real-time 3D rendering engine that was powered by an i7 processor.
It’s not Intel’s first stab at envisioning what the future of retail digital signage will be either. Earlier this year they published some renderings and documentation of another concept intended for stores. This one also seems designed for three simultaneous users, although it’s definitely more pedestrian.
Although not intended to actually hit the streets, Intel’s latest retail concept is an interesting vision of the relatively near future of digital signage installations in retail environments. In contrast to a lot of the digital media that has made its way into retailers thus far, Intel’s concept is highly interactive, and because it’s multi-user it offers the potential to build collaborative experiences as well. It incorporates a number of innovative technologies that we have been paying attention to – gestural interfaces, face tracking, augmented reality.
Along with presenting this concept, Intel also announced a partnership with Microsoft for its Windows Embedded Standard 2011 operating system and a stronger commitment to the digital signage space in general. That’s good news for those of us who depend on more capable and more reliable hardware platforms on which to develop the next generation of interactive experiences in public spaces.
For a fun diversion, the Schematic New York technology department invited Becky Stern to the office to give a soft circuits workshop. Becky works as associate editor for MAKE and CRAFT magazines and the pop-up NYC Gizmodo Gallery recently featured her TV-B-Gone Hoodie.
While companies like Philips are making high tech LED fabrics like Lumalive, the tools to make simple wearable electronics are extremely accessible. For our workshop, Becky provided conductive thread (from a fencing supply store), LEDs, a battery holder and a coincell battery. All it takes is simple electronics knowledge (no multimeter necessary) and a sewing needle to create a circuit. All of the materials, except the battery, are washable and can be made removable with everyday materials like hooks and eyes or button snaps, which are conductive. Becky also showed some more advanced examples using tools like the LilyPad Arduino, toggle switches and surface mount LEDs.
Pictures of our work are below. As you can tell, we sure like our flair.
In September, I had the opportunity to see James Powderly and Zach Lieberman present their work in progress, The EyeWriter at the 2009 Gadgetoff. The EyeWriter is an inexpensive open-source eye tracking headset and software created by members of Graffiti Research Lab and the team behind openFrameworks.
The EyeWriter was created to enable graffiti artist Tony Quan, aka Tempt One, to continue to write graffiti despite the fact that he is completely physically paralyzed due to ALS. DIY EyeWriter instructions were released in November by the team. Right after Powderly and Lieberman presented, magician Marco Tempest presented a live augmented reality magic performance.
Tempest wore an eyeglass rig similar to Tony Quan’s and created the augmented reality software through collaboration with Zach Lieberman and Theo Watson from openFrameworks. What struck me most, besides the truly inspiring story of Tony Quan, was how interesting to see one set of open source tools used for two such different purposes. Previous similar open source releases like L.A.S.E.R. Tag were made by artists for artists to entertain audiences whereas the eyeWriter is actually using these same creative technologies to fill a social need.
Comprehensive eyeWriter documentation can be found at www.eyewriter.org.
The Fun Theory is an experiential marketing campaign produced by VW to help make the connection between the importance of fun in everyday life and how fun can be a transformative power for improvement. This translates well for VW since they have long established themselves as the ambassadors of driving fun, an everyday task easily taken for granted. To help communicate this message VW has developed several digital out of home experiences that are targeted at everyday activities. The fact that they digitally enabled activities that do not require instructions to conduct ensured a high usage.
The one DOOH activity that I enjoy the most is the Piano Staircase. This digital, audio experience is part musical, part physical with an element of onstage performance. The experience can be summarized as a digital keyboard placed on top of a staircase, encouraging the audience to play music as they walk, run or hop up the stairs instead of using the escalator. The first time I saw this I immediately thought of Tom Hanks in the movie Big.
Without this video most of us would not know about this experience, but it also helps with the theory/evidence part of this story – to see the foot traffic over time using the staircase really makes the point.
The decision to make a compelling DOOH activity that are rolled out in a single market for a short period of time with the sole intention to extend that experience worldwide via online video is smart. The activities are easily interpreted by watching others use them and in some ways you get the point more by watching a well-produced summary clip. This also keeps the cost down compared to a multi-location deployment. I think you lose much of the buzz factor that DOOH is good for, but that buzz comes at a price.
One thought… since the main character of this story is the piano (a popular musical instrument worldwide) I would like to have seen Piano Staircases deployed in several major markets simultaneously e.g. London, New York, Tokyo, etc. and facilitated as a public platform for group-based performances. Everything would be captured and shared via audio and video streams so that every note could be heard as part of a worldwide continuous public concert. Now that has legs.
RSS Feed
Email