Designing Gestural Interfaces

T-Mobile unveiled this transparent (!) wall interface at their CeBit 2008 booth a few days ago. Brief Wired article on it:

So how did the companies keep everyone’s attention? By letting people play with touchscreens, and believe it, nothing is hotter right now than touchscreens. In the T-Mobile area, an extended Media Wall was placed in the middle of the space, but the most interesting part about it was its transparency, which we haven’t seen before. We’re not sure if this was a Han Wall or not, but it seemed to work as a good advertising model.

ZD Net rumor:

Microsoft officials have been talking up the company’s plans to introduce more consumer-focused form factors of its Surface multi-touch tabletop. They’ve said future iterations of Microsoft’s Surface multi-touch technology will be available as part of next-generation PCs, cellphones, desks, kitchen counters and even walls in consumers’ homes over the next five to ten years.

Microsoft hasn’t been talking as publicly about even more offbeat Surface form factors, such as a sphere. But sources say the company is showing a spherical Surface prototype around campus lately.”

Crystal ball anyone?

BBC Video about large touchscreen technology that responds to a variety of styluses: “Anthony Uhrick from Next Window describes a new multi-touch display that uses light-sensors to allow users to “hold” their work in their hands.”

From a Wired article on the GSMA:

Most apparent at the show, the biggest mobile conference in the western world, is that nobody is doing touch screens properly.

Sure, Apple didn’t invent touch screens, but it was arguably the first company to do it right. Sony Ericsson, no latecomers to the touch game, showed myriad new phones today, and of all of those we tried out, the UI was invariably clunky, counter-intuitive, or downright hard to navigate.

Flashy, animated icons are great, but not if they come at the expense of usability. It feels like everyone is scrambling to add touch capabilities because they feel they have to, ease-of-use be damned. The point of the iPhone is being missed: It’s a pleasure to use because of the fancy UI, designed from scratch to be intuitive, attractive and easy.

As part of Chapter 2, there is a (by necessity incomplete) section on basic human gestures that designers can flip through for inspiration when designing gestural interfaces. My photographer Sarah Nelson and my model Ellen Ho did a great job with the photo shoot, and I think the images are a real asset to the book. A sample:

In advance of my Tap is the New Click presentation at ETech, O’Reilly has graciously allowed me to post a draft of the first chapter of this book (now titled) Interactive Gestures: Designing Gestural Interfaces. It’s pretty much my unedited first draft, but I think it reads pretty well. Comments welcome, of course!

Download Chapter 1 (5.4mb pdf)

Elliptic Labs is showing off a new sensor technology that detects motion in 3D, and requires no special worn-sensors for operation. From the Gizmodo post:

By simply pointing at the screen, users can manipulate the object being displayed in all three dimensions. Sensors are mounted around the screen that is being used, by interacting in the line-of-sight of these sensors the motion is detected and interpreted into on-screen movements. What is to stop unintentional gestures being used as input is not entirely clear, but it looks promising nonetheless. The best part? Elliptic Labs says their technology will be easily small enough to be implemented into cellphones and the like. iPod Touchless anyone?

Neat. One wonders if in ten years anyone will have a mouse at all.

Not sure exactly what’s going on here, but it sure looks interesting. To whit:

Not only Nokia has found a way around most of the things that Apple may get IP protection on, they seem to be on the way to develop user interface technology that goes well beyond touch.

How about the multi-touch like mobile handset UI that does not actually require touch, can track your hand movements in three dimensional space, and even recognize 3D hand gestures?

…

Different finger combinations can be used also as a shortcuts to various applications. Make a letter C with you fingers and your calendar application launches, use a “V” sign to launch an address book. And if the operation of the device grinds to a halt and starts to really piss you off, just give him a “finger” and it resets to start anew.

Nifty. I’ve done some projects with Nokia and never caught a whiff of this. Cool.

If you haven’t seen this, take a look at this interactive store window brought to you by Orange and The Alternative. “UK based agency ‘The Alternative’ has developed a gesture based interaction screen for Orange. It is for the first time that such a technology is showcased to the general public says a spokesman for the agency. The large projected image has a touch free interface and provides users a new way to activate news, film clips and music videos.”

As I write the book, I’m also preparing an introductory talk about interactive gestures that I am pleased to be presenting in a number of awesome venue. With touchscreen interfaces exploding at CES and everywhere else, this information should get out sooner rather than later!

I’m presenting Tap is the New Click at:

• O’Reilly’s ETech on March 4. Use my discount code of et08fds and get 20% off.
• Indiana University School of Informatics on April 2
• The always-fun WebVisions on May 22-23
• Design IT in Tokyo in June

If you’d like me to speak at your event, contact me and we’ll try to arrange it.