Interactive Gestures: Designing Gestural Interfaces

From a Wired article on the GSMA:

Most apparent at the show, the biggest mobile conference in the western world, is that nobody is doing touch screens properly.

Sure, Apple didn’t invent touch screens, but it was arguably the first company to do it right. Sony Ericsson, no latecomers to the touch game, showed myriad new phones today, and of all of those we tried out, the UI was invariably clunky, counter-intuitive, or downright hard to navigate.

Flashy, animated icons are great, but not if they come at the expense of usability. It feels like everyone is scrambling to add touch capabilities because they feel they have to, ease-of-use be damned. The point of the iPhone is being missed: It’s a pleasure to use because of the fancy UI, designed from scratch to be intuitive, attractive and easy.

As part of Chapter 2, there is a (by necessity incomplete) section on basic human gestures that designers can flip through for inspiration when designing gestural interfaces. My photographer Sarah Nelson and my model Ellen Ho did a great job with the photo shoot, and I think the images are a real asset to the book. A sample:

In advance of my Tap is the New Click presentation at ETech, O’Reilly has graciously allowed me to post a draft of the first chapter of this book (now titled) Interactive Gestures: Designing Gestural Interfaces. It’s pretty much my unedited first draft, but I think it reads pretty well. Comments welcome, of course!

Download Chapter 1 (5.4mb pdf)

Elliptic Labs is showing off a new sensor technology that detects motion in 3D, and requires no special worn-sensors for operation. From the Gizmodo post:

By simply pointing at the screen, users can manipulate the object being displayed in all three dimensions. Sensors are mounted around the screen that is being used, by interacting in the line-of-sight of these sensors the motion is detected and interpreted into on-screen movements. What is to stop unintentional gestures being used as input is not entirely clear, but it looks promising nonetheless. The best part? Elliptic Labs says their technology will be easily small enough to be implemented into cellphones and the like. iPod Touchless anyone?

Neat. One wonders if in ten years anyone will have a mouse at all.

Not sure exactly what’s going on here, but it sure looks interesting. To whit:

Not only Nokia has found a way around most of the things that Apple may get IP protection on, they seem to be on the way to develop user interface technology that goes well beyond touch.

How about the multi-touch like mobile handset UI that does not actually require touch, can track your hand movements in three dimensional space, and even recognize 3D hand gestures?

…

Different finger combinations can be used also as a shortcuts to various applications. Make a letter C with you fingers and your calendar application launches, use a “V” sign to launch an address book. And if the operation of the device grinds to a halt and starts to really piss you off, just give him a “finger” and it resets to start anew.

Nifty. I’ve done some projects with Nokia and never caught a whiff of this. Cool.

If you haven’t seen this, take a look at this interactive store window brought to you by Orange and The Alternative. “UK based agency ‘The Alternative’ has developed a gesture based interaction screen for Orange. It is for the first time that such a technology is showcased to the general public says a spokesman for the agency. The large projected image has a touch free interface and provides users a new way to activate news, film clips and music videos.”

As I write the book, I’m also preparing an introductory talk about interactive gestures that I am pleased to be presenting in a number of awesome venue. With touchscreen interfaces exploding at CES and everywhere else, this information should get out sooner rather than later!

I’m presenting Tap is the New Click at:

• O’Reilly’s ETech on March 4. Use my discount code of et08fds and get 20% off.
• Indiana University School of Informatics on April 2
• The always-fun WebVisions on May 22-23
• Design IT in Tokyo in June

If you’d like me to speak at your event, contact me and we’ll try to arrange it.

From BusinessWeek: A Touching Story at CES

The consumer electronics industry came unbuttoned in Las Vegas this week. At the annual Consumer Electronics Show, which kicked off Jan. 7, companies from around the world unveiled myriad devices boasting touch-sensitive technology, rather than traditional push-button controls.

Motorola (MOT), Sony (SNE), and LG Electronics all showed off touch-screen phones this week. Microsoft (MSFT) Chairman Bill Gates touted a touch-screen computer, the Surface, akin to a flat panel TV for a tabletop. Even camera makers, such as Kodak (EK), included touch-screen LCDs in their devices. Touch-screen tech has been in use for years, but the impetus to develop and weave it into consumer electronics gathered steam with the June introduction of Apple’s (AAPL) iPhone. “The touch-screen industry is really getting hot,” says Jennifer Colegrove, a senior analyst at iSuppli. “Since the iPhone came out, there are a lot of companies that did not have a relationship with touch screens before that decided, ‘O.K., I want to make a touch screen, too.’”

I’m in the middle of writing chapter one of the book, which is an introduction to interactive gestures and gestural interfaces. Chapter one currently includes:

• Introduction
• Direct and Indirect Manipulation
• A Brief History of Gestural Interfaces
• The Mechanics of Touchscreens and Gestural Controllers
• The Attributes of Gestures
• The Ergonomics and Usability of Gestures
• and How to Determine the Appropriate Gesture

Is there anything else you’d like to see in an introductory chapter on this subject?

I am very pleased to announce that I am writing another book: Designing Gestural Interfaces for O’Reilly, scheduled for publication in Fall 2008. The book is based in part on the Interactive Gestures wiki that I started several months ago with A Call to Arms. This site will document the book’s progress and any news about publication.

What the book is about: Nintendo’s Wii and Apple’s iPhone have introduced the public to the power and grace of using gestures to control devices and interactive systems. But how exactly do you design for this new interaction paradigm? It isn’t like designing for traditional websites or software, but until now, there has been little written about these new interfaces, which will only grow in number and variety over the next few years.

Designing Gestural Interfaces will examine current patterns and trends in this new area, discussing emerging patterns of use, a taxonomy of human gestures, how to design and document interactive gestures, an overview of the technologies surrounding touchscreens and interactive environments, communicating to users how to use these new systems, and ways to prototype gestural interfaces.