Gestural Interaction

Essay: Mending Spaces

The highest bandwidth medium for communication available to human beings hasn't changed much, ever. It's a simple interface: the air between two people talking. Interface inventors have been playing catch-up to this magic for years without really knowing it. We've been praising telecentric eyes and single-fingered hands; attaching them to our bodies; calling them screens and mice. The effect of these prosthetics and their interconnections within the human organism is not yet clear—while reducing the bandwidth of some conversations they have extended the range of others; while strengthening the public voice they have lowered the level of discourse.

We are coevolving with technology, just like always. And just like always, through evolution we gain some things and lose others. We have gained information networks of great speed and portholes that fit in our pockets and sit on our laps. But the rich distributed forum that these intermediators (our cellphones and laptops) provide access to is subject to an unplanned and unwelcome gatekeeper. All those who enter must come alone.

We are being strangled by single-user interfaces. Consider Manuel Castells' Space of Place and Space of Flows [link]. Simply, Castells posits that the divide between local, real world spaces (like cafes, libraries, and sidewalks) and global, digital spaces (like websites, tweets, and phone calls) constitutes a fracture in our information processing capacity. The places where we eat, vote, and stroll are separated from the places where we read, write, and talk. The digitally mediated world where so much of our discourse now echos is too distant from the every day places where we buy bagels and chat with waitresses. And this is a problem.

Our computers connect people globally and separate them locally. Part of the reason why they separate us in physical space is because the physical languages that we use to communicate with them are too different from the physical languages that we use to communicate with each other. Small screens, tiny mouse movements, and social conventions make it easy to read private emails in public, difficult for collocated collaboration, and nearly impossible to create weak tie relationships [link].

Go to a local cafe and look around. Pick someone working on their laptop. How do you know if they are writing a novel, designing toothbrushes, watching a movie, or in the middle of a business meeting? The richness behind those cafe laptops is private; it's called up from the internet and hard drive platter, flashed out 83 times a second by LED illumination, filtered through thin matrices of liquid crystal, and delivered to the retina of a couple dozen individual users.

Luckily today pixels are cheap and computers are fast—after 40 years we are almost ready to move past the mouse. With new gestural interfaces like g-speak [link] we can create digital environments that are nearly seamless with architectural space. We can create hybrid spaces where groups of people can interact with digital objects, physical objects, and each other using a shared physical language that is both legible and scalable. At the MIT Media Lab, where we've been working with g-speak for a few months, we're very interested in understanding these potentials.

We are experimenting and getting revved up. What is so exciting is the ability to breakdown the essential dichotomy of interface and destroy the gatekeeper, freeing humans to do what they do best and stop the crippling effect of tools that disregard ganglia by the fistful (for example the little bundles of neurons that allow us to express ourselves bodily, receive nuanced impressions through touch, and navigate 3D space).

We can now imagine a future where there are no more interfaces, where Weiser's vision of calm computing becomes possible—a vision of human computer interactions that are both as peaceful and information dense as a walk in the woods [link]. We can imagine a mature interface language where we will no longer have to learn two dozen new commands for every gadget we buy, where instead we use a universal physical/gestural language that is body/space centric and not device/screen obsessed. It's a nice dream; a nascent future, full of possibility . . . in this dream the space of place and space of flows will no longer be so different—the digital world will emerge from its cloisters and take full citizenship in physical life. Simple things will change: intimate cellphone conversations on buses will only be as rude as the susurrus of married couples on airplanes, every sweater in your closet will have a url, and hands-free headsets will no longer be so dorky. We'll have shiny new eyes and limber new fingers. But eventually these too, like the ones before, will need mending.