During my undergraduate work at Tufts I spent a lot of my time developing a theory of contemporary user interfaces with my advisor, Robert J. K. Jacob. His research in this area is called Reality-Based Interaction.
Here is an excerpt from our CHI 2008 paper:
Over the past two decades, HCI researchers have developed a broad range of new interfaces that diverge from the "window, icon, menu, pointing device" (WIMP) or Direct Manipulation interaction style. Development of this new generation of post-WIMP interfaces has been fueled by advances in computer technology and improved understanding of human psychology. Some examples of these post-WIMP interaction styles are: virtual, mixed and augmented reality, tangible interaction, ubiquitous and pervasive computing, context-aware computing, handheld, or mobile interaction, perceptual and affective computing as well as lightweight, tacit or passive interaction. Although some may see these interaction styles as disparate innovations proceeding on unrelated fronts, we propose that they share salient and important commonalities, which can help us understand, connect, and analyze them. We believe that all of these new interaction styles draw strength by building on users' pre-existing knowledge of the everyday, non-digital world to a much greater extent than before.
- Jacob, R. J. K., Girouard, A., Hirshfield, L. M., Horn, M. S., Shaer, O., Treacy, E. S., and Zigelbaum, J. Reality-Based Interaction: A Framework for Post-WIMP Interfaces. Proc. CHI ‘08, ACM Press (2008), 201–210.
- Jacob, R.J.K., Girouard, A., Horn, M., Miller, L., Shaer, O., Treacy, E., and Zigelbaum, J. What Is the Next Generation of Human-Computer Interaction? interactions, 14, 3, (May 2007), 53–58.
- Jacob, R.J.K., Girouard, A., Hirshfield, L.M., Horn, M.S., Shaer, O., Solovey, E.T., and Zigelbaum, J. Reality-Based Interaction: Unifying the New Generation of Interaction Styles. Ext. Abstracts CHI ‘07, ACM Press (2007), 2465–2470.