by Scott Reinhard

Last year at this time, I was deeply thinking about the feeling of interactions. Coming off the first part of my career working in publication and print design, the luxury of using materials to convey ideas in physical form was something I missed in interactive, non-material–based design work. I never quite had the vocabulary to pair the form and behavior of a digital interaction with its subject matter the way that I could when working with paper, ink, and binding.

While graphic designers have long had a consistent kit of parts to work with (typography, grids, images, etc.), we risk losing a major opportunity going forward if we don’t develop an expressive interaction vocabulary as well.

Considering how people connect with digital interactions, I used my final year in graduate school attempting to fill in the holes of our interactive language. The joy of being a graphic designer is bringing a subject to life. With a deep understanding of the materials we work with, we have the ability to create a space where people derive meaning from the artifacts we create. The choices we make signal to the reader, viewer, or user something about the subject of our work. When we have greater means of expression and a wider vocabulary, we can provide richer experiences. To date, the field hasn’t developed a formal vocabulary that takes advantage of the affordances of digital interactions. We are left with our intuition. But within Human Computer Interaction (HCI) literature, it has long been theorized that people attribute certain emotions with the way interactive systems react to input. When all of the elements of an interface combine to create a larger experience than the individual components it’s called an interaction gestalt (Youn-kyong, 2007).

Through user tests, these researchers demonstrated their theories, although in neutral, lab-based experiments. The interactive studies that I developed in my graduate research built upon these HCI tests and confirmed that, in practice, manipulating the components of an interface does in fact produce experiences that elicit certain types of emotions and feelings. In short, through manipulating simple combinations of user interface behaviors, I could make an interaction feel heavy or light, soft or hard, and complicated or simple.

 

The implications of this research are potentially wide ranging. One could imagine that alongside typical branding guidelines—typography, logos, colors, and so on—an institution or organization could also develop interactive brand guidelines. They could define the behaviors of their interactive artifacts so that interactive experiences would feel appropriate to that particular organization across platforms and devices. For instance, the experience of buying a plane ticket through Southwest would feel different than the experience of buying a ticket with American Airlines.

In the research I did working towards my thesis, I took great influence from the discipline of visual rhetoric. An image contains certain associations and culturally understood meaning, and there is great power in how an image is used. I eventually started to think about considered decisions that affect meaning in an interaction to be interaction rhetoric. Essentially, what does it mean by how it acts? But now I realize that these questions can go on and on. What does the choice of a particular piece of technology mean to the user? How can designers use that to their advantage? How can a user experience be tailored to affect meaning? What part of the process of designing and producing interactive experiences has the greatest effect on meaning?

With the goal of a certain type of expressive interaction in mind, each step would inform the other

In my thesis work, my assumption was that a main way to shape an interaction was by manipulating the behavior of interface elements, the nitty gritty details of producing interactions. I assumed that the physical materials were out of my control and therefore did not have influence on the meaning that a person takes away from his or her experience. After graduating, my work on the interactive media team in the multi-disciplinary, New York-based design studio, 2×4, changed this assumption. The focus of our work at 2×4 is interactive experiences in architectural spaces, be that in retail, cultural, or other types of public arenas. Through working with architects, strategists, technologists, environmental graphic designers, and interaction and user experience designers, our interactive and technological contributions weave into the greater project experience. Our impact isn’t in the vacuum of an interface. There’s a physicality to these types of interactions that expands my understanding of interactive experiences. Once again, I’m faced with decisions of physical materials and technology that have bearing on the meaning that a person takes away from his or her experience. User experience in these instances isn’t just the journey through an immaterial virtual space but a physical one, where digital interventions aid and enhance the journey.

Many of the decisions about the feeling of particular interactions often are saved towards the end of the process, when the user experience, user journeys, technology, and visual design are already in place. At that point, I wonder whether relying on the interface behaviors to create an overall mood of the interaction is effective enough. In an ideal setting, discussions of these areas of a project would take place simultaneously. With the goal of a certain type of expressive interaction in mind, each step would inform the other.

The idea of a gestalt remains the most important principle in my understanding of interactions. Every choice influences the overall experience. So now with added complexity to interactions in physical spaces, the question remains the same: What does each design decision do to affect meaning for a person? Explorations into formalizing the vocabulary of user experience and technology are certainly warranted, but designers do not need to wait for this research to begin asking important questions of meaning to the user in every detail of their work.

References

Lim Youn-Kyung, Erik Stolterman, Heekyoung Jung, and Justin Donaldson. “Interaction Gestalt and the Design of Aesthetic Interactions.” Proceedings of the 2007 Conference on Designing Pleasurable Products and Interfaces – DPPI ‘07 (2007): n. pag. Web.