The bad news side of a great conference experience is there is a lot to process in a short period of time. After a week at CHI 2007 followed by a week at Kosmix for a user experience internship (mostly telecommuting), I’m only now opening up by big red book of notes to figure out how what I saw in San Jose is relevant to me. The one talk that is most on my mind at the moment is the “Research through design” talk given by three CMU profs, John Zimmerman, Jodi Forlizzi and Shelley Evenson. It was particularly nice to hear as a companion in the same session to Eli Blevis’ paper on sustainable design.
The CMU trio were clear in their foundational definitions of designer, design research and design thinking. Appropriating the understanding within the design community, designer is someone with training or extensive practical experience in architecture, product design, graphic design, or interaction design. Design research is the act of producing knowledge through design, rather than focusing on improving commercial product. Design thinking is about framing a problem through a process of grounding, ideation and iteration. Their framework for defining design research within the HCI community is culled from four years of workshops, interviews and a review of literature. And like Eli’s work, they believe design to be the intended transfer of a current world state to a preferred one.
The HCI Research Model they put forth uses the artifacts of design research—models, prototypes, products and documentation—to bound the HCI Researchers to the HCI Practitioners. The artifacts in this model are the main means of communication between the two groups. Researchers create many kinds of artifacts in many different ways, and practitioners take the conclusions of those artifacts to apply to product creation and improvement. Because this is specifically targeted at HCI folk, the Interaction Designers are the focal point of the research process, communicating with three other groups—engineers, anthropologists and behavioral scientists—to create these artifacts. Engineers are interested in how technology works, generating new implementations of technology in response to the technical opportunities seen by the interaction designers. Anthropologists collect field data that describe the reality of the world, some of which involves or is provided by interaction designers, too. Models and theory is generated by behavioral scientists, who seek to prove the dynamics and factors influencing the world. Interaction designers play with these ideas and report back the gaps and unanticipated effects the models produce.
According to Zimmerman, Forlizzi and Evenson, there should be a criteria to help the HCI community better understand how knowledge is being contributed. This evaluation has four components: process, invention, relevance and extensibility.
- Process—A process is something that is reproducible by others. Unlike hard science, it isn’t the actual result that should be predictable but rather the way at arriving at contextual results in a reliable manner. Process should be documented and include a rationale for decisions made about methodology.
- Invention—Significant inventions are those that can articulate their contribution in a specific context. That context must be culled from an examination of the existing literature to explain why the concept or design is new.
- Relevance—To be relevant, research must understand both the existing and the preferred state achieved through design. The CMU trio believe that this is an underrepresented aspect of current HCI research, as authors fail to explain why the community should accept the outcome of the intended design as preferred.
- Extensibility—If work is extensible, it means the ideas can be employed in future design problems, extending the knowledge available to practitioners to make better products.
Three of these criteria are vital, in my opinion. If work is not both relevant and extensible, it will be difficult if not impossible to apply any of the conclusions. If research does not clearly state the methods used, there is not incentive for the process to be duplicated in future study. These tools for evaluation will be helpful in writing future papers.
The part of this framework that may be most in question for me is the idea of invention being a requirement. If there is a new contribution to make, it should most certainly be articulated. However, is it also “new” to take an older study and attempt to repeat it, confirming or refuting it’s outcome in a modern context? If one can imagine this framework used by CHI reviewers, for example, the concept of new is relative to their own expectations and past experiences.
As was clarified in the Q&A, this isn’t intended to necessarily create a new set of activities for people to do, but rather to better explain the contributions we are making in a way that others can understand. Nor is this framework cast in stone. Some of the terminology—like “model”—are overloaded and add confusion. However, this framework for evaluating our work seems like it might be effective both to focus research and explain it to the next generation of researchers currently matriculating in institutions.
