Robin Adams, assistant professor of Engineering Education from Purdue University, showed up on campus today to talk about Iterative Design. Robin, a former colleague of Josh at the University of Washington, is interested in such things as design thinking and learning, strategies for connecting research on learning and teaching practice, and interdisciplinarity. In 2003, Robin won the Design Studies Award by the Design Research Society for “Educating Effective Engineering Designers: The role of reflective practice.” Other receipients include Erik Stolterman (“The Character of Things,” 1997) and Donald Schön (“Designing: Rules, types and worlds,” 1998).
Adams interest in design learning includes the pathways one takes to get from novice to expert. One of the ways she describes design is as three stages. The problem includes a process of definition and information gathering. The solution is where ideas are generated and modeled, and feasibility is studies. Finally, the realization includes evaluations, decision-making and communication about the design.
There aren’t many studies on iteration as a design practice (although Adams points to work by Bucciarelli, Radcliff & Lee, Dorst and Cross, Ball & Ormerod, Atman, Badke-Schaub and Frankenberger, Tjandra, and Brereton). Traditionally, engineers see iteration as something done during a specific point in the process of building something. But Adams’ work shows iteration occurs everywhere, with a higher frequency and diversity of iteration demonstrated by high-quality designers.
She describes her study as the “playground problem” — individual students are asked to design a playground in a three-hour session. They provide feedback to the researchers through “talk aloud” method, describing what they are thinking about as they go. There are two groups of a dozen students from different experience levels (freshmen and seniors). Their actions are videotaped, and multiple coders (85% inter-rater reliability) operationalize iteration in what they see on tape. Iterations begin with an information processing activity (monitoring, concepts, evaluation, etc) and result in a change to the design state. They are bounded by the moment where designer paused to reflect on some aspect of the design and where changes occurred. Reflective moments that did not lead to change were discarded.
The coding distinguishes between diagnostic and transformative actions, which basically means the design was either corrected or made inherently different. Coupled revisions are iterations that change both the process and the solution. The time, frequency and type of iterations were tracked for each participant in the study. Those stats were then correlated to a quality rating that reflected several factors, including meeting design requirements, matching to an expert model, and subjective assessment of original ideas (The high end of the normalized quality rating for each project was about .7).
“Effective” iteration is tied to student performance. A high-quality senior spent 40% of her time iterating and had a .585 quality rating. There were more places in the overall process (14) where iterations linked back to previous stages, and iteration took place throughout the process. The student had a willingness to locate potential failures and a high tolerance for the ambiguity of the project. On the other end of the spectrum, a low-quality freshman spent only 23% of his time iterating, generating a quality rating of .373 and forming few (3) iterative links between process stages. There was more evidence of bunched iteration late in the three-hour time block.
This all reflects, in a formal analysis, what designers arleady know: frequent reflection-in-action leads to better designs. The most interesting insight for me was the description of iteration as a conversation. I’m not sure if that was meant to mean a conversation between the designer and the design, or between problem and solution as facilitated by the designer. Either way, framing design as a conversation is a powerful hook for me.
