Liveblog: Phoebe Sengers “Representation and Response”

NOTE: This is a liveblog of today’s School of Informatics colloquium by Phoebe Sengers (Cornell), “Representation and Response.” Content will be updated throughout the talk, which begins at 3p at Informatics 107.

SUMMARY:
Really important to recognize that representations as mirroring reality is a metaphor (with limitations)
1) recognize and work around typical breakdowns
2) Open a wider design space (think of other metaphors)
3) Raise and answer questions about whose interpretation is correct (comes from a point a view, ask questions about who’s perspective and why we choose representations)

Notes:
Phoebe Sengers
Cornell University
“Representation and Response”

Any faculty who expected to arrive at 3 and get a seat needed to be in Informatics a half hour ago. The small room is packed with students, including a number of undergraduates. … Students are now being asked to volunteer to take seats on the floor (by the fan!) to free up chairs for faculty.

talk based on a book Phoebe is working on with Kirsten Boehner
Shif from seeing computational representations as directly mirroring the objective state of the world …

Examples:
* Proactive Home: embed sensors in an older person’s home to monitor activities, communicate that state to the caregiver
* Home Health: sensors embedded in a person’s home, detect patterns of activity, guess emotional and social well-being of the home, presenting info back to users in the form of a horoscope (ambiguous, permits reflection)

Different goals: reporting of state (mirroring) to reporting on context (let people interpret)
– limitations of mirroring – example: Facebook friends (lots of flavors, peers, intimidated, potential, etc)
Example: use of network analysis to find potential terrorists

Suggest shifting representations as a response from the computer … suggest something sensible, rather than stick the entire world in the computer / correspond with something actually happening.

Goals for today:
1) how did we get here (transparent mirroring)
2) recurrent patterns of trouble around this
3) how to design using representation as response
4) How to evaluate using representation as response

1) History of computing:
* goes back to 1949 Shannon/Weaver theory off information (source-transmitter-signal-noise-reception-receiver-destination message)
* information is not meaning
* Shannon: fundamental problem of communication is reproducing on the other end
* Weaver: engineering aspects is not irrelevant to the semantic aspects (Shannon didn’t mean to solve the problem of meaning, but did solve it)
* conduit theory of language – if all goes well, the representation in my head can become the representation in your head … we apply this to human-computer interactions, too (pervasive metaphor across computing sciences)
* Why is this metaphor pervasive? … because it works (it is a useful model)
– gives guidance in approaching a design problem, find real world representations and model in computer
– check results in computer against real world (evaluation)

2) Trouble:
– works well in artificial environments, but much more complex in the real world (ex. robot Shaky – took him forever to make it across the room, 1970s, had to represent everything)
– Rodney Brooks robots … modeled little of the world, but responded in real time … “The world is its own best representation”
a) it’s hard to maintain correspondence between representations and the world … have to keep representation updated all the time
b) arose out of Suchman’s work with situated actions … assumption that the representation of a copier is the same as the users, but the users were reacting to what they see, not following a plan
– CSCW … Bannon (Politics of Design)
– shift authority from situated activity to rigid representations (ex. telephone menu help systems)
From Boehmer et al … notion of emotional state is formal and often simplistic notion of what people actually feel in the real world (happy may not be happy, sad may not be sad)
c) flattens complexity and richness of human experience

ex. viet nam automated war plane – faith in computer representation (detect motion with sensors, send plane to bomb) didn’t match reality (viet cong fooled the system with tape records, planes bombed the wrong places)

What should we do today?
* “Third Wave” HCI – looking at tech in work settings, already have structures that channel people’s activities, now activities are very contextual
* move away from computational rep. as MIRRORING the world … to constructing contingent representations to respond in useful and interesting ways to what is happening

3) Strategies:
– avoid direct representation … Ian Lee’s MoodJam (communicate mood to you by selecting colors into stripes, changing moods by changing patterns over time, requires some knowledge of the person to interpret)
– take a stance … mateas/Bohlen’s Office Plant #1 (reads email, classifies it by social state [spam, chatty, etc], changes its state based on its interpretation of your email state, slowly and not perceived, not an alert system)
– use ambiguity to open representation … Home Health (has built-in ambiguity by presenting state as a horoscope, suggests an axis to think about without taking a stand on where you are in the axis)
– open representations directly to interpretation … Leahu and Schwenk Biomaps (track people’s GSR skin reactions as they walk around campus, sweaty palms, detects how aroused people are, asked users what the heat map meant … started with “it doesn’t mean anything” but later came up with meaning, can still discount things that don’t have meaning)

Range from not representing to over representing to allow for interpretation

4) Evaluating with representation as response
Evaluation: How do we know the system worked? … notion of “working” shifts (no longer that the representations match)
– the Bill Clinton it-depends-on-what-is-is
Kaye: Epistemologies of evaluation
– who are we? what is known?

Evaluation with representation as mirror:
– requires a fundamental faith in a world that is available and can correspond 1:1 with what is in the computer … job of evaluator: establish objective facts …
– know = ascertaining objective facts about the world
– we = the expert

How HCI interprets the cultural probes:
– create highly designed artifacts (dream recorder, listening glass, etc) … people react in unanticipated ways, lead to design inspiration
– original intention of methodology: express through design of artifacts, users would respond with what they think is pertinent or useful, designers in turn create new designs (using user response as inspiration)
– but, instead of responding to what they heard from users … try to fix the interpretation of meaning (shift between these perspectives)

Evaluation as Response:
-also experts, spectators, evaluation -> dialogical notion of truth (together, there is a co-construction of what the truth may be)
– know = dialogical construction of truth
– we = many participants
– evaluator is a provoceteur

Strategies:
– elicit multiple truths … cultural commentators (Gaver) … multiple ways of approaching the same situation
– present all data for co-interpretation … Boehner’s Dynamic Feedback (interactive diary, report back to the evaluation and explain what is going on in the system, allow all participants to view all the data, users see how other users react, use of the diary exploded compared to before interactivity)
– collect meaningless data … Isbister et al. Sensual Evaluation Instrument (worked with sculpter to suggest different kind of emotions, not clear what they mean in itself, ask players to grab the blob based on what they are feeling, interview based on transcript of the game, becomes a tool/trigger for discussing how they feel, discussion then matters more than how closely the representation fit the feeling)

Questions:
1) Talk about accountability, how wrong applications/misinterpretations come into play … if someone interprets something in a horrific manner, what are the consequences.

The risk is less if you recognize the nature of the representations. … if there is a misunderstanding, more likely to say “your system told me I’m happy” … personal responsibility for the interpretation. Not inherently true that transparency does away with authority.

2) Many of the things here are in practice

I do not believe that what I am describing is a shift for which I am personally responsible. Not, “looks here’s an idea that no one has thought of” but more that we need to recognize the history and that there is a fundamental difference in how we view and use representations. … Not just a design shift that can be understood in the same ways we always thought of design.

3) This audience is asked to distinguish between science, art, design … Do you see a distinction between these techs and evaluation we might use for art?

Biomaps were inspired by digital art, out of a critique of a digital map of London that turned emotions into static representations. … mirroring is only one strategy among many. … I don’t think the distinctions comes down to art one way, science another way.

4) In organizational dynamics, situation where culturally diverse groups brought together. Privileged actor is a designer. Is there a way to bridge gaps between culturally diverse groups? Shared representations.

In evaluations we have done, our goal is not to start with a fixed system but to bring other perspectives in. … I can understand how this talk looks like a design point of view. … It already works without a designer. In a cultural mode where designer originates technology.

Abstract:
Information technologies are often designed based on a metaphor of one-to-one correspondence between internal representations and the world of human activity. This metaphor structures commonly-chosen strategies for design (i.e., sense, represent, and manipulate symbolic states corresponding to real-world entities) and evaluation (i.e., check whether the internal representation is a true representation of the human world). Yet at least since roboticist Rodney Brooks’s cry that “the world is its own best representation” led to walking robotic insects in the 1980’s, questions have been raised not only about the accuracy but also the value of designing systems by explicitly modeling the environment in which they will be embedded. Using examples from what Boedker terms ‘third wave’ HCI research, I will explore the implications for HCI of explicitly and systematically shifting from a metaphor of representation to one of response. What design strategies are useful if we shift from accurately acquiring, representing, and reasoning about human activity to responding evocatively to human activity? How can we evaluate the resulting systems if we no longer count on them to have correctly understood and processed human activity? I will argue that not only our methods, but our notion of HCI as a science must be rethought.

Biography:
Phoebe Sengers is an assistant professor in Information Science and Science & Technology Studies at Cornell, where she leads the Culturally Embedded Computing group. Her work hybridizes HCI research and critical studies of technology. She uses insights from cultural analysis of IT to identify and rethink the assumptions underlying technologies; to build new applications for computing, including systems to support reflection on the environment; and to develop new techniques for designing systems, including design and evaluation methods to support open interpretation. Sengers received her Ph.D. in Artificial Intelligence and Cultural Theory from CMU in 1998.