When our research babies fail

For this weeks blog I chose the paper “Anatomy of a Failure How we knew when our design went wrong, and what we learned from it” by Graver et al.

 

It was refreshing someone critically reflecting on the unsuccessful aspects of their design. The paper highlight the difficulty in assessing the success of interpretive systems. Methods for doing that are open to relativism that make it difficult to firmly say “This project was a success”. The paper groups evaluation of user engagement based on (1) engagement, (2) reference, (3) accommodation and (4) surprise and insight and highlights that evaluation based on these features is inconclusive. They do this by evaluating their prototype the Home Health System which is based on the first Gaver paper we read the household horoscope. The technology measures activities around the house such as the time that a door is spent closed and then an algorithm would output it as a score measuring the wellbeing of a house, photographs and short statements. However, the outputs are open to interpretation by the household members. For example the participants did discuss the outputs and that could be seen as successful engagement but most of the time they were puzzling over the results. They were also uncomfortable discussing it as they worried to offend the authors. Additionally, when they referred to the technology in relation to other technologies that they disliked. Additionally, the participants did not accommodate for engaging with the technology in their daily lives as part of their routine. It also failed to occasionally surprise the participants.

 

The authors attributed this failure in that the outputs were either perceived as meaningless or dull. Additionally, the sensors were obvious and did not fit in within the house. The authors state that this was because they had a design for research approach rather than a design as research.

 

The ability to critically analyze our own research (or I would like to call them our own work babies) is essential and it allows other researchers to learn from the mistakes of others and therefore broadening the idea of building on the work of others.

 

I chose this paper because my experience in research has taught me that research projects rarely go as planned so it was interesting seeing how the likes of Gaver express this by connecting it with theory.