Let’s get Tangible… or not?

Critical Review of Brygg Ullmer’s Thesis Defence on Tangible User Interfaces: What is Tangible, Physical and Embodied Computing?

Based on Brygg’s thesis defence, physical or tangible user interfaces can broadly be described as “spatially reconfigurable physical objects […] as representations and controls for digital information” (Brygg’s defence 10:50). So while most other areas in HCI are dealing with screen-based interactions, Tangible, Physical or Embodied Computing goes beyond that and includes various physical objects in the interaction. Looking back in history, the invention of graphical user interfaces (GUIs) moved physicality of early computing systems to the screen. Brygg’s concept wants to bring back some of the qualities of physical interactions. In particular, tangible user interfaces support direct manipulation of digital information using one’s hand. They also invite two-handed and multi-user interaction, either simultaneously or as a communication medium. Finally, they are situated in the physical world and thus have better support for the context they are used in.

Brygg’s concept consists of physical tokens and constraints. Tokens represent information that can be associated with a physical constraint, e.g. a block is put into rack where it can slide left or right, or be rotated. Tokens are then manipulated within this constraint. Brygg presents two systems that illustrate this concept. The first, called mediaBlocks, uses physical objects as containers for sequences of online media elements, as illustrated in the video below.

The second system is called Tangible Query Interfaces and uses container and parameter blocks to manipulate large amounts of data, in particular to express queries across a database. This video explains the concept:

Brygg concludes his presentation with the words “don’t be dogmatic”, meaning that TUIs are not to replace GUIs, but they have value in areas where the above mentioned qualities matter. Many approaches (including his own) might also be hybrid. While I can see the benefits of TUIs, I also see many open questions. Overall it seems that due to their physical parts, a TUI system is always limited to a very specific application area. It is hard to see how an “ecology of objects” with a specific “grammar”, as Brygg envisages, could be established to allow more general purpose TUI computing.

Overall, I got the feeling that the “Tangible Computing story” sounds a lot like the “Ubicomp story”, especially in the mediaBlocks video. In fact, tangible interfaces could be a way to realise the low-cost computing vision of Ubicomp. Instead of having hundreds of “tags” and “pads” lying around, just use “dumb” objects with markers and move the computing power to the cloud. But as already discussed in the case of Ubicomp, how realistic is this vision? It seems tangible computing suffers from the same kind of colonialism as Ubicomp, imagining our future life in the lab but not being really concerned with people’s live in the present. It is telling that Brygg developed his Tangible Query Interface system for a real estate application but never even talked to real estate person to find out if this is useful for their work. After all, almost 15 years have past since his thesis defence, but TUIs haven’t taken off – yet?

Conference Papers

I picked the following three papers for our fictional conference session on “Histories and Futures”. They exemplify Tangible Interaction systems to facilitate reflecting on the past and planning for the future.

Bennett, P. et al. 2012. ChronoTape: Tangible Timelines For Family History. Proceedings of TEI ’12 (2012), 49–56.

Chu, J.H. et al. 2015. Mapping Place : Supporting Cultural Learning through a Lukasa-inspired Tangible Tabletop Museum Exhibit. Proceedings of TEI ’15 (2015), 261–268.

Riedenklau, E. et al. 2012. An integrated multi-modal actuated tangible user interface for distributed collaborative planning. Proceedings of TEI ’12 (2012), 169.