i:LAB learn Kitchen

Ambient Kitchen

The Ambient Kitchen is a platform for research in pervasive computing that was installed at Culture Lab in 2007. It is a proof-of-concept context-aware computing environment, originally designed to demonstrate the potential for technology to support older adults live independently for longer, but since developed to explore the role of context-aware computing to support healthier eating and also task-based language learning (i.e. learning a language through cooking). The application within the Ambient Kitchen that was developed to explore prompting of people with dementia preparing food and drinks was done is collaboration with Jesse Hoey (University of Waterloo) and Andrew Monk (then University of York but now a visiting professor at Newcastle University).


Sensing Technologies: The current version of the Ambient Kitchen uses RFID technology (embedded in the worktops and the cupboards), a pressure-sensitive floor (under the laminate flooring), multiple flat LCDs screens (behind tinted glass wall covering), and numerous wireless accelerometers embedded into specially adapted utensils. Through this sensing infrastructure the behaviour of users in the kitchen can be tracked and reasoned about.

utensils    knife

Collaborators: Jesse Hoey (University of Waterloo); Andrew Monk (University of York); Guangyou Xu (Tsinghua University).



[mendeley type=”folder” id=”40642671″ filter=”title=assistance systems”]
[mendeley type=”folder” id=”40642671″ filter=”title=Rapid specification and automated generation of prompting systems to assist people with dementia”]
[mendeley type=”folder” id=”40642671″ filter=”title=kitchen”]
[mendeley type=”folder” id=”40642671″ filter=”title=food”]



Language Learning in the Wild

Foreign languages are generally taught within a classroom setting using textbook exercises. Despite its wide usage, there are a number of problems with this approach. Students are only able to “rehearse” the language, rather than use it practically. The classroom setting also makes it difficult for students to immerse themselves in the foreign culture. Studies have shown that Task-Based Learning, in which students utilise the language they have learned within a practical setting, is a far more effective way to learn languages.

For this project, we worked with researchers from the School of Education, Communication and Language Sciences to turn the Ambient Kitchen into a platform for Task-Based Learning of French. Originally created to assist older people and those with cognitive impairments in their day-to-day lives, the Ambient Kitchen technology was adapted to offer users a recipe in French. It uses motion sensors to ensure that they have understood each step, and supports the user by repeating or translating instructions where necessary. Students can apply their skills practically and with tangible results, making the experience of this Task-Based Learning more rewarding than the traditional classroom setting. With cookery as the context, they are also able to experience something of French culture, as the recipes offered by the kitchen are classic French dishes.

We carried out a study among 46 staff and students from the university to assess whether the kitchen had a positive, negative, or neutral impact on the way they learned French, and to see how people with varying levels of fluency in the language responded to the learning environment. This provided insight into how we might improve the technology and influenced its adaptation for the ongoing European kitchen project.

See also: iLAB: Learn Kitchen

Press release: French kitchen is recipe for success


Date: Jun 2010 – Nov 2011

Funding: EPSRC: Engineering and Physical Sciences Research Council, Digital Economy Programme £162,525

Researchers: Paul Seedhouse (PI) – School of Education Communication and Language Sciences, Patrick Olivier (CI).                                                                                     Dan Jackson, Thomas Ploetz, Jack Weeden, Saandia Ali – School of Education Communication and Language Sciences.

Collaborators: Newcastle College, and CILT: The National Centre for Languages.

Human Activity Recognition for Pervasive Interaction

In this project, we developed a Human Activity Recognition (HAR) framework using sensors embedded into kitchen utensils. The first version of HAR framework, Slice&Dice, was developed to detect 11 low-level, fine-grained food preparation activities using modified Wii Remotes integrated into three knives and one serving spoon. This was followed by the real-time version of HAR, which works with Culture Lab’s wireless accelerometers and a new set of utensils including knives, a spoon, a whisk, a ladle and a peeler. The real-time HAR framework was integrated into the Ambient Kitchen and iLAB Learn kitchen.

We also developed a chopping board that used fibre optic technology to detect food ingredients. A webcam camera and a microphone were integrated into the chopping board. A computer vision algorithm based on colour and shape was developed for food ingredient classification; this was more than 78% accurate in a pilot study we carried out with twelve different foods, showing our approach to be very promising for food recognition. A later version of this algorithm was based on fusion sensing data: colour and feature to detect food before it is chopped and audio and acceleration data intensities to detect food being chopped on the fibre chopping board.

This was followed by automatic recipe tracking and video summarisation applications, which were developed based on the HAR framework. Such applications can monitor which steps of a recipe the user is doing or has done, and are thus able to advise the next step to the user. There is also potential for these applications to assist in calorie intake monitoring or planning meals.

Start Date: February 2008

Project Supervisor: Patrick Olivier, Thomas Ploetz

Funding: Ministry of Education and Training of Vietnam

[mendeley type=”folder” id=”8070971″ groupby=”year” filter=”title=Real-Time Activity Recognition”]
[mendeley type=”folder” id=”8070971″ groupby=”year” filter=”title=Slice&Dice”]