Measuring Cooking Competence

Lack of cooking competence is often a contributing factor to poor diet. In this project I aim to find an objective measure of people’s cooking competence using the Ambient Kitchen as a platform to carry out my research. This information can be used to provide personalised and situated support to help the user improve both their cooking competence and diet.

Start Date: 2007

Project Supervisor: Patrick Olivier, Thomas Ploetz

Collaborators: Philips Research, Eindhoven.


The aim of this project was to understand the changes in motor skill that take place during the early phases of learning a new fine motor skill task. For this project specifically, that motor skill was suturing, which we measured by attaching sensors to the surgical suturing instruments that the participants (medical students) used, in order to track and record movements of instruments while participants practice suturing. The participants were trained in the correct technique by use of a video demonstrating the task, as well as an information sheet that described the steps. They were then asked to perform the suturing task. We also captured hand movements during the tasks with a video recording device.

A novice group of participants who had no prior training in suturing were used to carry out the study, in order to allow us to record motor skill acquisition with practice of a novel task. We also used an ‘experienced’ group of participants: this was composed of medical students who had been shown the technique before.

The suturing was carried out with proper surgical instruments and suture material. We used the same artificial suture pads that are frequently used for training purposes in hospitals.

Date: Nov 2010 – July 2011

Funding: EPSRC: Engineering and Physical Sciences Research Council (KTA) £27,074

Researchers: Patrick Olivier (PI), Andrew McCaskie (CI) – Institute of Cellular Medicine, Nils Hammerla, Thomas Ploetz, Roisin McNaney, Sandeep Deshmukh -Institute of Cellular Medicine.

Ambient Kitchen

The Ambient Kitchen is a platform for research in pervasive computing that was installed at Culture Lab in 2007. It is a proof-of-concept context-aware computing environment, originally designed to demonstrate the potential for technology to support older adults live independently for longer, but since developed to explore the role of context-aware computing to support healthier eating and also task-based language learning (i.e. learning a language through cooking). The application within the Ambient Kitchen that was developed to explore prompting of people with dementia preparing food and drinks was done is collaboration with Jesse Hoey (University of Waterloo) and Andrew Monk (then University of York but now a visiting professor at Newcastle University).


Sensing Technologies: The current version of the Ambient Kitchen uses RFID technology (embedded in the worktops and the cupboards), a pressure-sensitive floor (under the laminate flooring), multiple flat LCDs screens (behind tinted glass wall covering), and numerous wireless accelerometers embedded into specially adapted utensils. Through this sensing infrastructure the behaviour of users in the kitchen can be tracked and reasoned about.

utensils    knife

Collaborators: Jesse Hoey (University of Waterloo); Andrew Monk (University of York); Guangyou Xu (Tsinghua University).



[mendeley type=”folder” id=”40642671″ filter=”title=assistance systems”]
[mendeley type=”folder” id=”40642671″ filter=”title=Rapid specification and automated generation of prompting systems to assist people with dementia”]
[mendeley type=”folder” id=”40642671″ filter=”title=kitchen”]
[mendeley type=”folder” id=”40642671″ filter=”title=food”]



TEDDI: Building Management and Energy Demand

TEDDI-001This research project involves the design and development of a sensing infrastructure that consists of networked physical (e.g. presence sensors, power consumption sensors) and virtual (e.g. calendar and room booking sensors, application usage sensors) sensors, which will provide fine-grained information about how much energy is being used, for what purpose and by whom. By applying techniques from knowledge engineering, activity recognition and machine learning, the project involves the use of high level information in order to link usage patterns to real-world activities and workflows.

This information is used to parameterise building models used in building management to more accurately predict energy usage and to optimise (decentralised) energy consumption, generation and storage and to assist in the development of a decision support tool that visualises the collected data as well as the expected impact of energy saving strategies such as organisational changes and policies or the rescheduling of activities. This will enable decision makers to identify where energy is being wasted and to formulate and evaluate strategies to reduce energy consumption.  The data will enable a better understanding of the way the building is used and how heat is wasted. Through a combination of physical and virtual sensors, a more accurate measurement of thermal comfort of the building’s occupants will be established and thus assist in resolving ever occurring complaints and potential conflicts associated with the diverse needs for occupant comfort in buildings that can result in unnecessary overheating.

Date: Sept 2010 – Sept 2013

Funding: EPSRC (Engineering and Physical Sciences Research Council) £606,679

Researchers: Tony Roskilly (PI) – Sir Joseph Swan Centre for Research on Energy. Patrick Olivier, and Thomas Ploetz (CIs).
Dan Jackson, Cas Ladha, Karim Ladha, Aftab Khan, Sam Mitchell Finnigan, Chris Kray, Mohammad Royapoor, Zhiwei Gao – Sir Joseph Swan Institute for Energy Research.

Collaborators: John Counsell, and James Johnston (Strathclyde University). David Dunn.

Cueing Technology for Parkinsons

Approximately 70% of people with Parkinson’s Disease experience problems with swallowing. The resulting build-up of saliva can cause drooling, which is often a source of embarrassment and puts the person at risk of choking or pneumonia if the saliva gets into the lungs. Therefore, it is important that people who suffer from drooling have a way to combat the problem.

Many therapies offered to tackle this issue work by decreasing saliva production, but this can result in problems relating to eating or oral hygiene. In addition, therapies such as Botox involve injections and can cause discomfort. One form of management that does not cause negative physical side effects is the use of a cueing device, which is worn on the body and reminds users to swallow their saliva by giving them cues at regular intervals, allowing them to manage the drooling behaviourally. However, this form of management is not widely used, and there are numerous issues with available devices, such as that the cues are noticeable to observers, that manually-operated devices are difficult to use, and that the designs are not aesthetically pleasing.

The aim of this project is to investigate the use of cueing to manage drooling in such a way that the cueing device itself is not embarrassing or uncomfortable for the wearer. We have developed a device through a participatory design process, ensuring that the design takes into account the needs and desires of people with Parkinson’s Disease so it is something they wish to use. This is an interdisciplinary project involving speech therapists, old age clinicians, interaction designers and hardware engineers.
Following feedback on the initial prototype, an improved version of the device is being trialled for one month by thirty NHS patients.

See also: Cueing for Swallowing

Funding: National Institute for Health Research (NIHR) £37,018
Researchers: Nick Miller (PI) -Institute of Health and Society, Patrick Olivier (CI), Roisin McNaney, Stephen LindsayKarim Ladha,  Thomas Ploetz, Nils Hammerla, Dan Jackson
Collaborators: Richard Walker – Northumbria NHS Foundation Trust

[mendeley type=”folder” id=”8070971″ groupby=”year” filter=”title=Cueing”]


The aim of this project is to help European citizens to obtain consumer solutions for hassle-free guidance towards a balanced lifestyle regarding meal planning and preparation and personal choice. We explore methods for inferring eating habits in an unobtrusive way, and seek to use this information to provide situated feedback on meal planning and preparation, as well as creating pervasive kitchen technology that will help to underpin this advice.

The research for the project takes place in the Ambient Kitchen and Experience Lab at Philips Research in Eindhoven, as these research facilities represent a real-world environment. We will also take advantage of the geographical distance between ourselves and our partners at Philips Research to explore the role of social networking in sharing cooking experiences and the promotion of balanced eating habits.

We are adopting a User-Centred Development approach in order to develop successful solutions, meaning the starting point for each of our developments will be the insights of the groups we intend to use it, and testing prototypes with users during further developments. The three groups we are specifically looking at are single-person households, two-person households, and families with young children.

Date: April 2009 – March 2013 (48 month Project)

Funding: Marie Curie Action under the European 7th Framework Program (FP7). £190,201

Researchers: Patrick Olivier (PI). Paula Moynihan – Institute of Ageing & Health, and Martyn Dade-Robertson – Architecture, Planning & Landscape (CIs). Juergen Wagner, Robert Comber, Jack Weeden.

Collaborators: Aart van Halternen, Jettie Hoonhout, Peggy Nachtigal, and Gijs Geleijnse (Philips Research, Eindhoven).

Press: Balancing Act”  – ResearchMediaLtd

See the Balance@Home project website


[mendeley type=”folder” id=”8070971″ groupby=”year” filter=”title=Ambient kitchen: designing situated services using a high fidelity prototyping environment”]

[mendeley type=”folder” id=”8070971″ groupby=”year” filter=”title=Ambient Intelligence. Slice&Dice”]

Human Activity Recognition for Pervasive Interaction

In this project, we developed a Human Activity Recognition (HAR) framework using sensors embedded into kitchen utensils. The first version of HAR framework, Slice&Dice, was developed to detect 11 low-level, fine-grained food preparation activities using modified Wii Remotes integrated into three knives and one serving spoon. This was followed by the real-time version of HAR, which works with Culture Lab’s wireless accelerometers and a new set of utensils including knives, a spoon, a whisk, a ladle and a peeler. The real-time HAR framework was integrated into the Ambient Kitchen and iLAB Learn kitchen.

We also developed a chopping board that used fibre optic technology to detect food ingredients. A webcam camera and a microphone were integrated into the chopping board. A computer vision algorithm based on colour and shape was developed for food ingredient classification; this was more than 78% accurate in a pilot study we carried out with twelve different foods, showing our approach to be very promising for food recognition. A later version of this algorithm was based on fusion sensing data: colour and feature to detect food before it is chopped and audio and acceleration data intensities to detect food being chopped on the fibre chopping board.

This was followed by automatic recipe tracking and video summarisation applications, which were developed based on the HAR framework. Such applications can monitor which steps of a recipe the user is doing or has done, and are thus able to advise the next step to the user. There is also potential for these applications to assist in calorie intake monitoring or planning meals.

Start Date: February 2008

Project Supervisor: Patrick Olivier, Thomas Ploetz

Funding: Ministry of Education and Training of Vietnam

[mendeley type=”folder” id=”8070971″ groupby=”year” filter=”title=Real-Time Activity Recognition”]
[mendeley type=”folder” id=”8070971″ groupby=”year” filter=”title=Slice&Dice”]