Activities

No upcoming events

Intelligent Systems Lab Project: Swipe Kinect Interaction

Participants

Supervisors

Motivation

The modern chef in the intelligent kitchen no longer wants to ruin the pages of grandmothers cookbook with greasy fingers. Instead, he would much rather scroll through a digital cookbook using gesture control. On the vacant space in the kitchen he can store more utensils and ingredients, so these are quickly at hand. Even at late hours he can still use the digital cookbook well as there are no additional light sources required due to the active display.

Cookbooks are indeed a handy means to provide information about the cooking process but they also have disadvantages. First, direct contact for navigation is necessary, as a result there is a hygiene problem with the cooking process by the contaminated hands. To counter this problem, one would have to wash his hands at an increased rate which on the one hand may yield sensitive skin and on the other hand is an unreasonable effort from an ecological perspective. The future lies in a digital cookbook - but how you can navigate through the virtual cooking process? Even in conventional human-machine interaction direct contact is necessary, whereby the cookbook of the future would have no benefits. On the contrary, modern hardware is even more sensitive than grandma's good cookbook, and the functionality of a touch screen would be adversely affected by contaminants very quickly. The solution to this problem is a gesture-controlled navigation through the virtual cookbook, which allows for a contactless scrolling through the recipe. Important here is the really simple and intuitive use of the cookbook, because the cooking process is supposed to be simplified and not unnecessarily complicated. Navigation through the cookbook should be realized with simple hand gestures that are easy to learn and have a certain connotation about there possible functionality. This is the only way to create a true competitor to grandma's cookbook and to clear away any obstacle for a broad acceptance in the world of chefs.

Application Szenario

Tom wants to prepare a special meal for his first official date with his new girlfriend Lilly. He has taken one day off for the big day and is calmly preparing for the evening.

When the two have small talk behind him, Tom decides to start cooking. He turns on his kitchen PC. The logo of the virtual cookbook and a selection of Tom's favorite recipes appear on the built-in kitchen display. With simple wipe gestures Tom navigates to the recipe of his choice. Tom begins, led by his cookbook to prepare the ingredients first. He watches as Lilly curiously studies his modern kitchen. Finally, Tom decides to fetch a bottle of wine from the basement, to counteract his excitement. He finishes his job and disappears into the basement. Lilly walks curiously in the kitchen and decides to help Tom. She has been observing Toms gestures and tries to imitate the gesture task confirmation. The system recognizes this and presents the next preparation step to Lilly. Lilly is thrilled about how simple the control is, and tries other gestures. She manages very quickly to gain a good overview of the cooking process. She has already started the preparation further when Tom comes back from the basement and is surprised how Lilly has managed to become familiar with its virtual cookbook so quickly. He helps Lilly peeling onions when he suddenly has to think about whether he has bought all the ingredients. He scrolls by hand gesture to the list of required ingredients and is pleased that neither his mouse nor his keyboard will smell like onions, since he simply has to run the appropriate hand gesture in front of Kinect.

Shortly before he pushes the meal for baking in the oven, he compares one last time the pictures of the nearly finished meal in the digital cookbook. The visual match is a positive indication that everything is OK. Now the meal has to stay for the specified time in the oven.

Objectives

The project goals are

Description

The panel offered by the developed library for Kinect interaction and its visualization.

Results

Discussion and Conclusion

The SKI system is able to recognize simple gestures in a stable way. The chosen gestures function mapping seems to be simple an intuitive for users without experience with the system. This has been tested several times with development independent users. The interaction was stable which means the error of failed gesture recognitions is nearly null. The hand tracking algorithm traces just one hand in one time frame so there is no double user interaction possible but a consecutively interaction is supported. The last but not least main feature is the user independent gesture recognition which allows an easy integration of the system in an unknown environment.

Outlook

References