Activities

No upcoming events

Intelligent Systems Lab Project: Hausgeist - an interactive assistant for the Intelligent Room

In synergy with "Teledienste für den intelligenten Raum" and "Information Lens"

Participants

Supervisors

Motivation

The idea of the project Intelligent room is to expand the rooms capabilities in each term. As time goes by more and more projects come along and try to achieve this goal. In a room like this as well as in other facilities it might be useful for people to have an artificial assistant who can offer information about the objects in the room. Often robots are used for this purpose, but they are typically bound by physical problems and cannot reach each object in the room as this would not allow the indexical gesture 'to point' to an object. Also just giving access to a database containing all information would not satisfy the user as he wants to get the information fast and accurately.

By using a lightspot as a personification of an artificial assistant called 'Hausgeist' this problem could be solved: this assistant would be able to reach every location in the room, given that there is no obstacle between lightspot and object, therefore he can show the position of objects and give accurate information about them. He would be capable of presenting a number of objects in the room by moving to the object and giving information about its functionality.

Application Scenario

There are two types of scenario. One main scenario in which a user comes into the room and a side scenario where our project is started via speech-input through another project - here "Teledienste für den intelligenten Raum".

Main scenario

Thomas comes into the room and wants to know what objects are in the room and what purpose they are there for. He catches a glimpse of the Hausgeist represented through a mysterious blue lightspot wobbling around a touchscreen showing the Hausgeist-User-Interface (HUI). Thomas goes to the touchscreen and starts the main tour by touching the play button. The Hausgeist changes the color of the spot and moves from one object to the other and presents Thomas the different objects including the coffee machine through speech-output. At the end of the tour the Hausgeist comes back to its starting point at the touchscreen and rests again in the blue wobbling pattern representing inactivity. Well informed about the objects Thomas starts to use the coffee machine.

Side scenario

The "Teledienste"-project is able to command the Hausgeist via speech-input to start the main tour or cancel a running tour.

Objectives

The project goals are the following:

Description

We used the following Hard- and Software to realise the project:

Design overview

Design overview

As this overview shows we have several controllers, most important the MainController controlling the program flow. All controllers are able to communicate via an event system. The Input- and OutputControllers are responsible to manage the communication with the HUI as well as with the LightCommunicator and the speech-output component that present objects. The DatabaseCommunicator is in contact with the database and fetches the object information during the program flow.

Hausgeist-User-Interface (HUI)

HUI

This screenshot shows the Hausgeist-User-Interface. On the left we see the interface to start and stop a tour. The right button opens the tour selection window, which enables the user to select objects for the tour as well as saving and loading predefined tours.

Basic path planning

The idea behind our basic path planning was to let the Hausgeist appear more lively and give it a ghost-like movement. On the technical side it is realised by the lightspot which pivots n-times around a fixed value above and below the last object z-Position, where n depends on the travel-time from the position of the last object to the new one.

for(unsigned int i = 1; i <= numberofmovements; i++) {
calculateAngles(objX, objY, lastObjZ+(50*pow(-1,i)));
//spot specific commands
setPan(pan);
setTilt(tilt);
writeBuffer();
msleep(sleeptime/numberofmovements);
}

numberofmovements = sleeptime (in seconds, mathematically rounded)
sleeptime (in milliseconds) = angle to go (in degree) * time (per degree)

Results

The system works as planned. In detail this means:

A demonstration is available as Hausgeist Video (mp4, 5,9 Mb, very low quality): The video shows a demonstration of the working system.

A Full-HD Version is available here:

Discussion and Conclusion

The realization of the project worked well and we were able to implement an assistant capable of the basic ability of presenting a room and its objects. However, there are some points that are not included in the current Hausgeist program. For example it only relies on the database, but does not have any information about the current state of the room: how many people are in there or where is a person standing? Perhaps someone wants to get to know something about an object he cannot name and thus cannot select it in the HUI. By integrating the assistant deeper into the room and enabling the Hausgeist to "see" its environment, a better adaptation of the assistant to the current situation might have been reached. For the current scenario and due to the time limit of the project this was not realizable. We decided to stay with the selection of tours as this allows the maximum flexibility without having to make the effort to evaluate visual informations about the room.

Some more practical problems have been encountered: The more distant the object the larger the Hausgeist's light spot since the light rays diverge at an angle that is not controllable by the light spot. In consequence we cannot react on object distances. Near objects get a small lightspot and far objects a bigger blured one. As we rely on the database of "Information Lens" we are not capable of adding or changing information in the database itself. For testing purpose and fall-back-strategy we implemented our own fake server containing some object definitions.

Outlook

For the future the project could be extended by using multiple individually adressable loudspeakers to intensify the personification and location of the Hausgeist. Another possibility is to implement a more complex path planning regarding the objects in the room to avoid discontinuous movements or occlusion, for example through a pillar between spot and object. To accentuate the idea of an "interactive agent" it would be nice if the speech-input could accept more complex commands, not only short ones like "start tour" or "abort tour". Additionally to the speech-output further information could be presented visually through a display or projector to give the user information which are hard to describe verbally, such as for instance pictures.