Activities

No upcoming events

Biosignal-basiertes Assistenzsystem zur Stimmungserkennung für Patienten mit psychischen Erkrankungen

Participants

• Patrick Blöbaum
• Frank Grimm
• Dennis Wigand

Project Supervisors

• Andrea Finke
• Nils Hachmeister
• Hannes Riechmann

Motivation

Monitoring levels of relaxation for users of the intelligent room could be relevant, since most mental tasks require some sort of concentration that can be supported by the user's surroundings through means of light or music control, as well as more direct feedback.
While normal room environments do not offer much support for concentrated attention phases or require manual interaction (to control ambient lights for example), an intelligent room could help to solve this problem by providing unobstrusive neurofeedback to the user.
An automated system that infers the user's mental states of concentration or relaxation to some degree can aid a user by making such an interaction unnecessary while retaining the benefits of softer/warmer light or environment sounds.

Application Scenario

This project scenario focusses on phases of concentrated attention when a user is performing office work that does not require full body movement. This environment allows Electroencephalography (EEG) measurements without requiring a strict lab setting. While the user is focussing on the task at hand, a low-cost commercial EEG headset is used to record his/her brainwave data.
Our BIOSIG system analyzes this data and derives the user's level of relaxation. Depending on whether the user is relaxed or focused, the user is aided by different auditive stimuli. These stimuli are presented as changing ambient sounds and aim to keep the user in a mild level of relaxation or alertness:

• Phases of focused work: Ambient sounds, which are selected recordings of a rain-forest, are played repeatedly and continously to keep the user's concentration on the current task. These sounds aid the user's concentration mostly by being very predictable and unobstrusive.
• Phases of relaxation: Short, loud sounds are played whenever the system registers 4 successive relaxed phases. These sounds are used to remind the user about his/her task and get him/her back to work.

The BIOSIG system records data for five seconds (one phase) and classifies the level of relaxation in this past phase. Supplemental to the auditive stimuli, recent classification results are visualized on a display in proximity to the user. This visualization is resembling a Chernoff-face , in case a more direct form of feedback is desired.

Objectives

The project goals were

• Integration of the lib_emokit hardware-interface for the Emotiv EPOC EEG headset into the Brain-Computer Interface Framework UBiCI framework.
  The framework was developed at AG Neuroinformatik of University Bielefeld.
• Implementation of "offline-training" sequences to gather baseline data on a user's relaxation. The data gathered during training sessions is used for classification while a participant uses the BIOSIG system.
• Online data-retrieval and binary online classification of levels of relaxation. This is the active part of the BIOSIG system which is used while a participant performs his/her task.
• Visualization of EEG activity and classification results.
• Auditive neuro-feedback depending on classification results.

Description

• For training and online classification we asked a participant to wear an Emotiv EPOC EEG headset.
  (Source: emotiv.com).
  The hardware is mobile and low-cost, especially when compared to clinical EEG hardware. As such, the hardware enables projects to be run outside of a laboratory environment.
  A wireless connection is used to connect the EEG headset to a computer. This makes it easier and more comfortable to wear compared to clinical EEG hardware, which uses wires to connect.
• One training session of approximately 40 minutes is performed per participant. During that time instructions are presented using PsychoPy 2. The instructions include textual content on a display in front of the user as well as beeping sounds, indicating a change of action. Short breaks between actions are included to keep the participant focused. These breaks are also used to prevent motor artifacts from contaminating the data.
  The training session provides baseline data for classification. By asking the participant to close his/her eyes data for a relaxed class is gathered. Data for the focused class is gathered by asking the participant to perform simple tasks with their eyes open. The theory behind this training sequence is that by closing his/her eyes, the EEG data gathered from the participant would show alpha wave patterns. These oscilations in a specific frequency range are considered as indicating relaxation.
  These classes are later used to classify the current level of relaxation.
  
  We need approximately 10-15 minutes of preparation time for each training or online session. The EEG headsets use wet electrodes that need to be prepared with a salt-water solution.
  For good classification results, we recorded the alignment of the headset on each participant's skull with a digital camera. These pictures were stored manually and are referenced every time a participant is prepared for a new session using existing training data.
• In the online-classification scenario, the participant is equipped with the same headset and instructed to perform mentally demanding tasks (focused reading). Afterwards, the BIOSIG system gathers new data and compares it to the data gathered on levels of relaxation during the training session.
  To achieve this the newly created Lib_BCI_EPOC component feeds the live EEG data into the UBiCI framework. The framework is using the classification parameters, previously determined in the offline training session, to distinguish relaxed and focused levels.
• Live-EEG data, as well as classification results, are then visualized. For that reason, all relevant data points are transferred over the local network to a workstation responsible for providing visual and auditive feedback. This workstation runs a Java-based tool responsible for controlling the audio environment and visualizations.

Results

• Integration: We have integrated the lib_emokit library into the UBiCI framework. The Lib_BCI_EPOC component provides a platform for follow-up projects using the same hardware.
We also provide a Java-based application that uses classification results for audiovisual-feedback and data visualizations. Instructors can choose to run this application showing a chernoff-face visualization to the participant or use a more technical view to assess the data themselves.
• Software: We implemented the necessary scripts and UBiCI configurations for offline training sessions and online classification.
  
• Performance: We performed a cross-validation on the recorded data for a training session. Data gathered in a training session contains information of phases of relaxation and focus, as described in the previous section. In the cross-validation the system has shown up to 80% correct classifications.
  Our project contains a cross-validation configuration that was used to automate testing different system parameters on a per-participant basis. This is necessary because patterns in EEG activity (and the alpha waves the system is focused on) vary between participants. We used it to optimize the classification results for the online classification scenario.
  The optimal parameters to get the best classification results are then used to analyze data in the training and online settings. These classification results may vary depending on the individual participant, room environment and setup.

Demonstration video illustrating the training sequence and showing extracts of several online classifications:

Conclusion

• The EEG hardware is still cumbersome to use and prone to errors. The wireless connection used to connect the headset with the USB dongle on the data processing workstation is sometimes unstable. Especially outside of a laboratory setting, external sources of disturbance have shown to interfer with the quality of measurements.
• The described classification in a binary problem space is working well enough to provide supportive neuro-feedback to participants. A participant can be asked to focus on reading a text in order to summarize a text later on. This task fits the training sequence especially well, as reading only requires minimal body movement.
• Although all planned goals have been achieved, there is still room for improvement. Future projects could experiment with changing different parameters of the experiment (temporal length of phases).
  Additionally, the classification is relatively limited. Other indicators can be derived from the EEG data or using other sources in a multimodal setting to improve how much a user can benefit from using the system.
  A classification that returns gradual results instead of binary decisions might make sense to make the ambience sounds more dynamic and provide faster feedback on the users level of relaxation.
• When a participant talks or moves, motor artifacts show up in the EEG data that contaminate the data. The hardware provides gyroscope measurements that the BIOSIG system can retrieve. Projects should use this data in order to clean up training data and discard samples that were recorded while the participant was moving.