Multi-Channel Image Data Analysis using Sonification

In biomedicine as well as in many other areas experimental data consists of topographically ordered multidimensional data arrays or images.

In our collaboration, multi parameter flourescence microscopy data of immunoflourescently labeled lymphocytes has to be analysed. One experimental data set consists of $n$ intensity images of the sample. As a result of a specific immunolabeling technique in each image different subsets of the lymphocytes appear with high intensity values, expressing the existence of a specific cell surface protein. Because the positions of the cells are not affected by the labeling process, the $n$ flourescence signals of a cell can be traced through the image stack at constant coordinates.

The analysis of such stacks of images by an expert user is limited to two strategies in most laboratories: the images are analyzed one after the other or up to three images are written into the RGB channels of a color map. Obviously, these techniques are not suitable for the analysis of higher dimensional data.

Here, Sonification of the stack of images allows to perceive the complete pattern of {\em all} markers.
The biomedical expert may probe specific cells on an auditory map \cite{Bla94} and listen to their flourescence patterns.
The sonification was designed to satisfy specific requirements:

Identification - Cells with identical patterns should very easily be perceived as identical sounds
Similarity - Similar cell flourescence patterns should lead to sonifications that sound similar
Extensibility - the sonification should be extensible, so that the future addition of markers does not change the sound characteristic, driven by the other markers
Short Duration - the whole sonification should last only a short time of about 1 sec, to allow a fast browsing of the image.

Such sonifications can be derived using several strategies. One is to play a tone for each marker if the corresponding flourescence intensity is more than a threshold. Thus a rythmic pattern emerges for each cell.
Another strategy is to use frequency to distinct markers. Thus each cell is a superposition of tones with different pitch and a chord or tone-cluster is the result. This leads to a harmonic presentation of each cell.
However, using both time and pitch, the result is a rythmical sequence of tones and thus a specific melody for a cell.
As our abbilities to memorize and recognice melodies or musical structures is better than recognizing visual presented histograms, this yields a promising approach for the inspection of such data by an expert. Now, an example sonification is presented using only five dimensional data images. However, the results are even good with much higher dimensionality - we tested the method with a stack of 12 images.
The following demonstration uses only 5 markers. A map is rendered to show all cells for browsing (shown right)

cd-02	cd-08		Identical patterns: Cell 1 Cell 2
cd-03	------	superposition	Similar pattern: Cell 3
cd-04	hla-dr		Very different pattern: Cell 4

A specific advantage of this method is, that it allows to examine the high-dimensional data vectors without the need to change the viewing direction. However, there are many other methods to present such data acoustically, e.g. by using different timbre classes for the markers, like percussive instruments, fluid sounds, musical instruments or the human voice. These alternatives and their applicability are currently investigated.

Literature

Contact

Thomas Hermann , Tim Nattkemper

Back to Sonification Page

Last modified: Tue 2001-05-22