1.6.7. Speech Visualizer

The Dialog-Visualizer is a collection of tools that contains at the moment a SpeechVisualizer and a GrammarHelper.
Their aim is to assist the interaction with the dialog.

1.6.7.1. Speech Hypotheses Visualizer

Used to show results from speech recognition.

1.6.7.1.2. Interfaces

Currently there are two interaction islands, the kitchen and entrance.

Scope (Listener) Type
<location>/speechhypotheses SpeechHypotheses , DialogAct
<location>/situation/result ClassificationResultMap
<location>/saytask/SAY TaskState
<location>/flobi/speakingstate String
<location>/speechhypotheses/visualization/debugmode String

1.6.7.1.3. Debug Mode

Debugmode can be enabled and disabled via cmdline-properties or an rsb-event. The scope of the debugmode-listener is always speech-hypotheses-scope + /visualization/debugmode. The rsb-event to handle debugmode must contain a string-message ON or OFF, not case-sensitive.

Enabling debugmode entails:

  • a textFrame for grammarTrees shows up.
  • grammarTrees (if available) are shown.
  • type of incoming dialogact is printed to terminal.

1.6.7.1.4. Examples

  • start some speech-recognition-software, that sends its sprechhypothese or dialogacts via rsb.
  • start visualizer with scope to listen to ./speechvisualizer -s /result_scope/
If you give no scope, default will be “/citec/csra/home/hallway/entrance/speechhypotheses”.
  • see what recognizer sends as result.

A small bar at the bottom of the screen appears. The results from the speech-recognition will be posted there. The current hypothesis is shown in the first line, below is the history of last hypotheses.

../../_images/speech-visualizer-text-final.png

If the result is not final yet, the text will be greyed out. If it is final, the text is black. Filler-words will only be shown as long as they are not final.

../../_images/speech-visualizer-text-not-final.png

You can also let the visualizer show the speaking and attention / listening state of the robot-system. Therefore you can invoke it with the command -p, which tells the visualizer to show if the robot speaks on a certain scope.

There is a small section at the right end which becomes colored if the system is speaking.

../../_images/speech-visualizer-system-speaking.png

Also, you can invoke it with the parameters -a and -d to to show the attention and addressee state of the system. When the backgroundcolour of the textbox is blue, the system has attention to something.

../../_images/speech-visualizer-has-attention.png

When the backgroundcolour of the textbox is green, the system has attention and an addressee.

../../_images/speech-visualizer-has-addressee.png

If you enable debug-mode, you will see a second frame above the recognition-bar. It will show grammar-trees from incoming SpeechHypotheses or DialogAct.

../../_images/speech-visualizer-grammar-tree-frame.png

1.6.7.2. Grammar Helper

Used to show possible speech commands for an interaction with the dialogsystem.

1.6.7.2.1. Related resources

Component repository:speech-visualization.

System startup:

The start script grammar-helper.sh can be found at <prefix>/opt/speech-visualizer/bin/grammar-helper.sh.

The program can be invoked with the following parameter:

-r, --scp-scope
 Set the rsb-scope where you want to call the Speech Command Proposer.
-d, --debug debug-mode, shows more information.
-l, --dialog-act
 Set the scope where dialogacts should be published.

Related projects:

1.6.7.2.2. Interfaces

The property of -r or --scp-scope serves as prefix.

Scope (Listener) Type
prefix/grammarhelper/debugmode String
prefix/grammarhelper/rulechange String

The scope for rulechange is used to inform the grammarhelper that active rules have changed. It is used by the Speech Command Proposer.

1.6.7.2.3. Debug Mode

Debugmode can be enabled and disabled via cmdline-properties or an rsb-event. The scope of the debugmode-listener is always scp-scope + /grammarhelper/debugmode. The rsb-event to handle debugmode must contain a string-message ON or OFF, not case-sensitive.

Enabling debugmode entails:

  • the grammar-helper frame will be switched to debugmode.
  • grammar control section lets you set and ask for the jsgf-grammar the Speech Command Proposer uses.
  • two getRules buttons to grab rule-lists from the Speech Command Proposer.

1.6.7.2.4. Examples and Screenshots

The appearance of the grammar-helper differs for the debug mode and no-debug mode.
Below the two frames are shown for comparison. Only one mode can be active at a time.

../../_images/overview.png

1.6.7.2.4.1. Basic usage

Start the Speech Command Proposer .
For example ./speech-command-proposer -s /ScopeToMyServer/ -g /path/to/my/grammar.gram
Start grammar-helper with scope to make remote-calls ./grammar-helper -r /ScopeToMyServer/
A frame at the right of the screen appears.
After the tool has started, it will try to grab a list of active rules from the Speech Command Proposer.
It will show an example-sentence for every rule in a list in the white list-box.
The rule is updated autonomous whenever the list of active rules or the grammar is changed.
The two checkboxes at the top-right enable their entitled action on click when checked.
So you can retrieve new examples or send a dialogAct for the clicked entry.

1.6.7.2.4.2. Example for debugmode

In debugmode you have more control about the grammar and what rules are shown.
To set a grammar to make suggestions from, simply type in the file-path in the top text-field and press setGrammar.
The label beneath the button will tell if setting the grammar worked fine.

If you want to check which grammar the Speech Command Proposer is using, simply click checkGrammar.
In this case, the text-field beneath the button will be filled with the grammar path.

../../_images/grammar.png
If your are content with the used grammar file, you can retrieve its rules by hitting the getAllRules or getActiveRules button.
It will show them up in a list in the white box below.
Active rules are autonomous updated whenever you change the grammarfile.
Now you can click on an entry in the list to get an example sentence for this rule (shown in the white box at the bottom).
Also there is a checkbox above the rule-list which will allow you to send dialogActs for the clicked list-entry.

../../_images/debug.png

Note

Active rules are handled by the Speech Command Proposer!

Note

The scope for sending dialogActs is passed through to the speech-command-proposer. The scp will also send the dialogActs.