Catch few words — Listen¶

Goal

In this tutorial we will use Listen and PhraseSet in order to make Pepper listen what a human says to him.

Prerequisites: Running Actions on Pepper.

Let’s start a new project

Start a new project, let’s call it ListenPepper.
Robotify it and make sure it implements the QiSDK & the Robot Life Cycle.

For further details, see: Creating a robot application.

Building PhraseSet¶

A PhraseSet represents a collection of Phrases that Pepper will be able to understand. Each Phrase object contains a text.

To build a PhraseSet instance, use the PhraseSetBuilder class.

Put the following code in the onRobotFocusGained method:

val phraseSetYes: PhraseSet = PhraseSetBuilder.with(qiContext) // Create the builder using the QiContext.
                                   .withTexts("yes", "OK", "alright", "let's do this") // Add the phrases Pepper will listen to.
                                   .build() // Build the PhraseSet.

val phraseSetNo: PhraseSet = PhraseSetBuilder.with(qiContext) // Create the builder using the QiContext.
                                  .withTexts("no", "Sorry", "I can't") // Add the phrases Pepper will listen to.
                                  .build() // Build the PhraseSet.

// Create the PhraseSet 1.
PhraseSet phraseSetYes = PhraseSetBuilder.with(qiContext) // Create the builder using the QiContext.
                                         .withTexts("yes", "OK", "alright", "let's do this") // Add the phrases Pepper will listen to.
                                         .build(); // Build the PhraseSet.

// Create the PhraseSet 2.
PhraseSet phraseSetNo = PhraseSetBuilder.with(qiContext) // Create the builder using the QiContext.
                                        .withTexts("no", "Sorry", "I can't") // Add the phrases Pepper will listen to.
                                        .build(); // Build the PhraseSet.

Here, we define the Phrases by passing directly each text as a String. In this example Pepper will be able to understand positive and negative answers, like “yes”, “no”, “OK”, etc.

Note

You can call withTexts multiple times on the same PhraseSetBuilder instance. All the specified texts will be added to the PhraseSet that will be built. This way, the content of the PhraseSet may evolve.

Let’s see now how we can use these PhraseSets to make Pepper understand the previously specified texts.

Listening¶

To make Pepper listen, use the Listen interface: create it with a ListenBuilder.

val listen: Listen = ListenBuilder.with(qiContext) // Create the builder with the QiContext.
                          .withPhraseSets(phraseSetYes, phraseSetNo) // Set the PhraseSets to listen to.
                          .build() // Build the listen action.

// Create a new listen action.
Listen listen = ListenBuilder.with(qiContext) // Create the builder with the QiContext.
                             .withPhraseSets(phraseSetYes, phraseSetNo) // Set the PhraseSets to listen to.
                             .build(); // Build the listen action.

We used the previously created PhraseSets to set what phrases Pepper will listen to.

We can now run the Listen:

// Run the listen action and get the result.
val listenResult: ListenResult = listen.run()

// Run the listen action and get the result.
ListenResult listenResult = listen.run();

If the Listen action succeeds, we will be able to access this ListenResult and make Pepper react accordingly.

Testing¶

For a quick test, display the heard phrase and the matched PhraseSet in the Android Logcat:

Log.i(TAG, "Heard phrase: ${listenResult.heardPhrase.text}")

Log.i(TAG, "Heard phrase: " + listenResult.getHeardPhrase().getText());

// Identify the matched phrase set.
val matchedPhraseSet: PhraseSet = listenResult.matchedPhraseSet
if (PhraseSetUtil.equals(matchedPhraseSet, phraseSetYes)) {
    Log.i(TAG, "Heard phrase set: yes")
} else if (PhraseSetUtil.equals(matchedPhraseSet, phraseSetNo)) {
    Log.i(TAG, "Heard phrase set: no")
}

// Identify the matched phrase set.
PhraseSet matchedPhraseSet = listenResult.getMatchedPhraseSet();
if (PhraseSetUtil.equals(matchedPhraseSet, phraseSetYes)) {
    Log.i(TAG, "Heard phrase set: yes");
} else if (PhraseSetUtil.equals(matchedPhraseSet, phraseSetNo)) {
    Log.i(TAG, "Heard phrase set: no");
}

Or better, display them on Pepper’s tablet!

Let’s try it¶

The sources for this tutorial are available on GitHub.

Step	Action
	Install and run the application. For further details, see: Running an application.
	Choose “Catch few words”.
	Try to say “Yes” or “No”. You should see the log “Heard phrase: yes” or “Heard phrase: no” . Note If you want Pepper to listen again, the Listen must be re-executed with the `run` method. Indeed, Pepper stops listening when he hears one of the targeted phrases. You can rerun the application or put it in background and then bring it back in foreground.
	Choose again “Catch few words”.
	Try to say any non expected word. When you try to say a phrase that was not contained in the PhraseSet, Pepper will not listen to it. For example, if you say “hey”, nothing will be logged.

You are now able to make Pepper listen and get information about what he heard!