Understanding LookAt¶

Goal

In this tutorial, we will make Pepper lower his head as if looking at the ground just in front of him.

To do this, we will:

Create the Frame where he should look at.
Use the LookAt Action.

Prerequisites

Before stepping in this tutorial, you should be familiar with the action notion. For further details, see: Running Actions on Pepper.

Let’s start a new project

Start a new project, let’s call it LookAtPepper.
Robotify it and make sure it implements the QiSDK & the Robot Life Cycle.

For further details, see: Creating a robot application.

Building the target frame¶

We will create the target Frame, 1 meter in front of Pepper. To retrieve this Frame, we will first need to get the robot frame.

The Robot Frame is provided by a service called Actuation. We can retrieve it if we use the getActuation method on the QiContext.

Put the following code in the onRobotFocusGained method:

// Get the Actuation service from the QiContext.
val actuation: Actuation = qiContext.actuation

// Get the Actuation service from the QiContext.
Actuation actuation = qiContext.getActuation();

Now we can get the robot frame:

// Get the robot frame.
val robotFrame: Frame = actuation.robotFrame()

// Get the robot frame.
Frame robotFrame = actuation.robotFrame();

Next, we create a 1 meter forward translation corresponding to the transform between the robot frame and the target frame:

// Create a transform corresponding to a 1 meter forward translation.
val transform: Transform = TransformBuilder.create().fromXTranslation(1.0)

// Create a transform corresponding to a 1 meter forward translation.
Transform transform = TransformBuilder.create().fromXTranslation(1);

We will now create the target frame. The Mapping service provides a method to create a FreeFrame:

 // Get the Mapping service from the QiContext.
val mapping: Mapping = qiContext.mapping

// Create a FreeFrame with the Mapping service.
val targetFrame: FreeFrame = mapping.makeFreeFrame()

// Get the Mapping service from the QiContext.
Mapping mapping = qiContext.getMapping();

// Create a FreeFrame with the Mapping service.
FreeFrame targetFrame = mapping.makeFreeFrame();

A FreeFrame represents a location free to be placed anywhere, that does not move when other frames move.

The global position of a FreeFrame can be updated by applying a Transform to a base Frame. The timestamp is left to 0, to update targetFrame relatively to the last known location of robotFrame.

// Update the target location relatively to Pepper's current location.
targetFrame.update(robotFrame, transform, 0L)

// Update the target location relatively to Pepper's current location.
targetFrame.update(robotFrame, transform, 0L);

We now have the target frame. We will use it to build the lookat action.

Looking at the target frame¶

You can make Pepper look at a Frame by using the LookAt interface.

Add a lookat field in your MainActivity:

// Store the LookAt action.
private var lookAt: LookAt? = null

// Store the LookAt action.
private LookAt lookAt;

We will create it with a LookAtBuilder in the onRobotFocusGained method:

// Create a LookAt action.
lookAt = LookAtBuilder.with(qiContext) // Create the builder with the context.
                             .withFrame(targetFrame.frame()) // Set the target frame.
                             .build() // Build the LookAt action.

// Create a LookAt action.
lookAt = LookAtBuilder.with(qiContext) // Create the builder with the context.
                             .withFrame(targetFrame.frame()) // Set the target frame.
                             .build(); // Build the LookAt action.

Next, we will set the LookAtMovementPolicy to use:

// Set the LookAt policy to look with the head only.
lookAt?.policy = LookAtMovementPolicy.HEAD_ONLY

// Set the LookAt policy to look with the head only.
lookAt.setPolicy(LookAtMovementPolicy.HEAD_ONLY);

The LookAtMovementPolicy parameter is set to HEAD_ONLY so that Pepper only moves his head when looking at the specified Frame.

The LookAt interface has a addOnStartedListener method that allows you to be notified when the LookAt action starts. We will use it to log to the console:

// Add an on started listener on the LookAt action.
lookAt?.addOnStartedListener { Log.i(TAG, "LookAt action started.") }

// Add an on started listener on the LookAt action.
lookAt.addOnStartedListener(() -> Log.i(TAG, "LookAt action started."));

Do not forget to remove this listener on LookAt in the onRobotFocusLost method:

// Remove on started listeners from the LookAt action.
lookAt?.removeAllOnStartedListeners()

// Remove on started listeners from the LookAt action.
if (lookAt != null) {
    lookAt.removeAllOnStartedListeners();
}

Store a Future<Void> in the MainActivity:

// Store the action execution future.
private var lookAtFuture: Future<Void>? = null

// Store the action execution future.
private Future<Void> lookAtFuture;

It will be used to keep a reference on the action execution.

We will now run the LookAt action asynchronously:

// Run the LookAt action asynchronously.
lookAtFuture = lookAt?.async()?.run()

// Run the LookAt action asynchronously.
lookAtFuture = lookAt.async().run();

We will finally chain the lookAtFuture with a lambda:

// Add a lambda to the action execution.
lookAtFuture?.thenConsume { future ->
    if (future.isSuccess) {
        Log.i(TAG, "LookAt action finished with success.")
    } else if (future.isCancelled) {
        Log.i(TAG, "LookAt action was cancelled.")
    } else {
        Log.e(TAG, "LookAt action finished with error.", future.error)
    }
}

// Add a lambda to the action execution.
lookAtFuture.thenConsume(future -> {
    if (future.isSuccess()) {
        Log.i(TAG, "LookAt action finished with success.");
    } else if (future.isCancelled()) {
        Log.i(TAG, "LookAt action was cancelled.");
    } else {
        Log.e(TAG, "LookAt action finished with error.", future.getError());
    }
});

Cancelling the action¶

The LookAt action needs to be cancelled to stop its execution.

The way to cancel this action is to call the requestCancellation method on the Future representing the action execution:

lookAtFuture?.requestCancellation()

lookAtFuture.requestCancellation();

We will use a Button to cancel the action.

In the activity_main.xml layout file, add the following Button:

<Button
    android:id="@+id/cancel_button"
    android:layout_width="wrap_content"
    android:layout_height="wrap_content"
    android:layout_centerInParent="true"
    android:text="Cancel action"/>

Set the button onClick listener in the onCreate method.

// Set the button onClick listener.
cancel_button.setOnClickListener {
    lookAtFuture?.requestCancellation()
}

// Find the button in the view in the onCreate method.
Button cancelButton = (Button) findViewById(R.id.cancel_button);

// Set the button onClick listener.
cancelButton.setOnClickListener(v -> {
    if (lookAtFuture != null) {
        lookAtFuture.requestCancellation();
    }
});

Let’s try it¶

The sources for this tutorial are available on GitHub.

Step

Action

Install and run the application.

For further details, see: Running an application.

Choose “Stare somewhere”.

the log trace “LookAt action started.” is displayed in the console,
Pepper looks at the ground.

If you click on the cancel button:

the log trace “LookAt action was cancelled.” is displayed in the console.
Pepper stops looking at the ground.

You are now able to make Pepper look at something!