Human-Robot Interaction

Preschoolers emerge themselves in language learning accompanied by an interactive agent


This project was a project work for Tampere University's User Experience in Robotics -course, client was a company that makes a language learning software on top of Softbanks's NAO robot. It was used as a teaching assistant with local elementary school and preschool pupils. In addition to the robot and the software, also a cloud-based web application has been developed to control the robot’s programs remotely.


What kind of interactional functionalities can be added or changed in the robot to support children’s learning? Design a concept based on your findings.


The designed concept consists of combined verbal and nonverbal cues. Solution consists of more natural dialogue, a button added into the application for extra feedback and finally, direction of gaze prioritizing children over the operator.


The report was initiated with an examination into the previous literature. This gave a foundation for the rest of the project. To begin our design process, we first needed data from the field. The communication with all included parties were fluent and the observation studies were successful. From the data received from the preliminary studies, a list of design implications was found. A concept was design based on our own study data and the data from the literature. Finally, the concept was evaluated and developed accordingly.

    0 step
  • Research
    • Observation & Interview x 2
  • 1 step
  • Design
    • Affinity Diagram
    • Design Implications
    • Concept specification
    • Dialoque
  • 2 step
  • Evaluation
    • Theatrical Robot


  • 3 Designers
Time Frame
  • March ’19 – April ’19
My Role
  • Interaction Design


We set out to observe the robot in use and form a concept to improve it's interaction with the primary target groups, a local preschool class learning Finnish as a second language and a preschool class familiarizing with English, French or German as a second language, (end users) and for the secondary target group their teachers, who operate the robot (the operators).

Observation frame and findings

Observation sessions were agreed upon in a meeting with the teachers and an observation form was prepared in advance with the focus of examining how children interact with the robot, but not excluding the teacher. Session was videoed and notes on the observation form were made. After the teachers present was immediately interviewed.

In the first observation, four children in the Finnish as a second language group and the teacher interacted with the robot for about 30 minutes and practiced various linguistic exercises in a closed classroom.

In the second observation session, we observed a class of English as a second language group where ~15 children were present and the robot told a story for the whole group. Multiple children were sitting in a circle with the robot and listened to the story. After which, volunteered pupils could interact with the robot in a smaller classroom in a group of three.

Screen capture of our game environment with a controller, a blue ball, and a yellow circle goal in the distance

Storytelling session

Similar results were found in observation sessions. The pupils were highly excited with the robot coming to their lesson and went bravely close to the robot. All the children had already seen the robot before each observation session. In addition to being interested and excited of the robot, most children had nursing behavior towards the robot: they were gentle with it, one child put pillow under it when the robot was sitting and they were shielding the robot when it was doing some movements in order to avoid it falling down. Adding to this, one of the children said that (s)he wanted to trip the robot if it would start walking, but it was not clear if the statement was a joke to amuse other children or actual urge.

The pupils had difficulties of getting the robot to understand them and the speech recognition was unsuccessful more frequently than successful. The children tried to fix the issue by spelling the command to the robot or saying it to near its ear, these countermeasures did not work. Also the children asked for the teacher to vocalize the commands, as they had observed better recognizing that way. In addition, the children looked for acceptance more from the teacher than the robot.

In the bigger group session, the robot was gazing at the teacher most of time rather than at the children who were trying to establish a contact with the robot. The teacher had to turn manually the robot in order to have it's face and eyes towards the children.

Interview frame and findings

Recorded semi-structured interviews held after the observation session took roughly 20 minutes each were afterwards transcribed.

Teachers had a positive attitude for using the robot in teaching. One stated that the robot might be the most useful motivational tool, rather than as an independent teacher independently. All found the robot a useful asset and agreed that the children like the robot very much and enjoy interacting with it. They were not especially concerned of ethical issues arising from using the robot, they just pointed out that children of this age might have a sense of animism towards the robot. For example, teacher had told the pupils that the robot will be joining them to the next class and the children were waiting for the robot in the window as if the robot would walk to the classroom by itself like a person.

The operators' main frustration were speech recognition problems. They recalled that children are persistent repeaters, but felt personally that the repetition is tiring.


When both observation session were done, the team gathered the notes on the observation form and the notes were utilized together with the interview results in the Affinity Diagram. We printed out observation and interview data, cut them to slices of single items, looked for related ideas, sorted them into piles and then organized them under themes on to a wall poster.