Robot Learning from Human Teachers. Sonia Chernova

Чтение книги онлайн.

      Figure 3.3: (a) Kinesthetic teaching with the iCub robot [13]. (b) User controlling the full-body motions of an Aldebaran Nao robot using the Xsens MVN inertial motion capture suit [141].

3.2 LEARNING BY DOING

      Teleoperation provides the most direct method for information transfer within demonstration learning. During teleoperation, the robot is operated by the teacher while recording from its own sensors. Demonstrations recorded through human teleoperation via a joystick have been used in a variety of applications, including flying a robotic helicopter [1], soccer kicking motions [40], robotic arm assembly tasks [64], and obstacle avoidance and navigation [118, 237]. Teleoperation has also been applied to a wide variety of simulated domains, such as mazes [70, 214], driving [3, 66], and soccer [7], and many other applications. Teleoperation interfaces vary in complexity from hand-held controllers to teleoperation suits [159]. Hand-written controllers have also been used to teleoperate the robot in the place of a human teacher [11, 102, 221, 237].

      Kinesthetic teaching offers another variant for teleoperation. In this method, the robot is not actively controlled, but rather its passive joints are moved through the desired motions while the robot records the trajectory [51]. Figure 3.3(a) shows a person teaching a humanoid robot to manipulate an object. This technique has been extensively used in motion trajectory learning, and many complementary computational methods are discussed in Chapter 4. A key benefit of teaching through this method of interaction is that it ensures that the demonstrations are constrained to actions that are within the robot’s abilities, and the correspondence problem is largely eliminated. Additionally, the user is able to directly experience the limitation of the robot’s movements, and thus gain greater understanding about the robot’s abilities.

      Another alternative to direct teleoperation is shadowing, in which the robot mimics the teacher’s demonstrated motions while recording from its own sensors. In comparison to teleoperation, shadowing requires an extra algorithmic component which enables the robot to track and actively shadow (rather than be passively moved by) the teacher. Body sensors are often used to track the teacher’s movement with a high degree of accuracy. Figure 3.3(b) shows an example setup used by [141], in which the Xsens MVN inertial motion capture suit worn by the user is used to control the robot’s pose. This example demonstrates tightly coupled interaction between the user and the robot, since almost every teacher movement is detected by the sensors.

      Shadowing also allows for loosely coupled interactions, and has even been applied to robotic teachers. Hayes and Demiris [109] perform shadowing with a robot teacher whose platform is identical to the robot learner; the learner follows behind the teacher as it navigates through a maze. Nehmzow et al. [187] present an algorithm for robot motion control in which the robot first records the human teacher’s execution of the desired navigation trajectory, and then shadows this execution. While repeating the teacher’s trajectory, the robot records data about its environment using its onboard sensors. The action and sensor data are then combined into a feedback controller that is used to reproduce future instances of the demonstrated task.

      Trajectory information collected through teleoperation, kinesthetic teaching or shadowing can be combined with other input modalities, such as speech. Nicolescu and Mataric [190] present an approach in which a robot learns by shadowing a robotic or human teacher. In addition to trajectory information, their technique enables the teacher to use simple voice cues to frame the learning (“here,” “take,” “drop,” “stop”), to provide informational cues about the relevance or irrelevance of observation inputs and indications of the desired behavioral output. In Rybski et al. [225], demonstration of the desired task is also performed through shadowing combined with dialog in which the robot is told specifically what actions to execute in various states. Meriçli et al. [175] present a similarly motivated approach which additionally supports repetitions (cycles) in the task representation and enables the user to modify and correct an existing task. Breazeal et al. [36] also explore this form of demonstration, enabling a robot to learn a symbolic high-level task within a social dialog.

      Finally, some learning methods pay attention only to the state sequences, without recording any actions. This makes it possible to communicate the task objective function to the learner without traditional action demonstrations. For example, by drawing a path through a 2-D representation of the physical world, Ratliff et al. provide high-level path planning demonstrations to a rugged outdoor robot [215] and a small quadruped robot [143, 216]. Human-controlled teleoperation demonstrations are also utilized with the same outdoor robot for lower-level obstacle avoidance [216]. Since actions are not provided in the demonstration data, at run time a learned state-action mapping does not exist to provide guidance for action selection. Instead, actions are selected by employing low level motion planners and controllers [215, 216], and provided transition models [143].

      Figure 3.4: (a) User teaching a forehand swing motion to a humanoid robot using the Sarcos Sen-Suit [115]. (b) Humanoid robot learning to play air hockey from observation of opponent player [25].

3.3 LEARNING FROM OBSERVATION

      In many situations, it is more effective or natural for the teacher to perform the task demonstration using their own body instead of controlling the robot directly. As discussed above, this form of demonstration introduces a correspondence problem with respect to the mapping between the teacher’s and robot’s state and actions. As a result, this technique is commonly used with humanoid or anthropomorphic robots, since the robot’s resemblance to a human results in a simpler and more intuitive mapping, though learning with other robot embodiments is also possible. Unlike in the use of shadowing, the robot does not simultaneously mimic the teacher’s actions during the observation.

      Accurately sensing the teacher’s actions is critical for the success of this approach. Traditionally, many techniques have relied on instrumenting the teacher’s body with sensors, including the use of motion capture systems and inertial sensors. Ijspeert et al. [114, 115] use a Sarcos Sen-Suit worn by the user to simultaneously record 35 DOF motion. The recorded joint angles were used to teach a 30-DoF humanoid to drum, reach, draw patterns, and perform tennis swings (Figure 3.3(a)). This work is extended in [184] to walking patterns. The same device, supplemented with Hall sensors, is used by Billard et al. to teach a humanoid robot to manipulate boxes in sequence [29]. In later work, Calinon and Billard combine demonstrations executed by human teacher via wearable motion sensors with kinesthetic teaching [50].

      Wearable sensors, and other forms of specialized recording devices, provide a high degree of accuracy in the observations. However, their use restricts the adoption of such learning methods beyond research laboratories and niche applications. A number of approaches have been designed to use only camera data. One of the earliest works in this area was the 1994 paper by Kuniyoshi et al. [152], in which a robot extracts the action sequence and infers and executes a task plan based on observations of a human hand demonstrating a blocks assembly task. Another example of this demonstration approach includes the work of Bentivegna et al. [25], in which a 37-DoF humanoid learns to play air hockey by tracking the position of the human opponent’s paddle (Figure 3.3(b)). Visual markers are also often used to improve the quality of visual information, such as in [30], where reaching patterns are taught to a simulated humanoid. Markers are similarly used to optically track human motion in [122, 123, 259] and to teach manipulation [209] and motion sequences [10]. In recent years, the availability of low-cost depth sensors (e.g., Microsoft Kinect) and their associated body pose tracking methods makes this a great СКАЧАТЬ