Keywords: Machine Learning methods for robot development, Development of skills in biological systems and robots, Action selection and planning
Abstract: Bio-inspired tendon-driven manipulators have the potential to achieve human-level dexterity. However, their control is more complex than prevailing robotic hands because the relation between actuation and hand motion (Jacobian) is hard to obtain. On the other hand, humans maneuver their complex hands skillfully and conduct adaptive object grasping and manipulation. We conjecture that the foundation of this ability is a visual posing of hands (i.e., a skill to make arbitrary hand poses with visual and proprioceptive feedback). Children develop this skill before or in parallel with learning grasping and manipulation. Inspired by this developmental process, this study explored a method to equip compliant tendon-driven manipulators with the visual posing. To overcome the complexity of the system, we used a learning-based approach. Specifically, we adopted PlaNet, model- based reinforcement learning that leverages a dynamics model on a compact latent representation. To further accelerate learning, we restricted the control space using the idea of muscle synergy found in the human body control. We validated the effectiveness of the proposed method in a simulation. We also demonstrated that the posing skill acquired using our method is useful for object grasping. This study will contribute to achieving human- level dexterity in manipulations.

Keywords: Machine Learning methods for robot development, Embodiment, General Principles of Development and Learning
Abstract: We propose to make the physical characteristics of a robot oscillate while it learns to improve its behavioral performance. We consider quantities such as mass, actuator strength, and size that are usually fixed in a robot, and show that when those quantities oscillate at the beginning of the learning process on a simulated 2D soft robot, the performance on a locomotion task can be significantly improved. We investigate the dynamics of the phenomenon and conclude that in our case, surprisingly, a high-frequency oscillation with a large amplitude for a large portion of the learning duration leads to the highest performance benefits. Furthermore, we show that morphological wobbling significantly increases exploration of the search space.

Keywords: Sensorimotor development, Development of skills in biological systems and robots, Neural Plasticity
Abstract: Song acquisition and production in songbirds is governed by a dedicated neural circuitry that involves two parallel pathways: a motor pathway for the production and a basal ganglia (BG) pathway for the acquisition. Juveniles imitate adult vocalizations by trial and error, evaluation being conveyed by a dopaminergic signal. The complex relationship between neural control, syrinx musculature and the produced sound makes song learning a difficult problem to solve. Reinforcement learning (RL) has been widely hypothesized to underlie such sensorimotor learning even though this can lead to sub-optimal solutions under uneven contours in continuous action spaces. In this article, we propose to re-interpret the role of the dual pathway architecture underlying avian vocal learning, that helps overcome these limitations. We posit that the BG pathway conducts exploration by inducing large daily shifts in vocal production while the motor pathway gradually consolidates song changes. This process shows striking similarities to simulated annealing, an optimisation technique in machine learning. Simulations on multiple performance landscapes with varying complexities are demonstrated and compared with standard approaches. Under realistic temporal constraints (total number of days and vocalizations), the model reaches the global optimum in complex landscapes and thus provides a sound insight into the role of the dual pathway architecture underlying vocal learning.

Keywords: Architectures for Cognitive Development and Open-Ended Learning, Intrinsic Motivation, Exploration and Play, Machine Learning methods for robot development
Abstract: Autonomous open-ended learning is a relevant approach in machine learning and robotics, allowing artificial agents to acquire a wide repertoire of goals and motor skills without the necessity of specific assignments. Leveraging intrinsic motivations, different works have developed systems that can autonomously allocate training time amongst different goals to maximise their overall competence. However, only few works in the field of intrinsically motivated open-ended learning focus on scenarios where goals have interdependent relations, and even fewer tackle scenarios involving non-stationary interdependencies. Building on previous works, we propose a new hierarchical architecture (H-GRAIL) that selects its own goals on the basis of intrinsic motivations and treats curriculum learning of interdependent tasks as a Markov Decision Process. Moreover, we provide H-GRAIL with a novel mechanism that allows the system to self-regulate its exploratory behaviour and cope with the non-stationarity of the dependencies between goals. The system is tested in a simulated and real robotic environment with different experimental scenarios involving interdependent tasks.

Keywords: Human-human and human-robot interaction and communication, Emergence of verbal and non-verbal communication, Embodiment
Abstract: Humans have an extraordinary ability to communicate and read the properties of objects by simply watching them being carried by someone else. This level of communicative skills and interpretation, available to humans, is essential for collaborative robots if they are to interact naturally and effectively. For example, suppose a robot is handing over a fragile object. In that case, the human who receives it should be informed of its fragility in advance, through an immediate and implicit message, i.e., by the direct modulation of the robot's action. This work investigates the perception of object manipulations performed with a communicative intent by two robots with different embodiments (an iCub humanoid robot and a Baxter robot). We designed the robots' movements to communicate carefulness or not during the transportation of objects. We found that not only this feature is correctly perceived by human observers, but it can elicit as well a form of motor adaptation in subsequent human object manipulations. In addition, we get an insight into which motion features may induce to manipulate an object more or less carefully.

Keywords: Development of skills in biological systems and robots, Baby robots, Social robots and social learning
Abstract: Promoting exploratory movements through contingent feedback can positively influence motor development in infancy. Our ongoing work gears toward the development of a robot-assisted contingency learning environment through the use of small aerial robots. This paper examines whether aerial robots and their associated motion controllers can be used to achieve efficient and highly-responsive robot flight for our purpose. Infant kicking kinematic data were extracted from videos and used in simulation and physical experiments with an aerial robot. The efficacy of two standard of practice controllers was assessed: a linear PID and a nonlinear geometric controller. The ability of the robot to match infant kicking trajectories was evaluated qualitatively and quantitatively via the mean squared error (to assess overall deviation from the input infant leg trajectory signals), and dynamic time warping algorithm (to quantify the signal synchrony). Results demonstrate that it is in principle possible to track infant kicking trajectories with small aerials robots, and identify areas of further development required to improve the tracking quality.

Keywords: Neural Plasticity, Cogntive vision, Machine Learning methods for robot development
Abstract: This article presents a comperehensive study of trade-offs between simulation steps and reconstruction performance for spiking autoencoders. We execute training and inference of spiking neural network to reconstruct FashionMNSIT dataset for several simulation step configurations and evaluate reconsutruction accuracies by mean squared error. Experiments showed that longer simulation step configuration indeed improves reconstruction accuracy while the improvement gets a peak at very long configuration. Flexible design on the training configuration will be applicable; for example, shorter steps could be aceptable for accurate-insensitible but latenct-restricted systems.

Keywords: Reasoning with abstract knowledge, Grounding of Knowledge and Development of Representations, Human-human and human-robot interaction and communication
Abstract: Assistive robots operating in household environments would require items to be available in the house to perform assistive tasks. However, when these items run out, the assistive robot must remind its user to buy the missing items. In this paper, we present a computational architecture that can allow a robot to learn personalized contextual knowledge of a household through interactions with its user. The architecture can then use the learned knowledge to make predictions about missing items from the household over a long period of time. The architecture integrates state-of-the-art perceptual learning algorithms, cognitive models of memory encoding and learning, a reasoning module for predicting missing items from the household, and a graphical user interface (GUI) to interact with the user. The architecture is integrated with the Fetch mobile manipulator robot and validated in a large indoor environment with multiple contexts and objects. Our experimental results show that the robot can adapt to an environment by learning contextual knowledge through interactions with its user. The robot can also use the learned knowledge to correctly predict missing items over multiple weeks and it is robust against sensory and perceptual errors.

Keywords: Reward and Value Systems, Multimodal perception, Social robots and social learning
Abstract: This study presents a robot trust model based on cognitive load that uses multimodal cues in a learning setting to assess the trustworthiness of heterogeneous interaction partners. As a test-bed, we designed an interactive task where a small humanoid robot, Nao, is asked to perform a sequential audio-visual pattern recall task while minimizing its cognitive load by receiving help from its interaction partner, either a robot, Pepper, or a human. The partner displayed one of three guiding strategies, reliable, unreliable, or random. The robot is equipped with two cognitive modules: a multimodal auto-associative memory and an internal reward module. The former represents the multimodal cognitive processing of the robot and allows a `cognitive load' or `cost' to be assigned to the processing that takes place, while the latter converts the cognitive processing cost to an internal reward signal that drives the cost-based behavior learning. Here, the robot asks for help from its interaction partner when its action leads to a high cognitive load. Then the robot receives an action suggestion from the partner and follows it. After performing interactive experiments with each partner, the robot uses the cognitive load yielded during the interaction to assess the trustworthiness of the partners --i.e., it associates high trustworthiness with low cognitive load. We then give a free choice to the robot to select the trustworthy interaction partner to perform the next task. Our results show that, overall, the robot selects partners with reliable guiding strategies. Moreover, the robot's ability to identify a trustworthy partner was unaffected by whether the partner was a human or a robot.

Keywords: Human-human and human-robot interaction and communication, Models of human motion and state, Body schema and body image
Abstract: When humans perform object handovers, the non-verbal communication implicit in the movement of the interaction partners mutually communicates information on how the handover will proceed. This intention communication allows both subjects to understand where the transfer of the object will occur, the speed of the gesture, and how careful the receiver of the object must be. In human-robot interaction, it is also desirable that the robot can read and transmit the same information. Bayesian Interaction Primitives (BIP) can be used to learn natural handover interactions from demonstrations performed between humans. In this work, we explore BIPs for handover interactions and compare a state representation obtained directly from a motion capture system with a representation using a statistical body pose model fitted to the motion capture data.

Keywords: Sensorimotor development, Architectures for Cognitive Development and Open-Ended Learning, Development of skills in biological systems and robots
Abstract: Reaching and grasping objects in 3D is still a challenging task in robotics because they have to be done in an integrated fashion, as it is for tool-use or during imitation with a human partner. The visuo-motor networks in the human brain exploit a neural mechanism known as gain-field modulation to adapt different circuits together with respect to the task and for parsimony purpose. In this paper, we show how gain- field neural networks achieve the learning of visuo-motor cells sensitive to the 3D direction of the arm motion (3D reaching), to the 3D reaching + 3D orientation of the hand (6D reaching) and to the 3D direction of tool tip (tool-use reaching) when this new information is added to the network. Experiments on robotic simulations demonstrate the accuracy of control and the efficient remapping to the new coordinate system.

Keywords: Sensorimotor development, Body schema and body image, Machine Learning methods for robot development
Abstract: When leaving the aquatic constrained environment of the womb, newborns are thrown into the world with essentially new laws and regularities that govern their interactions with the environment. Here, we study how spontaneous self-contacts can provide material for learning implicit models of the body and its action possibilities in the environment. Specifically, we investigate the space of only somatosensory (tactile and proprioceptive) activations during self-touch configurations in a simple model agent. Using biologically motivated overlapping receptive fields in these modalities, a variational autoencoder (VAE) in a denoising framework is trained on these inputs. The denoising properties of the VAE can be exploited to fill in the missing information. In particular, if tactile stimulation is provided on a single body part, the model provides a configuration that is closer to a previously experienced self-contact configuration. Iterative passes through the VAE reconstructions create a control loop that brings about reaching for stimuli on the body. Furthermore, due to the generative properties of the model, previously unsampled proprioceptive-tactile configurations can also be achieved. In the future, we will seek a closer comparison with empirical data on the kinematics of spontaneous self-touch in infants and the results of reaching for stimuli on the body.

Keywords: Sensorimotor development, Body schema and body image, Multimodal perception
Abstract: Postnatal development in infants involves building the body schema based on integrating information from different modalities. An early phase of this complex process involves coupling proprioceptive inputs with tactile information during self-touch enabled by motor babbling. Such functionality is also desirable in humanoid robots that can serve as embodied instantiation of cognitive learning. We describe a simple connectionist model composed of neural networks that learn the proprio-tactile representations in the simulated iCub humanoid robot. Input signals from both modalities -- joint angles and touch stimuli on both upper limbs -- are first self-organized in neural maps and then connected using a bidirectional associative network (UBAL). The model demonstrates the ability to predict touch and its location from proprioceptive information with relatively high accuracy. We also discuss limitations of the model and the ideas for future work.

Keywords: Social robots and social learning, Action selection and planning, Human-human and human-robot interaction and communication
Abstract: Social robots assisting in cognitive stimulation therapies, physical rehabilitation, or entertainment sessions have gained visibility in the last years. In these activities, users may present different features and needs, so personalization is essential. This manuscript presents a Preference Learning System for social robots to personalize Human-Robot Interaction during entertainment activities. Our system is integrated into Mini, a social robot dedicated to research with a wide repertoire of entertainment activities like games, displaying multimedia content, or storytelling. The learning model we propose consists of four stages. First, the robot creates a unique profile of its users by obtaining their defining features using interaction. Secondly, a Preference Learning algorithm predicts the users' favorite entertainment activities using their features and a database with the features and preferences of other users. Third, the prediction is adapted using Reinforcement Learning while entertainment sessions occur. Finally, the robot personalizes Human-Robot Interaction by autonomously selecting the users' favorite activities. Thus, the robot aims at promoting longer-lasting interactions and sustaining engagement.

Keywords: Social robots and social learning, Language acquisition, Models of emotions and internal states
Abstract: Most computational models of language development adopt a passive-learner view on language learning, and disregard the important role that motivation and affect play in the development of communication. In this paper, we present a motivation-grounded, active learning robot model of language acquisition that relies on social interaction with a caregiver. The robot learns multiple associations---between words and internal states, and between the latter and perceived objects--allowing it to have a “meaning potential” of the acquired language, which is in line with the functionalist view of language theory. We evaluate the model experimentally in different environments and with different levels of caregiver's responsiveness to study the impact of external factors on language acquisition.

Keywords: Statistical Learning, General Principles of Development and Learning, Language acquisition
Abstract: Machine learning models of visual action recognition are typically trained and tested on data from specific situations where actions are associated with certain objects. It is an open question how action-object associations in the training set influence a model's ability to generalize beyond trained situations. We set out to identify properties of training data that lead to action recognition models with greater generalization ability. To do this, we take inspiration from a cognitive mechanism called cross-situational learning, which states that human learners extract the meaning of concepts by observing instances of the same concept across different situations. We perform controlled experiments with various types of action-object associations, and identify key properties of action-object co-occurrence in training data that lead to better classifiers. Given that these properties are missing in the datasets that are typically used to train action classifiers in the computer vision literature, our work provides useful insights on how we should best construct datasets for efficiently training for better generalization.

Keywords: The Contributions of interaction to learning, Active learning
Abstract: The capability to interactively learn from human feedback would enable agents in new settings. For example, even novice users could train service robots in new tasks naturally and interactively. Human-in-the-loop Reinforcement Learning (HRL) combines human feedback and Reinforcement Learning (RL) techniques. State-of-the-art interactive learning techniques suffer from slow learning speed, thus leading to a frustrating experience for the human. We approach this problem by extending the HRL framework TAMER for evaluative feedback with the possibility to enhance human feedback with two different types of counterfactual explanations (action and state based). We experimentally show that our extensions improve the speed of learning.