Keywords: Legged Robots, Reinforcement Learning, Deep Learning in Robotics and Automation, AI-Based Methods
Abstract: Several earlier studies have shown impressive control performance in complex robotic systems by designing the controller using a neural network and training it with model-free reinforcement learning. However, these outstanding controllers with natural motion style and high task performance are developed through extensive reward engineering, which is a highly laborious and time-consuming process of designing numerous reward terms and determining suitable reward coefficients. In this work, we propose a novel reinforcement learning framework for training neural network controllers for complex robotic systems consisting of both rewards and constraints. The learning framework is applied to train locomotion controllers for several legged robots with different morphology and physical attributes to traverse challenging terrains. Extensive simulation and real-world experiments demonstrate that performant controllers can be trained with significantly less reward engineering, by tuning only a single reward coefficient. Furthermore, a more straightforward and intuitive engineering process can be utilized, thanks to the interpretability and generalizability of constraints.

Keywords: Legged Robots, Optimization and Optimal Control, Compliance and Impedance Control, Motion Control
Abstract: Introducing parallel elasticity in the hardware design endows quadrupedal robots with the ability to perform explosive and efficient motions. However, for this kind of articulated soft quadruped, realizing dynamic jumping with robustness against system uncertainties remains a challenging problem. To achieve this, we propose an impact-aware jumping planning and control approach. Specifically, an offline kino-dynamic-type trajectory optimizer is first formulated to achieve compliant 3D jumping motions, using a novel actuated spring-loaded inverted pendulum (SLIP) model. Then, an optimization-based online landing strategy, including the pre-impact leg motion modulation in the air and post-impact landing recovery after touch-down, is designed. The actuated SLIP model, with the capability of explicitly characterizing parallel elasticity, captures the jumping and landing dynamics, making the problem of motion generation/regulation more tractable. Finally, a hybrid torque control consisting of a feedback tracking loop and a feedforward compensation loop is employed for motion control. Experiments demonstrate the ability to accomplish robust 3D jumping motions with stable landing and recov

Keywords: Humanoid and Bipedal Locomotion, Motion and Path Planning, Legged Robots
Abstract: How many ways are there to climb a staircase in a given number of steps? Infinitely many, if we focus on the continuous aspect of the problem. A finite, possibly large number if we consider the discrete aspect, i.e. on which surface which effectors are going to step and in what order. We introduce NAS, an algorithm that considers both aspects simultaneously and computes all the possible solutions to such a contact planning problem, under standard assumptions. To our knowledge NAS is the first algorithm to produce a globally optimal policy, efficiently queried in real time for planning the next footsteps of a humanoid robot. Our empirical results (in simulation and on the Talos platform) demonstrate that, despite the theoretical exponential complexity, optimisations reduce the practical complexity of NAS to a manageable bilinear form, maintaining completeness guarantees and enabling efficient GPU parallelisation. NAS is demonstrated on a variety of scenarios for the Talos robot, both in simulation and on the hardware platform. Future work will focus on further reducing computation times and extending the algorithm’s applicability beyond gaited locomotion

Keywords: Humanoid and Bipedal Locomotion, Whole-Body Motion Planning and Control, Humanoid Robot Systems
Abstract: This paper presents a three-layered architecture that enables stylistic locomotion with online contact location adjustment. Our method combines an autoregressive Deep Neural Network (DNN) acting as a trajectory generation layer with a model-based trajectory adjustment and trajectory control layers. The DNN produces centroidal and postural references serving as an initial guess and regularizer for the other layers. Being the DNN trained on human motion capture data, the resulting robot motion exhibits locomotion patterns, resembling a human walking style. The trajectory adjustment layer utilizes non-linear optimization to ensure dynamically feasible center of mass (CoM) motion while addressing step adjustments. We compare two implementations of the trajectory adjustment layer: one as a receding horizon planner (RHP) and the other as a model predictive controller (MPC). To enhance MPC performance, we introduce a Kalman filter to reduce measurement noise. The filter parameters are automatically tuned with a Genetic Algorithm. Experimental results on the ergoCub humanoid robot demonstrate the system's ability to prevent falls, replicate human walking styles, and withstand disturbances up to 68 Newton.

Keywords: Multi-Contact Whole-Body Motion Planning and Control, Collision Avoidance, Motion and Path Planning
Abstract: Humanoid robots rely on multi-contact planners to navigate a diverse set of environments, including those that are unstructured and highly constrained. To synthesize stable multi-contact plans within a reasonable time frame, most planners assume statically stable motions or rely on reduced order models. However, these approaches can also render the problem infeasible in the presence of large obstacles or when operating near kinematic and dynamic limits. To that end, we propose a new multi-contact framework that leverages recent advancements in relaxing collision-free path planning into a convex optimization problem, extending it to be applicable to humanoid multi-contact navigation. Our approach generates near-feasible trajectories used as guides in a dynamic trajectory optimizer, altogether addressing the aforementioned limitations. We evaluate our computational approach showcasing three different-sized humanoid robots traversing a high-raised naval knee-knocker door using our proposed framework in simulation. Our approach can generate motion plans within a few seconds consisting of several multi-contact states, including dynamic feasibility in joint space.

Keywords: Optimization and Optimal Control, Whole-Body Motion Planning and Control, Humanoid Robot Systems
Abstract: Whole body model predictive control (WBMPC) is a powerful tool to generate complex robotics motion. Despite the recent increase in computational capabilities with new processors such as the Apple chipsets (M1, .. M3) or GPUs, WBMPC algorithms need a significant amount of computational time. This induces delay, which, if not properly accounted for, can have detrimental effects on the controller's performance. This paper conducts a detailed study to understand the impact of delay on WBMPC and proposes an efficient solution to handle it effectively. In this regard, a whole-body control task is formulated as an optimal control problem and solved using the Crocoddyl library. An extensive amount of numerical studies are carried out to understand the nature of the problem and thereby devise an effective solution. The proposed solution is found to be effective numerically, and it has been experimentally verified with the humanoid TALOS. Both numerical and experimental results are presented and discussed in this work to provide valuable incites.

Keywords: Humanoid and Bipedal Locomotion, Reinforcement Learning, Optimization and Optimal Control
Abstract: This paper proposes an online bipedal footstep planning strategy that combines model predictive control (MPC) and reinforcement learning (RL) to achieve agile and robust bipedal maneuvers. While MPC-based foot placement controllers have demonstrated their effectiveness in achieving dynamic locomotion, their performance is often limited by the use of simplified models and assumptions. To address this challenge, we develop a novel foot placement controller that leverages a learned policy to bridge the gap between the use of a simplified model and the more complex full-order robot system. Specifically, our approach employs a unique combination of an ALIP-based MPC foot placement controller for sub-optimal footstep planning and the learned policy for refining footstep adjustments, enabling the resulting footstep policy to capture the robot's whole-body dynamics effectively. This integration synergizes the predictive capability of MPC with the flexibility and adaptability of RL. We validate the effectiveness of our framework through a series of experiments using the full-body humanoid robot DRACO 3. The results demonstrate significant improvements in dynamic locomotion performance, including better tracking of a wide range of walking speeds, enabling reliable turning and traversing challenging terrains while preserving the robustness and stability of the walking gaits compared to the baseline ALIP-based MPC approach.

Keywords: Human-Robot Collaboration, Humanoid Robot Systems, Reactive and Sensor-Based Planning
Abstract: This paper presents Surena-V, a humanoid robot designed to enhance human-robot collaboration capabilities. The robot features a range of sensors, including barometric tactile sensors in its hands, to facilitate precise environmental interaction. This is demonstrated through an experiment showcasing the robot's ability to control a medical needle's movement through soft material. Surena-V's operational framework emphasizes stability and collaboration, employing various optimization-based control strategies such as Zero Moment Point (ZMP) modification through upper body movement and stepping. Notably, the robot's interaction with the environment is improved by detecting and interpreting external forces at their point of effect, allowing for more agile responses compared to methods that control overall balance based on external forces. The efficacy of this architecture is substantiated through an experiment illustrating the robot's collaboration with a human in moving a bar. This work contributes to the field of humanoid robotics by presenting a comprehensive system design and control architecture focused on human-robot collaboration and environmental adaptability.

Keywords: Optimization and Optimal Control, Learning from Demonstration, Modeling and Simulating Humans
Abstract: Inverse optimal control (IOC) allows the retrieval of optimal cost function weights, or behavioral parameters, from human motion. The literature on IOC uses methods that are either based on a slow bilevel process or a fast but noise-sensitive minimization of optimality conditions. Assuming equality-constrained optimal control models of human motion, this article presents a faster approach to solving IOC using a single-level reformulation of the bilevel method and yields equivalent results. Through numerical experiments in simulation, we analyze the robustness to noise of the proposed single-level reformulation to the bilevel IOC formulation with a human-like planar reaching task that is used across recent studies. The approach shows resilience to very large levels of noise and reduces the computation time of the IOC on this task by a factor of 15 when compared to a classical bilevel implementation.

Keywords: Hardware-Software Integration in Robotics, Legged Robots, Software-Hardware Integration for Robot Systems
Abstract: Quadruped robots that individual researchers can build by themselves are crucial for expanding the scope of research due to their high scalability and customizability. These robots must be easily ordered and assembled through e-commerce or DIY methods, have a low number of components for easy maintenance, and possess durability to withstand experiments in diverse environments. Various quadruped robots have been developed so far, but most robots that can be built by research institutions are relatively small and made of plastic using 3D printers. These robots cannot withstand experiments in external environments such as mountain trails or rubble, and they will easily break with intense movements. Although there is the advantage of being able to print parts by yourself, the large number of components makes replacing broken parts and maintenance very cumbersome. Therefore, in this study, we develop a metal quadruped robot MEVIUS, that can be constructed and assembled using only materials ordered through e-commerce. We have considered the minimum set of components required for a quadruped robot, employing metal machining, sheet metal welding, and off-the-shelf components only. Also, we have achieved a simple circuit and software configuration. Considering the communication delay due to its simple configuration, we experimentally demonstrate that MEVIUS, utilizing reinforcement learning and Sim2Real, can traverse diverse rough terrains and withstand outside experiments. All hardware and software components can be obtained from https://github.com/haraduka/mevius.

Keywords: Legged Robots, Multi-Contact Whole-Body Motion Planning and Control, Imitation Learning
Abstract: Legged robots have become capable of performing highly dynamic maneuvers in the past few years. However, agile locomotion in highly constrained environments such as stepping stones is still a challenge. In this paper, we propose a combination of model-based control, search, and learning to design efficient control policies for agile locomotion on stepping stones. In our framework, we use nonlinear model predictive control (NMPC) to generate whole-body motions for a given contact plan. To efficiently search for an optimal contact plan, we propose to use Monte Carlo tree search (MCTS). While the combination of MCTS and NMPC can quickly find a feasible plan for a given environment (a few seconds), it is not yet suitable to be used as a reactive policy. Hence, we generate a dataset for optimal goal-conditioned policy for a given scene and learn it through supervised learning. In particular, we leverage the power of diffusion models in handling multi-modality in the dataset. We test our proposed framework on a scenario where our quadruped robot Solo12 successfully jumps to different goals in a highly constrained environment.

Keywords: Gesture, Posture and Facial Expressions, Human and Humanoid Motion Analysis and Synthesis, Datasets for Human Motion
Abstract: Head movements are crucial for social human-human interaction. They can transmit important cues (e.g., joint attention, speaker detection) that cannot be achieved with verbal interaction alone. This advantage also holds for human-robot interaction. Even though modeling human motions through generative AI models has become an active research area within robotics in recent years, the use of these methods for producing head movements in human-robot interaction remains underexplored. In this work, we employed a generative AI pipeline to produce human-like head movements for a Nao humanoid robot. In addition, we tested the system on a real-time active-speaker tracking task in a group conversation setting. Overall, the results show that the Nao robot successfully imitates human head movements in a natural manner while actively tracking the speakers during the conversation. Code and data from this study are available at https://bosongding.github.io/Humanoids2024Air-Lab/.

Keywords: Reinforcement Learning, Humanoid Robot Systems, Social HRI
Abstract: This paper presents a novel approach to enhance the social interaction capabilities of the ARI humanoid robot using reinforcement learning. We focus on enabling ARI to imitate human attention/gaze behaviour by identifying salient points in dynamic environments, employing the Zero-Shot Transfer technique combined with domain randomisation and generalisation. Our methodology uses the Proximal Policy Optimisation algorithm, training the reinforcement learning agent in a simulated environment to maximise robustness in real-world scenarios. We demonstrated the efficacy of our approach by deploying the trained agent on the ARI humanoid and validating its performance in human-robot interaction scenarios. The results indicated that using the developed model, ARI can successfully identify and respond to salient points, exhibiting human-like attention/gaze behaviours, which is an important step towards acceptability and efficiency in human-robot interactions. This research contributes to advancing the capabilities of social robots in dynamic and unpredictable environments, highlighting the potential of combining Zero-Shot Transfer with domain randomisation and generalisation for robust real-world applications.

Keywords: Body Balancing, Dynamics, Contact Modeling
Abstract: Abstract--- In this paper, we propose a controller for legged robots that takes into account the individual contact compliance. This controller is able to achieve both balance stabilization and force control in the same loop in task space thanks to the use of an explicit contact flexibility model and simplified centroidal dynamics that allow exploiting the redundancy of the kinematic and force feedback, while making multiple contacts with the environment. The control problem is formulated as LQR based on a linearized model of the reduced non-linear model. The performance of the controller with regard to robustness to modeling error and external perturbations has been tested in experiments on the position-control robot HRP-2KAI, in legged contacts and multi-contact modes.

Keywords: Prosthetics and Exoskeletons, Actuation and Joint Mechanisms, Wearable Robotics
Abstract: Inducing proper ankle joint alignment at heel contact is important in the gait cycle in terms of smooth weight transfer and reduced burden on the knees and hips. Therefore, the insole-type device that makes the ankle angle neutral during heel contact has developed. However, the backlash in the drive mechanism caused a rattling during heel contact, resulting in the accuracy of the control angle and discomfort issues. In this study, we propose the insole-type walking support device equipped with a control method to eliminate rattling. The validation of effectiveness was conducted through functionality and durability experiments, and it was confirmed that the ankle angle could be controlled within 375ms with an accuracy of 0.1deg or less, and that there were no durability problems. In addition, a foot-stamping experiment with one healthy male showed that use of the device improved knee alignment.

Keywords: Whole-Body Motion Planning and Control, Humanoid Robot Systems, Collision Avoidance
Abstract: This paper proposes a novel self-collision avoidance (SCA) scheme for whole-body model predictive control (WB-MPC). Since WB-MPC deals with a large-scale optimization problem, which becomes larger as the target robot is with larger degrees of freedom, the derivatives of the dynamics and cost functions should be computed as fast as possible. As SCA using detailed collision bodies is computationally expensive, it is challenging to embed SCA in WB-MPC with such a requirement. One way to solve this open issue is to approximate the robot model with primitive shapes, but this method accumulates modeling errors. To make derivative calculations of a detailed collision model fast in real-time, we develop a frame-distance predictor using a deep neural network (DNN). The DNN-based frame-distance predictor estimates the minimum distance between all body frames based on the whole-body joint angles. The minimum distance estimated can be embedded into WB-MPC as one of the cost functions in a differentiable manner. As a result, the WB-MPC with the DNN-based frame-distance predictor is able to plan long-term self-collision-aware motions in real-time. The proposed method is evaluated using a dual-arm robot both in simulation and real environment. The results show that the proposed method achieves more accurate tracking motions while satisfying SCA than the case with the approximated collision model.

Keywords: Human-Centered Robotics, Social HRI, Multi-Modal Perception for HRI
Abstract: Socially Assistive Robots (SARs) integrate technology with social interaction to provide personalized assistance, especially in healthcare and social care settings. Seniors can use SARs to enhance their understanding of robots by interacting with them and learning about their different features and capabilities. However, ensuring effective senior- robot interactions necessitates addressing the specific needs and accessibility requirements of all users, an area that remains under-explored in Human-Robot Interaction (HRI). This research involved conducting one-hour interactive sessions in two care homes (N=12), where seniors engaged with SoftBank Robotics’ Pepper robot for playing bingo and LuxAI’s QTrobot for discussing data privacy. Following these senior-robot interactions, group interviews were conducted, and all sessions were video recorded for analysis. The study aimed to identify the key attributes that enhance the comprehensibility of the robot’s speech, and to assess the impact of combining speech and supporting pictures on senior-robot interactions. The study highlighted the importance of optimizing the robot’s speech characteristics, such as speed, volume, vocabulary, and pitch to ensure better understanding and more effective communication. Additionally, findings indicated that supplementing speech with supporting pictures enhances seniors’ task independence.

Keywords: Compliance and Impedance Control, Telerobotics and Teleoperation, Design and Human Factors
Abstract: In this paper, we present a design and evaluation of a novel finger-operated teleimpedance interface used to command stiffness ellipsoids to the remote robot. The proposed interface provides a practical alternative to the state-of-the-art teleimpedance interfaces based on physiological signals that can be impractical in daily use. On the other hand, as opposed to existing practical interfaces that lack in terms of controlled degrees of freedom, the proposed interface enables control of 3D aspects of the ellipsoid. The remote robot stiffness ellipsoid is controlled with a single hand using the thumb, index, and middle fingers to operate two scroll wheels, a joystick, and a force sensor. These combinations of inputs can be mapped to control different aspects of the stiffness ellipsoid, i.e., orientation and shape/size. To investigate different modes of input mapping, we perform a human factors experiment to evaluate the performance and user acceptance of the proposed interface modes. The results of the experiments indicate that the participants can successfully operate the interface to complete 3D stiffness configuration alignment tasks in different modes. To further demonstrate the functionality of the proposed teleimpedance interface, we performed an additional experiment utilising a Force Dimension Sigma7 haptic device to control the motion of a KUKA LBR iiwa robotic arm while performing a complex physical interaction task.

Keywords: Manipulation Planning, Reinforcement Learning
Abstract: Close and precise placement of irregularly shaped objects requires a skilled robotic system. The manipulation of objects that have sensitive top surfaces and a fixed set of neighbors is particularly challenging. To avoid damaging the surface, the robot has to grasp them from the side, and during placement, it has to maintain the spatial relations with adjacent objects, while considering the physical gripper extent. In this work, we propose a framework to learn an agent based on reinforcement learning that generates end-effector motions for placing objects as closely as possible to one another. During the placement, our agent considers the spatial constraints with neighbors defined in a given layout of the objects while avoiding collisions. Our approach learns to place compact object assemblies without the need for predefined spacing between objects, as required by traditional methods. We thoroughly evaluated our approach using a two-finger gripper mounted on a robotic arm with six degrees of freedom. The results demonstrate that our agent significantly outperforms two baseline approaches in object assembly compactness, thereby reducing the space required to position the objects while adhering to specified spatial constraints.

Keywords: Education Robotics, Product Design, Development and Prototyping
Abstract: This research focuses on developing a humanoid robot hardware platform that improves accessibility by allowing most mechanical and electronic components to be purchased from e-commerce sites and obtained as custom-made parts through external services. The SUSTAINA-OP™ series, developed through research and competition activities in the RoboCup Humanoid League is designed as a robust robot using metal and FRP and has influenced the development of various robots in RoboCup and other international competitions. The public availability of design data allows researchers to efficiently build and customize robots for diverse applications. The successor model, SUSTAINA-OP2™, adopts official developer kits for its computer boards and sensor modules to reduce the burden of software development and features new actuators that further enhance the robot’s robustness. All design data is available at github.com/SUSTAINA-OP2. The SUSTAINAOP2™ was evaluated at RoboCup 2024, held in Eindhoven in July 2024, where it secured first place in the Humanoid KidSize class soccer competition.

Keywords: Actuation and Joint Mechanisms, Passive Walking, Legged Robots
Abstract: Contemporary advances in humanoid bipeds include complicated mechanisms supporting fully actuated robotics. While these solutions offer unprecedented versatility that relies on sophisticated locomotion systems, the efficient traversing capabilities of natural, passive-inspired gait may be overlooked. In this paper, we focus on the development of a disengage-able, selectively lockable, affordable knee joint assembly suitable for either active or underactuated biped operation. The multi-modal nature of the design facilitates actuated bidirectional rotation as well as disengaged operation, while the knee lock ensures legs remain rigid during stance phase. Experimental validation of the performance of the knee assembly demonstrated both controllable knee joint actuation and fully passive rotation, various modes of operation, and torque evaluations. Further investigations include validation testing of operational and locking performance on a physical bipedal robot. System parameter optimizations can reduce energy consumption. The design proved feasible and is open-sourced to facilitate replication and co-development with others.

Keywords: Bioinspired Robot Learning, Imitation Learning, Legged Robots
Abstract: Soccer kicking is a complex whole-body motion that requires intricate coordination of various motor actions. To accomplish such dynamic motion in a humanoid robot, the robot needs to simultaneously: 1) transfer high kinetic energy to the kicking leg, 2) maintain balance and stability of the entire body, and 3) manage the impact disturbance from the ball during the kicking moment. Prior studies on robotic soccer kicking often prioritized stability, leading to overly conservative quasi-static motions. In this work, we present a biomechanics-inspired control framework that leverages trajectory optimization and imitation learning to facilitate highly dynamic soccer kicks in humanoid robots. We conducted an in-depth analysis of human soccer kick biomechanics to identify key motion constraints. Based on this understanding, we designed kinodynamically feasible trajectories that are then used as a reference in imitation learning to develop a robust feedback control policy. We demonstrate the effectiveness of our approach through a simulation of an anthropomorphic 25 DoF bipedal humanoid robot, named PresToe, which is equipped with 7 DoF legs, including a unique actuated toe. Using our framework, PresToe can execute dynamic instep kicks, propelling the ball at speeds exceeding 11 m/s in full dynamic simulation.

Keywords: Social HRI
Abstract: Physical exercise is crucial for promoting and maintaining the health of middle-aged and older adults. As the elderly population grows, effective coaching methods are increasingly necessary. Most current coaching systems lack clear reactive and objective feedback options. This study introduces a multimodal system featuring a socially assistive robot that guides individuals through exercise routines while providing encouragement and feedback. The system includes a heart rate sensor and physical motion monitoring, allowing the robot to offer real-time suggestions based on user performance. A graphical interface mirrors the user’s movements and displays information on heart rate, kinematic data, scores, and next steps. This study evaluates the performance of two robots (NAO and Pepper). It investigates whether different robot embodiments—such as physical appearance and size—influence users’ perceptions through a within-subjects approach. Questions from the Robotic Social Attribute Scales (ROSaS) assessed users’ feelings of competence, warmth, and discomfort towards the robots. Elements from the Unified Theory of Acceptance and Use of Technology (UTAUT) measured performance expectation, effort expectancy, and social influence. Nineteen adults aged over 40 completed a series of upper-limb exercises. Both robots effectively communicated the exercises and corrected participants’ movements. Participants found the system engaging and anticipated that it would be well-received by others, regardless of the robot used.

Keywords: Robotics and Automation in Life Sciences, Telerobotics and Teleoperation, Semantic Scene Understanding
Abstract: Robot systems capable of executing tasks based on language instructions have been actively researched. It is challenging to convey uncertain information that can only be determined on-site with a single language instruction to the robot. In this study, we propose a system that includes ambiguous parts as template variables in language instructions to communicate the information to be collected and the options to be presented to the robot for predictable uncertain events. This study implements prompt generation for each robot action function based on template variables to collect information, and a feedback system for presenting and selecting options based on template variables for user-to-robot communication. The effectiveness of the proposed system was demonstrated through its application to real-life support tasks performed by the robot.

Keywords: Wearable Robotics, Intention Recognition, Physical Human-Robot Interaction
Abstract: Controlling active exoskeletons for occupational assistance is a challenge. Unlike for rehabilitation exoskeletons, Electromyography (EMG) sensors can hardly be used for control in an industrial environment. The control of assistive exoskeletons needs to rely on onboard sensors to follow the human and assist when needed. This study explores the use of motion prediction, to enhance exoskeleton control in the absence of payloads. When no payloads are involved, the exoskeleton should be transparent meaning that the interaction forces between the exoskeleton and the user should be minimal. We conducted an experiment using a 3D-printed active elbow exoskeleton and compared exoskeleton control methodologies based on dynamic modeling and human motion prediction. Fifteen participants performed a repetitive pointing task under a baseline, two non-predictive controllers and two predictive controllers. The results demonstrated a significant reduction in interaction forces—up to 45%—with predictive controllers compared to non-predictive controllers. While motion prediction enhanced exoskeleton transparency, the force magnitude in this study was small, so users could hardly discern the improvement. Future research will investigate motion prediction for exoskeleton control in the context of load-handling assistance.

Keywords: Force and Tactile Sensing, Physical Human-Robot Interaction, Robot Safety, Additive Manufacturing
Abstract: Although many soft robotic skins have been introduced, their use has been hindered due to practical limitations such as difficulties in manufacturing, poor accessibility, and cost inefficiency. To solve this, we present a low-cost, easy-to-build soft robotic skin utilizing air-pressure sensors and 3D-printed pads. In our approach, we utilized digital fabrication and ROS to facilitate the creation and use of the robotic skin. The skin pad was fabricated by printing thermoplastic urethane (TPU) and post-processed with an organic solvent to secure air-tightness. Each pad consists of a TPU shell and infill, so the internal air pressure changes in response to tactile stimuli such as force and vibration. The internal pressure is measured and processed by a microcontroller and transmitted to the PC via a serial bus. We conducted experiments to investigate the characteristics of the skin pads, and the results showed that the developed robotic skins are capable of perceiving interaction force and dynamic stimuli. Finally, we developed the dedicated soft robotic skins for our custom robot designed in-house, and demonstrated safe and intuitive physical human-robot interaction.

Keywords: Telerobotics and Teleoperation, Mobile Manipulation, Whole-Body Motion Planning and Control
Abstract: Remote-controlled humanoid robots can revolutionize manufacturing, construction, and healthcare industries by performing complex or strenuous manual tasks traditionally done by humans. We refer to these behaviors as Dynamic Loco-Manipulation (DLM). To successfully complete these tasks, humans control the position of their bodies and contact forces at their hands. To enable similar whole-body control in humanoids, we introduce loco-manipulation retargeting strategies with switched position and force control modes in a bilateral teleoperation framework. Our proposed locomotion mappings use the pitch and yaw of the operator's torso to control robot position or acceleration. The manipulation retargeting maps the operator's arm movements to the robot's arms for joint-position or impedance control of the end-effector. A Human-Machine Interface captures the teleoperator's motion and provides haptic feedback to their torso, enhancing their awareness of the robot's interactions with the environment. In this paper, we demonstrate two forms of DLM. First, we show the robot slotting heavy boxes (5-10.5 kg), weighing up to 83% of the robot's weight, into desired positions. Second, we show human-robot collaboration for carrying an object, where the robot and teleoperator take on leader and follower roles.

Keywords: Haptics and Haptic Interfaces, Human-Centered Robotics, Social HRI
Abstract: Using non-verbal cues such as breathing and heartbeat signals as media for Human-Robot Interaction (HRI) enables subtle exchanges between humans and robots beyond explicit verbal or visual cues. This study describes the development of a novel pneumatic haptic interface that replicates human breathing and heartbeat animations for a humanoid robot and investigates its application for enhancing non-verbal HRI. The experimental results validate the effectiveness of this haptic device and highlight the significance of non-verbal cues in conveying emotional states. Introducing the haptic interface as a means for delivering involuntary, non-verbal emotional signals significantly enhances human-robot interaction compared to using gestures alone.

Keywords: Telerobotics and Teleoperation, Compliance and Impedance Control, RGB-D Perception
Abstract: This paper presents a method for semi-autonomous teleimpedance where the control is shared between the human operator and the robot. The human commands the position of the teleoperated robotic arm end-effector while the robot autonomously adjusts the impedance depending on the object with which the end-effector interacts. We developed a vision system that calculates the appropriate robot stiffness based on the detected object geometry and material and object's relation to the environment. This system uses an RGB-D camera near the robot's end-effector to capture different perspectives of the scene. To validate the proposed method, we conducted experiments on a teleoperation system where a Force Dimension Sigma7 haptic device was used to operate a KUKA LBR iiwa robotics arm. At the same time, the Intel RealSense D455 depth camera provided the visual input. We examined two practical tasks: engaging with bolts on a plate and polishing a stripe.

Keywords: Human and Humanoid Motion Analysis and Synthesis, Modeling and Simulating Humans, Humanoid and Bipedal Locomotion
Abstract: Gait analysis is essential in many scientific fields; to study it marker-based or markerless motion capture (MoCap) techniques are used. The latter have significantly benefited from the recent rise of research in deep learning (DL) and its applications on human mesh generation. However, insufficient and suboptimal camera viewpoint selections often lead to low-grade human mesh geometries. This paper presents an approach to consistently obtain accurate human meshes using DL-based avatar reconstruction algorithms (ARAs). Our framework provides a systematic approach, utilizing a simulated environment to inform decisions on the number of cameras and their spatial configuration to achieve optimal reconstruction results. These results are enhanced through mesh evaluation, mesh alignment, and surface reconstruction to remove poorly formed geometries and artifacts. Additionally, we present a gait analysis tool, tested in simulation and reality, that detects gait phase changes, extracts the significant human body joint angles and recreates the animation of the gait cycle in 3D space. The proposed approach is open-source, adjustable, and applicable to various research contexts where gait analysis is essential.

Keywords: Physical Human-Robot Interaction, Human-Centered Robotics, Task and Motion Planning
Abstract: Work-related musculoskeletal disorders (WMSDs) are among the most common injuries associated with industrial tasks. Repetitive tasks are a major WMSDs risk factor, because they load the same human joints over and over again. Collaborative robots can be used to induce movement variability in highly repetitive co-manipulation tasks by hanging the position of the co-manipulated object through time, thereby distributing the physical load over different body parts and reducing fatigue accumulation. This is even more beneficial when long-term consequences of the robot actions are considered. However, selecting the optimal action within the continuous robot workspace is not compatible with time constraints imposed by online planning in highly repetitive tasks, especially when the planning horizon increases. In this work we therefore propose an approach to automatically extract a set of actions from the continuous workspace, that combines two properties: planning speed (i.e. reduced number of actions in the set), and ability to induce a variety of fatigue distributions over the different human joints. The proposed approach combines a digital human simulation to estimate the fatigue induced by possible actions, with a repeated short-term planning (greedy-based selection approach) phase that explores the fatigue space and simultaneously identifies optimal actions from a large space for each visited state. By retaining actions used in the short-term planning, this process allows to extract a subset of relevant actions. We evaluate our approach in a simulated co-manipulation scenario, and show that the resulting action set robustly outperforms action sets extracted with benchmark methods, both in terms of planning time and human fatigue mitigation.

Keywords: Data Sets for Robotic Vision, Object Detection, Segmentation and Categorization, Perception for Grasping and Manipulation
Abstract: Despite the recent progress on 6D object pose estimation methods for robotic grasping, a substantial performance gap persists between the capabilities of these methods on existing datasets and their efficacy in real-world grasping and mobile manipulation tasks, particularly when robots rely solely on their monocular egocentric field of view (FOV). Existing real-world datasets primarily focus on table-top grasping scenarios, where a robot arm is placed in a fixed position and the objects are centralized within the FOV of fixed external camera(s). Assessing performance on such datasets may not accurately reflect the challenges encountered in everyday grasping and mobile manipulation tasks within kitchen environments such as retrieving objects from higher shelves, sinks, dishwashers, ovens, refrigerators, or microwaves. To address this gap, we present KITchen, a novel benchmark designed specifically for estimating the 6D poses of objects located in diverse positions within kitchen settings. For this purpose, we recorded a comprehensive dataset comprising around 205k real-world RGBD images for 111 kitchen objects captured in two distinct kitchens, utilizing a humanoid robot with its egocentric perspectives. Subsequently, we developed a semi-automated annotation pipeline, to streamline the labeling process of such datasets, resulting in the generation of 2D object labels, 2D object segmentation masks, and 6D object poses with minimal human effort. The benchmark, the dataset, and the annotation pipeline are publicly available upon acceptance at url{https://kitchen-dataset.github.io/KITchen}.

Keywords: Human-Robot Collaboration, Cognitive Control Architectures
Abstract: Large Language Models (LLMs) have been recently used in robot applications for grounding LLM common-sense reasoning with the robot's perception and physical abilities. In humanoid robots, memory also plays a critical role in fostering real-world embodiment and facilitating long-term interactive capabilities, especially in multi-task setups where the robot must remember previous task states, environment states, and executed actions. In this paper, we address incorporating memory processes with LLMs for generating cross-task robot actions, while the robot effectively switches between tasks. Our proposed dual-layered architecture features two LLMs, utilizing their complementary skills of reasoning and following instructions, combined with a memory model inspired by human cognition. Our results show a significant improvement in performance over a baseline of five robotic tasks, demonstrating the potential of integrating memory with LLMs for combining the robot's action and perception for adaptive task execution.

Keywords: Multifingered Hands, Incremental Learning
Abstract: We propose a model of Tactile Memory that integrates sequential touches to identify the shape and pose of an object. The memory also controls the next tactile action to approximately optimize its information gain at each step. The information fusion and determination of the next tactile action is achieved with the aid of an ensemble of hypotheses, each of which represents a possible shape and pose of the object. We assume a first touch event has already occurred, and focus on the process of shape and pose identification. In order to minimize the number of tactile actions required, the proposed method combines: 1) Tactile Memory: a record of the tactile event history from multiple fingers of a Shadow Hand along with the contact traces and hand location 2) the hypothesis ensemble as a distributed representation of the remaining shape and pose uncertainty and 3) Explorative Tactile Actions: a set of tactile event-specific heuristics that create proposals for hand location based on the tactile feedback. We analyze our approach in simulation and quantify its improvement of exploration over a baseline algorithm that does not use the contact traces. Also we compare Explorative Tactile Actions with a baseline that uses random hand locations. We also demonstrate our algorithm on a robot with Shadow Hand to show that we can estimate the shape and pose of an object in about ten tactile actions.

Keywords: In-Hand Manipulation, Multifingered Hands, Grasping
Abstract: In-hand tool manipulation is an operation that not only manipulates a tool within the hand (i.e., in-hand manipulation) but also achieves a grasp suitable for a task after the manipulation. This study aims to achieve an in-hand tool manipulation skill through deep reinforcement learning. The difficulty of learning the skill arises because this manipulation requires (A) exploring long-term contact-state changes to achieve the desired grasp and (B) highly-varied motions depending on the contact-state transition. (A) leads to a sparsity of a reward on a successful grasp, and (B) requires an RL agent to explore widely within the state-action space to learn highly-varied actions, leading to sample inefficiency. To address these issues, this study proposes Action Primitives based on Contact-state Transition (APriCoT). APriCoT decomposes the manipulation into short-term action primitives by describing the operation as a contact-state transition based on three action representations (detach, crossover, attach). In each action primitive, fingers are required to perform short-term and similar actions. By training a policy for each primitive, we can mitigate the issues from (A) and (B). This study focuses on a fundamental operation as an example of in-hand tool manipulation: rotating an elongated object grasped with a precision grasp by half a turn to achieve the initial grasp. Experimental results demonstrated that ours succeeded in both the rotation and the achievement of the desired grasp, unlike existing studies. Additionally, it was found that the policy was robust to changes in object shape.

Keywords: Multifingered Hands, Force and Tactile Sensing, Reinforcement Learning
Abstract: Enabling multi-fingered robots to grasp and manipulate objects with human-like dexterity is especially challenging during the dynamic, continuous hand-object interactions. Closed-loop feedback control is essential for dexterous hands to dynamically finetune hand poses when performing precise functional grasps. This work proposes an adaptive motion planning method based on deep reinforcement learning to adjust grasping poses according to real-time feedback from joint torques from pre-grasp to goal grasp. We find the multijoint torques of the dexterous hand can sense object positions through contacts and collisions, enabling real-time adjustment of grasps to generate varying grasping trajectories for objects in different positions. In our experiments, the performance gap with and without force feedback reveals the important role of force feedback in adaptive manipulation. Our approach, utilizing force feedback, preliminarily exhibits human-like flexibility, adaptability, and precision.

Keywords: Art and Entertainment Robotics, Perception for Grasping and Manipulation, Deep Learning in Grasping and Manipulation
Abstract: Humans are capable of continuously manipulating a wide variety of deformable objects into complex shapes. This is made possible by our intuitive understanding of material properties and mechanics of the object, for reasoning about object states even when visual perception is occluded. These capabilities allow us to perform diverse tasks ranging from cooking with dough to expressing ourselves with pottery-making. However, developing robot systems to robustly perform similar tasks remains challenging, as current methods struggle to effectively model volumetric deformable objects and reason about the complex behavior they typically exhibit. To study the robot systems and algorithms capable of deforming volumetric objects, we introduce a novel robot task of continuously deforming clay on a pottery wheel. We propose a pipeline for perception and pottery skill-learning, called RoPotter, wherein we demonstrate that structural priors specific to the task of pottery-making can be exploited to simplify the pottery skill-learning process. Namely, we can project the cross-section of the clay to a plane to represent the state of the clay, reducing dimensionality. We also demonstrate a mesh-based method of occluded clay state recovery, toward robot agents capable of continuously deforming clay. Our experiments show that by using the reduced representation with structural priors based on the deformation behaviors of the clay, RoPotter can perform the long-horizon pottery task with 44.4% lower final shape error compared to the state-of-the-art baselines. Supplemental materials, experiment data, and visualizations are available at https://robot-pottery.github.io.

Keywords: Wearable Robotics, Tendon/Wire Mechanism, Mechanism Design
Abstract: Numerous wearable robots have been developed to meet the demands of physical assistance and entertainment. These wearable robots range from body-enhancing types that assist human arms and legs to body-extending types that have extra arms. This study focuses specifically on wearable robots of the latter category, aimed at bodily extension. However, they have not yet achieved the level of powerfulness and reachability equivalent to that of human limbs, limiting their application to entertainment and manipulation tasks involving lightweight objects. Therefore, in this study, we develop an body-extending wearable robot, Vlimb, which has enough powerfulness to lift a human and can perform manipulation. Leveraging the advantages of tendon-driven mechanisms, Vlimb incorporates a wire routing mechanism capable of accommodating both delicate manipulations and robust lifting tasks. Moreover, by introducing a passive ring structure to overcome the limited reachability inherent in tendon-driven mechanisms, Vlimb achieves both the powerfulness and reachability comparable to that of humans. This paper outlines the design methodology of Vlimb, conducts preliminary manipulation and lifting tasks, and verifies its effectiveness.

Keywords: Soft Sensors and Actuators, Wearable Robotics, Performance Evaluation and Benchmarking
Abstract: Research and development on large-area robotic skins has seen considerable progress recently. Providing high resolution, non-invasive imaging of conductivity changes in material, electrical impedance tomography (EIT) has been pivotal in developing robotic skins that can detect strain, pressure, and other stimuli. Despite these advances, EIT-based robotic skins have been limited by bulky and complex electronics. In this paper, we introduce DecisioNova, a compact battery-powered development board for advanced sensor data acquisition and processing. DecisioNova enables fast and accurate EIT measurements, making EIT-based robotic skins viable for use in robots and prosthetic devices that demand compact and low-energy electronics. To examine DecisioNova’s performance, we fabricated a hydrogel robotic skin and conducted both conventional and data-driven EIT for pressure mapping. We evaluated the system’s performance in contact localization and force quantification. The system achieved a temporal resolution of 0.25 seconds, with data-driven EIT yielding a contact localization error of 0.57 mm over a 70×70 mm sensing area. Future work will focus on practical applications of DecisioNova as a wearable device for smart prostheses. Source files are available on GitHub: https://github.com/Decisionics/DecisioNova

Keywords: Mapping, Visual-Inertial SLAM, Humanoid Robot Systems
Abstract: We present a novel algorithm for real-time planar semantic mapping tailored for humanoid robots navigating complex terrains such as staircases. Our method is adaptable to any odometry input and leverages GPU-accelerated processes for planar extraction, enabling the rapid generation of globally consistent semantic maps. We utilize an anisotropic diffusion filter on depth images to effectively minimize noise from gradient jumps while preserving essential edge details, enhancing normal vector images’ accuracy and smoothness. Both the anisotropic diffusion and the RANSAC-based plane extraction processes are optimized for parallel processing on GPUs, significantly enhancing computational efficiency. Our approach achieves real-time performance, processing single frames at rates exceeding 30 Hz, which facilitates detailed plane extraction and map management swiftly and efficiently. Extensive testing underscores the algorithm’s capabilities in real-time scenarios and demonstrates its practical application in humanoid robot gait planning, significantly improving its ability to navigate dynamic environments.

Keywords: Robotics and Automation in Life Sciences, Human Detection and Tracking
Abstract: This study investigated the possibility of using machine learning to estimate 3D lower-limb joint kinematics during a rehabilitation squat exercise from force plate data, that can be collected very simply outside of a laboratory and does not pose privacy issues. The proposed approach is based on a bidirectional-Long-Short-Term-Memory (Bi-LSTM) associated to a Multi-Layer-Perceptron (MLP) model. The use of MLP allows fast training and evaluation time. The model was trained and validated on nineteen healthy young volunteers using a stereophotogrammetric motion capture system to collect ground truth data. Volunteers performed squats in normal conditions and using an ankle brace to simulate pathological motion. Also additional loads were added onto lower limbs segments to study the influence atypical mass distribution. The root mean square differences between the estimated joint angles and those reconstructed with the stereophotogrammetric system were lower than 6deg with correlation coefficients higher than 0.9 in average. Furthermore, the inference time of the proposed approach was as low as 12us paving the way of future reliable real-time measurement tools.

Keywords: Legged Robots, Whole-Body Motion Planning and Control, Optimization and Optimal Control
Abstract: Thanks to recent advancements in accelerating nonlinear model predictive control (NMPC), it is now feasible to deploy whole-body NMPC at real-time rates for humanoid robots. However, enforcing inequality constraints in real time for such high-dimensional systems remains challenging due to the need for additional iterations. This paper presents an implementation of whole-body NMPC for legged robots that provides low-accuracy solutions to NMPC with general equality and inequality constraints. Instead of aiming for highly accurate optimal solutions, we leverage the alternating direction method of multipliers to rapidly provide low-accuracy solutions to quadratic programming subproblems. Our extensive simulation results indicate that real robots often cannot benefit from highly accurate solutions due to dynamics discretization errors, inertial modeling errors and delays. We incorporate control barrier functions (CBFs) at the initial timestep of the NMPC for the self-collision constraints, resulting in up to a 26-fold reduction in the number of selfcollisions without adding computational burden. The controller is reliably deployed on hardware at 90 Hz for a problem involving 32 timesteps, 2004 variables, and 3768 constraints. The NMPC delivers sufficiently accurate solutions, enabling the MIT Humanoid to plan complex crossed-leg and arm motions that enhance stability when walking and recovering from significant disturbances.

Keywords: Telerobotics and Teleoperation, Networked Robots, Autonomous Agents
Abstract: In this paper, an autonomy allocation approach is proposed for bilateral teleoperation systems to improve user performance under suboptimal communication networks. To achieve this, time-varying communication quality metrics such as delay and jitter are continuously monitored on the teleoperated side, and the autonomy level is dynamically adjusted based on the communication network quality. An autonomous agent is also deployed on the teleoperated side, leveraging pre-existing task knowledge for shared autonomy. Additionally, a time-domain passivity approach is employed to maintain communication channel passivity, mitigating the impact of adverse network behavior on task performance. The proposed approach is validated through extensive experiments and user studies, and the result shows our approach significantly improved the performance of the subjects (p < 0.01).

Keywords: Physical Human-Robot Interaction, Touch in HRI, Soft Robot Materials and Design
Abstract: Recent developments of vision-based tactile sensors primarily focus on small-sized and fixed-structured robot bodies, while the application of this sensing technique for whole-body, shape-changing robots remains a challenge. To address this problem, this study introduces a soft, changeable structured robot with integrated whole-body tactile sensing. The proposed robot system features a compact mechanism for adjusting the robot's height, and a camera system for vision-based tactile sensing, which are collectively covered by a marker-embedded soft robot skin. Additionally, a data-driven tactile sensing approach is developed to enable the robot to detect contacts across the entire robot body with varying heights. The preliminary results and showcases demonstrate the promise of the proposed system in facilitating robot operations in unstructured environments and enhancing human-robot interaction scenarios.

Keywords: Prosthetics and Exoskeletons, Underactuated Robots, Wearable Robots, Series Elastic Actuator (SEA)
Abstract: Robotic lower-limb prostheses have the power to revolutionize mobility by enhancing gait efficiency and facilitating movement. While several design approaches have been explored to create lightweight and energy-efficient devices, the potential of underactuation remains largely untapped in lower-limb prosthetics. Taking inspiration from the natural harmony of walking, we have developed an innovative active transfemoral prosthesis. By incorporating underactuation, our design uses a single power actuator placed near the knee joint and connected to a differential mechanism to drive both the knee and ankle joints. We conducted comprehensive benchtop tests and evaluated the prosthesis with three individuals who have above-knee amputations, assessing its performance in walking, stair climbing, and transitions between sitting and standing. Our evaluation focused on gathering position and torque data recorded from sensors integrated into the prosthesis and comparing these measurements to biomechanical data of able-bodied locomotion. Our findings highlight the promise of underactuation in advancing lower-limb prosthetics and demonstrate the feasibility of our knee-ankle design

Keywords: Grasping, Multifingered Hands, Deep Learning in Grasping and Manipulation
Abstract: We introduce DexGanGrasp, a dexterous grasping synthesis method that generates and evaluates grasps with single view in real time. DexGanGrasp comprises a Conditional Gen- erative Adversarial Networks (cGANs)-based DexGenerator to generate dexterous grasps and a discriminator-like DexEvalautor to assess the stability of these grasps. Extensive simulation and real-world expriments showcase the effectiveness of our proposed method, outperforming the baseline FFHNet with an 18.57% higher success rate in real-world evaluation. We further extend DexGanGrasp~to DexAfford-Prompt, an open-vocabulary affordance grounding pipeline for dexterous grasping leveraging Multimodal Large Language Models (MLLMs) and Vision Language Models (VLMs), to achieve task-oriented grasping with successful real-world deployments. For the code and data, visit our website (https://david-s-martinez.github.io/DexGANGrasp.io)

Keywords: Deep Learning in Grasping and Manipulation, Autonomous Agents, Reinforcement Learning
Abstract: Learning skills that interact with objects is of major importance for robotic manipulation. These skills can indeed serve as an efficient prior to solving various manipulation tasks. We propose a novel Skill Learning approach that discovers composable behaviors by solving a large and diverse number of autonomously generated tasks. Our method learns skills allowing the robot to consistently and robustly interact with objects in its environment. The discovered behaviors are embedded in primitives which can be composed with Hierarchical Reinforcement Learning to solve unseen manipulation tasks. In particular, we leverage Asymmetric Self-Play to discover behaviors and Multiplicative Compositional Policies to embed them. We compare our method to Skill Learning baselines and find that our skills are more interactive. Furthermore, the learned skills can be used to solve a set of unseen manipulation tasks, in simulation as well as on a real robotic platform.

Keywords: AI-Based Methods, Recognition, Mobile Manipulation
Abstract: State recognition of the environment and objects, such as the open/closed state of doors and the on/off of lights, is indispensable for robots that perform daily life support and security tasks. Until now, state recognition methods have been based on training neural networks from manual annotations, preparing special sensors for the recognition, or manually programming to extract features from point clouds or raw images. In contrast, we propose a robotic state recognition method using a pre-trained vision-language model, which is capable of Image-to-Text Retrieval (ITR) tasks. We prepare several kinds of language prompts in advance, calculate the similarity between these prompts and the current image by ITR, and perform state recognition. By applying the optimal weighting to each prompt using black-box optimization, state recognition can be performed with higher accuracy. Experiments show that this theory enables a variety of state recognitions by simply preparing multiple prompts without retraining neural networks or manual programming. In addition, since only prompts and their weights need to be prepared for each recognizer, there is no need to prepare multiple models, which facilitates resource management. It is possible to recognize the open/closed state of transparent doors, the state of whether water is running or not from a faucet, and even the qualitative state of whether a kitchen is clean or not, which have been challenging so far, through language.

Keywords: Force Control, Machine Learning for Robot Control, Model Learning for Control
Abstract: In this study, we propose learning-based force control of twisted string actuators (TSAs) using a neural network-based inverse model. A learning-based force controller is designed using the input and output data of TSAs without a dynamic model of TSAs. Furthermore, the neural network-based inverse model is utilized to reduce model errors and handle nonlinearities between the inputs and outputs of the TSAs. The trained neural network-based inverse model is directly implemented as a force controller for the TSAs. Additionally, we propose data collection methods utilizing three types of inputs to improve the performance of the proposed controller. To verify the improved performance resulting from the proposed data collection methods, we compared the performance of the learning-based force controller for each dataset in the TSA hardware. We then selected the dataset with the best performance among the proposed inputs through experiments. Additionally, to verify the performance of the learning-based force controller, a reference force tracking experiment was performed and compared with a proportional-integral-derivative (PID) controller and feedback linearization. The learning-based force controller utilizing the selected input-based dataset demonstrated higher force tracking performance than the other controllers. Consequently, the TSA’s learning-based force control demonstrates robust force control performance.

Keywords: Dexterous Manipulation, Manipulation Planning, Multifingered Hands
Abstract: In-hand manipulation of tools using dexterous hands in real-world is an underexplored problem in the literature. In addition to more complex geometry and larger size of the tools compared to more commonly used objects like cubes or cylinders, task oriented in-hand tool manipulation involves many sub-tasks to be performed sequentially. This may involve reaching to the tool, picking it up, reorienting it in hand with or without regrasping to reach to a desired final grasp appropriate for the tool usage, and carrying the tool to the desired pose. Research on long-horizon manipulation using dexterous hands is rather limited and the existing work focus on learning the individual sub-tasks using a method like reinforcement learnNing (RL) and combine the policies for different subtasks to perform a long horizon task. However, in general a single method may not be the best for all the sub tasks, and this can be more pronounced when dealing with multi-fingered hands manipulating objects with complex geometry like tools. In this paper, we investigate the use of a mixed-method approach to solve for the long-horizon task of tool usage and we use imitation learning, reinforcement learning and model based control. We also discuss a new RL-based teacher-student framework that combines real world data into offline training. We show that our proposed approach for each subtask outperforms the commonly adopted reinforcement learning approach across different subtasks and in performing the long horizon task in simulation. Finally we show the successful transferability to real world.

Keywords: Human-Robot Collaboration, Grasping, Physically Assistive Devices
Abstract: This study introduces i-GRIP, an innovative movement goal estimator designed to facilitate the control of assistive devices for grasping tasks in individuals with upper-limb impairments. The algorithm operates within a collaborative control paradigm, eliminating the need for specific user actions apart from naturally moving their hand toward a desired object. i-GRIP analyzes the hand's movement in an object-populated scene to determine its target and select an appropriate grip. In an experimental study involving 11 healthy participants, i-GRIP exhibited promising estimation performances (success rates of 89.9% for target identification and 94.8% for grip selection) and responsiveness (mean delays of 0.53s for target identification and 0.39s for grip selection), showing its potential to facilitate the daily use of grasping assistive devices for individuals with upper-limb impairments.

Keywords: Calibration and Identification, Kinematics, Mobile Manipulation
Abstract: This paper aims to address two intrinsic phenomena encountered in mobile manipulator robots, but often neglected, with the objective of improving the overall accuracy of end-effector pose estimation. Firstly, after performing state-of-the-art geometric calibration of the arm, we propose two identifiable mathematical models to account for non-geometric effects: a model for the mobile base suspension system and a model of non-linear inaccuracies of joint angles estimates. The latter is due to backlash and misaligned encoders mounting. Then, the proposed models were experimentally validated on the mobile manipulator TIAGo using a stereophotogrammetric system. Overall, the end-effector pose accuracy was improved by 60% when compared to the nominal manufacturer model, with root mean square errors (RMSE) of 5.7mm and 2.7deg for positional and orientational errors, respectively.

Keywords: In-Hand Manipulation, Reinforcement Learning, Multifingered Hands
Abstract: In-hand manipulation with multi-fingered hands is a challenging problem that recently became feasible with the advent of deep reinforcement learning methods. While most contributions to the task brought improvements in robustness and generalization, this paper addresses the critical performance measure of the speed at which an in-hand manipulation can be performed. We present reinforcement learning policies that can perform in-hand reorientation significantly faster than previous approaches for the complex setting of goal-conditioned reorientation in SO(3) with permanent force closure and tactile feedback only (i.e., using the hand's torque and position sensors). Moreover, we show how policies can be trained to be speed-adjustable, allowing for setting the average orientation speed of the manipulated object during deployment. To this end, we present suitable and minimalistic reinforcement learning objectives for time-optimal and speed-adjustable in-hand manipulation, as well as an analysis based on extensive experiments in simulation. We also demonstrate the zero-shot transfer of the learned policies to the real DLR-Hand II with a wide range of target speeds and the fastest dextrous in-hand manipulation without visual inputs.

Keywords: Biologically-Inspired Robots, Tendon/Wire Mechanism, Soft Robot Materials and Design
Abstract: For robots to become more versatile and expand their areas of application, their bodies need to be suitable for contact with the environment. When the human body comes into contact with the environment, it is possible for it to continue to move even if the positional relationship between muscles or the shape of the muscles changes. We have already focused on the effect of geometric deformation of muscles and proposed a drive system called wire-wound Muscle-Tendon Complex (ww-MTC), an extension of the wire drive system. Our previous study using a robot with a two-dimensional configuration demonstrated several advantages: reduced wire loosening, interference, and wear; improved robustness during environmental contact; and a muscular appearance. However, this design had some problems, such as excessive muscle expansion that hindered inter-muscle movement, and confinement to planar motion. In this study, we develop the ww-MTC into a three-dimensional shape. We present a fundamental construction method for a muscle exterior that expands gently and can be contacted over its entire surface. We also apply the three-dimensional ww-MTC to a 2-axis 3-muscle robot, and confirm that the robot can continue to move while adapting to its environment.

Keywords: Multi-Contact Whole-Body Motion Planning and Control, Whole-Body Motion Planning and Control, Humanoid Robot Systems
Abstract: Many humanoid and multi-legged robots are controlled in positions rather than in torques, which prevents direct control of contact forces, and hampers their ability to create multiple contacts to enhance their balance, such as placing a hand on a wall or a handrail. This paper introduces the SEIKO (Sequential Equilibrium Inverse Kinematic Optimization) pipeline, and proposes a unified formulation that exploits an explicit model of flexibility to indirectly control contact forces on traditional position-controlled robots. SEIKO formulates whole-body retargeting from Cartesian commands and admittance control using two quadratic programs solved in real time. Our pipeline is validated with experiments on the real, full-scale humanoid robot Talos in various multi-contact scenarios, including pushing tasks, far-reaching tasks, stair climbing, and stepping on sloped surfaces. Code and videos are available at https://hucebot.github.io/seiko_controller_website/.

Keywords: Humanoid and Bipedal Locomotion, Legged Robots, Optimization and Optimal Control
Abstract: Developing sophisticated control architectures has endowed robots, particularly humanoid robots, with numerous capabilities. However, tuning these architectures remains a challenging and time-consuming task that requires expert intervention. In this work, we propose a methodology to automatically tune the gains of all layers of a hierarchical control architecture for walking humanoids. We tested our methodology by employing different gradient-free optimization methods: Genetic Algorithm (GA), Covariance Matrix Adaptation Evolution Strategy (CMA-ES), Evolution Strategy (ES), and Differential Evolution (DE). We validated the parameter found both in simulation and on the real ergoCub humanoid robot. Our results show that GA achieves the fastest convergence (10 × 10^3 function evaluations vs 25 × 10^3 needed by the other algorithms) and 100% success rate in completing the task both in simulation and when transferred on the real robotic platform. These findings highlight the potential of our proposed method to automate the tuning process, reducing the need for manual intervention.

Keywords: Whole-Body Motion Planning and Control, Optimization and Optimal Control, Simulation and Animation
Abstract: This paper proposes a method that realizes dynamic dancing motion of humanoid robots based on hybrid model predictive control. The proposed control method runs two types of model predictive controllers with different fidelity and time scale in parallel; one performs long-horizon prediction by making use of a closed-form solution of the centroidal dynamics, and the other performs short-horizon prediction based on the whole-body dynamics. A reference key-pose sequence of more than 100 key frames including stepping and fast upper-body movement was edited using Choreonoid and input to the controller. In closed-loop simulation of a torque-controlled 32-DoF humanoid robot, the controller was able to track the reference sequence by attenuating large angular momentum.

Keywords: Legged Robots, Humanoid and Bipedal Locomotion
Abstract: Legged robotic systems can play an important role in real-world applications due to their superior load-bearing capabilities, enhanced autonomy, and effective navigation on uneven terrain. They offer an optimal trade-off between mobility and payload capacity, excelling in diverse environments while maintaining efficiency in transporting heavy loads. However, planning and optimizing gaits and gait sequences for these robots presents significant challenges due to the complexity of their dynamic motion and the numerous optimization variables involved. Traditional trajectory optimization methods address these challenges by formulating the problem as an optimization task, aiming to minimize cost functions, and to automatically discover contact sequences. Despite their structured approach, optimization-based methods face substantial difficulties, particularly because such formulations result in highly nonlinear and difficult to solve problems. To address these limitations, we propose CrEGOpt, a bi-level optimization method that combines traditional trajectory optimization with a black-box optimization scheme. CrEGOpt at the higher level employs the Mixed Distribution Cross-Entropy Method to optimize both the gait sequence and the phase durations, thus simplifying the lower level trajectory optimization problem. This approach allows for fast solutions of complex gait optimization problems. Extensive evaluation in simulated environments demonstrates that CrEGOpt can find solutions for biped, quadruped, and hexapod robots in under 10 seconds. This novel bi-level optimization scheme offers a promising direction for future research in automatic contact scheduling.

Keywords: Telerobotics and Teleoperation, Virtual Reality and Interfaces, Human-Robot Teaming
Abstract: High-quality demonstrations are necessary when learning complex and challenging manipulation tasks. In this work, we introduce an approach to puppeteer a robot by controlling a virtual robot in an augmented reality setting. Our system allows for retaining the advantages of being intuitive from a physical leader-follower side while avoiding the unnecessary use of expensive physical setup. In addition, the user is endowed with additional information using augmented reality. We validate our system with a pilot study n=10 on a block stacking and rice scooping tasks where the majority rates the system favorably. Oculus App and corresponding ROS code are available on the project website https://ar-puppeteer.github.io/.

Keywords: Emotional Robotics, Human-Robot Collaboration
Abstract: Abstract—The humanoid design of robots can maximize the imitation of real human work, facilitate integrated control with other devices, and offer good safety and human-machine interaction. These attributes have led to their widespread application in various fields. The most critical component of the robot’s structure, the arm, can be categorized into three types based on stiffness characteristics: rigid robotic arms, flexible robotic arms, and rigid-flexible coupled robotic arms. Rigid robotic arms are characterized by high load capacity, high precision, high repeatability, and high speed. Their development is relatively mature across different fields. However, due to their low degrees of freedom, poor environmental adaptability, and unsafe human-machine interaction, they are difficult to apply in complex, unstructured scenarios. Currently, the design and manufacturing of flexible robotic arms have become a research hotspot in many engineering disciplines. Compared to traditional bulky rigid robotic arms, flexible robotic arms offer advantages such as smaller size, lower energy consumption, and reduced cost. However, their lower stiffness can lead to structural fatigue, and their precision is affected by elastic deformation. During motion or when subjected to external disturbances, flexible robotic arms are prone to vibration issues, which can reduce control accuracy and compromise the stability of the system. This paper focuses on the scenario of human-robot hugging and designs a safe and reliable rigid-flexible coupled dual-arm robot that can provide psychological comfort. The rigid-flexible coupled robotic arm primarily consists of a rigid skeleton, variable stiffness joints, and flexible muscles. Driven by pneumatic actuators, it can achieve sensing and human-machine interaction. The rigid-flexible coupled design ensures both the end-effector output force and safety. The research content mainly includes the structural design of the dual-arm robot, model establishment, simulation and control scheme of the hugging motion, and human-robot interaction hugging experiments.

Keywords: Perception for Grasping and Manipulation, Force and Tactile Sensing, Bioinspired Robot Learning
Abstract: Humans perceive objects and environments actively and dynamically using tactile sensing facilitated by the skin. However, due to the gap in sensing capabilities between electronic skin and human skin, it remains a challenge for robots to achieve intricate tactile perception. For example, the task of liquid properties estimation within containers, demands sensing and comprehension of complex dynamic tactile signals during contact. This paper introduces a novel tactile fingertip inspired by human skin enabling both static tactile sensing facilitated by a layer of porous piezoresistive elastomer and dynamic tactile sensing enabled by condenser microphones. A data-driven methodology is employed to analyze liquids through multimodal signals devoid of physical modelling. The proposed robotic system engages in shaking various bottles filled with different volumes of water, allowing the tactile fingertip to detect changes in grasping force caused by shaking and small vibrations caused by the collision of the container with liquid. Five machine learning models trained on these tactile signals are analyzed and compared. Experimental results demonstrate the proficiency of the tactile system in distinguishing liquid fill percentages across bottles of different shapes and capacities with a remarkable accuracy of 98%. This paper holds promise for advancing the embodied intelligence of robots, enhancing their ability to perceive mixed tactile signals dynamically, thereby facilitating tasks such as liquid estimation and other intricate tactile operations in environments like kitchens and hospitals.

Keywords: Human-Robot Collaboration, Gesture, Posture and Facial Expressions
Abstract: This paper investigates the effectiveness and efficiency of incorporating pointing gestures as well as hand-speech synchronization policies into instruction delivery, as would be used in an industrial case with a cobot. Through brick assembly tasks, our study explores the integration of pointing gestures into human-robot interaction, extending prior research on verbal instruction efficacy. Results show that pointing gestures significantly reduce errors compared to verbal instructions alone, especially for complex tasks. However, this improvement comes at the cost of increased task completion time. We also show that depending on this synchronization, the user might delay its action until all information is presented instead of exploiting the information as it arrived. This study emphasizes the potential of pointing gestures and hand-speech synchronization in improving human-robot interaction and suggests further research for optimal integration.

Keywords: Humanoid and Bipedal Locomotion, Legged Robots, Reinforcement Learning
Abstract: This paper addresses the challenge of terrain-adaptive dynamic locomotion in humanoid robots, traditionally tackled by optimization-based methods or reinforcement learning (RL). Optimization-based methods, such as model-predictive control, excel in finding optimal reaction forces and achieving agile locomotion, but struggle with the nonlinear hybrid dynamics of legged systems and the real-time computation of step location, timing, and reaction forces. Conversely, RL-based methods show promise in navigating dynamic and rough terrains but are limited by their extensive data requirements. We introduce a novel locomotion architecture that integrates a neural network policy, trained through RL in simplified environments, with a state-of-the-art motion controller combining model-predictive control (MPC) and whole-body impulse control (WBIC). The policy efficiently learns high-level locomotion strategies, such as gait selection and step positioning, without the need for full dynamics simulations. This control architecture enables humanoid robots to dynamically navigate discrete terrains, making strategic locomotion decisions (e.g., walking, jumping, and leaping) based on ground height maps. Our results demonstrate that this integrated control architecture achieves dynamic locomotion with significantly fewer training samples than conventional RL-based methods and can be transferred to different humanoid platforms without additional training. The control architecture has been extensively tested in dynamic simulations, accomplishing terrain height-based dynamic locomotion for three different robots.

Keywords: Compliant Joints and Mechanisms, Actuation and Joint Mechanisms, Robust/Adaptive Control
Abstract: Series Elastic Actuator (SEA) has widely been used in various robotic applications due to its ability to provide safe and accurate force. However, conventionally, securing the high torque sensitivity of the SEA of the spring is challenging due to the limitation of natural frequency and torque capacity, making it less applicable in certain situations. Therefore, this paper proposes a novel Series Spring-Embraced Elastic Actuator (SSE-EA), which uses a Transmitted Force-Sensing SEA (TFSEA) structure and a vertical torsional spring to high torque sensitivity. The design method of the spring is proposed to maximize the advantages of the structure. In addition, mechanisms and controller designs are presented to achieve sensitive torque control for heavy load tasks. Through several validations, we evaluate the performance and applicability of the SSE-EA.

Keywords: Optimization and Optimal Control
Abstract: Most interesting problems in robotics (e.g., locomotion and manipulation) are realized through intermittent contact with the environment. Due to the perception and modeling errors, assuming an exact time for establishing contact with the environment is unrealistic. On the other hand, handling uncertainties in contact timing is notoriously difficult as it gives rise to either handling uncertain complementarity systems or solving combinatorial optimization problems at run-time. This work presents a novel optimal control formulation to find robust control policies under contact timing uncertainties. Our main novelty lies in casting the stochastic problem to a deterministic optimization over the uncertainty set that ensures robustness criterion satisfaction of candidate pre-contact states and optimizes for contact-relevant objectives. This way, we only need to solve a manageable standard nonlinear programming problem without complementarity constraints or combinatorial explosion. Our simulation results on multiple simplified locomotion and manipulation tasks demonstrate the robustness of our uncertainty-aware formulation compared to the nominal optimal control formulation.

Keywords: Humanoid and Bipedal Locomotion, Legged Robots, Reinforcement Learning
Abstract: Achieving precise gait planning and high robustness in locomotion control is crucial for the development and application of humanoid robots. In this paper, a novel control strategy is proposed, which combines dynamics control and reinforcement learning (RL), leveraging the precision of dynamics control and the robustness of RL. Specifically, foot placements for each step of the humanoid robot are designed, and the trajectories of the center of mass (CoM) and feet are obtained using a 3D linear inverted pendulum model (3D LIPM). Subsequently, joint angles during motion are calculated based on the trajectories of the CoM and feet using inverse kinematics equations. Finally, the obtained joint angles are trained as baseline actions using RL algorithms. Parameter domain randomization is introduced during the training process to enhance control robustness. By employing this control strategy, simulations of various single-step gaits, such as walking forward, walking to the right, and making right turns, are achieved. Additionally, trajectory tracking, locomotion tests on different terrains, and disturbance resistance are conducted. The simulation results demonstrate that the proposed control strategy enables precise gait control and exhibits strong robustness in humanoid robots.