Keywords: SLAM, Localization, Range Sensing
Abstract: Frequency-Modulated Continuous-Wave (FMCW) lidar is a recently emerging technology that additionally enables per-return instantaneous relative radial velocity measurements via the Doppler effect. In this letter, we present the first continuous-time lidar-only odometry algorithm using these Doppler velocity measurements from an FMCW lidar to aid odometry in geometrically degenerate environments. We apply an existing continuous-time framework that efficiently estimates the vehicle trajectory using Gaussian process regression to compensate for motion distortion due to the scanning-while-moving nature of any mechanically actuated lidar (FMCW and non-FMCW). We evaluate our proposed algorithm on several real-world datasets, including publicly available ones and datasets we collected. Our algorithm outperforms the only existing method that also uses Doppler velocity measurements, and we study difficult conditions where including this extra information greatly improves performance. We additionally demonstrate state-of-the-art performance of lidar-only odometry with and without using Doppler velocity measurements in nominal conditions. Code for this project can be found at: https://github.com/utiasASRL/steam_icp.

Keywords: SLAM, Sensor Fusion, Perception for Grasping and Manipulation
Abstract: Quantification of uncertainty in point cloud matching is critical in many tasks such as pose estimation, sensor fusion, and grasping. Iterative closest point (ICP) is a commonly used pose estimation algorithm which provides a point estimate of the transformation between two point clouds. There are many sources of uncertainty in this process that may arise due to sensor noise, ambiguous environment, initial condition, and occlusion. However, for safety critical problems such as autonomous driving, a point estimate of the pose transformation is not sufficient as it does not provide information about the multiple solutions. Current probabilistic ICP methods usually do not capture all sources of uncertainty and may provide unreliable transformation estimates which can have a detrimental effect in state estimation or decision making tasks that use this information. In this work we propose a new algorithm to align two point clouds that can precisely estimate the uncertainty of ICP's transformation parameters. We develop a Stein variational inference framework with gradient based optimization of ICP's cost function. The method provides a non-parametric estimate of the transformation, can model complex multi-modal distributions, and can be effectively parallelized on a GPU. Experiments using 3D kinect data as well as sparse indoor/outdoor LiDAR data show that our method is capable of efficiently producing accurate pose uncertainty estimates.

Keywords: SLAM, Localization, Computer Vision for Medical Robotics
Abstract: Deformable Monocular SLAM algorithms recover the localization of a camera in an unknown deformable environment. Current approaches use a template-based deformable tracking to recover the camera pose and the deformation of the map. These template-based methods use an underlying global deformation model. In this paper, we introduce a novel deformable camera tracking method with a local deformation model for each point. Each map point is defined as a single textured surfel that moves independently of the other map points. Thanks to a direct photometric error cost function, we can track the position and orientation of the surfel without an explicit global deformation model.

In our experiments, we validate the proposed system and observe that our local deformation model estimates more accurately the targeted deformations of the map in both laboratory-controlled experiments and in-body scenarios undergoing quasi-isometric deformations, with changing topology or discontinuities.

Keywords: SLAM, RGB-D Perception, Sensor Fusion, Evolution Strategy
Abstract: High-dimensional nonlinear state estimation is at the heart of inertial-aided navigation systems (INS). Traditional methods usually rely on good initialization and find difficulty in handling large inter-frame transformations due to fast camera motion. We opt to tackle these challenges by solving the depth inertial odometry (DIO) problem with random optimization. To address the exponentially increased amount of candidate states sampled for the high-dimensional state space, we propose a highly efficient variant of random optimization based on the idea of active subspace. Our method identifies the active dimensions which contribute the most significantly to the decrease of the cost function in each iteration, and samples candidate states only within the corresponding subspace. This allows us to efficiently explore the 18D state space of DIO and achieve good optimality by sampling and evaluating only thousands of candidate states. Experiments show that our method attains highly robust and accurate DIO under fast camera motions and low light conditions, without needing a slow-motion warm-up for initialization.

Keywords: SLAM, Localization, Mapping
Abstract: We describe a general approach for maximum a posteriori (MAP) inference in a class of discrete-continuous factor graphs commonly encountered in robotics applications. While there are openly available tools providing flexible and easy-to-use interfaces for specifying and solving inference problems formulated in terms of either discrete or continuous graphical models, at present, no similarly general tools exist enabling the same functionality for hybrid discrete-continuous problems. We aim to address this problem. In particular, we provide a library, DC-SAM, extending existing tools for inference problems defined in terms of factor graphs to the setting of discrete-continuous models. A key contribution of our work is a novel solver for efficiently recovering approximate solutions to discrete-continuous inference problems. The key insight to our approach is that while joint inference over continuous and discrete state spaces is often hard, many commonly encountered discrete-continuous problems can naturally be split into a “discrete part” and a “continuous part” that can individually be solved easily. Leveraging this structure, we optimize discrete and continuous variables in an alternating fashion. In consequence, our proposed work enables straightforward representation of and approximate inference in discrete-continuous graphical models. We also provide a method to approximate the uncertainty in estimates of both discrete and continuous variables.

Keywords: SLAM, Localization, Mapping
Abstract: Iterated Extended Kalman Filter is a promising and widely-used estimator for real-time localization applications. It iterates the observation equation to find a better linearization point and, simultaneously, only maintains the state estimation in a single time to save the computation resources. Inspired by the recent development of the iterative closest point algorithm, this paper investigates an acceleration approach to the iterations in iterative error state Kalman filters (IESKFs). We show that the IESKF can be seen as a fixed point problem, and the Anderson acceleration (AA) can be elegantly applied to the iterations of IESKF since the error state naturally lies in the tangent space and does not require additional transforms. However, the tangent space of the current estimation may change during the iterations, so we should switch the tangent space to the starting point to perform Anderson acceleration. We propose the AA-IEKF and apply it to the lidar-inertial odometry (LIO) systems to estimate the ego-motion of a lidar. The experiments show that the Anderson acceleration can efficiently reduce the number of iterations in ESKF and achieve a lower computational cost.

Keywords: SLAM, Localization, Mapping
Abstract: This paper presents a LiDAR odometry estimation framework called Generalized LOAM. Our proposed method is generalized in that it can seamlessly fuse various local geometric shapes around points to improve the position estimation accuracy compared to the conventional LiDAR odometry and mapping (LOAM) method. To utilize continuous geometric features for LiDAR odometry estimation, we incorporate tiny neural networks into a generalized iterative closest point (GICP) algorithm. These neural networks improve the data association metric and the matching cost function using local geometric features. Experiments with the KITTI benchmark demonstrate that our proposed method reduces relative trajectory errors compared to the GICP and LOAM methods.

Keywords: SLAM, Localization
Abstract: Loop closing is a fundamental part of simultaneous localization and mapping (SLAM) for autonomous mobile systems. In the field of visual SLAM, bag of words (BoW) has achieved great success in loop closure. The BoW features for loop searching can also be used in the subsequent 6-DoF loop correction. However, for 3D LiDAR SLAM, the state-of-the-art methods may fail to effectively recognize the loop in real time, and usually cannot correct the full 6-DoF loop pose. To address this limitation, we present a novel Bag of Words for real-time loop closing in 3D LiDAR SLAM, called BoW3D. Our method not only efficiently recognizes the revisited loop places, but also corrects the full 6-DoF loop pose in real time. BoW3D builds the bag of words based on the 3D LiDAR feature LinK3D, which is efficient, pose-invariant and can be used for accurate point-to-point matching. We furthermore embed our proposed method into 3D LiDAR odometry system to evaluate loop closing performance. We test our method on public dataset, and compare it against other state-of-the-art algorithms. Our BoW3D shows better performance in terms of F1 max and extended precision scores in most scenarios with superior real-time performance. It is noticeable that BoW3D takes an average of 50 ms to recognize and correct the loops on KITTI 00 (includes 4K+ 64-ray LiDAR scans), when executed on a notebook with an Intel Core i7 @2.2 GHz processor. We release the implementation of our method here: https://github.com/YungeCui/

Keywords: SLAM, Sensor Fusion, Object Detection, Segmentation and Categorization
Abstract: In this letter, we propose a novel method to merge a Gaussian mixture on matrix Lie groups and present its application for a simultaneous localization and mapping problem with symmetric objects. The key idea is to predetermine the weighted mean called a midway point and merge Gaussian mixture components at the associated tangent space. Through this rule, the covariance matrix captures the original density more accurately, and the need for the back-projection is spared when compared to the conventional merge. We highlight the midway-merge by numerically evaluating dissimilarity metrics of density functions before and after the merge on the rotational group. Furthermore, we experimentally discover that the rotational error of symmetric objects follows heavy-tailed behavior and formulate the Gaussian sum filter to model it by a Gaussian mixture noise. The effectiveness of our approach is validated through virtual and real-world datasets.

Keywords: Soft Robot Applications, Additive Manufacturing, Hydraulic/Pneumatic Actuators
Abstract: Although research studies in pneumatic soft robots develop rapidly, most pneumatic actuators are still controlled by rigid valves and conventional electronics. The existence of these rigid, electronic components sacrifices the compliance and adaptability of soft robots. Current electronics-free valve designs based on soft materials are facing challenges in behaviour consistency, design flexibility, and fabrication complexity. Taking advantages of soft material 3D printing, this paper presents a new design of a bi-stable pneumatic valve, which utilises two soft, pneumatically-driven, and symmetrically-oriented conical shells with structural bistability to stabilise and regulate the airflow. The critical pressure required to operate the valve can be adjusted by changing the design features of the soft bi-stable structure. Multi-material printing simplifies the valve fabrication, enhances the flexibility in design feature optimisations, and improves the system repeatability. In this work, both a theoretical model and physical experiments are introduced to examine the relationships between the critical operating pressure and the key design features. Results with valve characteristic tuning via material stiffness changing show better effectiveness compared to the change of geometry design features (demonstrated largest tunable critical pressure range from 15.3 to 65.2 kPa and fastest response time <= 1.8 s).

Keywords: Soft Robot Applications, AI-Based Methods, Machine Learning for Robot Control
Abstract: Programmable surfaces (PSs) consist of a 2D array of actuators that can deform in the third dimension, providing the ability to create continuous 3D profiles. Discrete PSs can be realized using an array of independent solid linear actuators. Continuous PSs consist of actuators that are mechanically coupled, providing deformation states that are more similar to real surfaces with reduced complexity of the control electronics. However, continuous PSs have been limited in size by the lack of the control systems required to take into account the complex internal coupling between actuators in the array. In this work, we computationally explore the deformation of a fully continuous PS with 81 independent actuation pixels based on ionic bending actuator. We establish a control strategy using machine learning (ML) regression models. Both forward and inverse control are achieved based on the training datasets which are derived from the finite element analysis (FEA) data of our PS. The prediction of surface deformation achieved by forward control with accuracy under 1% is 15000 times faster than FEM. And the real-time inverse control of continuous PSs that is to reproduce any arbitrary pre-defined surfaces, which possess high practical value for tactile display or human-machine interactive devices, is first proposed in the letter.

Keywords: Soft Robot Applications, Soft Sensors and Actuators, Multifingered Hands
Abstract: In this letter, we propose a physics-based framework to exploit magnets

in robotic manipulation. More specifically, we suggest equipping soft and underactuated hands with magnetic elements, which can generate a magnetic actuation able to synergistically interact with tendon-driven and pneumatic actuations, engendering a complementarity that enriches the capabilities of the actuation system. Magnetic elements can act as additional Degrees of Actuation (DoAs), robustifying the motion control

of the device and augmenting the hand manipulation capabilities. We investigate the interaction of a soft hand with itself for enriching possible hand shaping, and the interaction of the hand with the environment for enriching possible grasping capabilities. Physics laws and notions reported in the manuscript can be used as a guidance for DoAs augmentation and can provide tools for the design of novel soft hands.

Keywords: Origami robot, Soft Robot Materials and Design, Kinematics, Compliant Joint/Mechanism
Abstract: Origami processes can generate both rigid and compliant structures from the same homogeneous sheet material. We advance the origami robotics literature by showing that it is possible to construct an arbitrary rigid kinematic chain with prescribed joint compliance from a single tubular sheet. Our "Kinegami" algorithm converts a Denavit-Hartenberg specification into a single-sheet crease pattern for an equivalent serial robot mechanism by composing origami modules from a catalogue. The algorithm arises from the key observation that tubular origami linkage design reduces to a Dubins path planning problem. The automatically generated structural connections and movable joints that realize the specified design can also be endowed with independent user-specified compliance. We apply the Kinegami algorithm to a number of common robot mechanisms and hand-fold their algorithmically generated single-sheet crease patterns into functioning kinematic chains. We believe this is the first completely automated end-to-end system for converting an abstract manipulator specification into a physically realizable origami design that requires no additional human input.

Keywords: Rehabilitation Robotics, Wearable Robotics, Soft Robot Applications
Abstract: A gait entrainment study was conducted using a lightweight soft robotic hip exosuit (SR-HExo) that can apply perturbations at the hip joint during treadmill walking. Periodic perturbations were applied by flat fabric Pneumatic Artificial Muscle actuators starting at a subject’s preferred gait frequency and increasing up to 15% higher in 3% increments. Anterior hip flexion perturbations and posterior hip extension perturbations were tested in two separate experiments. All 11 healthy participants showed successful entrainment in all 12 experimental conditions (i.e., from preferred gait frequency to 15% higher in both flexion and extension perturbation directions). This study confirmed that there exists a single stable point attractor during gait entrainment to unilateral, unidirectional hip perturbations, which is consistent with previous ankle studies. Phase-locking was consistently observed around toe-off phase of the gait cycle (GC). Group averaged results showed gait synchronization with extension perturbations occurred earlier in the gait cycle (around 50% GC where the hip angle reaches maximum extension) than with flexion perturbations (just after 60% GC where the transition from maximum hip extension towards hip flexion occurs). Other gait entrainment characteristics (success rate of entrainment, basin of entrainment, and transient response) observed in this study posits the potential of the SR-HExo for entrainment-based gait training in rehabilitation contexts.

Keywords: Soft Robot Materials and Design, Soft Robot Applications, Aerial Systems: Mechanics and Control
Abstract: This paper presents the first design of a soft, 3D-printed in flexible filament, lightweight UAV, capable of performing full-body perching using soft tendons, specifically landing and stabilizing on pipelines and irregular surfaces without the need for an auxiliary system. The flexibility of the UAV can be controlled during the additive manufacturing process by adjusting the infill rate distribution. However, the increase in flexibility implies difficulties in controlling the UAV, as well as structural, aerodynamic, and aeroelastic effects. This article provides insight into the dynamics of the system and validates the flyability of the vehicle for densities as low as 6%. Within this range, quasi-static arm deformations can be considered, thus the autopilot is fed back through a static arm deflection model. At lower densities, strong non-linear elastic dynamics appear, which translates to complex modeling, and it is suggested to switch to data-based approaches.

Keywords: Soft Robot Materials and Design, Biologically-Inspired Robots, Climbing Robots
Abstract: Most current in-pipe robots are usually designed for pipes of a specific size. In this paper, we propose a novel inchworm-like in-pipe robot based on the concept of tensegrity for moving in pipes with varying diameters. Firstly, a tensegrity-based robotic module capable of two kinds of shape change is designed. One kind is extension in the axial direction accompanied by contrac-tion in the radial direction, which is the basis for the wave-like crawling movement of the in-pipe robot. The other kind is ex-pansion in the radial direction while keeping changeless in the axial direction, enabling the module adaptable to pipes with different diameters. Then, the geometrical equilibrium configu-ration of the tensegrity module is determined, followed by kinematic analysis using force density method. By cascading three modules, the in-pipe crawling robot is developed. Finally, a series of experiments are performed to test the shape change-ability and friction force of the tensegrity module, and the mo-bility, load capacity, and adaptability of the in-pipe robot. The results validate that the robot can crawl in horizontal pipes, vertical pipes, and elbow pipes under the control of a simple actuation sequence. Furthermore, the robot has the abilities to adapt to pipes with different diameters varying from 100 mm to 180 mm. It is suggested that the usage of tensegrity structures brings about higher adaptability, flexibility, and mobility to the in-pipe crawling robot.

Keywords: Soft Robot Materials and Design, Biologically-Inspired Robots, Soft Robot Applications
Abstract: Mobile robots that can adapt to an extensive range of terrains play essential roles in many applications. Millipedes are one of the most terrain-adaptive creatures in nature due to their multi-legged locomotion and flexible body. Inspired by natural millipedes, we report an untethered robotic millipede with a 6-segments soft-rigid hybrid body that can actively bend and 24 legs driven by low-pressure microfluidic actuators. The 24 microfluidic actuators are driven by two independent low-pressure sources from miniature pumps, which allows the untethered locomotion of the robotic millipede in small size (length, 23 cm; width, 5 cm; height, 4 cm) and lightweight (150 g). Using a pre-defined gait for the multi-legs, the robotic millipede can locomote with a maximum speed of 30.96 cm/min (1.35 body length per minute) and a minimum turning radius of 15 cm (0.65 body length). Experiments also demonstrated that the robot was able to locomote effectively in various uneven terrains. Utilizing its passive or active mode of its flexible body, the robot could also achieve adaptive moves. The robotic millipede has the potential to perform a variety of environment exploration tasks by remotely controlling and transmitting real time images wirelessly.

Keywords: Soft Robot Materials and Design, Modeling, Control, and Learning for Soft Robots, Soft Robot Applications
Abstract: Soft robotics modeling is a research topic that is evolving fast. Many techniques are present in literature but most of them require analytical models with a lot of equations that are time-consuming, hard to resolve, and not so easy to handle. For this reason, the help of a soft mechanics simulator is essential in this field. In fact, this paper presents a tutorial on how to build a soft-robot model using an open-source Finite Element Analysis (FEA) simulator, called SOFA. This software is able to generate a simulation scene from a code written in Python or XML, so it can be used by people that with different fields of competence like mechanical knowledge, knowledge of material properties and programming skills. As a case study, a Python simulation of a cable-driven soft actuator that makes contact with a rigid object is considered. The basic working principles of SOFA required to make a scene are explained step by step. In particular, it shows how to simulate the mechanics and animate the bending behavior of the actuator. Furthermore, it will be shown also how to retrieve and save data from simulation, demonstrating that SOFA can easily adapt to a multi-disciplinary subject as the research in soft-robotics, but also be useful for teaching simulation and programming language principles to engineering students.

Keywords: Soft Robot Materials and Design, Mechanism Design, Compliant Joints and Mechanisms
Abstract: Inflatable structures used in soft robotics applications have unique characteristics. In particular, the tip-extension structure, which extends the structure from its tip, can grow without creating friction with the environment. However, these inflatable structures need high pressure to maintain their stiffness under various conditions. Excessive inner pressure limits their application in that it prevents the structure from maintaining its curved shape and from complying with specifications. This study aimed to simultaneously lower the pressure and increase the rigidity of the structure. Our work resulted in the proposal of a mechanism that combines a skeleton structure consisting of multi-joint links with functions to increase the rigidity. Insertion of this mechanism into an inflatable structure obviates the need for high inner pressure, yet enables the structure to bend and maintain the intended shape. We devised a design based on rigid articulated links and combined it with a membrane structure that utilizes the advantages of the tip-extension structure. The experimental results show that the payload of the structure designed to operate at low pressure increases compared to that of the membrane-only structure. The findings of this research can be applied to long robots that can be extended into open space without drooping and to mechanisms that enable structures to wrap around the human body.

Keywords: Mechanism Design, Biologically-Inspired Robots, Medical Robots and Systems
Abstract: This paper analyzes the crawling locomotion of a wave-like robot in curved tubes. We use an energy-based approach to determine the optimal crawling orientation of the robot that minimizes the surface energy while advancing. The results showed that the robot rotated its body along the roll direction so that the wave motion would be in the same plane as the curvature plane of the tube. The incorporation of a passive bending joint along the plane of the wave motion decreased the surface energy and enhanced the robot’s ability to advance in even tighter curves. Given these findings we designed and manufactured two new robots with either one or two passive bending joints. We molded custom flexible surfaces and tubes and experimentally tested our robots in them. These validating experiments indicated that the bending joints substantially improved the robots’ ability to traverse curved tubes (see video).

Keywords: Mechanism Design, Soft Robot Applications
Abstract: This paper proposes a palm-sized omnidirectional mobile robot with two torus wheels. A single torus wheel is made of an elastic elongated coil spring in which the two ends of the coil connected each other and is driven by a piezoelectric actuator (stator) that can generate 2-degrees-of-freedom (axial and angular) motions. The stator converts its thrust force and torque into longitudinal and meridian motions of the torus wheel, respectively, making the torus work as an omnidirectional wheel on a plane. In this paper, we build a control system of a piezo-driven 2-degrees-of-freedom torus wheel and evaluate its performance measures, such as the transient characteristics, the orientation accuracy and the payload capacity. An omnidirectional robot with the two torus wheels is constructed, and the feedback control for a desired planar motion is demonstrated. The design inspired by a ring torus represents the possibility toward the creation of an unprecedentedly simple, light, and compact 2-wheel omnidirectional robot.

Keywords: Cellular and Modular Robots, Actuation and Joint Mechanisms, Biologically-Inspired Robots
Abstract: Modular robotic units that can change their size at will presents an exciting pathway for modular robotics. However, current attempts have been relatively limited, requiring tethers, complex fabrication or slow cycle times. In this work, we present AuxBots: an auxetic-based approach to create high force, fast cycle time self-contained modules. By driving the auxetic shell's inherent mathematical expansion with a motor and leadscrew, these robots are capable of expanding their volume by 274% in 0.7 seconds with a maximum strength to weight ratio of 76x. These force and expansion properties enable us to use these modules in conjunction with flexible wire constraints to get shape changing behavior and independent locomotion. We demonstrate the power of this modular system by using a limited number of AuxBots to mimic the flipper-style locomotion of mudskippers and sea turtles. These structures are entirely untethered and can still move forward even as some AuxBots stall out, achieving the key modular robotics goal of versatility and robustness.

Keywords: Multifingered Hands, Methods and Tools for Robot System Design, Product Design, Development and Prototyping
Abstract: How to design an anthropomorphic hand imitating specific human hand grasps with as few actuators as possible is still a challenge. This paper presents a method for obtaining a simplified configuration of anthropomorphic hand imitating specific human hand grasps based on the motion analyses of the human hand. A participation matrix which characterizes a human hand grasp on joint motion level is constructed according to the motion participation of each finger joint. By adding all participation matrices of expected human hand grasps together a total participation matrix can be derived, and through mathematical processing a simplified anthropomorphic hand configuration can be obtained. Following the proposed method, a simplified anthropomorphic hand configuration that imitates six basic human hand grasps was obtained. A series of grasp experiments with the anthropomorphic hand prototype were conducted to validate the grasping capability as well as the proposed simplified configuration design method. This method can help to obtain a reasonably simplified configuration of an anthropomorphic hand when expected human hand grasps are definite.

Keywords: Reinforcement Learning, Mechanism Design, Legged Robots
Abstract: The process of robot design is a complex task and the majority of design decisions are still based on human intuition or tedious manual tuning. A more informed way of facing this task is computational design methods where design parameters are concurrently optimized with corresponding controllers. Existing approaches, however, are strongly influenced by predefined control rules or motion templates and cannot provide end-to-end solutions. In this paper, we present a design optimization framework using model-free meta reinforcement learning, and its application to the optimizing kinematics and actuator parameters of quadrupedal robots.

We use meta reinforcement learning to train a locomotion policy that can quickly adapt to different designs. This policy is used to evaluate each design instance during the design optimization. We demonstrate that the policy can control robots of different designs to track random velocity commands over various rough terrains. With controlled experiments, we show that the meta policy achieves close-to-optimal performance for each design instance after adaptation. Lastly, we compare our results against a model-based baseline and show that our approach allows higher performance while not being constrained by predefined motions or gait patterns.

Keywords: Robot Safety, Cooperating Robots, Mechanism Design
Abstract: In recent years, human-robot cooperation has enhanced productivity and achieved high payload, speed, and accuracy. Integrating typical industrial robots in human-robot cooperation is challenging because their arms may cause serious injuries to humans during a collision due to malfunction or errors due to robot operators. Therefore, counterbalance robot arms that are capable of counterbalancing the gravitational torques due to the robot mass have been developed to decrease the required capacity of the motors and speeds of these robots. In this research, we propose an advanced counterbalance mechanism using gear units and springs to improve the durability and reliability compared to the previously proposed wire-based counterbalance mechanism, which is difficult to apply to a commercialized product because it can easily be broken or stretched when an excessive force is applied for a long period. Moreover, our proposed method was extended to a multi-DOF system using a parallelogram mechanism based on a timing belt and pulleys to achieve multi-DOF robotic arms. A 2-DOF counterbalanced arm was designed to verify the effectiveness of the proposed mechanism. The simulations and

experimental results showed that the proposed mechanism effectively reduced the gravitational torques of each joint of the multi-DOF arm.

Keywords: Actuation and Joint Mechanisms, Force Control, Mechanism Design
Abstract: Machines such as robotic grippers use powerful actuators or gearboxes to exert large loads at the expense of energy consumption, volume, and mass. We propose a stepless force amplification mechanism that assists clamping by a pair of permanent magnets, in which the external control force required to adjust their distance, and thus the output force, is suppressed by compensation springs. For further sophistication, we invented a new width adjuster using a lever. By separating the actuation of fingers and compensated magnets temporarily, the adjuster eliminated the nonlinear influence of the object width on the clamping force. The prototype gripper for proof of concept revealed that the adjuster successfully linearized the width-force characteristic with an inclination of 0.15 N/mm, which is sufficiently insignificant compared to the major output force of approximately 50 N. The force amplification effect coexisted with this phenomenon, such that the clamping force was amplified to 137.5% while maintaining the energy consumption of a DC motor, and the force-energy efficiency was multiplied by 1.39. Thus, able to be driven by a weaker, smaller, and lighter actuator, the gripper contributes to extension of the operation time of robots with limited power supply.

Keywords: Compliant Joint/Mechanism, Mechanism Design, Micro/Nano Robots, Biologically-Inspired Robots
Abstract: This paper proposes the concept design of a novel bio-inspired dual-axis compliant micromanipulator with millimeter working strokes dedicated to fiber alignment. It subtly mimics the gripping and rubbing function of the human hand consisting of the forefinger, purlicue, and thumb. Compared with traditional dual-axis grippers, its advantages lie in millimeter-level stroke, bi-directional rotation, less slippage, and comprehensive force sensing. To achieve dexterous and reliable manipulation, a two-degree-of-freedom (2-DOF) flexible decoupling mechanism and a displacement reversing mechanism based on the leaf-shaped flexible hinge are introduced. A prototype driven by two voice coil motors is fabricated for experimental testing. Three high-precision strain gauges with temperature compensation are glued on the sensitive region to measure the gripping force and rubbing force. Experimental results show that the gripping and rubbing strokes of the manipulator are up to 2.3 mm and 2.1 mm, respectively. For a custom-made fiber flag with a diameter of 200 um, the rotation stroke of more than 1000 degrees has been achieved, which cannot be realized by previous work with the same level of compact mechanism design.

Keywords: Biologically-Inspired Robots, Actuation and Joint Mechanisms, Aerial Systems: Mechanics and Control
Abstract: Flapping wing robots show promise as platforms for safe and efficient flight in near-human operations, thanks to their ability to agile maneuver or perch at a low Reynolds number. The growing trend in the automatization of these robots has to go hand in hand with an increase in the payload capacity. This work provides a new passive morphing wing prototype to increase the payload of this type of UAV. The prototype is based on a biased elastic joint and the holistic research also includes the modelling, simulation and optimization scheme, thus allowing to adapt the prototype for any flapping wing robot. This model has been validated through flight experiments on the available platform, and it has also been demonstrated that the morphing prototype can increase the lift of the robot under study by up to 16% in real flight and 10% of estimated consumption reduction.

Keywords: Motion and Path Planning, Collision Avoidance, Multi-Robot Systems
Abstract: We propose a variant of alternating direction method of multiplier (ADMM) to solve constrained trajectory optimization problems. Our ADMM framework breaks a joint optimization into small sub-problems, leading to a low iteration cost and decentralized parameter updates. Starting from a collision-free initial trajectory, our method inherits the theoretical properties of primal interior point method (P-IPM), i.e., guaranteed collision avoidance and homotopy preservation throughout optimization, while being orders of magnitude faster. We have analyzed the convergence and evaluated our method for time-optimal multi-UAV trajectory optimizations and simultaneous goal-reaching of multiple robot arms, where we take into consider kinematics-, dynamics-limits, and homotopy-preserving collision constraints. Our method highlights an order of magnitude's speedup, while generating trajectories of comparable qualities as state-of-the-art P-IPM solver.

Keywords: Planning under Uncertainty, Search and Rescue Robots, Object Detection, Segmentation and Categorization
Abstract: Frontier-based exploration is widely adopted for exploring an unknown region. The conventional frontier-based exploration for a mobile robot may collide with three-dimensional (3D) obstacles or can suffer from a slower exploration time because the robot may move to another place before completely exploring the current area. To solve this problem, in this paper, we propose a new exploration algorithm by considering a path traveled by a mobile robot and segmenting a two-dimensional (2D) map. The segmented 2D map is generated in real-time by using the position of the robot and the location of the detected frontiers. To apply our algorithm to the actual experiment, we develop an object detection-based exploration algorithm that can remarkably reduce the probability of collision with 3D obstacles. To verify the effectiveness of our proposed algorithm, we perform simulations (Gazebo) and experiments (in the real world) to compare the conventional approach and our algorithm in a cluttered environment. The simulation and experiment results show that our algorithm can satisfactorily shorten the exploration path and time.

Keywords: Motion and Path Planning, Collision Avoidance, Robot Safety
Abstract: In this paper, we propose a Wasserstein metric-based random path planning algorithm. Wasserstein Safe RRT (W-Safe RRT) provides finite-sample probabilistic guarantees on the safety of a returned path in an uncertain obstacle environment. Vehicle and obstacle states are modeled as distributions based upon state and model observations. We define limits on distributional sampling error so the Wasserstein distance between a vehicle state distribution and obstacle distributions can be bounded. This enables the algorithm to return safe paths with a confidence bound through combining finite sampling error bounds with calculations of the Wasserstein distance between discrete distributions. W-Safe RRT is compared against a baseline minimum encompassing ball algorithm, which ensures balls that minimally encompass discrete state and obstacle distributions do not overlap. The improved performance is verified in a 3D environment using single, multi, and rotating non-convex obstacle cases, with and without forced obstacle error in adversarial directions, showing that W-Safe RRT can handle poorly modeled complex environments.

Keywords: Motion and Path Planning, Reinforcement Learning, Collision Avoidance
Abstract: Obstacle avoidance for robotic manipulators can be challenging when they operate in unstructured environments. This problem is probed with the sim-to-real (sim2real) deep reinforcement learning, such that a moving policy of the robotic arm is learnt in a simulator and then adapted to the real world. However, the problem of sim2real adaptation is notoriously difficult. To this end, this work proposes (1) a unified representation of obstacles and targets to capture the underlying dynamics of the environment while allowing generalization to unseen goals and (2) a flexible end-to-end model combining the unified representation with the deep reinforcement learning control module that can be trained by interacting with the environment. Such a representation is agnostic to the shape and appearance of the underlying objects, which simplifies and unifies the scene representation in both simulated and real worlds. We implement this idea with a vision-based actor-critic framework by devising a bounding box predictor module. The predictor estimates the 3D bounding boxes of obstacles and targets from the RGB-D input. The features extracted by the predictor are fed into the policy network, and all the modules are jointly trained. Our experiments in simulated environment and the real-world show that the end-to-end model of the unified representation achieves better sim2real adaption and scene generalization than state-of-the-art techniques.

Keywords: Motion and Path Planning, Autonomous Agents, Dynamics, Sampling-Based Motion Planning
Abstract: Bidirectional path and motion planning approaches decrease planning time, on average, compared to their unidirectional counterparts. In single-query feasible motion planning, using bidirectional search to find a continuous motion plan requires an edge connection between the forward and the reverse search tree. Such a tree–tree connection requires solving a two-point boundary value problem (BVP). However, obtaining a closed-form two-point BVP solution can be difficult or impossible for many systems. While numerical methods can provide a reasonable solution in many cases, they are often computationally expensive or numerically unstable for the purposes of single-query sampling-based motion planning. To overcome this challenge, we present a novel bidirectional search strategy that does not require solving the two-point BVP. Instead of connecting the forward and reverse trees directly, the reverse tree’s cost information is used as a guiding heuristic for forward search. This enables the forward search to quickly grow down the reverse tree—converging to a fully feasible solution without the solution to a two-point BVP. In this article, we propose two algorithms that use this strategy for single-query feasible motion planning for various dynamical systems, performing experiments in both simulation and hardware testbeds. We find that these algorithms perform better than or comparable to existing state-of-the-art methods with respect to quickly finding an initial feasible solution.

Keywords: Manipulation Planning, Task Planning, Robotics in Construction, Multi-Robot Systems
Abstract: Robotic construction assembly planning aims to find feasible assembly sequences as well as the corresponding robot-paths and can be seen as a special case of task and motion planning (TAMP). As construction assembly can well be parallelized, it is desirable to plan for multiple robots acting concurrently. Solving TAMP instances with many robots and over a long time-horizon is challenging due to coordination constraints, and the difficulty of choosing the right task assignment.

We present a planning system which enables parallelization of complex task and motion planning problems by iteratively solving smaller subproblems. Combining optimization methods to jointly solve for manipulation constraints with a sampling-based bi-directional space-time path planner enables us to plan cooperative multi-robot manipulation with unknown arrival-times. Thus, our solver allows for completing subproblems and tasks with differing timescales and synchronizes them effectively. We demonstrate the approach on multiple construction case-studies to show the robustness over long planning horizons and scalability to many objects and agents. % of our algorithm. Finally, we also demonstrate the

Keywords: Motion and Path Planning, Constrained Motion Planning, Optimization and Optimal Control
Abstract: By limiting the planning domain to “L2 Informed Set”, some sampling-based motion planner (SBMP) (e.g. Informed RRT*, BIT*) can solve the geometric motion planning problems efficiently. However, the construction of informed set (IS) will be very challenging, when further differential constraints are considered. For the time-optimal kinodynamic motion planning problem, this paper defines a modified time informed set (MTIS) to limit the planning domain. Due to drawing inspiration from Hamilton-Jacobi-Bellman (HJB) reachability analysis, MTIS, compared with the original TIS, can not only help save the running time of SBMP, but also extend the applicable scope from linear systems to polynomial nonlinear systems with control constrains. On this basis, a spatio-temporal sampling strategy adapted to MTIS is proposed. Firstly, MTIS is used to estimate the optimal cost and the valid tree structure is reused, so that we do not need to provide a solution trajectory in advance. Secondly, this strategy is generic, allowing it to be combined with common SBMP (such as SST, etc.) to accelerate convergence and reduce memory complexity. Several simulation experiments also demonstrate the effectiveness of proposed method.

Keywords: Autonomous Vehicle Navigation, Integrated Planning and Learning, Motion and Path Planning
Abstract: Safely interacting with other traffic participants is one of the core requirements for autonomous driving, especially in intersections and occlusions. Most existing approaches are designed for particular scenarios and require significant human labor in parameter tuning to be applied to different situations. To solve this problem, we first propose a learning-based Interaction Point Model (IPM), which describes the interaction between agents with the protection time and interaction priority in a unified manner. We further integrate the proposed IPM into a novel planning framework, demonstrating its effectiveness and robustness through comprehensive simulations in highly dynamic environments.

Keywords: Motion and Path Planning, Reactive and Sensor-Based Planning, Nonholonomic Motion Planning, Field Robots
Abstract: We propose and experimentally demonstrate a reactive planning system for bipedal robots on unexplored, challenging terrain. The system includes: a multi-layer local map for assessing traversability; an anytime omnidirectional Control Lyapunov Function (CLF) for use with a Rapidly Exploring Random Tree Star (RRT*) that generates a vector field for specifying motion between nodes; a sub-goal finder when the final goal is outside of the current map; and a finite-state machine to handle high-level mission decisions. The system also includes a reactive thread that copes with robot deviations via a vector field, defined by a closed-loop feedback policy. The vector field provides real-time control commands to the robot's gait controller as a function of instantaneous robot pose. The system is evaluated on various challenging outdoor terrains and cluttered indoor scenes in both simulation and experiment on Cassie Blue, a bipedal robot with 20 degrees of freedom. All implementations are coded in C++ with the Robot Operating System (ROS) and are available at https://github.com/UMich-BipedLab/CLF_reactive_planning_system.

Keywords: Task Planning, Reactive and Sensor-Based Planning, Planning, Scheduling and Coordination
Abstract: Robots often need to accomplish complex tasks in unknown environments, which is a challenging problem, involving autonomous exploration for acquiring necessary scene knowledge and task planning. In traditional approaches, the agent first explores the environment to instantiate a complete planning domain and then invokes a symbolic planner to plan and perform high-level actions. However, task execution is inefficient since the two processes involve many repetitive states and actions. Hence, this paper proposes a framework to co-optimize robot exploration and task planning in unknown environments. To afford robot exploration and symbolic planning not being independent and separated, we design a unified structure named subtask, which is exploited to decompose the robot exploration and planning phases. To select the appropriate subtask each time, we develop a value function and a value-based scheduler to co-optimize exploration and task processing. Our framework is evaluated in a photo-realistic simulator with three complex household tasks, increasing task efficiency by 25%-29%.

Keywords: Reinforcement Learning, Embedded Systems for Robotic and Automation, Hardware-Software Integration in Robotics
Abstract: This paper explores a Deep Reinforcement Learning (DRL) approach for designing image-based control for edge robots to be implemented on Field Programmable Gate Arrays (FPGAs). Although FPGAs are more power-efficient than CPUs and GPUs, a typical (DRL) method cannot be applied since they are composed of many Logic Blocks (LBs) for high-speed logical operations but low-speed real-number operations. To cope with this problem, we propose a novel DRL algorithm called Binarized P-Network (BPN), which learns image-input control policies using Binarized Convolutional Neural Networks (BCNNs). To alleviate the instability of reinforcement learning caused by a BCNN with low function approximation accuracy, our BPN adopts a robust value update scheme called Conservative Value Iteration, which is tolerant of function approximation errors. We confirmed the BPN's effectiveness through applications to a visual tracking task in simulation and real-robot experiments with FPGA.

Keywords: Reinforcement Learning, Incremental Learning, Autonomous Agents
Abstract: Deep reinforcement learning has enabled robots to learn motor skills from environmental interactions with minimal to no prior knowledge. However, existing reinforcement learning algorithms assume an episodic setting, in which the agent resets to a fixed initial state distribution at the end of each episode, to successfully train the agents from repeated trials. Such reset mechanism, while trivial for simulated tasks, can be challenging to provide for real-world robotics tasks. Resets in robotic systems often require extensive human supervision and task-specific workarounds, which contradicts the goal of autonomous robot learning. In this paper, we propose an extension to conventional reinforcement learning towards greater autonomy by introducing an additional agent that learns to reset in a self-supervised manner. The reset agent preemptively triggers a reset to prevent manual resets and implicitly imposes a curriculum for the forward agent. We apply our method to learn from scratch on a suite of simulated and real-world continuous control tasks and demonstrate that the reset agent successfully learns to reduce manual resets whilst also allowing the forward policy to improve gradually over time.

Keywords: Reinforcement Learning, Robust/Adaptive Control, Robot Safety
Abstract: A reinforcement learning (RL) control policy could fail in a new/perturbed environment that is different from the training environment, due to the presence of dynamics variations. For controlling systems with continuous state and action spaces, we propose an add-on approach to robustifying a pre-trained RL policy by augmenting it with an L1 adaptive controller (L1AC). Leveraging the capability of an L1AC for fast estimation and active compensation of dynamic variations, the proposed approach can improve the robustness of an RL policy that is trained either in a simulator or in the real world without consideration of a broad class of dynamics variations. Numerical and real-world experiments empirically demonstrate the efficacy of the proposed approach in robustifying RL policies trained using both model-free and model-based methods.

Keywords: Reinforcement Learning
Abstract: Many hierarchical reinforcement learning algorithms utilise a series of independent skills as a basis to solve tasks at a higher level of reasoning. These algorithms don't consider the value of using skills that are cooperative instead of independent. This paper proposes the Cooperative Consecutive Policies (CCP) method of enabling consecutive agents to cooperatively solve long time horizon multi-stage tasks. This method is achieved by modifying the policy of each agent to maximise both the current and next agent's critic. Cooperatively maximising critics allows each agent to take actions that are beneficial for its task as well as subsequent tasks. Using this method in a multi-room maze domain and a peg in hole manipulation domain, the cooperative policies were able to outperform a set of naive policies, a single agent trained across the entire domain, as well as another sequential HRL algorithm.

Keywords: Formal Methods in Robotics and Automation, Learning from Demonstration, Reinforcement Learning
Abstract: In the paradigm of robot learning-from-demonstrations (LfD), understanding and evaluating the demonstrated behaviors plays a critical role in extracting control policies for robots. Without this knowledge, a robot may infer incorrect reward functions that lead to undesirable or unsafe control policies. Prior work has used temporal logic specifications, manually ranked by human experts based on their importance, to learn reward functions from imperfect/suboptimal demonstrations. To overcome reliance on expert rankings, we propose a novel algorithm that learns from demonstrations, a partial ordering of provided specifications in the form of a performance graph. Through various experiments, including simulation of industrial mobile robots, we show that extracting reward functions with the learned graph results in robot policies similar to those generated with the manually specified orderings. We also show in a user study that the learned orderings match the orderings or rankings by participants for demonstrations in a simulated driving domain. These results show that we can accurately evaluate demonstrations with respect to provided task specifications from a small set of imperfect data with minimal expert input.

Keywords: AI-Based Methods, Agent-Based Systems, Machine Learning for Robot Control
Abstract: Tumbling robots are simple platforms that are able to traverse large obstacles relative to their size, at the cost of being difficult to control. Existing control methods apply only a subset of possible robot motions and make the assumption of flat terrain. Reinforcement learning allows for the development of sophisticated control schemes that can adapt to diverse environments. By utilizing domain randomization while training in simulation, a robust control policy can be learned which transfers well to the real world. In this paper, we implement autonomous setpoint navigation on a tumbling robot prototype and evaluate it on flat and uneven terrain. The flexibility of our system demonstrates the viability of nontraditional robots for navigational tasks.

Keywords: Reinforcement Learning, AI-Enabled Robotics, Transfer Learning
Abstract: Recent successes aside, reinforcement learning still faces significant challenges in its application to the real-world robotics domain. Guiding the learning process with additional knowledge offers a potential solution, thus leveraging the strengths of data- and knowledge-driven approaches. However, this field of research encompasses several disciplines and hence would benefit from a structured overview.

In this paper, we propose the concept of guided reinforcement learning that provides a systematic approach towards accelerating the training process and improving the performance for real-world robotic settings. We introduce a classification that structures guided reinforcement learning approaches and shows how different sources of knowledge can be integrated into the learning pipeline in a practical way. Based upon this, we describe available approaches in this field and evaluate their specific impact in terms of efficiency, effectiveness, and sim-to-real transfer within the robotics domain.

Keywords: Reinforcement Learning, Machine Learning for Robot Control
Abstract: Reinforcement learning needs to learn policies through trial and error. The unstable policies in the early stage of training make it expensive (and time-consuming) to train directly in the real environment, which may cause disastrous consequences. The popular solution is to use the simulator to train the policy and deploy it in a real environment. However, the modeling error and external disturbance between the simulation and the real environment may fail the physical deployment, resulting in the sim2real transfer problem. In this letter, we propose a novel robust adversarial reinforcement learning framework, which uses the ensemble training of multi-adversarial agents that can adaptively adjust adversaries’ strength to enhance RL policy’s robustness. More specifically, we take the accumulative reward as feedback and construct a PID controller to adjust the adversary’s output magnitude to perform the adversarial training well. Experiments in the simulated and the real environment show that our algorithm improves the generalization ability of the policy for the modeling error and the uncertain disturbance simultaneously, outperforming the next best prior methods across all domains. The algorithm was further proven to be effective in a sim2real transfer task through the load experiment of a real racing drone, and the tracking performance is better than the PID-based flight controller.

Keywords: Reinforcement Learning, Integrated Planning and Learning, Robot Safety
Abstract: Applying reinforcement learning to autonomous driving entails particular challenges, primarily due to dynamically changing traffic flows. To address such challenges, it is necessary to quickly determine response strategies to the changing intentions of surrounding vehicles. This paper proposes a new policy optimization method for safe driving using graph-based interaction-aware constraints. In this framework, the motion prediction and control modules are trained simultaneously while sharing a latent representation that contains a social context. To reflect social interactions, we illustrate the movements of agents in graph form and filter the features with the graph convolution networks. This helps preserve the spatiotemporal locality of adjacent nodes. Furthermore, we create feedback loops to combine these two modules effectively. As a result, this approach encourages the learned controller to be safe from dynamic risks and renders the motion prediction robust to abnormal movements. In the experiment, we set up a navigation scenario comprising various situations with CARLA, an urban driving simulator. The experiments show state-of-the-art performance in navigation strategy and motion prediction compared to the baselines. The code is available online.

Keywords: Reinforcement Learning
Abstract: Accurate value estimates are important for off-policy reinforcement learning. Algorithms based on temporal difference learning typically are prone to an over- or underestimation bias building up over time. In this paper, we propose a general method called Adaptively Calibrated Critics (ACC) that uses the most recent high variance but unbiased on-policy rollouts to alleviate the bias of the low variance temporal difference targets. We apply ACC to Truncated Quantile Critics [1], which is an algorithm for continuous control that allows regulation of the bias with a hyperparameter tuned per environment. The resulting algorithm adaptively adjusts the parameter during training rendering hyperparameter search unnecessary and sets a new state of the art on the OpenAI gym continuous control benchmark among all algorithms that do not tune hyperparameters for each environment. ACC further achieves improved results on different tasks from the Meta-World robot benchmark. Additionally, we demonstrate the generality of ACC by applying it to TD3 [2] and showing an improved performance also in this setting.

Keywords: Parallel Robots, Legged Robots, Motion and Path Planning, Power consumption minimization
Abstract: Legged robots have great potential to be extraterrestrial exploration rovers of extraordinary versatility. Minimizing power consumption is of vital importance in the scenarios of extraterrestrial explorations. The actuation pattern, which refers to the combination of necessary actuators that output torque, has a significant influence on the power consumption of legged robots. This article seeks to investigate the effect of actuation patterns on the power consumption of legged robots that perform motion in a quasi-static manner. The power consumption model of legged robots considering actuation patterns is deduced. Based on that, the effect of the actuation pattern on mechanical power and heat power, which are the main power-loss terms, is investigated. The lowest power consumption under various conditions achieved by different actuation patterns is investigated. Simulation results show that the power consumption can be reduced by choosing the actuation pattern properly. Furthermore, the principles of selecting the optimal actuation pattern from the perspective of power consumption are summarized, which are expected to facilitate the minimal power consumption motion planning of legged robots.

Keywords: Marine Robotics, Autonomous Vehicle Navigation, Collision Avoidance
Abstract: All vessels operating in a marine environment are required to comply with the international regulations for preventing collisions at sea (COLREGs), which provide the guidelines and evasive procedures required to resolve potential conflicts between vessels. However, not all vessels strictly abide by COLREGs, often leading to dangerous situations. This paper presents a novel approach for robust collision avoidance in encounter situations involving COLREG-violating vessels. A probabilistic velocity obstacle algorithm based on intent inference is designed and implemented with consideration of the tradeoff between the adherence to traffic rules and the proactive evasive actions for safety. One-to-one and multi-ship encounter situations in the presence of rule-violating vessels are examined through Monte-Carlo simulations, and the results are discussed to demonstrate the feasibility and performance of the proposed approach.

Keywords: Marine Robotics, Motion Control, Field Robots
Abstract: The distinct design concepts of the vehicles operating in air and water is one of the tremendous challenges that constrain the development of the hybrid aerial underwater vehicle (HAUV). This incompatibility consequently results in the enlarging volume and weight of the existing prototypes, as well as the unmatched maneuvering characteristics in both domains. This letter presented a novel miniaturized and lightweight HAUV, "Nezha-mini", which weighs 953g and is only A4-scaled. Besides, the low cost and high modularity allow the convenient repair and remanufacturing. Nezha-mini reconciles the complete multi-domain maneuverability within 50m aerially and 6m underwater whilst sufficing for the rapid and stable cross-domain locomotion, which benefits from the selection and unique layout of the propulsion system, as well as our proposed multi-modal control strategy and the cross-domain triggering mechanism. The results of the field experiments are in good agreement with the dynamics simulation, demonstrating the performance of multi-domain locomotion in real environments. The preliminary exploration in this letter provides a referential solution for the miniaturization of the highly maneuverable HAUVs for practical applications and creates a feasible platform for the future clustering and networking of HAUVs.

Keywords: Marine Robotics, Legged Robots, Motion Control
Abstract: Most of the existing underwater legged robots are capable of moving on small-angled slopes, but few of them can climb the large-angled slope or transition from one plane to another, such as transition from horizontal plane to vertical plane. In this paper, we propose a motion planning method of a hybrid underwater hexapod robot (HUHR) driven by six C-shape legs and eight thrusters. By analyzing the relationship between rotation and displacement of the hip joint, we establish a single-leg kinematic model. By analyzing the force at the touchpoint, we propose a locomotion mechanism to ensure no slip of the C-shape leg. Based on the central pattern generator (CPG) and tripod gait, we design an aperiodic mapping between the oscillator outputs and the desired rotation angles of hip joints. Overall, a gait planning and control method for our robot is proposed to realize continuous legged locomotion from one plane to another, including directional climbing and transition between them. Furthermore, the effectiveness of the proposed method has been verified on HUHR.

Keywords: Marine Robotics, Sensor Fusion, SLAM, Commercial off-the-shelf (COTS) systems
Abstract: Accurate, self-consistent bathymetric maps are needed to monitor changes in subsea environments and infrastructure. These maps are increasingly collected by underwater vehicles, and mapping requires an accurate vehicle navigation solution. Commercial off-the-shelf (COTS) navigation solutions for underwater vehicles often rely on external acoustic sensors for localization, however survey-grade acoustic sensors are expensive to deploy and limit the range of the vehicle. Techniques from the field of simultaneous localization and mapping, particularly loop closures, can improve the quality of the navigation solution over dead-reckoning, but are difficult to integrate into COTS navigation systems. This work presents a method to improve the self-consistency of bathymetric maps by smoothly integrating loop-closure measurements into the state estimate produced by a commercial subsea navigation system. Integration is done using a white-noise-on-acceleration motion prior, without access to raw sensor measurements or proprietary models. Improvements in map self-consistency are shown for both simulated and experimental datasets, including a 3D scan of an underwater shipwreck in Wiarton, Ontario, Canada.

Keywords: Marine Robotics, Localization, Autonomous Vehicle Navigation
Abstract: Underwater positioning is critical to autonomous underwater vehicles (AUVs) for navigation and geo-referencing. The rapid attenuation of the electromagnetic wave in the underwater environment prevents the use of traditional positioning methods such as the Global Positioning System, whereupon acoustic methods like ultra-short baseline (USBL) positioning systems play an important role in AUV navigation. However, the high cost and complexity of classical USBL systems have stifled the democratization of these technologies, which leads to a new method called passive inverted ultra-short baseline (piUSBL) positioning. In a typical piUSBL system, a single beacon is placed at a reference point, periodically broadcasting a positioning signal. A passive USBL receiver, time-synchronized to the beacon, is mounted on an AUV to get one-way travel-time (OWTT) slant range and azimuth estimates. The passive nature of the receiver means the system is inexpensive, low-power, and lightweight. Particularly, the omnidirectional broadcasted signals offer a feasible solution for concurrent multi-AUV navigation. This letter demonstrates a full-stack design and development of a piUSBL positioning system, and presents evaluations of the accuracy and reliability of the system through a series of experiments. More significantly, a successful sea trial of a disc-shaped AUV outfitted with our piUSBL was conducted in the South China Sea.

Keywords: Field Robots, Tendon/Wire Mechanism, Cooperating Robots
Abstract: Reduced traction limits the ability of mobile robotic systems to resist or apply large external loads, such as tugging a massive payload. One simple and versatile solution is to wrap a tether around naturally occurring objects to leverage the capstan effect and create exponentially-amplified holding forces. Experiments show that an idealized capstan model explains force amplification experienced on common irregular outdoor objects – trees, rocks, posts. Robust to variable environmental conditions, this exponential amplification method can harness single or multiple capstan objects, either in series or in parallel with a team of robots. This adaptability allows for a range of potential configurations especially useful for when objects cannot be fully encircled or gripped. This versatility is demonstrated with teleoperated mobile platforms to (1) control the lowering and arrest of a payload, (2) to achieve planar control of a payload, and (3) to act as an anchor point for a more massive platform to winch towards. We show the simple addition of a tether, wrapped around shallow stones in sand, amplifies holding force of a low-traction platform by up to 774x.

Keywords: Soft Robot Materials and Design, Soft Robot Applications, Mechanism Design
Abstract: Inflatable structures have attracted considerable research attention in many fields owing to their numerous advantages, such as being light and able to engage in interactions safely. However, in most cases, the inflatable structure can only have one stable configuration, which is undesirable for robotic arms. This study proposes a novel inflatable structure that can be easily reconfigured into multiple stable configurations, even with single-body inflation. In the proposed mechanism, the structure length can be freely adjusted, and its respective joints can be set in the desired directions to facilitate the reconfiguration of its pose. An additional advantage of the proposed mechanism is that it can withstand external forces as well as its own weight. This study analyzes and experimentally validates the shape locking and load-carrying properties of the proposed mechanism. Further, the fabrication process and design guidelines for the proposed mechanism are presented. Through a suitable demonstration, the proposed mechanism is shown to exhibit multiple stable configurations and lock its poses.

Keywords: Soft Robot Materials and Design, Tendon/Wire Mechanism, Hydraulic/Pneumatic Actuators
Abstract: Soft continuum robots have shown tremendous potential for medical and industrial applications owing to their flexibility and continuous deformability. However, their telescopic and bending capabilities and variable stiffness are still limited. This study proposes a novel origami-inspired soft continuum robot to possess large telescopic and bending capabilities while improving stiffness based on the principle of antagonistic actuation. The soft robot consists of dual origami structures. The inner forms an air chamber actuated by pneumatics, and the outer is controlled by nine tendon-driven actuators. The proposed design uses the advantages of a hybrid actuation to achieve motion and stiffness control. The performance of the soft robot is studied experimentally based on single and three robot modules. Results show that the robot has an excellent stretch ratio and a maximum bending angle of 180°. The robot can also increase stiffness to resist the bending deformation induced by self-weight and loads.

Keywords: Modeling, Control, and Learning for Soft Robots
Abstract: The Finite Element Method (FEM) is a powerful modeling tool for predicting the behavior of soft robots. However, its use for control can be difficult for non-specialists of numerical computation: it requires an optimization of the computation to make it real-time. In this paper, we propose a learning-based approach to obtain a compact but sufficiently rich mechanical representation. Our choice is based on non- linear compliance data in the actuator/effector space provided by a condensation of the FEM model. We demonstrate that this compact model can be learned with a reasonable amount of data and, at the same time, be very efficient in terms of modeling, since we can deduce the direct and inverse kinematics of the robot. We also show how to couple some models learned individually in particular on an example of a gripper composed of two soft fingers. Other results are shown by comparing the inverse model derived from the full FEM model and the one from the compact learned version. This work opens new perspectives, namely for the embedded control of soft robots, but also for their design. These perspectives are also discussed in the paper.

Keywords: Modeling, Control, and Learning for Soft Robots, Hydraulic/Pneumatic Actuators, Neural and Fuzzy Control
Abstract: One of the recent developments in physical reservoir computing, which uses the complex dynamics of a physical system as a computational resource, is the use of a pneumatic pipeline system as a computational resource. This uses the dynamics of air for computation, and because it is lightweight and power-saving, it is used for gait-assist control using a soft exoskeleton with pneumatic rubber artificial muscles. In this study, we verified that by feeding back the estimated information to a pneumatic pipeline system, the pneumatic physical reservoir computing can generate periodic pressure changes as a stable limit cycle, such as those seen in walking. A pneumatic reservoir with feedback loops was modeled to generate limit cycles in the simulation, and it was confirmed that the system could generate limit cycles with high accuracy even from initial positions far from the target limit cycle. This system is expected to be applied to assist walking movements with a soft exoskeleton with a lightweight computational device.

Keywords: Modeling, Control, and Learning for Soft Robots, Deep Learning for Visual Perception, Soft Robot Materials and Design
Abstract: Pneumatic soft robots present many advantages in manipulation tasks. Notably, their inherent compliance makes them safe and reliable in unstructured and fragile environments. However, full-body shape sensing for pneumatic soft robots is challenging because of their high degrees of freedom and complex deformation behaviors. Vision-based proprioception sensing methods relying on embedded cameras and deep learning provide a good solution to proprioception sensing by extracting the full-body shape information from the high-dimensional sensing data. But the current training data collection process makes it difficult for many applications. To address this challenge, we propose and demonstrate a robust sim-to-real pipeline that allows the collection of the soft robot's shape information in high-fidelity point cloud representation. The model trained on simulated data was evaluated with real internal camera images. The results show that the model performed with averaged Chamfer distance of 8.85 mm and tip position error of 10.12 mm even with external perturbation for a pneumatic soft robot with a length of 100.0 mm. We also demonstrated the sim-to-real pipeline’s potential for exploring different configurations of visual patterns to improve vision-based reconstruction results. The code and dataset are available at https://github.com/DeepSoRo/DeepSoRoSim2Real.

Keywords: Modeling, Control, and Learning for Soft Robots, Transfer Learning, Probabilistic Inference
Abstract: Recently, data-driven models such as deep neural networks have shown to be promising tools for modelling and state inference in soft robots. However, voluminous amounts of data are necessary for deep models to perform effectively, which requires exhaustive and quality data collection, particularly of state labels. Consequently, obtaining labelled state data for soft robotic systems is challenged for various reasons, including difficulty in the sensorization of soft robots and the inconvenience of collecting data in unstructured environments. To address this challenge, in this paper, we propose a semi-supervised sequential variational Bayes (DSVB) framework for transfer learning and state inference in soft robots with missing state labels on certain robot configurations. Considering that soft robots may exhibit distinct dynamics under different robot configurations, a feature space transfer strategy is also incorporated to promote the adaptation of latent features across multiple configurations. Unlike existing transfer learning approaches, our proposed DSVB employs a recurrent neural network to model the nonlinear dynamics and temporal coherence in soft robot data. The proposed framework is validated on multiple setup configurations of a pneumatic-based soft robot finger. Experimental results on four transfer scenarios demonstrate that DSVB performs effective transfer learning and accurate state inference amidst missing state labels.

Keywords: Computer Vision for Automation, Modeling, Control, and Learning for Soft Robots, Visual Tracking
Abstract: State estimation from measured data is crucial for robotic applications as autonomous systems rely on sensors to capture the motion and localize in the 3D world. Among sensors that are designed for measuring a robot's pose, or for soft robots, their shape, vision sensors are favorable because they are information-rich, easy to set up, and cost-effective. With recent advancements in computer vision, deep learning-based methods no longer require markers for identifying feature points on the robot. However, learning-based methods are data-hungry and hence not suitable for soft and prototyping robots, as building such bench-marking datasets is usually infeasible. In this work, we achieve image-based robot pose estimation and shape reconstruction from camera images. Our method requires no precise robot meshes, but rather utilizes a differentiable renderer and primitive shapes. It hence can be applied to robots for which CAD models might not be available or are crude. Our parameter estimation pipeline is fully differentiable. The robot shape and pose are estimated iteratively by back-propagating the image loss to update the parameters. We demonstrate that our method of using geometrical shape primitives can achieve high accuracy in shape reconstruction for a soft continuum robot and pose estimation for a robot manipulator.

Keywords: Modeling, Control, and Learning for Soft Robots, Underactuated Robots, Robust/Adaptive Control
Abstract: In this article we investigate the discrete-time model based control of a planar soft continuum manipulator with proprioceptive sensing provided by fiber Bragg gratings. A control algorithm is designed with a discrete-time energy shaping approach which is extended to account for control-related lag of digital nature. A discrete-time nonlinear observer is employed to estimate the uncertain bending stiffness of the manipulator and to compensate constant matched disturbances. Simulations and experiments demonstrate the effectiveness of the controller compared to a continuous time implementation.

Keywords: Soft Sensors and Actuators, Soft Robot Materials and Design, Soft Robot Applications
Abstract: Soft optical sensing strategies are rapidly developing for soft robotic systems as a means to increase the controllability of soft compliant robots. In this paper, we present a roughness tuning strategy for the fabrication of soft optical sensors to achieve the dual functionality of shape sensing combined with contact recognition within a single multi-modal sensor. The molds used to fabricate the soft sensors are roughened via laser micromachining to achieve asymmetrical sensor responses when bent in opposite directions. We demonstrate the integration of these sensors into a fully soft robotic platform consisting of a multi-directional bending module with integrated 3D shape sensing and a gripper with tip position monitoring along with contact force recognition. We show the accuracy of our sensing strategy in validation experiments and a pick-andplace task is performed to demonstrate the robot’s functionality.

Keywords: Soft Sensors and Actuators, Surgical Robotics: Steerable Catheters/Needles, Medical Robots and Systems
Abstract: Following biology’s lead, soft robotics has emerged as a perfect candidate for actuation within complex environments. While soft actuation has been developed intensively over the last few decades, soft sensing has so far slowed to catch up. A largely unresearched area is the change of the soft material properties through prestress to achieve a degree of mechanical sensitivity tunability within soft sensors. Here, a new 3D force sensor which employs novel hydraulic filament artificial muscles capable of sensitivity tunability is introduced. Using a neural network (NN) model, the new soft 3D sensor can precisely detect external forces based on the change of the hydraulic pressures with error of ~1.0, ~1.3, and ~0.94 % in the x, y, and z-axis directions, respectively. The sensor is also able to sense large force ranges, comparable to other similar sensors available in the literature. The sensor is then integrated into a soft robotic surgical arm for monitoring the tool-tissue interaction during the ablation process.

Keywords: Soft Sensors and Actuators, Soft Robot Materials and Design, Soft Robot Applications
Abstract: Vibration perception is essential for robotic sensing and dynamic control. Nevertheless, due to the rigorous demand for sensor conformability and stretchability, enabling soft robots with proprioceptive vibration sensing remains challenging. This paper proposes a new liquid metal-based stretchable e-skin via a kirigami-inspired design to enable soft robot proprioceptive vibration sensing. The e-skin is fabricated into 0.1mm ultrathin thickness, ensuring its negligible influence on the overall stiffness of the soft robot. Moreover, the working mechanism of the e-skin is based on the ubiquitous triboelectrification effect, which transduces mechanical stimuli without external power supply. To demonstrate the practicability of the e-skin, we built a soft gripper consisting of three soft robotic fingers with proprioceptive vibration sensing. Our experiment shows that the gripper can accurately distinguish the grain category (six grains with the same mass, 99.9% accuracy) and the packaging quality (100% accuracy) by simply shaking the gripped bottle. In summary, a soft robotic proprioceptive vibration sensing solution is proposed; it helps soft robots to have a more comprehensive awareness of their self-state and may inspire further research on soft robotics.

Keywords: Soft Sensors and Actuators, Modeling, Control, and Learning for Soft Robots, Soft Robot Materials and Design
Abstract: As soft robotic systems become increasingly complex, there is a need to develop sensory systems which can provide rich state information to the robot for feedback control. Multi-axis force sensing and control is one of the less explored problems in this domain. There are numerous challenges in the development of a multi-axis soft sensor: from the design and fabrication to the data processing and modelling. This work presents the design and development of a novel multi-axis soft sensor using a gelatin-based ionic hydrogel and 3D printing technology. A learning-based modelling approach coupled with sensor redundancy is developed to model the environmentally dependent soft sensors. Numerous real-time experiments are conducted to test the performance of the sensor and its applicability in closed-loop control tasks. Our results indicate that the soft sensor can predict force values and orientation angle within 4% and 7% of their total range, respectively.

Keywords: Soft Sensors and Actuators, Wearable Robotics
Abstract: Soft, wearable systems hold promise for a wide variety of new or enhanced applications in the realm of human-computer interaction, physiological monitoring, wearable robotics, and a host of other human-centric devices. Soft sensor systems have been developed concurrently in order to allow these wearable systems to respond intelligently with their surroundings. A recently reported sensing mechanism based on the strain-mediated contact in anisotropically resistive structures (SCARS) is an attractive solution due to its high sensing resolution, low-profile nature, and high mechanical resilience. Furthermore, the resistance-based output provides a simple electronic readout, facilitating its use in a wide variety of applications. However, previous iterations of the sensing mechanism have exhibited stress relaxation and hysteretic behaviors that limit the scope of its use. Here, we report an iteration of the SCARS mechanism that uses silicone-based materials with low mechanical loss in order to improve the sensor signal stability and bandwidth. A new fabrication approach is developed which permits the incorporation of a liquid elastomer adhesive layer while also preserving the SCARS sensing functionality. The silicone-based SCARS sensors exhibited fast stress relaxation response (< 1 s) and reduced cyclic drift properties by more than half that of previously reported designs. A physiological monitoring demonstration is presented, validating that the new sensor design is mechanically resilient to such applications and has potential for use in real-world wearable use cases.

Keywords: Biologically-Inspired Robots, Force and Tactile Sensing, Soft Sensors and Actuators
Abstract: Whisker-based tactile sensors have the potential to perform fast and accurate 3D mappings of the environment, complementing vision-based methods under conditions of glare, reflection, proximity, and occlusion. However, current algorithms for mapping with whiskers make assumptions about the conditions of contact, and these assumptions are not always valid and can cause significant sensing errors. Here we introduce a new whisker sensing system with a tapered, flexible whisker. The system provides inputs to two separate algorithms for estimating radial contact distance on a whisker. Using a Gradient-Moment (GM) algorithm, we correctly detect contact distance in most cases (within 4% of the whisker length). We introduce the Z-Dissimilarity score as a new metric that quantifies uncertainty in the radial contact distance estimate using both the GM algorithm and a Moment-Force (MF) algorithm that exploits the tapered whisker design. Combining the two algorithms ultimately results in contact distance estimates more robust than either algorithm alone.

Keywords: Soft Sensors and Actuators, Modeling, Control, and Learning for Soft Robots, Machine Learning for Robot Control
Abstract: Distributed sensor arrays capable of detecting multiple spatially distributed stimuli are considered an important element in the realisation of exteroceptive and proprioceptive soft robots. This paper expands upon the previously presented idea of decoupling the measurements of pressure and location of a local indentation from global deformation, using the overall stretch experienced by a soft capacitive e-skin. We employed machine learning methods to decouple and predict these highly coupled deformation stimuli, collecting data from a soft sensor e-skin which was then fed to a machine learning system comprising of linear regressor, gaussian process regressor, SVM and random forest classifier for stretch, force, detection and localisation respectively. We also studied how the localisation and forces are affected when two forces are applied simultaneously. Soft sensor arrays aided by appropriately chosen machine learning techniques can pave the way to e-skins capable of deciphering multi-modal stimuli in soft robots.

Keywords: Soft Sensors and Actuators, Software-Hardware Integration for Robot Systems, Soft Robot Applications
Abstract: This paper presents the novel use of air gaps in flexible optical light pipes to create coded segments for use in bend localization. The OptiGap sensor system allows for the creation of extrinsic intensity modulated bend sensors that function as flexible absolute linear encoders. Coded segment patterns are identified by a Gaussian naive Bayes classifier running on an STM32 microcontroller. Fitting of the classifier is aided by a custom software suite that simplifies data collection and processing from the sensor. The sensor model is analyzed and verified through simulation and experiments, highlighting key properties and parameters that aid in the design of OptiGap sensors using different light pipe materials and for various applications. This system allows for realtime and accurate bend localization in many robotics and automation applications, in wet and dry conditions.

Keywords: Soft Robot Materials and Design, Soft Robot Applications, Modeling, Control, and Learning for Soft Robots
Abstract: Soft devices employ variable stiffness to ensure safety and improve the robustness in the interaction between robots and objects. Using soft materials is one of the most popular approaches to design a variable-stiffness device, while the use of silicone sponge remains less explored in this field. Here we present a novel silicone-sponge-based variable-stiffness device (SVD). The SVD is easy-to-make and low-cost, and fabricated by an air-tight bellow enclosing a silicone sponge core. This allows easy access to the hyper-elastic response of the porous sponge whilst stiffness tuning of the device via pneumatic pressure difference. A detailed mathematical model of the SVD is proposed, by which the stiffness can be precisely controlled by the pressure difference applied. The stiffness of SVD can be tuned in the range of [1.55, 22.82]×10^3 N/m, up to 14.7 times increase. The high stiffness is easily triggered by a low pressure difference (ΔP < 12 kPa). The SVD is a versatile and compact module, with small axial size (10 mm height) and light weight (14.3 g), making it highly suitable for integration in a wide range of robotics and industrial applications. This, in addition to its easy-to-fabricate and low-cost features, may appeal to the robotics community at large. We further detail its working principle, fabrication processes, mathematical model and automated control methods to show its versatility.

Keywords: Soft Sensors and Actuators, Compliance and Impedance Control, Modeling, Control, and Learning for Soft Robots
Abstract: We propose a novel design for a lightweight and compact tunable stiffness actuator capable of stiffness changes up to 20x. The design is based on the concept of a coiled spring, where changes in the number of layers in the spring change the bulk stiffness in a near linear fashion. We present an elastica nested rings model for the deformation of the proposed actuator and empirically verify that the designed stiffness-changing spring abides by this model. Using the resulting model, we design a physical prototype of the tunable-stiffness coiled-spring actuator and discuss the effect of design choices on the resulting achievable stiffness range and resolution. In the future, this actuator design could be useful in a wide variety of soft robotics applications, where fast, controllable, and local stiffness change is required over a large range of stiffnesses.

Keywords: Soft Robot Materials and Design, Modeling, Control, and Learning for Soft Robots, Soft Sensors and Actuators
Abstract: Electrostatic actuators provide a promising approach to creating soft robotic sheets, due to their flexible form factor, modular integration, and fast response speed. However, their control requires kilo-Volt signals and understanding of complex dynamics resulting from force interactions by on-board and environmental effects. In this work, we demonstrate an untethered planar five-actuator piezoelectric robot powered by batteries and on-board high-voltage circuitry, and controlled through a wireless link. The scalable fabrication approach is based on bonding different functional layers on top of each other (steel foil substrate, actuators, flexible electronics). The robot exhibits a range of controllable motions, including bidirectional crawling (up to ~0.6 cm/s), turning, and in-place rotation (at ~1 degree/s). High-speed videos and control experiments show that the richness of the motion results from the interaction of an asymmetric mass distribution in the robot and the associated dependence of the dynamics on the driving frequency of the piezoelectrics. The robot's speed can reach 6 cm/s with specific payload distribution.

Keywords: Modeling, Control, and Learning for Soft Robots, Soft Robot Materials and Design, Soft Sensors and Actuators
Abstract: Soft robots have long been attractive to robotic engineers due to their remarkable dexterity; however, reports that standardize soft actuators into modularized off-shelf devices akin to rigid robots are still rare, and the mechanical efficiency of existing designs is still limited. This work identifies origami folding to enable the design of LEGO-like modularized soft actuators with high mechanical efficiency in terms of payload capability and workspace. Herein, three modularized origami actuators that can generate translational, bending, and twisting motion are designed, prototyped, and tested. The translational actuator can contract to 40% of its original length, and the twisting and bending actuators can exert 31° and 52° angular motions, respectively. The translational actuator can exert a blocked force of about 821 times self-weight. The motion of origami soft actuators is accurately modeled using rigid body kinematics, and complex systems built by them are captured by homogeneous transformation. Finally, the modularized design and efficient kinematic model are verified on a manipulator and a reconfigurable letter. Benefiting from the unprecedented modularity and mechanical efficiency, these LEGO-like origami actuators are promising for practical applications like food handling and healthcare.

Keywords: Soft Robot Materials and Design, Soft Robot Applications, Soft Sensors and Actuators
Abstract: Soft robotic structures may play a major role in the 4th industrial revolution. Researchers have successfully demonstrated the advantages of soft robotics over traditional robots made of rigid links and joints in many application areas. Variable stiffness links (VSL) and joints (VSJ) have been investigated to achieve on-demand forces and, at the same time, be inherently safe in interactions with humans. However, a thorough characterisation of soft and rigid robotic components is still required.

This paper investigates the influence of antagonistically actuated, stiffness-controllable joint-link units (JLUs) on the performance of collaborative robots (i.e. stiffness, load capacity, repetitive precision) and characterizes the difference compared with rigid units. A JLU is made of a combination of a VSL, a VSJ, and their rigid counterparts. Experimental results show that the VSL has minor differences in terms of stiffness (0.62 ~ 0.95), output force (0.93 ~ 0.94), and repetitive precision compared with the rigid link. For the VSJ, our results show a significant gap compared with the servo motor with regards to maximum stiffness (0.14 ~ 0.21) and repetitive position precision (0.07 ~ 0.25). However, similar performance on repetitive force precision and better performance on the maximum output force (1.54 ~ 1.55 times) are demonstrated.

Keywords: Soft Sensors and Actuators, Soft Robot Materials and Design, Compliance and Impedance Control
Abstract: One of the biggest problems with soft robots is precisely the fact that they are soft. Indeed the softer they are, the less force they can exert on the environment. Researchers have proposed a number of stiffening methods, but all of them have drawbacks, such as locking the shape of the device in a way that precludes further adjustments. In this paper we propose a stiffening method inspired by the internal structure of the mammalian penis. The soft actuation chamber is divided into small compartments that trap the actuation fluid, leading to locally amplified pressure increase under certain conditions. At the same time, the proposed solution does not affect the actuation mechanism, allowing the actuator to be adjusted in one direction just as if it was in non-stiffened mode, while offering a stiff response in the opposite direction. Our prototype achieves an increase in stiffening of approximately a factor of two. The paper describes the concept, the mathematical justification of the working principle, the prototype design, its implementation and our experimental results.

Keywords: Modeling, Control, and Learning for Soft Robots, Tendon/Wire Mechanism
Abstract: This paper discusses the effect of axial backbone compression on tendon-driven continuum robots. A new mechanics model for compensating for this effect that does not require tendon tension sensing or knowledge of manipulator material properties/stiffnesses is introduced and analyzed. In addition, we provide an analytical expression for the minimum preload on the tendons to achieve a given bend, a quantity determined empirically thus far. Our model is computationally efficient and achieves real time control on low cost hardware. The analysis is supported by experimental results demonstrating significant improvement over kinematics in open loop control of a tendon-driven continuum hose robot.

Keywords: Computer Vision for Transportation, Deep Learning for Visual Perception, Sensor Fusion
Abstract: This article proposes a novel semi-supervised learning framework FourStr} (Four-Stream formed by two two-stream models) that focuses on the improvement of fusion and labeling efficiency for 3D multi-sensor detector. FourStr adopts a multi-sensor single-stage detector named adaptive fusion network (AFNet) as the backbone and trains it through the semi-supervision learning (SSL) strategy Stereo Fusion. Note that multi-sensor AFNet and SSL Stereo Fusion can benefit each other. On the one hand, the Four-stream composed of two AFNets naturally provides rich inputs and large models for SSL Stereo Fusion. While other SSL works have to use massive augmentation to obtain rich inputs, and deepen and widen the network for large models. On the other hand, by the novel three fusion stages and Loss Pruning, Stereo Fusion improves the fusion and labeling efficiency for AFNet. Finally, extensive experiments demonstrate that FourStr performs excellently on outdoor dataset (KITTI and Waymo Open Dataset) and indoor dataset (SUN RGB-D), especially for the small contour objects. And compared to the fully-supervised methods, FourStr achieves similar accuracy with only 2% labeled data on KITTI (or with 50% labeled data on SUN RGB-D).

Keywords: Object Detection, Segmentation and Categorization, Sensor Fusion, RGB-D Perception
Abstract: We present a robust method to visually segment scenes into objects based on motion and appearance. Both these cues provide complementary information that we fuse using two interconnected recursive estimators: One estimates object segmentation from motion as a probabilistic clustering of tracked 3D points, and the other estimates object segmentation from appearance as a probabilistic image segmentation. The interconnected estimators provide a probabilistic and consistent object segmentation in real time, which makes them well suited for many downstream robotic tasks. We evaluate our method on one such task, kinematic structure estimation, on a dataset of interactions with articulated objects and show that our fusion improves object segmentation by 70% and in turn estimated kinematic joints by 26% over a purely motion-based approach. Furthermore, we show the necessity of probabilistic modeling for downstream robotic tasks, achieving 339% of the performance of a recent multimodal but deterministic RNN for object segmentation on the estimation of kinematic structure.

Keywords: Object Detection, Segmentation and Categorization, Sensor Fusion, Visual Tracking
Abstract: Accurate and timely onboard perception is a prerequisite for mobile robots to operate in highly dynamic scenarios. The bio-inspired event camera can capture more motion details than a traditional camera by triggering each pixel asynchronously and therefore is more suitable in such scenarios. Among various perception tasks based on the event camera, ego-motion removal is one fundamental procedure to reduce perception ambiguities. Recent ego-motion removal methods are mainly based on optimization processes and may be computationally expensive for robot applications. In this paper, we consider the challenging perception task of detecting fast-moving objects from an aggressively operated platform equipped with an event camera, achieving computational cost reduction by directly employing IMU motion measurement. First, we design a nonlinear warping function to capture rotation information from an IMU and to compensate for the camera motion during an asynchronous events stream. The proposed nonlinear warping accuracy by 10%-15%. Afterward, we segmented the moving parts on the warped image through dynamic threshold segmentation and optical flow calculation, and clustering. Finally, we validate the proposed detection pipeline on public datasets and real-world data streams containing challenging light conditions and fast-moving objects.

Keywords: Robot Audition, Sensor Fusion
Abstract: Robots need to understand articulated objects, such as drawers. The state of articulated structures is commonly estimated using vision, but visual perception is limited when objects are occluded, have few salient features, or are not in the camera's field of view. Audio sensing does not face these challenges, since sound propagates in a fundamentally different way than light. Therefore we propose to fuse vision and audio sensing to overcome the challenges faced by vision alone. We estimate motion in several drawers and show that an audio-visual approach estimates drawer motion more reliably than only vision -- even in settings where the purely visual approach completely breaks down. Additionally, we perform an in-depth analysis of the regularities that govern how motion in drawers shapes their sound.

Keywords: Robot Audition, Sensor Fusion, Multi-Modal Perception for HRI
Abstract: Developing embodied agents in simulation has been a key research topic in recent years. Exciting new tasks, algorithms, and benchmarks have been developed in various simulators. However, most of them assume deaf agents in silent environments, while we humans perceive the world with multiple senses. We introduce Sonicverse, a multisensory simulation platform with integrated audio-visual simulation for training household agents that can both see and hear. Sonicverse models realistic continuous audio rendering in 3D environments in real-time. Together with a new audio-visual VR interface that allows humans to interact with agents with audio, Sonicverse enables a series of embodied AI tasks that need audio-visual perception. For semantic audio-visual navigation in particular, we also propose a new multi-task learning model that achieves state-of-the-art performance. In addition, we demonstrate Sonicverse's realism via sim-to-real transfer, which has not been achieved by other simulators: an agent trained in Sonicverse can successfully perform audio-visual navigation in real-world environments. Sonicverse is available at: https://github.com/StanfordVL/Sonicverse.

Keywords: Semantic Scene Understanding, Sensor Fusion, Autonomous Agents
Abstract: Semantic grids can be useful representations of the scene around an autonomous system. By having information about the layout of the space around itself, a robot can leverage this type of representation for crucial tasks such as navigation or tracking. By fusing information from multiple sensors, robustness can be increased and the computational load for the task can be lowered, achieving real time performance. Our multi-scale LiDAR-Aided Perspective Transform network uses information available in point clouds to guide the projection of image features to a top-view representation, resulting in a relative improvement in the state of the art for semantic grid generation for human (+8.67%) and movable object (+49.07%) classes in the nuScenes dataset, as well as achieving results close to the state of the art for the vehicle, drivable area and walkway classes, while performing inference at 25 FPS.

Keywords: Sensor Fusion
Abstract: In this paper, we address the problem of estimating the in-hand 6D pose of an object in contact with multiple vision-based tactile sensors. We reason on the possible spatial configurations of the sensors along the object surface. Specifically, we filter contact hypotheses using geometric reasoning and a Convolutional Neural Network (CNN), trained on simulated object-agnostic images, to promote those that better comply with the actual tactile images from the sensors. We use the selected sensors configurations to optimize over the space of 6D poses using a Gradient Descent-based approach. We finally rank the obtained poses by penalizing those that are in collision with the sensors. We carry out experiments in simulation using the DIGIT vision-based sensor with several objects, from the standard YCB model set. The results demonstrate that our approach estimates object poses that are compatible with actual object-sensor contacts in 87.5% of cases while reaching an average positional error in the order of 2 centimeters. Our analysis also includes qualitative results of experiments with a real DIGIT sensor.

Keywords: Sensor Fusion
Abstract: LiDAR-Camera online calibration is of great significance for building a stable autonomous driving perception system. For online calibration, a key challenge lies in constructing a unified and robust representation between multimodal sensor data. Most methods extract features manually or implicitly with an end-to-end deep learning method. The former suffers poor robustness, while the latter has poor interpretability. In this paper, we propose CalibDepth, which uses depth maps as the unified representation for image and LiDAR point cloud. CalibDepth introduces a sub-network for monocular depth estimation to assist online calibration tasks. To further improve the performance, we regard online calibration as a sequence prediction problem, and introduce global and local losses to optimize the calibration results. CalibDepth shows excellent performance in different experimental setups.

Keywords: Visual Servoing
Abstract: In this paper we propose a visual servoing approach that controls the deformation of a suspended tether cable subject to gravity from visual data provided by a RGB-D camera. The cable shape is modelled with a parabolic curve together with the orientation of the plane containing the tether. The visual features considered are the parabolic coefficients and the yaw angle of that plane. We derive the analytical expression of the interaction matrix that relates the variation of the visual features to the velocities of the cable extremities. Singularities are demonstrated to occur if and only if the cable is taut horizontally or vertically. An image processing algorithm is also developed to extract in real-time the current features fitting the parabola to the cable from the observed point cloud. Simulations and experimental results demonstrate the efficiency of our visual servoing approach to deform the tether cable toward a desired shape configuration.

Keywords: Visual Servoing
Abstract: We propose a new visual servoing method that controls a robot's motion in a latent space. We aim to extract the best properties of two previously proposed servoing methods: we seek to obtain the accuracy of photometric methods such as Direct Visual Servoing (DVS), as well as the behavior and convergence of pose-based visual servoing (PBVS). Photometric methods suffer from limited convergence area due to a highly non-linear cost function, while PBVS requires estimating the pose of the camera which may introduce some noise and incurs a loss of accuracy. Our approach relies on shaping (with metric learning) a latent space, in which the representations of camera poses and the embeddings of their respective images are tied together. By leveraging the multimodal aspect of this shared space, our control law minimizes the difference between latent image representations thanks to information obtained from a set of pose embeddings. Experiments in simulation and on a robot validate the strength of our approach, showing that the sought out benefits are effectively found.

Keywords: Visual Servoing, Dual Arm Manipulation, Deep Learning in Grasping and Manipulation
Abstract: This paper proposes CNN-based visual servoing for simultaneous positioning and flattening of a soft fabric part placed on a table by a dual manipulator system. We propose a network for multimodal data processing of grayscale images captured by a camera and force/torque applied to force sensors. The training dataset is collected by moving the real manipulators, which enables the network to map the captured images and force/torque to the manipulator’s motion in Cartesian space. We apply structured lighting to emphasize the features of the surface of the fabric part since the surface shape of the non-textured fabric part is difficult to recognize by a single grayscale image. Through experiments, we show that the fabric part with unseen wrinkles can be positioned and flattened by the proposed visual servoing scheme.

Keywords: Visual Servoing, Learning from Demonstration, Imitation Learning
Abstract: Nowadays ubiquitous robots must be adaptive and easy to use. To this end, dynamical system-based imitation learning plays an important role. In fact, it allows to realize stable and complex robotic tasks without explicitly coding them, thus facilitating the robot use. However, the adaptation capabilities of dynamical systems have not been fully exploited due to the lack of closed-loop implementations making use of visual feedback. In this regard, the integration of visual information allows higher flexibility to cope with environmental changes. This work presents a dynamical system-based imitation learning for visual servoing, based on the large projection task priority formulation. The proposed scheme enables complex and stable visual tasks, as demonstrated by a simulation analysis and experiments with a robotic manipulator.

Keywords: Visual Servoing, Motion Control, Parallel Robots
Abstract: Cable-Driven Parallel Robots (CDPRs) are well-adapted to large workspaces since they replace rigid links by cables. However, they lack in positioning accuracy and new control methods are necessary to achieve profile-following tasks. This paper presents a control scheme designed for these tasks, relying on a combination of accurate boarded distance sensors and of a less accurate remote camera. The profile-following task is divided into two subtasks that are partially conflicting: maintaining a parallel orientation and a constant distance with the surface to follow, and following a trajectory between two points on the surface. The data fusion to solve the redundancy is based on the Gradient Projection Method. This control scheme is validated experimentally on a CDPR prototype and shown to provide the expected behaviour.

Keywords: Visual Servoing, Sensor-based Control, Optimization and Optimal Control
Abstract: This paper presents a spectral domain registration-based visual servoing scheme that works on 3D point clouds. Specifically, we propose a 3D model/point cloud alignment method, which works by finding a global transformation between reference and target point clouds using spectral analysis. A 3D Fast Fourier Transformation (FFT) in R3 is used for the translation estimation, and the real spherical harmonics in SO(3) are used for the rotations estimation. Such an approach allows us to derive a decoupled 6 degrees of freedom (DoF) controller, where we use gradient ascent optimisation to minimise translation and rotational costs. We then show how this methodology can be used to regulate a robot arm to perform a positioning task. In contrast to the existing state-of-the-art depth-based visual servoing methods that either require dense depth maps or dense point clouds, our method works well with partial point clouds and can effectively handle larger transformations between the reference and the target positions. Furthermore, the use of spectral data (instead of spatial data) for transformation estimation makes our method robust to sensor-induced noise and partial occlusions. We validate our approach by performing experiments using point clouds acquired by a robot-mounted depth camera. Obtained results demonstrate the effectiveness of our visual servoing approach.

Keywords: Visual Servoing, Surgical Robotics: Laparoscopy, Medical Robots and Systems
Abstract: This paper presents a novel autonomous endoscope control method for the dVRK’s Endoscopic Camera Manipulator (ECM), which allows the camera to track the surgical instruments on the Patient Side Manipulator (PSM). An Image-based Visual Servoing (IBVS) is enforced by the addition of a visibility constraint that ensures the identified surgical tool remains in the camera’s Field Of View (FOV) for the continued availability of image feedback and a joint limits avoidance constraint that prevents the ECM from exceeding its joint limits. The work relies on an optimization approach, with constraints performed using the Control Barrier Functions concept (CBFs). The goal is to minimize the surgeon’s cognitive and physical workload by removing the time-consuming job of camera reorientation, offering an enforced method compared to the traditional IBVS endoscopic camera controller.

Keywords: Machine Learning for Robot Control, Vision-Based Navigation, RGB-D Perception
Abstract: Safe motion control in unknown environments is one of the challenging tasks in robotics, such as autonomous navigation. Control Barrier Function (CBF), as a strong mathematical tool, has been widely used in many safety-critical systems to satisfy safety requirements. However, there are only a handful of recent studies on safety controllers with perception inputs. Common assumptions in most of the works are that the CBF is already known and obstacles have predefined shapes. In this work, we introduce a novel Vision-based Control Barrier Function (V-CBF), which enables generalization to new environments and obstacles of arbitrary shapes. We then derive CBF safety conditions over RGB-D space and relate those to actual robot control inputs. To train the CBF function, we introduce a method to generate ground truth with desired properties complying with CBF and a method to generate part of the CBF as an image-to-image translation problem. We finally demonstrate the efficacy of V-CBF on the safe control of an autonomous car in CARLA simulator.

Keywords: Visual Tracking, Aerial Systems: Applications, AI-Based Methods
Abstract: In this paper, we propose a multi-object tracking framework for videos captured by UAVs, considering motion imperfection in the following two aspects: 1) motion blurring of objects due to high-speed motion of the UAV and the objects, deteriorating the performance of the detector; 2) motion coupling of the global movement of the UAV camera with the object motion, resulting in the objects trajectory in adjacent frames more difficult to predict. For motion blurring, this paper proposes a hybrid deblurring module that deals with the blurred frames while retaining the clear frames, trading off between video tracking performance and spatio-temporal consistency. For motion coupling, we proposed a motion compensation module to align adjacent frames by feature matching, and the corrected target position is obtained in the next frame to alleviate the interference of camera movement with tracking. We evaluate the proposed methods on VisDrone dataset and validate that our framework achieves new state-of-the-art performance on UAV-based MOT systems.

Keywords: Hardware-Software Integration in Robotics, Computer Vision for Automation, Visual-Inertial SLAM
Abstract: Motion deblurring is a critical ill-posed problem that is important in many vision-based robotics applications. The recently proposed event-based double integral (EDI) provides a theoretical framework for solving the deblurring problem with the event camera and generating clear images at high frame-rate. However, the original EDI is mainly designed for offline computation and does not support real-time requirement in many robotics applications. In this paper, we propose the fast EDI, an efficient implementation of EDI that can achieve real-time online computation on single-core CPU devices, which is common for physical robotic platforms used in practice. In experiments, our method can handle event rates at as high as 13 million event per second in a wide variety of challenging lighting conditions. We demonstrate the benefit on multiple downstream real-time applications, including localization, visual tag detection, and feature matching.

Keywords: Visual Tracking, Computer Vision for Automation, SLAM
Abstract: This work addresses the issue of motion compensation and pattern tracking in event camera data. An event camera generates asynchronous streams of events triggered independently by each of the pixels upon changes in the observed intensity. Providing great advantages in low-light and rapid-motion scenarios, such unconventional data present significant research challenges as traditional vision algorithms are not directly applicable to this sensing modality. The proposed method decomposes the tracking problem into a local SE(2) motion-compensation step followed by a homography registration of small motion-compensated event batches. The first component relies on Gaussian Process (GP) theory to model the continuous occupancy field of the events in the image plane and embed the camera trajectory in the covariance kernel function. In doing so, estimating the trajectory is done similarly to GP hyperparameter learning by maximising the log marginal likelihood of the data. The continuous occupancy fields are turned into distance fields and used as templates for homography-based registration. By benchmarking the proposed method against other state-of-the-art techniques, we show that our open-source implementation performs high-accuracy motion compensation and produces high-quality tracks in real-world scenarios.

Keywords: Visual Tracking, Deep Learning for Visual Perception, Data Sets for Robotic Vision
Abstract: Current robotic hand manipulation narrowly operates with objects in predictable positions in limited environments. Thus, when the location of the target object deviates severely from the expected location, a robot sometimes responds in an unexpected way, especially when it operates with a human. For safe robot operation, we propose the EXit-aware Object Tracker (EXOT) on a robot hand camera that recognizes an object's absence during manipulation. The robot decides whether to proceed by examining the tracker's bounding box output containing the target object. We adopt an out-of-distribution classifier for more accurate object recognition since trackers can mistrack a background as a target object. To the best of our knowledge, our method is the first approach of applying an out-of-distribution classification technique to a tracker output. We evaluate our method on the first-person video benchmark dataset, TREK-150, and on the custom dataset, RMOT-223, that we collect from the UR5e robot. Then we test our tracker on the UR5e robot in real-time with a conveyor-belt sushi task, to examine the tracker's ability to track target dishes and to determine the exit status. Our tracker shows 38% higher exit-aware performance than a baseline method. The dataset and the code will be released at https://github.com/hskAlena/EXOT.

Keywords: Visual Tracking, RGB-D Perception
Abstract: We present Mono-STAR, the first real-time RGB-D 3D reconstruction system that simultaneously supports semantic fusion, fast motion tracking, non-rigid object deformation, and topological change under a unified framework. The proposed system solves a new optimization problem incorporating optical-flow-based 2D constraints to deal with fast motion and a novel semantic-aware deformation graph (SAD-graph) for handling topology change. We test the proposed system under various challenging scenes and demonstrate that it significantly outperforms existing state-of-the-art methods.

Keywords: Visual Tracking, Sensor Fusion, Localization
Abstract: Persistent multi-object tracking (MOT) allows autonomous vehicles to navigate safely in highly dynamic environments. One of the well-known challenges in MOT is object occlusion when an object becomes unobservant for subsequent frames. The current MOT methods store objects information, like objects' trajectory, in internal memory to recover the objects after occlusions. However, they retain short-term memory to save computational time and avoid slowing down the MOT method. As a result, they lose track of objects in some occlusion scenarios, particularly long ones. In this paper, we propose DFR-FastMOT, a light MOT method that uses data from a camera and LiDAR sensors and relies on an algebraic formulation for object association and fusion. The formulation boosts the computational time and permits long-term memory that tackles more occlusion scenarios. Our method shows outstanding tracking performance over recent learning and non-learning benchmarks with about 3% and 4% margin in MOTA, respectively. Also, we conduct extensive experiments that simulate occlusion phenomena by employing detectors with various distortion levels. The proposed solution enables superior performance under various distortion levels in detection over current state-of-art methods. Our framework processes about 7,763 frames in 1.48 seconds, which is seven times faster than recent benchmarks. The framework will be available at https://github.com/MohamedNagyMostafa/DFR-FastMOT.

Keywords: Visual Tracking, Vision-Based Navigation, Localization
Abstract: Event cameras are asynchronous neuromorphic vision sensors with high temporal resolution and no motion blur, offering advantages over standard frame-based cameras especially in high-speed motions and high dynamic range conditions. However, event cameras are unable to capture the overall context of the scene, and produce different events for the same scenery depending on the direction of the motion, creating a challenge in data association. Standard camera, on the other hand, provides frames at a fixed rate that are independent of the motion direction, and are rich in context. In this paper, we present a robust feature tracking method that employs 8-DOF warping model in minimizing the difference between brightness increment patches from events and frames, exploiting the complementary nature of the two data types. Unlike previous works, the proposed method enables tracking of features under complex motions accompanying distortions. Extensive quantitative evaluation over publicly available datasets was performed where our method shows an improvement over state-of-the-art methods in robustness with greatly prolonged feature age and in accuracy for challenging scenarios.

Keywords: Visual Tracking, Autonomous Vehicle Navigation, Human Detection and Tracking
Abstract: We propose a method for joint detection and tracking of multiple objects in 3D point clouds, a task conventionally treated as a two-step process comprising object detection followed by data association. Our method embeds both steps into a single end-to-end trainable network eliminating the dependency on external object detectors. Our model exploits temporal information employing multiple frames to detect objects and track them in a single network, thereby making it a utilitarian formulation for real-world scenarios. Computing affinity matrix by employing features similarity across consecutive point cloud scans forms an integral part of visual tracking. We propose an attention-based refinement module to refine the affinity matrix by suppressing erroneous correspondences. The module is designed to capture the global context in affinity matrix by employing self-attention within each affinity matrix and cross-attention across a pair of affinity matrices. Unlike competing approaches, our network does not require complex post-processing algorithms, and processes raw LiDAR frames to directly output tracking results. We demonstrate the effectiveness of our method on the three tracking benchmarks: JRDB, Waymo, and KITTI. Experimental evaluations indicate the ability of our model to generalize well across datasets.

Keywords: Learning from Demonstration, Intelligent and Flexible Manufacturing, Task Planning
Abstract: In this work, we present an inverse reinforcement learning approach for solving the problem of task sequencing for robots in complex manufacturing processes. Our proposed framework is adaptable to variations in process and can perform sequencing for entirely new parts. We prescribe an approach to capture feature interactions in a demonstration dataset based on a metric that computes feature interaction coverage. We then actively learn the expert's policy by keeping the expert in the loop. Our training and testing results reveal that our model can successfully learn the expert's policy. We demonstrate the performance of our method on a real-world manufacturing application where we transfer the policy for task sequencing to a manipulator. Our experiments show that the robot can perform these tasks to produce human-competitive performance. Code and video can be found at: https://sites.google.com/usc.edu/irlfortasksequencing

Keywords: Representation Learning, Learning from Demonstration, Telerobotics and Teleoperation
Abstract: Identifying an appropriate task space can simplify solving robotic manipulation problems. One solution is deploying control algorithms in a learned low-dimensional action space. Linear and nonlinear action mapping methods have trade-offs between simplicity and the ability to express motor commands outside of a single low-dimensional subspace. We propose that learning local linear action representations can achieve both of these benefits. Our state-conditioned linear maps ensure that for any given state, the high-dimensional robotic actuation is linear in the low-dimensional actions. As the robot state evolves, so do the action mappings, so that necessary motions can be performed during a task. These local linear representations guarantee desirable theoretical properties by design. We validate these findings empirically through two user studies. Results suggest state-conditioned linear maps outperform conditional autoencoder and PCA baselines on a pick-and-place task and perform comparably to mode switching in a more complex pouring task.

Keywords: Machine Learning for Robot Control, Reinforcement Learning, Deep Learning Methods
Abstract: Recent works in robotic manipulation through reinforcement learning (RL) or imitation learning (IL) have shown potential for tackling a range of tasks e.g., opening a drawer or a cupboard. However, these techniques generalize poorly to unseen objects. We conjecture that this is due to the high-dimensional action space for joint control. In this paper, we take an alternative approach and separate the task of learning 'what to do' from 'how to do it' i.e., whole-body control. We pose the RL problem as one of determining the skill dynamics for a disembodied virtual manipulator interacting with articulated objects. The whole-body robotic kinematic control is optimized to execute the high-dimensional joint motion to reach the goals in the workspace. It does so by solving a quadratic programming (QP) model with robotic singularity and kinematic constraints. Our experiments on manipulating complex articulated objects show that the proposed approach is more generalizable to unseen objects with large intra-class variations, outperforming previous approaches. The evaluation results indicate that our approach generates more compliant robotic motion and outperforms the pure RL and IL baselines in task success rates. Additional information and videos are available at https://kl-research.github.io/decoupskill.

Keywords: Reinforcement Learning, Machine Learning for Robot Control
Abstract: Model Free Reinforcement Learning (MFRL) has shown significant promise for learning dexterous robotic manipulation tasks, at least in simulation. However, the high number of samples, as well as the long training times, prevent MFRL from scaling to complex real-world tasks. Model-Based Reinforcement Learning (MBRL) emerges as a potential solution that, in theory, can improve the data efficiency of MFRL approaches. This could drastically reduce the training time of MFRL, and increase the application of RL for real-world robotic tasks. This article presents a study on the feasibility of using the state-of-the-art MBRL to improve the training time for two real-world dexterous manipulation tasks. The evaluation is conducted on a real low-cost robot gripper where the predictive model and the control policy are learned from scratch. The results indicate that MBRL is capable of learning accurate models of the world, but does not show clear improvements in learning the control policy in the real world as prior literature suggests should be expected.

Keywords: Machine Learning for Robot Control, Reinforcement Learning, Optimization and Optimal Control
Abstract: Reinforcement learning (RL) has recently proven great success in various domains. Yet, the design of the reward function requires detailed domain expertise and tedious fine-tuning to ensure that agents are able to learn the desired behaviour. Using a sparse reward conveniently mitigates these challenges. However, the sparse reward represents a challenge on its own, often resulting in unsuccessful training of the agent. In this paper, we therefore address the sparse reward problem in RL. Our goal is to find an effective alternative to reward shaping, without using costly human demonstrations, that would also be applicable to a wide range of domains. Hence, we propose to use model predictive control MPC as an experience source for training RL agents in sparse reward environments. Without the need for reward shaping, we successfully apply our approach in the field of mobile robot navigation both in simulation and real-world experiments with a Kuboki Turtlebot 2. We furthermore demonstrate great improvement over pure RL algorithms in terms of success rate as well as number of collisions and timeouts. Our experiments show that MPC as an experience source improves the agent's learning process for a given task in the case of sparse rewards.

Keywords: Reinforcement Learning, Machine Learning for Robot Control, Vision-Based Navigation
Abstract: Embodied AI agents in large scenes often need to navigate to find objects. In this work, we study a naturally emerging variant of the object navigation task, hierarchical relational object navigation (HRON), where the goal is to find objects specified by logical predicates organized in a hierarchical structure—objects related to furniture and then to rooms—such as finding an apple on top of a table in the kitchen. Solving such a task requires an efficient representation to reason about object relations and correlate the relations in the environment and in the task goal. HRON in large scenes (e.g. homes) is particularly challenging due to its partial observability and long horizon, which invites solutions that can compactly store the past information while effectively exploring the scene. We demonstrate experimentally that scene graphs are the best-suited representation compared to conventional representations such as images or 2D maps. We propose a solution that uses scene graphs as part of its input and integrates graph neural networks as its backbone, with an integrated task-driven attention mechanism, and demonstrate better scalability and learning efficiency than state-of-the-art baselines.

Keywords: Machine Learning for Robot Control, Reinforcement Learning, Deep Learning Methods
Abstract: Programming manipulation behaviors can become increasingly difficult with a growing number and complexity of manipulation tasks, particularly in a dynamic and unstructured environment. Recent progress in unsupervised skill discovery algorithms has shown great promise in learning an extensive collection of behaviors without extrinsic supervision. On the other hand, safety is one of the most critical factors for real-world robot applications. As skill discovery methods typically encourage exploratory and dynamic behaviors, it can often be the case that a large portion of learned skills remain too dangerous and unsafe. In this paper, we introduce the novel problem of Safety-Aware Skill Discovery, which aims to learn, in a task-agnostic fashion, a repertoire of reusable skills that are inherently safe to be composed for solving downstream tasks. We present a computationally tractable algorithm that learns a latent-conditioned skill policy that maximizes intrinsic rewards regularized with a safety-critic that can model any user-defined safety constraints. Using the pretrained safe skill repertoire, hierarchical reinforcement learning can solve multiple downstream tasks without the need for explicit consideration of safety during training and testing. We evaluate our algorithm on a collection of force-controlled robotic manipulation tasks in simulation and show promising downstream task performance while satisfying safety constraints.

Keywords: Formal Methods in Robotics and Automation, Software, Middleware and Programming Environments, Software Tools for Robot Programming
Abstract: The spatial distribution of sensed objects strongly influences the behavior of mobile robots. Yet, as robots evolve in complexity to operate in increasingly rich environments, it becomes much more difficult to specify the underlying relations between sensed object spatial distributions and robot behaviors. We aim to address this challenge by leveraging system trace data to automatically infer relations that help to better characterize these spatial associations. In particular, we introduce SpRInG, a framework for the unsupervised inference of system specifications from traces that characterize the spatial relationships under which a robot operates. Our method builds on a parameterizable notion of reachability to encode relationships of spatial neighborship, which are used to instantiate a language of patterns. These patterns provide the structure to infer, from system traces, the connection between such relationships and robot behaviors. We show that SpRInG can automatically infer spatial relations over two distinct domains: autonomous vehicles in traffic and a surgical robot. Our results demonstrate the power and expressiveness of SpRInG, in its ability to learn existing specifications as machine-checkable first-order logic, uncover previously unstated specifications that are rich and insightful, and reveal contextual differences between executions.

Keywords: Computer Vision for Automation, Imitation Learning, Machine Learning for Robot Control
Abstract: The ability to specify robot commands by a non-expert user is critical for building generalist agents capable of solving a large variety of tasks. One convenient way to specify the intended robot goal is by a video of a person demonstrating the target task. While prior work typically aims to imitate human demonstrations performed in robot environments, here we focus on a more realistic and challenging setup with demonstrations recorded in natural and diverse human environments. We propose Video-conditioned Policy learning (ViP), a data-driven approach that maps human demonstrations of previously unseen tasks to robot manipulation skills. To this end, we learn our policy to generate appropriate actions given current scene observations and a video of the target task. To encourage generalization to new tasks, we avoid particular tasks during training and learn our policy from unlabelled robot trajectories and corresponding robot videos. Both robot and human videos in our framework are represented by video embeddings pre-trained for human action recognition. At test time we first translate human videos to robot videos in the common video embedding space, and then use resulting embeddings to condition our policies. Notably, our approach enables robot control by human demonstrations in a zero-shot manner, i.e., without using robot trajectories paired with human instructions during training. We validate our approach on a set of challenging multi-task robot manipulation environments and outperform state of the art. Our method also demonstrates excellent performance in a new challenging zero-shot setup where no paired data is used during training.

Keywords: Agricultural Automation, Perception for Grasping and Manipulation, Domestic Robotics
Abstract: Food picking is trivial for humans but not for robots, as foods are fragile. Presetting foods' physical properties does not help robots much due to the objects' inter- and intra-category diversity. A recent study proved that learning-based fracture anticipation with tactile sensors could overcome this problem; however, the method trains the model for each food to deal with intra-category differences, and tuning robots for each food leads to an undesirable amount of food consumption. This study proposes a novel framework for learning food-picking tasks without consuming foods. The key idea is to leverage the object-breaking experiences of several reusable fragile objects instead of consuming real foods while making the picking ability object-invariant with domain generalization (DG). In real-robot experiments, we trained a model with reusable objects (toy blocks, ping-pong balls, and jellies), which are selected by three typical fracture types (crack, rupture, and crush). We then tested the model with four real food objects (tofu, bananas, potato chips, and tomatoes). The results showed that the proposed combination of reusable objects' breaking experiences and DG is effective for the food-picking task.

Keywords: Field Robots, Learning from Demonstration, Reinforcement Learning
Abstract: The process of designing costmaps for off-road driving tasks is often a challenging and engineering-intensive task. Recent work in costmap design for off-road driving focuses on training deep neural networks to predict costmaps from sensory observations using corpora of expert driving data. However, such approaches are generally subject to overconfident mis-predictions and are rarely evaluated in-the-loop on physical hardware. We present an inverse reinforcement learning-based method of efficiently training deep cost functions that are uncertainty-aware. We do so by leveraging recent advances in highly parallel model-predictive control and robotic risk estimation. In addition to demonstrating improvement at reproducing expert trajectories, we also evaluate the efficacy of these methods in challenging off-road navigation scenarios. We observe that our method significantly outperforms a geometric baseline, resulting in 44% improvement in expert path reconstruction and 57% fewer interventions in practice. We also observe that varying the risk tolerance of the vehicle results in qualitatively different navigation behaviors, especially with respect to higher-risk scenarios such as slopes and tall grass.

Keywords: Deep Learning Methods, Field Robots, Vision-Based Navigation
Abstract: Estimating terrain traversability in off-road environments requires reasoning about complex interaction dynamics between the robot and these terrains. However, it is challenging to create informative labels to learn a model in a supervised manner for these interactions. We propose a method that learns to predict traversability costmaps by combining exteroceptive environmental information with proprioceptive terrain interaction feedback in a self-supervised manner. Additionally, we propose a novel way of incorporating robot velocity into the costmap prediction pipeline. We validate our method in multiple short and large-scale navigation tasks on challenging off-road terrains using two different large, all-terrain robots. Our short-scale navigation results show that using our learned costmaps leads to overall smoother navigation, and provides the robot with a more fine-grained understanding of the robot-terrain interactions. Our large-scale navigation trials show that we can reduce the number of interventions by up to 57% compared to an occupancy-based navigation baseline in challenging off-road courses ranging from 400 m to 3150 m. Appendix and full experiment videos can be found in our website: https://mateoguaman.github.io/hdif.

Keywords: Machine Learning for Robot Control, Optimization and Optimal Control
Abstract: Motion generation seeks to produce safe and feasible robot motion from start to goal. Various tools at different levels of granularity have been developed. On one extreme, sampling-based motion planners focus on completeness -- a solution, if it exists, would eventually be found. However, produced paths are often of low quality, and contain superfluous motion. On the other, reactive methods optimise the immediate cost to obtain the next controls, producing smooth and legible motion that can quickly adapt to perturbations, uncertainties, and changes in the environment. However, reactive methods are highly local, and often produce motion that become trapped in non-convex regions of the environment. This paper contributes, Geometric Fabric Command Sequences, a method that lies in the middle ground. It can produce globally optimal motion that is smooth and intuitive, while being also reactive. We model motion via a reactive Geometric Fabric policy that ingests a sequence of attractor states, or commands, and then apply global optimisation over the space of commands. We postulate that solutions for different problems and scenes are highly transferable when conditioned on environmental features. Therefore, an implicit generative model is trained on solutions from optimisation and environment features in a self-supervised manner. That is, faced with multiple motion generation problems, the learning and optimisation are contained within the same loop: the optimisation generates labels for learning, while the learning improves the optimisation for the next problem, which in turn provides higher quality labels. We empirically validate our method in both simulation and on a real-world 6-DOF JACO arm.

Keywords: Machine Learning for Robot Control, Optimization and Optimal Control
Abstract: Reinforcement learning (RL) and trajectory optimization (TO) present strong complementary advantages. On one hand, RL approaches are able to learn global control policies directly from data, but generally require large sample sizes to properly converge towards feasible policies. On the other hand, TO methods are able to exploit gradient-based information extracted from simulators to quickly converge towards a locally optimal control trajectory which is only valid within the vicinity of the solution. Over the past decade, several approaches have aimed to adequately combine the two classes of methods in order to obtain the best of both worlds. Following on from this line of research, we propose several improvements on top of these approaches to learn global control policies quicker, notably by leveraging sensitivity information stemming from TO methods via Sobolev learning, and Augmented Lagrangian (AL) techniques to enforce the consensus between TO and policy learning. We evaluate the benefits of these improvements on various classical tasks in robotics through comparison with existing approaches in the literature.

Keywords: Machine Learning for Robot Control, Model Learning for Control, Optimization and Optimal Control
Abstract: We study the problem of generating control laws for systems with unknown dynamics. Our approach is to represent the controller and the value function with neural networks, and to train them using loss functions adapted from the Hamilton-Jacobi-Bellman (HJB) equations. In the absence of a known dynamics model, our method first learns the state transitions from data collected by interacting with the system in an offline process. The learned transition function is then integrated to the HJB equations and used to forward simulate the control signals produced by our controller in a feedback loop.

In contrast to trajectory optimization methods that optimize the controller for a single initial state, our controller can generate near-optimal control signals for initial states from a large portion of the state space. Compared to recent model-based reinforcement learning algorithms, we show that our method is more sample efficient and trains faster by an order of magnitude. We demonstrate our method in a number of tasks, including the control of a quadrotor with 12 state variables.

Keywords: Robot Safety, Machine Learning for Robot Control, Collision Avoidance
Abstract: We present Learned Risk Metric Maps (LRMM) for real-time estimation of coherent risk metrics of high-dimensional dynamical systems operating in unstructured, partially observed environments. LRMM models are simple to design and train---requiring only procedural generation of obstacle sets, state and control sampling, and supervised training of a function approximator---which makes them broadly applicable to arbitrary system dynamics and obstacle sets. In a parallel autonomy setting, we demonstrate the model's ability to rapidly infer collision probabilities of a fast-moving car-like robot driving recklessly in an obstructed environment; allowing the LRMM agent to intervene, take control of the vehicle, and avoid collisions. In this time-critical scenario, we show that LRMMs can evaluate risk metrics 20-100x times faster than alternative safety algorithms based on control barrier functions (CBFs) and Hamilton-Jacobi reachability (HJ-reach), leading to 5-15% fewer obstacle collisions by the LRMM agent than CBFs and HJ-reach. This performance improvement comes in spite of the fact that the LRMM model only has access to local/partial observation of obstacles, whereas the CBF and HJ-reach agents are granted privileged/global information. We also show that our model can be equally well trained on a 12-dimensional quadrotor system operating in an obstructed indoor environment. All software for training and experiments is provided at https://github.com/mit-drl/pyrmm

Keywords: Model Learning for Control, Machine Learning for Robot Control, Intelligent Transportation Systems
Abstract: Near the limits of adhesion, the forces generated by a tire are nonlinear and intricately coupled. Efficient and accurate modelling in this region could improve safety, especially in emergency situations where high forces are required. To this end, we propose a novel family of tire force models based on neural ordinary differential equations and a neural-texttt{ExpTanh} parameterization. These models are designed to satisfy physically insightful assumptions while also having sufficient fidelity to capture higher-order effects directly from vehicle state measurements. They are used as drop-in replacements for an analytical brush tire model in an existing nonlinear model predictive control framework. Experiments with a customized Toyota Supra show that scarce amounts of driving data -- less than three minutes -- is sufficient to achieve high-performance autonomous drifting on various trajectories with speeds up to 45mph. Comparisons with the benchmark model show a 4 times improvement in tracking performance, smoother control inputs, and faster and more consistent computation time.

Keywords: Model Learning for Control, Deep Learning Methods, Motion Control
Abstract: Autonomous driving has attracted lots of attention in recent years. For some tasks, e.g., trajectory prediction, motion planning, and trajectory tracking, an accurate vehicle model can reduce the difficulty of these tasks and improve task completion performance. Prior works focused on parameter estimation of physical models or modeling nonlinear dynamics using neural networks. Still, these methods rely on internal parameters of vehicles or are not friendly for control due to the strong nonlinearity of models. This paper proposes a data-driven method to approximate vehicle dynamics based on the Koopman operator. The resulting model is an interpretable linear time-invariant model, facilitating controller design and solving related optimization problems. In the proposed approach, the state transition matrix is constructed based on the learned Koopman eigenvalues, while the input matrix is trained as a tensor. Based on the resulting model, a linear model predictive controller is designed to implement coupled longitudinal and lateral trajectory tracking. Simulations and experiments, including vehicle dynamics modeling and coupled longitudinal and lateral trajectory tracking, are performed in a high-fidelity CarSim environment and a real vehicle platform. An oil-driven D-Class SUV is selected in the simulation, while a real electric SUV is utilized in the experiment. Simulation and experiment results illustrate that the model of the nonlinear vehicle dynamics can be identified effectively via the proposed method, and high-quality trajectory tracking performance can be obtained with the resulting model.

Keywords: Modeling, Control, and Learning for Soft Robots, Physical Human-Robot Interaction, Force Control
Abstract: Safe and efficient robot-environment interaction is a critical but challenging problem as robots are being increasingly employed to operate in unstructured and unpredictable environments. Soft robots are inherently compliant to safely interact with environments but their high nonlinearity exacerbates control difficulties. Meta-learning provides a powerful tool for fast online model adaptation because it can learn an efficient model from data across different environments. Thus, this work applies the idea of meta-learning for the control of soft robotics. In particular, a target-oriented proactive search strategy is firstly performed to collect environment-specific data efficiently when a new interaction environment occurs. Then meta-learning exploits past experience to train a data-driven probabilistic model prior, and the model prior is online updated to be fast adapted to the new environment. Lastly, a model-based optimal control policy is utilized to drive the robot to desired performance. Our approach controls a soft robotic manipulator to achieve the desired position and contact force simultaneously when interacting with unknown changing environments. Experimental results demonstrate that the tracking error could be reached within 1mm for position and 0.01N for contact force. Overall, this work provides a viable control approach for soft robots to interact with unknown environments.

Keywords: AI-Based Methods, AI-Enabled Robotics
Abstract: In this paper, we present a novel Heavy-Tailed Stochastic Policy Gradient (HT-PSG) algorithm to deal with the challenges of sparse rewards in continuous control problems. Sparse rewards are common in continuous control robotics tasks such as manipulation and navigation and make the learning problem hard due to the non-trivial estimation of value functions over the state space. This demands either reward shaping or expert demonstrations for the sparse reward environment. However, obtaining high-quality demonstrations is quite expensive and sometimes even impossible. We propose a heavy-tailed policy parametrization along with a modified momentum-based policy gradient tracking scheme (HT-SPG) to induce a stable exploratory behavior to the algorithm. The proposed algorithm does not require access to expert demonstrations. We test the performance of HT-SPG on various benchmark tasks of continuous control with sparse rewards such as 1D Mario, Pathological Mountain Car, Sparse Pendulum in OpenAI Gym, and Sparse MuJoCo environments (Hopper-v2, Half-Cheetah, Walker-2D). We show consistent performance improvement across all tasks in terms of high average cumulative reward without requiring access to expert demonstrations. We further demonstrate that a navigation policy trained using HT-SPG can be easily transferred into a Clearpath Husky robot to perform real-world navigation tasks.

Keywords: Optimization and Optimal Control, Sensor-based Control, Machine Learning for Robot Control
Abstract: Model predictive control is a powerful tool to generate complex motions for robots. However, it often requires solving non-convex problems online to produce rich behaviors, which is computationally expensive and not always practical in real time. Additionally, direct integration of high dimensional sensor data (e.g. RGB-D images) in the feedback loop is challenging with current state-space methods.This paper aims to address both issues. It introduces a model predictive control scheme, where a neural network constantly updates the cost function of a quadratic program based on sensory inputs, aiming to minimize a general non-convex task loss without solving a non-convex problem online. By updating the cost, the robot is able to adapt to changes in the environment directly from sensor measurement without requiring a new cost design. Furthermore, since the quadratic program can be solved efficiently with hard constraints, a safe deployment on the robot is ensured. Experiments with a wide variety of reaching tasks on an industrial robot manipulator demonstrate that our method can efficiently solve complex non-convex problems with high-dimensional visual sensory inputs, while still being robust to external disturbances.

Keywords: Formal Methods in Robotics and Automation, Optimization and Optimal Control, Machine Learning for Robot Control
Abstract: We propose a novel Branch-and-bound method for reachability analysis of neural networks. Our idea is to first compute accurate bounds on the Lipschitz constant of the neural network in specific directions of interest offline using a convex program. We then use these computations to obtain an instantaneous but conservative polyhedral approximation of the reachable set online using Lipschitz continuity arguments. To reduce conservatism, we incorporate our bounding algorithm within a branching strategy to decrease the over-approximation error within an arbitrary accuracy. We then extend our method to reachability analysis of control systems with neural network controllers. Finally, to capture the shape of the reachable sets as accurately as possible, we use sample trajectories to inform the directions of the reachable set over-approximations using Principal Component Analysis (PCA). We evaluate the performance of the proposed method in several open-loop and closed-loop settings.

Keywords: Machine Learning for Robot Control, Motion and Path Planning
Abstract: Trajectory optimization methods have achieved an exceptional level of performance on real-world robots in recent years. These methods heavily rely on accurate analytical models of the dynamics, yet some aspects of the physical world can only be captured to a limited extent. An alternative approach is to leverage machine learning techniques to learn a differentiable dynamics model of the system from data. In this work, we use trajectory optimization and model learning for performing highly dynamic and complex tasks with robotic systems in absence of accurate analytical models of the dynamics. We show that a neural network can model highly nonlinear behaviors accurately for large time horizons, from data collected in only 25 minutes of interactions on two distinct robots: (i) the Boston Dynamics Spot and an (ii) RC car. Furthermore, we use the gradients of the neural network to perform gradient-based trajectory optimization. In our hardware experiments, we demonstrate that our learned model can represent complex dynamics for both the Spot and Radio-controlled (RC) car, and gives good performance in combination with trajectory optimization methods.

Keywords: Machine Learning for Robot Control, Deep Learning Methods, Model Learning for Control
Abstract: Model Predictive Control (MPC) is a state-of-the-art (SOTA) control technique which requires solving hard constrained optimization problems iteratively. In the event of uncertain dynamics (typically encountered in real-life), analytical model based MPC requires setting conservative bounds on disturbances to obtain robust controllers. This however, increases the hardness of the problem, as more constraint satisfactions are required. The problem exacerbates in performance-critical applications, when more compute is required in lesser time. Data-driven regression methods such as Neural Networks have been proposed in the past to approximate system dynamics. However, such models rely on high volumes of labeled data, in the absence of symbolic analytical priors. This incurs non-trivial training overheads. Physics-informed Neural Networks (PINNs) have gained traction for approximating non-linear system of ordinary differential equations (ODEs), with reasonable accuracy. In this work, we propose a Robust Adaptive MPC framework via PINNs (RAMP-Net), which uses a neural network trained partly from simple ODEs and partly from data. A physics loss is used to learn simple ODEs (representing ideal dynamics). Having access to analytical functions inside the loss function acts as a regularizer, enforcing robust behavior for parametric uncertainties. On the other hand, a regular data loss is used for adapting to residual disturbances (non-parametric uncertainties), unaccounted during mathematical modelling. Experiments are performed in a simulated environment for trajectory tracking of a quadrotor. We report 7.8% to 43.2% and 8.04% to 61.5% reduction in tracking errors for speeds ranging from 0.5 to 1.75 m/s compared to two SOTA regression based MPC methods.

Keywords: Marine Robotics, Field Robots, Mapping
Abstract: This paper proposes a novel underwater Multi-View Photometric Stereo (MVPS) framework for reconstructing scenes in 3-D with a non-stationary low-cost robot equipped with a monocular camera and fixed lights. The underwater realm is the primary focus of study here, due to the challenges in utilizing underwater camera imagery and lack of low-cost reliable localization systems. Previous underwater PS approaches provided accurate scene reconstruction results, but assumed that the robot was stationary at the bottom. This assumption is limiting, as many artifacts, reefs, and man-made structures are large and meters above the bottom. Our proposed MVPS framework relaxes the stationarity assumption by utilizing a monocular SLAM system to estimate small robot motions and extract an initial sparse feature map. To compensate for the scale inconsistency in monocular SLAM output, our MVPS optimization scheme collectively estimates a high-quality, dense 3-D reconstruction and corrects the camera pose estimates. We also present an attenuation and camera-light extrinsic parameter calibration method for non-stationary robots. Finally, validation experiments with a BlueROV2 demonstrated the low-cost capability of producing high-quality scene reconstructions. Overall, this work is the foundation of an active perception pipeline for robots (i.e., underwater, ground, and aerial) to explore and map complex structures in high accuracy and resolution with an inexpensive sensor-light configuration.

Keywords: Marine Robotics, Mapping
Abstract: Acoustic perception in underwater environments is challenging due to the low frequency of the acquisition system and multiple and huge sources of noise. Therefore, point clouds built by profiling sonars mounted on Autonomous Underwater Vehicles (AUV) are sparse and noisy. To solve the mapping task, AUVs need a registration algorithm to prevent maps from inconsistencies. Many scan matching algorithms are available, however, a few of them are specialized in acoustic data. In this paper, a probabilistic scan matching methodology based on Gaussian Mixtures Models (GMM) is presented and, for the first time, the Bayesian-GMM algorithm is applied in this context to model acoustic data. The scan matching problem is properly formulated using Lie groups to define pose. In addition, this methodology can return an uncertainty measure for the matching result, which is fundamental in Pose SLAM applications. This tool is implemented in a public C++ library that can process in real-time 2D and 3D scans acquired by a profiling sonar. Theoretical justification and results with real data are provided to benchmark our method against the state-of-the-art Normal Distributions Transforms (NDT) technique. The library repository can be found in https://bitbucket.org/gmmregistration/gmm_registration.

Keywords: Marine Robotics, Mapping, Field Robots
Abstract: We present a technique for dense 3D reconstruction of objects using an imaging sonar, also known as forward-looking sonar (FLS). Compared to previous methods that model the scene geometry as point clouds or volumetric grids, we represent the geometry as a neural implicit function. Additionally, given such a representation, we use a differentiable volumetric renderer that models the propagation of acoustic waves to synthesize imaging sonar measurements. We perform experiments on real and synthetic datasets and show that our algorithm reconstructs high-fidelity surface geometry from multi-view FLS images at much higher quality than was possible with previous techniques and without suffering from their associated memory overhead.

Keywords: Marine Robotics, Mapping, Field Robots
Abstract: Underwater robots typically rely on acoustic sensors like sonar to perceive their surroundings. However, these sensors are often inundated with multiple sources and types of noise, which makes using raw data for any meaningful inference with features, objects, or boundary returns very difficultly. While several conventional methods of dealing with noise exist, their success rates are unsatisfactory. This paper presents a novel application of conditional Generative Adversarial Networks to train a model to produce noise-free sonar images, outperforming several conventional filtering methods. Estimating free space is crucial for autonomous robots performing active exploration and mapping. Thus we apply our approach to the task of underwater occupancy mapping and show superior free and occupied space inference when compared to conventional methods.

Keywords: Planning under Uncertainty, Marine Robotics, Field Robots
Abstract: Autonomous surface vessels (ASV) represent a promising technology to automate water-quality monitoring of lakes. In this work, we use satellite images as a coarse map and plan sampling routes for the robot. However, inconsistency between the satellite images and the actual lake, as well as environmental disturbances such as wind, aquatic vegetation, and changing water levels can make it difficult for robots to visit places suggested by the prior map. This paper presents a robust route-planning algorithm that minimizes the expected total travel distance given these environmental disturbances, which induce uncertainties in the map. We verify the efficacy of our algorithm in simulations of over a thousand Canadian lakes and demonstrate an application of our algorithm in a 3.7 km-long real-world robot experiment on a lake in Northern Ontario, Canada.

Keywords: Marine Robotics, Motion and Path Planning, Optimization and Optimal Control
Abstract: We present a navigation framework to perform autonomous underwater docking to a wave energy converter (WEC) under various ocean conditions by incorporating flow state estimation into the design of model predictive control (MPC). Existing methods lack the ability to perform dynamic rendezvous and autonomously dock in energetic conditions. The use of exteroceptive sensors or high performing acoustic sensors have been previously investigated to obtain or estimate the flow states. However, the use of such sensors increases the overall cost of the system and expects the vehicle to navigate close to the seafloor or other landmarks. To overcome these limitations, our method couples an active perception framework with MPC to estimate the flow states simultaneously while moving towards the dock. Our simulation results demonstrate the robustness and reliability of the proposed framework for autonomous docking under various ocean conditions. Furthermore, we conducted laboratory trials with a BlueROV2 docking with an oscillating dock and achieved a greater than 70% success rate.

Keywords: Marine Robotics, Autonomous Vehicle Navigation, Motion and Path Planning
Abstract: Vessel transit in ice-covered waters poses unique challenges in safe and efficient motion planning. When the concentration of ice is high, it may not be possible to find collision-free paths. Instead, ice can be pushed out of the way if it is small or if contact occurs near the edge of the ice. In this work, we propose a real-time navgiation framework that minimizes collisions with ice and distance travelled by the vessel. We exploit a lattice-based planner with a cost that captures the ship interaction with ice. To address the dynamic nature of the environment, we plan motion in a receding horizon manner based on updated vessel and ice state information. Further, we present a novel planning heuristic for evaluating the cost-to-go, which is applicable to navigation in a channel without a fixed goal location. The performance of our planner is evaluated across several levels of ice concentration both in simulated and in real-world experiments.

Keywords: Field Robots, Marine Robotics, Motion and Path Planning
Abstract: We present the results of experiments performed using a small autonomous underwater vehicle to determine the location of an isobath within a bounded area. The primary contribution of this work is to implement and integrate several recent developments real-time planning for environmental mapping, and to demonstrate their utility in a challenging practical example. We model the bathymetry within the operational area using a Gaussian process and propose a reward function that represents the task of mapping a desired isobath. As is common in applications where plans must be continually updated based on real-time sensor measurements, we adopt a receding horizon framework where the vehicle continually computes near-optimal paths. The sequence of paths does not, in general, inherit the optimality properties of each individual path. Our real-time planning implementation incorporates recent results that lead to performance guarantees for receding-horizon planning.

Keywords: Marine Robotics, Localization, SLAM
Abstract: Place recognition using SOund Navigation and Ranging (SONAR) images is an important task for simultaneous localization and mapping (SLAM) in underwater environments. This paper proposes a robust and efficient imaging SONAR-based place recognition, SONAR context, and loop closure method. Unlike previous methods, our approach encodes geometric information based on the characteristics of raw SONAR measurements without prior knowledge or training. We also design a hierarchical searching procedure for fast retrieval of candidate SONAR frames and apply adaptive shifting and padding to achieve robust matching on rotation and translation changes. In addition, we can derive the initial pose through adaptive shifting and apply it to the iterative closest point (ICP)-based loop closure factor. We evaluate the SONAR context’s performance in the various underwater sequences such as simulated open water, real water tank, and real underwater environments. The proposed approach shows the robustness and improvements of place recognition on various datasets and evaluation metrics. Supplementary materials are available at https://github.com/sparolab/sonar_context.

Keywords: Marine Robotics, Deep Learning for Visual Perception, Data Sets for Robotic Vision
Abstract: Underwater depth estimation is essential for safe Autonomous Underwater Vehicles (AUV) navigation. While there has been recent advances in out-of-water monocular depth estimation, it is difficult to apply these methods to the underwater domain due to the lack of well-established datasets with labelled ground truths. In this paper, we propose a novel method for self-supervised underwater monocular depth estimation by leveraging a low-cost single-beam echosounder (SBES). We also present a synthetic dataset for underwater depth estimation to facilitate visual learning research in the underwater domain, available at https://github.com/hdacnw/ sbes-depth. We evaluated our method on the proposed dataset with results outperforming previous methods and tested our method in a dataset we collected with an inexpensive AUV. We further investigated the use of SBES as an additional component in our self-supervised method for up-to-scale depth estimation providing insights on next research directions.

Keywords: Marine Robotics, RGB-D Perception, Vision-Based Navigation
Abstract: Depth estimation is critical for any robotic system. In the past years estimation of depth from monocular images have shown great improvement, however, in the underwater environment results are still lagging behind due to appearance changes caused by the medium. So far little effort has been invested on overcoming this. Moreover, underwater, there are more limitations for using high resolution depth sensors, this makes generating ground truth for learning methods another enormous obstacle. So far unsupervised methods that tried to solve this have achieved very limited success as they relied on domain transfer from dataset in air. We suggest training using subsequent frames self-supervised by a reprojection loss, as was demonstrated successfully above water. We suggest several additions to the self-supervised framework to cope with the underwater environment and achieve state-of-the-art results on a challenging forward-looking underwater dataset.

Keywords: Marine Robotics, Vision-Based Navigation, SLAM
Abstract: The recent development of high-precision subsea optical scanners allows for 3D keypoint detectors and feature descriptors to be leveraged on point cloud scans from subsea environments. However, the literature lacks a comprehensive survey to identify the best combination of detectors and descriptors to be used in these challenging and novel environments. This paper aims to identify the best detector/descriptor pair using a challenging field dataset collected using a commercial underwater laser scanner. Furthermore, studies have shown that incorporating texture information to extend geometric features adds robustness to feature matching on synthetic datasets. This paper also proposes a novel method of fusing images with underwater laser scans to produce coloured point clouds, which are used to study the effectiveness of 6D point cloud descriptors.

Keywords: Biologically-Inspired Robots, Legged Robots
Abstract: Quadruped animal locomotion emerges from the interactions between the spinal central pattern generator (CPG), sensory feedback, and supraspinal drive signals from the brain. Computational models of CPGs have been widely used for investigating the spinal cord contribution to animal locomotion control in computational neuroscience and in bio-inspired robotics. However, the contribution of supraspinal drive to anticipatory behavior, i.e. motor behavior that involves planning ahead of time (e.g. of footstep placements), is not yet properly understood. In particular, it is not clear whether the brain modulates CPG activity and/or directly modulates muscle activity (hence bypassing the CPG) for accurate foot placements. In this paper, we investigate the interaction of supraspinal drive and a CPG in an anticipatory locomotion scenario that involves stepping over gaps. By employing deep reinforcement learning (DRL), we train a neural network policy that replicates the supraspinal drive behavior. This policy can either modulate the CPG dynamics, or directly change actuation signals to bypass the CPG dynamics. Our results indicate that the direct supraspinal contribution to the actuation signal is a key component for a high gap crossing success rate. However, the CPG dynamics in the spinal cord are beneficial for gait smoothness and energy efficiency. Moreover, our investigation shows that sensing the front feet distances to the gap is the most important and sufficient sensory information for learning gap crossing. Our results support the biological hypothesis that cats and horses mainly control the front legs for obstacle avoidance, and that hind limbs follow an internal memory based on the front limbs' information. Our method enables the quadruped robot to cross gaps of up to 20 cm (50% of body-length) without any explicit dynamics modeling or Model Predictive Control (MPC).

Keywords: Biologically-Inspired Robots, Biomimetics
Abstract: The high speed and low energy cost are two conflicting objectives in the motion optimization of bio-inspired underwater robots, but playing a very important role. To this end, this paper proposes an optimization strategy for swimming speed and power cost using an improved NSGAII for a flexible robotic fish. A dynamic model involving flexible deformation is established for speed prediction with the hydrodynamic parameters identified. A back propagation (BP) neural network is applied to perform compensation of power cost prediction with the dynamic model’s prediction as input. In particular, an NSGA-II-AMS method is developed to improve the efficiency of solving the two-objective optimization problem based on NSGA-II. Finally, extensive simulations and experimental results demonstrate the effectiveness of the proposed optimization strategy, which offers promising prospects for the flexible robotic fish performing aquatic tasks with different performance constraints.

Keywords: Search and Rescue Robots, Biologically-Inspired Robots, Autonomous Agents
Abstract: Swarm robotics plays a non-negligible role in actual practice because of its scalability and robustness. Besides some specific studies, there is still a lack of an overall approach to solving the search and rescue problem in a communication-denied environment. This paper presents a bee-inspired swarm cooperation approach without information exchange, including a target grouping method suitable for multi-objective and multi-robot, a finite behavior state machine, and the corresponding control law. Finally, the effectiveness of the proposed approach is shown via simulation. The overall approach proposed in this paper requires no global position of the swarm and two-way information exchange, making swarm robotics search and rescue in a communication-denied environment possible.

Keywords: Biologically-Inspired Robots, Mechanism Design
Abstract: Robots that use impulsive mechanisms to achieve high-speed and high-powered motion are becoming more common and better understood, but control of these systems remains relatively rudimentary. Among robots that use spring actuation to generate motion, robot actuation and mechanisms are usually not controlled intentionally in order to achieve variation in the system's behavior, or they are controlled only roughly via adjustments made to the amount of energy stored in the mechanism. We describe the development, construction, and test of an impulsive catapult mechanism whose design is inspired by the grasshopper leg and for which extensive variation in the projectile trajectory is achieved by force control of the actuator that restrains the spring. As a step toward future controlled jumping robots, we give a detailed model of this system, validate this model experimentally, and explain how the actuator dynamics are critical to our ability to vary the system's trajectory using this approach. This work represents a novel approach to the control of spring actuated robots and illustrates how they can be controlled even under highly limiting actuator constraints.

Keywords: Biologically-Inspired Robots, Legged Robots, Motion Control
Abstract: Falling cat problem is well-known where cats show their super aerial reorientation capability and can land safely. For their robotic counterparts, a similar falling quadruped robot problem, has not been fully addressed, although achieving safe landing as the cats has been increasingly investigated. Unlike imposing the burden on landing control, we approach to safe landing of falling quadruped robots by effective flight phase control. Different from existing work like swinging legs and attaching reaction wheels or simple tails, we propose to deploy a 3-DoF morphable inertial tail on a medium-size quadruped robot. In the flight phase, the tail with its maximum length can self-right the body orientation in 3D effectively; before touch-down, the tail length can be retracted to about 1/4 of its maximum for impressing the tail's side-effect on landing. To enable aerial reorientation for safe landing in the quadruped robots, we design a control architecture, which is verified in a high-fidelity physics simulation environment with different initial conditions. Experimental results on a customized flight-phase test platform with comparable inertial properties are provided and show the tail's effectiveness on 3D body reorientation and its fast retractability before touch-down. An initial falling quadruped robot experiment is shown, where the robot Unitree A1 with the 3-DoF tail can land safely subject to non-negligible initial body angles.

Keywords: Soft Robot Applications, Biologically-Inspired Robots, Marine Robotics
Abstract: We present SunBot, a robotic system for the study and implementation of fish-inspired tearing manipulations. Various fish species -- such as the sunburst butterflyfish -- feed on prey fixed to substrates, a maneuver previously not demonstrated by robotic fish which typically specialize for open water swimming and surveillance. Biological studies indicate that a dynamic ``head flick'' behavior may play a role in tearing off soft prey during such feeding. In this work, we study whether the robotic tail is an effective means to generate such head motions for ungrounded tearing manipulations in water. We describe the function of SunBot and compare the forces that it applies to a fixed prey in the lab while varying tail speeds and ranges of motion. A simplified dynamic template model for the tail-driven head flick maneuver matches peak force magnitudes from experiments, indicating that inertial effects of the fish's body play a substantial role. Finally we demonstrate a tearing scenario and evaluate a free-swimming trial of SunBot -- this is important to show that the actuator that enables swimming also provides the new dual purpose of forceful tearing manipulation.

Keywords: Bioinspired Robot Learning, Reinforcement Learning, Biologically-Inspired Robots
Abstract: Central Pattern generators (CPG) are a biologically inspired, decentralized control architecture that enables modefree, but yet adaptively stable and computational lightweight locomotion capabilities on complex robots. Nevertheless, no unified design guidelines for closed-loop CPG controllers are available in the literature. Therefore, we propose a task-distributed, end-to-end trainable, closed-loop CPG control policy by generalizing and extending Tegotae control. The Tegotae approach modulates CPG activity by quantifying the discrepancy between internal belief states and environmental reactions. Spontaneous and adaptive gait formation towards situationally efficient locomotion patterns are intrinsic properties of Tegotae control. The Tegotae control policy is trained and benchmarked in simulation on a 1D hopping robot. We found that our approach can learn efficient and adaptive locomotion on minimal feedback information, while outperforming unstructured, classic reinforcement learning policies of equal complexity. To the best of our knowledge, this is the first study to fully generalize the Tegotae approach and construct unimpeded, end-to-end trainable Tegotae control policies.

Keywords: Legged Robots, Biologically-Inspired Robots, Biomimetics
Abstract: Most legged robots are built with leg structures from serially mounted links and actuators and are controlled through complex controllers and sensor feedback. In comparison, animals developed multi-segment legs, mechanical coupling between joints, and multi-segmented feet. They run agile over all terrains, arguably with simpler locomotion control. Here we focus on developing foot mechanisms that resist slipping and sinking also in natural terrain. We present first results of multi-segment feet mounted to a bird-inspired robot leg with multi-joint mechanical tendon coupling. Our one- and two-segment, mechanically adaptive feet show increased viable horizontal forces on multiple soft and hard substrates before starting to slip. We also observe that segmented feet reduce sinking on soft substrates compared to ball-feet and cylinder-feet. We report how multi-segmented feet provide a large range of viable centre of pressure points well suited for bipedal robots, but also for quadruped robots on slopes and natural terrain. Our results also offer a functional understanding of segmented feet in animals like ratite birds.

Keywords: Cyborgs, Motion Control, Autonomous Agents
Abstract: Terrestrial cyborg insects are biohybrid systems integrating living insects as mobile platforms. The insects’ locomotion is controlled by the electrical stimulation of their sensory, muscular, or neural systems, in which continuous pulse trains are usually chosen as the stimulation waveform. Although this waveform is easy to generate and can elicit graded responses from the insects, its locomotion control efficiency has not been consistent among existing literature. This study demonstrates the improvement of locomotion control by using a new stimulation protocol, named Burst Stimulation, to stimulate a cyborg beetle’s antennae (Zophobas morio). Modulating the continuous pulse train into multiple bursts enhanced the beetle’s turning responses. At the same stimulation intensity (amplitude, pulse width, and active duration), the Burst Stimulation improved the turning angle by up to 50% compared to the continuous waveform. Moreover, the beetle’s graded response was preserved. Increasing the stimulation frequency from 10 Hz to 40 Hz raised the turning rate by 40 deg/s. In addition, the initial implementation of this protocol in the feedback control-based navigation achieved a success rate of 81%, suggesting its potential use to optimize further the autonomous navigation of terrestrial cyborg insects.

Keywords: Legged Robots, Biologically-Inspired Robots, Optimization and Optimal Control
Abstract: Modern legged robot morphologies assign most of their actuated degrees of freedom (DoF’s) to the limbs and designs continue to converge to twelve DoF quadrupeds with three actuators per leg and a rigid torso often modeled as a Single Rigid Body (SRB). This is in contrast to the animal kingdom, which provides tantalizing hints that core actuation of a jointed torso confers substantial benefit for efficient agility. Unfortunately, the limited specific power of available actuators continues to hamper roboticists' efforts to capitalize on this bio-inspiration. This paper presents the initial steps in a comparative study of the costs and benefits associated with a traditionally neglected torso degree of freedom: a twisting spine. We use trajectory optimization to explore how a one-DoF, axially twisting spine might help or hinder a set of axially-active (twisting) behaviors: trots, sudden turns while bounding, and parkour-style wall jumps. By optimizing for minimum electrical energy or average power, intuitive cost functions for robots, we avoid hand-tuning the behaviors and explore the activation of the spine. Initial evidence suggests that for lower energy behaviors the spine increases the electrical energy required when compared to the rigid torso, but for higher energy runs the spine trends toward having no effect or reducing the electrical work. These results support future, more bio-inspired versions of the spine with inherent stiffness or dampening built into their mechanical design.

Keywords: Legged Robots, Motion Control, Biologically-Inspired Robots
Abstract: As an active spine introduces higher degree of freedoms (DOFs) as well as time-varying inertia, locomotion control of spined quadruped robots is challenging. Direct optimization on the full dynamics model causes prohibitive calculation time and is difficult to apply to embedded platforms. Model predictive control (MPC)-based on centroidal dynamics is a prevalent approach for ordinary quadruped robots, regarding the whole robot as a single rigid body (SRB). However, the approach ignores the changes of the center of mass (CoM) and inertia, which seriously affects the robot’s stability and could not be used in spined quadruped robots directly. To resolve the above issue, this paper presents an MPC approach that considers the movements of the spine in the SRB model. Since the mass of the robot is concentrated on its body, the whole robot is modelled as an unactuated 3D SRB with fully-actuated internal spine joints. MPC finds the optimal ground reaction forces (GRFs) based on the SRB centroidal dynamics, in which the missing spine part is complemented by the pre-defined spine joints’ states and corresponding inertia sequence. According to the GRFs, the full dynamic model calculates the precise joint torques. In addition, a quadruped robot with a 3-DOF active spine, Yat-sen Lion, is developed. With the presented approach, experimental results illustrate that Yat-sen Lion freely achieves bending, arching, and turning behaviors while trotting at speeds of 3.8 m/s in simulations and 0.5 m/s in real-world experiments.

Keywords: Swarm Robotics, Reinforcement Learning, Collision Avoidance
Abstract: In recent years, reinforcement learning and its multi-agent analogue have achieved great success in solving various complex control problems. However, multi-agent reinforcement learning remains challenging both in its theoretical analysis and empirical design of algorithms, especially for large swarms of embodied robotic agents where a definitive toolchain remains part of active research. We use emerging state-of-the-art mean-field control techniques in order to convert many-agent swarm control into more classical single-agent control of distributions. This allows profiting from advances in single-agent reinforcement learning at the cost of assuming weak interaction between agents. However, the mean-field model is violated by the nature of real systems with embodied, physically colliding agents. Thus, we combine collision avoidance and learning of mean-field control into a unified framework for tractably designing intelligent robotic swarm behavior. On the theoretical side, we provide novel approximation guarantees for general mean-field control both in continuous spaces and with collision avoidance. On the practical side, we show that our approach outperforms multi-agent reinforcement learning and allows for decentralized open-loop application while avoiding collisions, both in simulation and real UAV swarms. Overall, we propose a framework for the design of swarm behavior that is both mathematically well-founded and practically useful, enabling the solution of otherwise intractable swarm problems.

Keywords: Aerial Systems: Applications, Constrained Motion Planning, Motion and Path Planning
Abstract: In constrained solution spaces with a huge number of homotopy classes, stand-alone sampling-based kinodynamic planners suffer low efficiency in convergence. Local optimization is integrated to alleviate this problem. In this paper, we propose to thrive the trajectory tree growing by optimizing the tree in the forms of deformation units, and each unit contains one tree node and all the edges connecting it. The deforming proceeds both spatially and temporally by optimizing the node state and edge time durations efficiently. Deforming the unit only changes the tree locally yet improves the overall quality of a corresponding subtree. Further, to consider the computation burden and optimizing level, patterns to deform different tree parts in combination of different deformation units are studied and compared, all showing much faster convergence. The proposed deformation can be easily integrated into different RRT-based kinodynamic planning methods, and numerical experiments show that integrating the spatio-temporal deformation greatly accelerates the convergence and outperforms the spatial-only deformation.

Keywords: Aerial Systems: Perception and Autonomy, Integrated Planning and Learning, Motion and Path Planning
Abstract: Autonomous UAV path planning for 3D reconstruction has been actively studied in various applications for high-quality 3D models. However, most existing works have adopted explore-then-exploit, prior-based or exploration-based strategies, demonstrating inefficiency with repeated flight and low autonomy. In this paper, we propose PredRecon, a prediction-boosted planning framework that can autonomously generate paths for high 3D reconstruction quality. We obtain inspiration from humans can roughly infer the complete construction structure from partial observation. Hence, we devise a surface prediction module (SPM) to predict the coarse complete surfaces of the target from the current partial reconstruction. Then, the uncovered surfaces are produced by online volumetric mapping waiting for observation by UAV. Lastly, a hierarchical planner plans motions for 3D reconstruction, which sequentially finds efficient global coverage paths, plans local paths for maximizing the performance of Multi-View Stereo (MVS), and generates smooth trajectories for image-pose pairs acquisition. We conduct benchmarks in the realistic simulator, which validates the performance of PredRecon compared with the classical and state-of-the-art methods. The open-source code is released at https://github.com/HKUST-Aerial-Robotics/PredRecon.

Keywords: Aerial Systems: Perception and Autonomy, Collision Avoidance, Motion and Path Planning
Abstract: Navigating dynamic environments requires the robot to generate collision-free trajectories and actively avoid moving obstacles. Most previous works designed path planning algorithms based on one single map representation, such as the geometric, occupancy, or ESDF map. Although they have shown success in static environments, due to the limitation of map representation, those methods cannot reliably handle static and dynamic obstacles simultaneously. To address the problem, this paper proposes a gradient-based B-spline trajectory optimization algorithm utilizing the robot's onboard vision. The depth vision enables the robot to track and represent dynamic objects geometrically based on the voxel map. The proposed optimization first adopts the circle-based guide-point algorithm to approximate the costs and gradients for avoiding static obstacles. Then, with the vision-detected moving objects, our receding-horizon distance field is simultaneously used to prevent dynamic collisions. Finally, the iterative re-guide strategy is applied to generate the collision-free trajectory. The simulation and physical experiments prove that our method can run in real-time to navigate dynamic environments safely.

Keywords: Swarm Robotics, Aerial Systems: Mechanics and Control
Abstract: We introduce Spatial Predictive Control (SPC), a technique for solving the following problem: given a collection of robotic agents with black-box positional low-level controllers (PLLCs) and a mission-specific distributed cost function, how can a distributed controller achieve and maintain cost-function minimization without a plant model and only positional observations of the environment? Our fully distributed SPC controller is based strictly on the position of the agent itself and on those of its neighboring agents. This information is used in every time step to compute the gradient of the cost function and to perform a spatial look-ahead to predict the best next target position for the PLLC. Using a simulation environment, we show that SPC outperforms Potential Field Controllers, a related class of controllers, on the drone flocking problem. We also show that SPC works on real hardware, and is therefore able to cope with the potential sim-to-real transfer gap. We demonstrate its performance using as many as 16 Crazyflie 2.1 drones in a number of scenarios, including obstacle avoidance.

Keywords: Aerial Systems: Perception and Autonomy, Vision-Based Navigation, Deep Learning Methods
Abstract: Visual navigation plays an important role for Unmanned Aerial Vehicles(UAVs). In some applications, the landmark image and the real-time image may be heterogeneous, like near-infrared and visible images. In this work, we propose a multimodal image registration method to deal with near-infrared and visible images so that it can be applied to visual navigation system for the localization of UAVs in GPS-denied environments. At first, a new feature extraction strategy is developed to embed different modalities of images into the common feature space based on disentangled representations. Such common space is independent of the image modality, and this can eliminate the modality differences. Meanwhile, an intensity loss is introduced to measure the similarity of mono-modal images. In the proposed method, we can directly predict the transformation parameters and thus accelerates the localization of UAVs. Extensive experiments on synthetic datasets are conducted to demonstrate the validity of our method, and the experimental results show that the proposed method can effectively improve the localization accuracy.

Keywords: Aerial Systems: Applications, Aerial Systems: Perception and Autonomy, Motion and Path Planning
Abstract: We address the problem of efficient 3-D exploration in indoor environments for micro aerial vehicles with limited sensing capabilities and payload/power constraints. We develop an indoor exploration framework that uses learning to predict the occupancy of unseen areas, extracts semantic features, samples viewpoints to predict information gains for different exploration goals, and plans informative trajectories to enable safe and smart exploration. Extensive experimentation in simulated and real-world environments shows the proposed approach outperforms the state-of-the-art exploration framework by 24% in terms of the total path length in a structured indoor environment and with a higher success rate during exploration.

Keywords: Aerial Systems: Mechanics and Control, Motion and Path Planning, Underactuated Robots
Abstract: The use of bidirectional propellers provides quadrotors with greater maneuverability which is advantageous in constrained environments. This paper addresses the development of a trajectory planning algorithm for quadrotors with bidirectional motors. Previous work has shown that the property of differential flatness can be leveraged for efficient trajectory planning. However, planners that leverage flatness for quadrotors fail at points where the acceleration of the center of mass is equal to gravity, i.e., when the vehicle experiences free fall. The central contribution of this paper is a flatness-based trajectory planning method that allows quadrotors to use bidirectional propellers and pass through the so-called free-fall singularity. We model our system as a differentially flat hybrid system with the aid of coordinate charts derived from the Hopf fibration and develop an algorithm that computes forward and reverse thrusts for each propeller, resulting in smooth trajectories everywhere in SE(3). We demonstrate the planner’s versatility by planning knife-edge maneuvers and trajectories passing through the free-fall singularity, while transitioning from forward to reverse thrust.

Keywords: Aerial Systems: Perception and Autonomy, View Planning for SLAM, Environment Monitoring and Management
Abstract: Small uncrewed aerial systems are great for 3D reconstruction due to their speed, ease of use, and ability to access high-utility viewpoints. Today, most aerial survey approaches generate a preplanned coverage pattern assuming a planar target region. However, this is inefficient since it results in superfluous overlap and suboptimal viewing angles and does not utilize the entire flight envelope. In this work, we propose active path planning for photogrammetric reconstruction. Our main contribution is a view utility function based on Fisher information approximating the offline reconstruction uncertainty. The metric enables online path planning to make in-flight decisions to collect geometrically informative image data in complex terrain. We evaluate our approach in a photorealistic simulation. A viewpoint selection study shows that our metric leads to faster and more precise reconstruction than state-of-the-art active planning metrics and adapts to different camera resolutions. Comparing our online planning approach to an ordinary fixed-wing aerial survey yields 3.2 times faster coverage of 16 ha undulated terrain without sacrificing precision.

Keywords: Aerial Systems: Mechanics and Control, Motion Control, Integrated Planning and Control
Abstract: In this work, we present an Integrated Guidance and Controller (IGC) scheme to drive quadcopters in path-following tasks with obstacle avoidance and constant uncertainties rejection. This scheme is based on the combination of a time-varying artificial vector field and Backstepping with integral action control. The vector field switches between two behaviors: (i) path-following; and (ii) obstacle circumnavigation to allow collision avoidance. This vector field is then integrated into a non-linear controller designed via Backstepping with Integral Action to deal with the quadcopter vehicle dynamics and reject constant uncertainties. The considered vehicle model is based on quaternion algebra. The control inputs are considered to be the total thrust and torques. Stability is proved by using Lyapunov's Theory and Matrosov's Theorem. To illustrate our proposed solution, we show computational simulations.

Keywords: Aerial Systems: Applications, Machine Learning for Robot Control, AI-Enabled Robotics
Abstract: This paper proposes an adaptive near-hover position controller for quadcopters, which can be deployed to quadcopters of very different mass, size and motor constants, and also shows rapid adaptation to unknown disturbances during runtime. The core algorithmic idea is to learn a single policy that can adapt online at test time not only to the disturbances applied to the drone, but also to the robot dynamics and hardware in the same framework. We achieve this by training a neural network to estimate a latent representation of the robot and environment parameters, which is used to condition the behaviour of the controller, also represented as a neural network. We train both networks exclusively in simulation with the goal of flying the quadcopters to goal positions and avoiding crashes to the ground. We directly deploy the same controller trained in the simulation without any modifications on two quadcopters in the real world with differences in mass, size, motors, and propellers with mass differing by 4.5 times. In addition, we show rapid adaptation to sudden and large disturbances up to one-third of the mass of the quadcopters. We perform an extensive evaluation in both simulation and the physical world, where we outperform a state-of-the-art learning-based adaptive controller and a traditional PID controller specifically tuned to each platform individually.

Keywords: Aerial Systems: Applications, Cooperating Robots, Multi-Robot Systems
Abstract: The use of cables for aerial manipulation has shown to be a lightweight and versatile way to interact with objects. However, fastening objects using cables is still a challenge and human is required. In this work, we propose a novel way to secure objects using hitches. The hitch can be formed and morphed in midair using a team of aerial robots with cables. The hitch’s shape is modeled as a convex polygon, making it versatile and adaptable to a wide variety of objects. We propose an algorithm to form the hitch systematically. The steps can run in parallel, allowing hitches with a large number of robots to be formed in constant time. We develop a set of actions that include different actions to change the shape of the hitch. We demonstrate our methods using a team of aerial robots via simulation and actual experiments.

Keywords: Aerial Systems: Perception and Autonomy, Object Detection, Segmentation and Categorization, Visual Tracking
Abstract: Detect-and-Avoid (DAA) capabilities are critical for safe operations of unmanned aircraft systems (UAS). This paper introduces, AirTrack, a real-time vision-only detect and tracking framework that respects the size, weight, and power (SWaP) constraints of sUAS systems. Given the low Signal-to-Noise ratios (SNR) of far away aircraft, we propose using full resolution images in a deep learning framework that aligns successive images to remove ego-motion. The aligned images are then used downstream in cascaded primary and secondary classifiers to improve detection and tracking performance on multiple metrics. We show that AirTrack outperforms state-of-the art baselines on the Amazon Airborne Object Tracking (AOT) Dataset. Multiple real world flight tests with a Cessna 172 interacting with general aviation traffic and additional near-collision flight tests with a Bell helicopter flying towards a UAS in a controlled setting showcase that the proposed approach satisfies the newly introduced ASTM F3442/F3442M standard for DAA. Empirical evaluations show that our system has a probability of track of more than 95% up to a range of 700m.

Keywords: Failure Detection and Recovery, Aerial Systems: Applications, Model Learning for Control
Abstract: This paper proposes a new model for onboard physical fault detection on autonomous unmanned aerial vehicles (UAV) through machine learning (ML) techniques. The proposal performs the detection task with high accuracies and minimal processing requirements,while signaling an unreliable ML model to the operator, implemented in two main phases. First, a wrapper-based feature selection is performed to decrease the feature extraction computational costs, coped with a classification assessment technique to identify ML model unreliability. Second, physical UAV faults are signaled through a multi-view rationale that evaluates a variety of UAV sensors, while triggering alerts based on a sliding window scheme. Experiments performed on a real quadcopter UAV with a broken propeller use case, shows the proposal's feasibility. Our model can decrease the false-positive rates up to only 0.4%, while also decreasing the computational costs by at least 43% when compared to traditional techniques. Notwithstanding, it can identify ML model unreliability, signaling the UAV operator when model fine-tuning is needed.

Keywords: Reinforcement Learning, Contact Modeling
Abstract: A compliant control model based on reinforcement learning (RL) is proposed to allow robots to interact with the environment more effectively and autonomously execute force control tasks. The admittance model learns an optimal adjustment policy for interactions with the external environment using RL algorithms. The model combines energy consumption and trajectory tracking of the agent state using a cost function. Therein, an Unmanned Aerial Vehicle (UAV) can operate stably in unknown environments where interaction forces exist. Furthermore, the model ensures that the interaction process is safe, comfortable, and flexible while protecting the external structures of the UAV from damage. To evaluate the model performance, we verified the approach in a simulation environment using a UAV in three external force scenes. We also tested the model across different UAV platforms and various low-level control parameters, and the proposed approach provided the best results.

Keywords: Aerial Systems: Applications, Aerial Systems: Mechanics and Control
Abstract: Aerial robots have a wide range of applications, such as collecting data in hard-to-reach areas. This requires the longest possible operation time. However, because currently available commercial batteries have limited specific energy of roughly 300 Whkg^{−1}, a drone's flight time is a bottleneck for sustainable long-term data collection. Inspired by birds in nature, a possible approach to tackle this challenge is to perch drones on trees, and environmental or man-made structures, to save energy whilst in operation. In this paper, we propose an algorithm to automatically generate trajectories for a drone to perch on a tree branch, using the proposed tethered perching mechanism with a pendulum-like structure. This enables a drone to perform an energy-optimised, controlled 180-degree flip to disarm upside down safely. To fine-tune a set of reachable trajector

Keywords: Aerial Systems: Mechanics and Control, Model Learning for Control, Machine Learning for Robot Control
Abstract: Hybrid unmanned aerial vehicles (H-UAVs) are highly versatile platforms with the ability to transition between rotary- and fixed-wing flight. However, their (aero)dynamics tend to be highly nonlinear which increases the risk of introducing safety-critical modeling errors in a controller. Designing a safe, yet not too cautious controller, requires a credible model which provides accurate dynamics uncertainty quantification. We present a data-efficient, probabilistic semi-parametric dynamics modeling approach that allows for online, filter-based inference. The proposed model leverages prior knowledge using a nominal parametric model, and combines it with residuals in form of sparse Gaussian processes (GP) to account for possibly unmodeled forces and moments. Uncertain nominal and residual parameters are jointly estimated using Bayesian filtering. The resulting model accuracy and the reliability of its predicted uncertainty are analyzed for both a simulated and a real example, where we learn the 6DoF nonlinear dynamics of a tiltwing H-UAV from a few minutes of flight data. Compared to a residual-free nominal model, the proposed semi-parametric approach provides increased model accuracy in relevant parts of the flight envelope and substantially higher credibility overall.

Keywords: Aerial Systems: Applications, Aerial Systems: Perception and Autonomy, Surveillance Robotic Systems
Abstract: We propose a novel approach for aerial video action recognition. Our method is designed for videos captured using UAVs and can run on edge or mobile devices. We present a learning-based approach that uses customized auto zoom to automatically identify the human target and scale it appropriately. This makes it easier to extract the key features and reduces the computational overhead. We also present an efficient temporal reasoning algorithm to capture the action information along the spatial and temporal domains within a controllable computational cost. Our approach has been implemented and evaluated both on the desktop with high-end GPUs and on the low power Robotics RB5 Platform for robots and drones. In practice, we achieve 6.1-7.4% improvement over SOTA in Top-1 accuracy on the RoCoG-v2 dataset, 8.3-10.4% improvement on the UAV-Human dataset and 3.2% improvement on the Drone Action dataset.

Keywords: Imitation Learning, Aerial Systems: Applications, Formal Methods in Robotics and Automation
Abstract: Seamlessly integrating rules in Learning-from-Demonstrations (LfD) policies is a critical requirement to enable the real-world deployment of AI agents. Recently Signal Temporal Logic (STL) has been shown to be an effective language for encoding rules as spatio-temporal constraints. This work uses Monte Carlo Tree Search (MCTS) as a means of integrating STL specification into a vanilla LfD policy to improve constraint satisfaction. We propose augmenting the MCTS heuristic with STL robustness values to bias the tree search towards branches with higher constraint satisfaction. While the domain-independent method can be applied to integrate STL rules online into any pre-trained LfD algorithm, we choose goal-conditioned Generative Adversarial Imitation Learning as the offline LfD policy. We apply the proposed method to the domain of planning trajectories for General Aviation aircraft around a non-towered airfield. Results using the simulator trained on real-world data showcase 60% improved performance over baseline LfD methods that do not use STL heuristics.

Keywords: Visual Learning, Aerial Systems: Perception and Autonomy, Aerial Systems: Applications
Abstract: Unmanned aerial vehicle (UAV) tracking is crucial for autonomous navigation and has broad applications in robotic automation fields. However, reliable UAV tracking remains a challenging task due to various difficulties like frequent occlusion and aspect ratio change. Additionally, most of the existing work mainly focuses on explicit information to improve tracking performance, ignoring potential interframe connections. To address the above issues, this work proposes a novel framework with continuity-aware latent interframe information mining for reliable UAV tracking, i.e., ClimRT. Specifically, a new efficient continuity-aware latent interframe information mining network (ClimNet) is proposed for UAV tracking, which can generate highly-effective latent frame between two adjacent frames. Besides, a novel location-continuity Transformer (LCT) is designed to fully explore continuity-aware spatial-temporal information, thereby markedly enhancing UAV tracking. Extensive qualitative and quantitative experiments on three authoritative aerial benchmarks strongly validate the robustness and reliability of ClimRT in UAV tracking performance. Furthermore, real-world tests on the aerial platform validate its practicability and effectiveness. The code and demo materials are released at https://github.com/vision4robotics/ClimRT.

Keywords: Machine Learning for Robot Control, Aerial Systems: Mechanics and Control
Abstract: Recently, Model Predictive Contouring Control (MPCC) has arisen as the state-of-the-art approach for model-based agile flight. MPCC benefits from great flexibility in trading-off between progress maximization and path following at runtime without relying on globally optimized trajectories. However, finding the optimal set of tuning parameters for MPCC is challenging because (i) the full quadrotor dynamics are non-linear, (ii) the cost function is highly non-convex, and (iii) of the high dimensionality of the hyperparameter space. This paper leverages a probabilistic Policy Search method, Weighted Maximum Likelihood (WML), to automatically learn the optimal objective for MPCC. WML is sample-efficient due to its closed-form solution for updating the learning parameters. Additionally, the data efficiency provided by the use of a model-based approach allows us to directly train in a high-fidelity simulator, which in turn makes our approach able to transfer zero-shot to the real world. We validate our approach in the real world, where we show that our method outperforms both the previous manually tuned controller and the state-of-the-art auto-tuning baseline reaching speeds of 75 km/h.

Keywords: Machine Learning for Robot Control, Aerial Systems: Mechanics and Control, Reinforcement Learning
Abstract: Recently, learning-based controllers have been shown to push mobile robotic systems to their limits and provide the robustness needed for many real-world applications. However, only classical optimization-based control frameworks offer the inherent flexibility to be dynamically adjusted during execution by, for example, setting target speeds or actuator limits. We present a framework to overcome this shortcoming of neural controllers by conditioning them on an auxiliary input. This advance is enabled by including a feature-wise linear modulation layer (FiLM). We use model-free reinforcement-learning to train quadrotor control policies for the task of navigating through a sequence of waypoints in minimum time. By conditioning the policy on the maximum available thrust or the viewing direction relative to the next waypoint, a user can regulate the aggressiveness of the quadrotor's flight during deployment. We demonstrate in simulation and in real-world experiments that a single control policy can achieve close to time-optimal flight performance across the entire performance envelope of the robot, reaching up to 60km/h and 4.5g in acceleration. The ability to guide a learned controller during task execution has implications beyond agile quadrotor flight, as conditioning the control policy on human intent helps safely bringing learning based systems out of the well-defined laboratory environment into the wild.

Keywords: Machine Learning for Robot Control, Aerial Systems: Mechanics and Control, Robust/Adaptive Control
Abstract: Control design for robotic systems is complex and often requires solving an optimization to follow a trajectory accurately. Online optimization approaches like Model Predictive Control (MPC) have been shown to achieve great tracking performance, but require high computing power. Conversely, learning-based offline optimization approaches, such as Reinforcement Learning (RL), allow fast and efficient execution on the robot but hardly match the accuracy of MPC in trajectory tracking tasks. In systems with limited compute, such as aerial vehicles, an accurate controller that is efficient at execution time is imperative. We propose an Analytic Policy Gradient (APG) method to tackle this problem. APG exploits the availability of differentiable simulators by training a controller offline with gradient descent on the tracking error. We address training instabilities that frequently occur with APG through curriculum learning and experiment on a widely used controls benchmark, the CartPole, and two common aerial robots, a quadrotor and a fixed-wing drone. Our proposed method outperforms both model-based and model-free RL methods in terms of tracking error. Concurrently, it achieves similar performance to MPC while requiring more than an order of magnitude less computation time. Our work provides insights into the potential of APG as a promising control method for robotics. To facilitate the exploration of APG, we open-source our code and make it available at https://github.com/lis-epfl/apg_trajectory_tracking.

Keywords: Reinforcement Learning, Collision Avoidance, Aerial Systems: Perception and Autonomy
Abstract: Reinforcement learning (RL) is an agent-based approach for teaching robots to navigate within the physical world. Gathering data for RL is known to be a laborious task, and real-world experiments can be risky. Simulators facilitate the collection of training data in a quicker and more cost-effective manner. However, RL frequently requires a significant number of simulation steps for an agent to become skilful at simple tasks. This is a prevalent issue within the field of RL-based visual quadrotor navigation where state dimensions are typically very large and dynamic models are complex. Furthermore, rendering images and obtaining physical properties of the agent can be computationally expensive. To solve this, we present a simulation framework, built on AirSim, which provides efficient parallel training. Building on this framework, Ape-X is modified to incorporate parallel training of AirSim environments to make use of numerous networked computers. Through experiments we were able to achieve a reduction in training time from 3.9 hours to 11 minutes, for a toy problem, using the aforementioned framework and a total of 74 agents and two networked computers.

Keywords: Path Planning for Multiple Mobile Robots or Agents, Planning, Scheduling and Coordination, Multi-Robot Systems
Abstract: Parking in large metropolitan areas is often a time-consuming task with further implications toward traffic patterns that affect urban landscaping. Reducing the premium space needed for parking has led to the development of automated mechanical parking systems. Compared to regular garages having one or two rows of vehicles in each island, automated garages can have multiple rows of vehicles stacked together to support higher parking demands. Although this multi-row layout reduces parking space, it makes the parking and retrieval more complicated. In this work, we propose an automated garage design that supports near 100% parking density. Modeling the problem of parking and retrieving multiple vehicles as a special class of multi-robot path planning problem, we propose associated algorithms for handling all common operations of the automated garage, including (1) optimal algorithm and near-optimal methods that find feasible and efficient solutions for simultaneous parking/retrieval and (2) a novel shuffling mechanism to rearrange vehicles to facilitate scheduled retrieval at rush hours. We conduct thorough simulation studies showing the proposed methods are promising for large and high-density real-world parking applications

Keywords: Education Robotics, Hardware-Software Integration in Robotics, Wheeled Robots
Abstract: From both an educational and research point of view, experiments on hardware are a key aspect of robotics and control. In the last decade, many open-source hardware and software frameworks for wheeled robots have been presented, mainly in the form of unicycles and car-like robots, with the goal of making robotics accessible to a wider audience and to support control systems development. Unicycles are usually small and inexpensive, and therefore facilitate experiments in a larger fleet, but they are not suited for high-speed motion. Car-like robots are more agile, but they are usually larger and more expensive, thus requiring more resources in terms of space and money. In order to bridge this gap, we present Chronos, a new car-like 1/28th scale robot with customized open-source electronics, and CRS, an open-source software framework for control and robotics. The CRS software framework includes the implementation of various state-of-the-art algorithms for control, estimation, and multi-agent coordination. With this work, we aim to provide easier access to hardware and reduce the engineering time needed to start new educational and research projects.