Keywords: Optimization and Optimal Control, Machine Learning for Robot Control, Human-Aware Motion Planning
Abstract: For a companion robot that follows a person as an assistant, predicting human walking is important to produce a proactive movement that is helpful to maintain an appropriate area decided by the human personal space. However, fully trusting the prediction may result in obstructing human walking because it is not always accurate. Hence, we consider the estimation of uncertainty (i.e., entropy) of the prediction to enable the robot to move without causing overconfident motion and without being late for the person it follows. To consider this uncertainty of the prediction to the controller, we introduce a reliability value that changes based on the entropy of the prediction. This value expresses the extent the controller should trust the prediction result, and it affects the cost function of our controller. We propose an uncertainty-aware robot controller based on nonlinear model predictive control to realize natural human-followings. We found that our uncertainty-aware control system can produce an appropriate robot movement, such as not obstructing the human walking and avoiding delay, in both simulations using actual human walking data and real-robot experiments.

Keywords: Marine Robotics, Motion and Path Planning
Abstract: We present a path planning framework for marine robots subject to uncertain ocean currents that exploits data from ensemble forecasting, which is a technique for current prediction used in oceanography. Ensemble forecasts represent a distribution of predicted currents as a set of flow fields that are considered to be equally likely. We show that the typical approach of computing the vector-wise mean and variance over this set can yield meaningless results, and propose an alternative approach that considers each flow field in the ensemble simultaneously. Our framework finds a sequence of vehicle controls that minimises the root-mean-square error distance (RMSE) over the full set of ensemble-induced trajectories. The key to achieving computational efficiency in this approach is our use of Monte Carlo tree search (MCTS) with a specialised heuristic that improves convergence rate while preserving asymptotic optimality and the anytime property. We demonstrate our results using real ensemble forecasts provided by the Australian Bureau of Meteorology, and provide comparisons with the deterministic mean-based approach where we observe RMSE reductions of 92% and 43% in two example scenarios. Further, we argue that the framework can be used in a plan-as-you-go manner where ensemble forecasts change over time. These results help to introduce ensemble forecasts as a viable source of data to improve path planning in marine robotics.

Keywords: Intelligent Transportation Systems, Distributed Robot Systems, Motion and Path Planning
Abstract: The implementation of optimization-based motion coordination approaches in real world multi-agent systems remains challenging due to their high computational complexity and potential deadlocks. This paper presents a distributed model predictive control (MPC) approach based on convex feasible set (CFS) algorithm for multi-vehicle motion coordination in autonomous driving. By using CFS to convexify the collision avoidance constraints, collision-free trajectories can be computed in real time. We analyze the potential deadlocks and show that a deadlock can be resolved by changing vehicles' desired speeds. The MPC structure ensures that our algorithm is robust to low-level tracking errors. The proposed distributed method has been tested in multiple challenging multi-vehicle environments, including unstructured road, intersection, crossing, platoon formation, merging, and overtaking scenarios. The numerical results and comparison with other approaches (including a centralized MPC and reciprocal velocity obstacles) show that the proposed method is computationally efficient and robust, and avoids deadlocks.

Keywords: Deep Learning Methods, Collision Avoidance, Motion and Path Planning
Abstract: Modern navigation algorithms based on deep reinforcement learning (RL) have proven to be efficient and robust. However, most deep RL algorithms operate in a risk-neutral manner, making no special attempt to shield users from `outcomes that may hurt the most', even if such shielding might cause little loss of performance. Furthermore, such algorithms typically make no provisions to ensure safety in the presence of inaccuracies in the models on which they were trained, beyond adding a cost-of-collision and some domain randomization while training, in spite of the formidable complexity of the environments in which they operate. In this paper, we present a novel distributional RL algorithm that not only learns an uncertainty-aware policy, but can also change its risk measure without expensive fine-tuning or retraining. Our method shows superior performance and safety over baselines in partially-observed navigation tasks. We also demonstrate that agents trained using our method can adapt their policies to a wide range of risk measures in a zero-shot manner.

Keywords: Motion Control, Collision Avoidance
Abstract: We present NEO, a fast and purely reactive motion controller for manipulators which can avoid static and dynamic obstacles while moving to the desired end-effector pose. Additionally, our controller maximises the manipulability of the robot during the trajectory, while avoiding joint position and velocity limits. NEO is wrapped into a strictly convex quadratic programme which, when considering obstacles, joint limits, and manipulability on a 7 degree-of-freedom robot, is generally solved in a few ms. While NEO is not intended to replace state-of-the-art motion planners, our experiments show that it is a viable alternative for scenes with moderate complexity while also being capable of reactive control. For more complex scenes, NEO is better suited as a reactive local controller, in conjunction with a global motion planner. We compare NEO to motion planners on a standard benchmark in simulation and additionally illustrate and verify its operation on a physical robot in a dynamic environment. We provide an open-source library which implements our controller.

Keywords: Legged Robots, Motion Control
Abstract: A method for planning and controlling the somersault motion of a quadruped robot is proposed in this paper. The method divides the somersault motion into 5 stages according to intuitive understanding. Based on the simplified dynamic model, the linear programming method is used to obtain the maximum ground reaction force under the constraints of joint torque and friction cone, and then the optimal leg thrusting trajectory is obtained by double integration of the acceleration. In order to achieve the buffered landing of the robot after somersault, a whole body controller based on null space projection is used to obtain the optimal joint torque under the constraints of the robot's foot position, torso position and torso posture. The somersault motion control method proposed in this paper has been verified by the dynamics simulation software Webots and quadruped robot platform Yobogo. The results show that the robot can complete stable front flip and back flip under the constraints of joint output torque and foot motion space constraints.

Keywords: Surgical Robotics: Steerable Catheters/Needles, Surgical Robotics: Laparoscopy, Medical Robots and Systems
Abstract: Multi-segment continuum robots, that offer inherent compliance and dexterity, are suitable for deployment in minimally invasive surgical procedures. Cable-driven mechanism is commonly used in continuum surgical robots but it leads to motion coupling between different segments. In this paper, we present a coupled mechanics model for a two-segment notched continuum robot to analyze the coupled deflection in the proximal segment due to the distal cable force. The model has been developed for two different conditions in which the proximal segment is initially bent (general condition) and initially straight (special condition). It allows us to introduce a decoupled design methodology that systematically determines a stiffness parameter in each of the segments, based on the desired coupled bending angle and other design requirements. Using the method, we fabricated a decoupled notched continuum robot and evaluated the model accuracy with mean errors of 0.57 degree and 0.61 degree for general and special conditions, respectively, throughout the 90 degree distal segment bending angle. It was also demonstrated in a maxillary sinus phantom that the distal segment was capable of independently performing omnidirectional steering without the proximal segment getting in contact with its surrounding nasal wall.

Keywords: Visual-Inertial SLAM, SLAM, Localization
Abstract: In this paper, we develop a novel visual-inertial navigation system with motion constraint (VINS-Motion), which extends the visual-inertial navigation system (VINS) to incorporate vehicle motion constraints for improving the autonomous vehicles localization accuracy. Besides the prior information, IMU measurement residual, and visual measurement residual utilized in VINS, vehicle orientation/velocity constraint is first exploited to constitute motion residual. We minimize the sum of priors and Mahalanobis norms of three kinds of residuals to obtain a maximum posteriori estimation, thus increasing system consistency and accuracy. Stop detection is also added to help eliminate the abnormal jitter of the estimated poses during stopping, thus ensuring reasonability of the trajectory. The proposed approach is validated on public datasets and compared against state-of-the-art algorithms, which demonstrates that VINS-Motion achieves significantly higher positioning accuracy.

Keywords: Soft Sensors and Actuators, Surgical Robotics: Laparoscopy, Medical Robots and Systems
Abstract: In this letter, we propose a novel 3D shape sensing algorithm for flexible endoscopic surgery using multi-core fiber Bragg grating (FBG) sensors. Considering the signal noises and environmental perturbations, the direct use of FBG measurements for a real-time shape sensing and position estimation is regarded as far from accurate and stable, especially when utilized in the sensing of long and flexible surgical instruments. To solve this problem, a novel and generic model-based filtering technique for the iterative curvature/twist estimation by taking advantage of the configurations of the multi-core FBGs in the optical fiber is introduced to remedy the sensory noises. Besides, we introduce an enhanced moving average approach to smooth the estimated curvatures and twists spatially on the fiber. We extensively validate our algorithm by conducting shape sensing tasks in simulations under varying conditions, and in experiments using a robotic-assisted colonoscope system integrated with a multi-core FBG fiber. The results prove our method substantially outperforms the conventional approach in the aspect of estimation accuracy and robustness, showing the superiority and application feasibility.

Keywords: Medical Robots and Systems, Embedded Systems for Robotic and Automation
Abstract: A computed tomography (CT)-guided robotic needle requires registration to transfer coordinates between the robot and CT image for accurate insertion. In our previous work, we proposed a geometric marker that allows direct registration between a CT image and robot and demonstrated its proof of concept. In this paper, we present a registration algorithm for calculating the six-degrees-of-freedom error of one CT scan, and we demonstrated the registration with our needle insertion robot. We obtained its geometric shape from differences between multiple images, which we converted into the coordinate axes of the marker, to calculate the posture of the marker. Since the algorithm uses changes in the cross-sectional shapes of images, it can be adapted for markers of different sizes. In the evaluation, the registration error for insertion by the CT-guided robotic needle with the proposed algorithm was calculated to be 1.8 mm and 0.35°. The accuracy of the proposed algorithm is sufficient for lower abdominal insertion and shows its potential for clinical applications. When the markers are half the size of the original, the rotational error is the same as the original, but the positional error is about half. This suggested the possibility of miniaturizing the marker.

Keywords: Medical Robots and Systems, Surgical Robotics: Steerable Catheters/Needles, Soft Sensors and Actuators
Abstract: Shape sensors are important for safer and more dexterous manipulation of the medical catheters. Among the electromagnetic based shape sensors, a voice coil shape sensor measures the variation of a mutual inductance between the coils placed along the tube due to the bending of the tube. Owing to the design flexibility of a voice coil, it offers the small size without the external magnetic field generator outside patient body. This paper presents an improved voice coil shape sensor in terms of modeling and measurement. Analytic model that incorporates the bending of an exciter coils is used to improve the accuracy of the sensor and band-pass filter is applied to simplify the measurements. Bending angle from sensors placed in multiple locations can be measured through a single channel from frequency sweep input. The simulation and experimental results verify the improvement and demonstrate that the sensor system can reconstruct a shape of a catheter placed through larynx.

Keywords: Surgical Robotics: Laparoscopy, Surgical Robotics: Planning, Computer Vision for Medical Robotics
Abstract: Vessel and neurovascular bundle localization plays an essential role in endoscopic and robotic surgery. It still remains challenging to spare vessels and neurovascular bundles to avoid inadvertent injury due to limited visual and tactile perception of surgeons. This work assumes that surgeons have great difficulty in intuitively perceiving small pulsatile motion of vessels and neurovascular bundles from complex surgical field provided by endoscopic videos, and proposes a new surgical video pulsatile motion magnification method to help surgeons easily and precisely recognize vessels or neurovascular bundles by their visual systems. The new method consists of robust hybrid temporal filtering and deeply learned spatial decomposition. The proposed hybrid temporal filtering can significantly magnify pulsatile motion more consistent with reality and simultaneously keep non-pulsating regions in magnified videos almost identical to original videos, while learning-based spatial decomposition can reduce noise and ring artifacts in magnified videos. We evaluate our method on surgical videos acquired from robotic prostatectomy, with the experimental results showing that our method essentially outperforms current motion magnification approaches. In particular, visual quality and quantitative assessment of our method are certainly better than these methods.

Keywords: Additive Manufacturing
Abstract: Additive Manufacturing, has evolved beyond prototyping to manufacturing end-products. The authors are involved in developing a large-scale extrusion-based 3D printer to print mining equipment - a Gravity Separation Spiral, and embedding sensors to monitor the operational conditions remotely. This paper presents a temperature-compensated strain sensor that can be 3D printed inline within large-scale 3D printed equipment. The sensor is printed using conductive carbon filament and embedded in a Polylactic acid (PLA) base. A half-bridge setup is proposed to reduce the impact of temperature variations. Temperature-controlled tests have been conducted with the proposed half-bridge and compared with a non-temperature compensated quarter-bridge setup. Results show that the half-bridge configuration reduces the temperature impact on the strain measurement significantly (68%) compared to the quarter-bridge, in the range of 25-40 °C. Deflection testing conducted on the printed sensor shows a near-linear relationship between bending strain and voltage. Multiple bending cycles have shown that there is no significant hysteresis. ANSYS simulations are used to accurately estimate the internal temperature since embedding a temperature sensor would affect the structural integrity. Although carbon black material is naturally brittle, steps have been taken in the design to avoid undesirable cracking. Results from laser microscopy analysis of the printed traces showed no crack defects.

Keywords: Mechanism Design, Marine Robotics, Simulation and Animation
Abstract: The recovery of autonomous underwater vehicles (AUVs) has been a challenging mission due to the limited localization accuracy and movement capability of the AUVs. To overcome these limitations, we propose a novel design of a deployable underwater robot (DUR) for the recovery mission. Utilizing the origami structure, the DUR can transform between open and closed states to maximize the performance at different recovery stages. At the approaching stage, the DUR will remain closed state to reduce the drag force. While at the capturing state, the DUR will deploy to form a much larger opening to improve the success rate of docking. Meanwhile, the thrusters’ configuration also changes with the transformation of the robot body. The DUR can achieve a high driven force in the forward direction with the closed state which leads to a fast-approaching speed. While with the open state, the DUR can achieve more balanced force and torque maneuverability to prepare for agile position adjustment for the docking. CFD simulation has been used to analyze the drag forces and identify the hydrodynamic coefficients. A prototype of the robot has been fabricated and tested in an indoor water pool. Both simulation and experiment results validate the feasibility of the proposed design.

Keywords: Wearable Robotics, Prosthetics and Exoskeletons, Physically Assistive Devices
Abstract: In this paper we present a methodology for optimising the design of a metamaterial structure with one degree of freedom that is able to simultaneously bend and stretch. The structure is intended for assisting flexion-extension of the wrist joint. The metamaterial is comprised of serially connected, individually designed cells. The design parameters can be chosen to optimally fit a desired planar curve such as the curvature of the skin on a plane normal to the flexion/extension axis of the wrist joint. A tool for the optimised design is described and the experimental validation of the output is conducted, to show the ability of the mechanism to conform to a 2D curve, also exhibiting a change in length, which is desirable for reducing sliding and shear on the skin. The design tool allows the generation of metamaterials optimised for a multitude of other applications where actuated mechanisms must be worn by a user for rehabilitation or assistance purposes.

Keywords: Human-Centered Robotics, Human-Centered Automation, Prosthetics and Exoskeletons
Abstract: In this paper, we present a low-damping active knee prosthesis with low noise and high backdrivability. The proposed prosthesis is driven by a motor and then decelerated by a four-stage synchronous belt. High backdrivability given by this structural form accelerates the response of the prosthesis. Also, a control system containing several sensors are embedded in the prototype of the designed prosthesis to distinguish different situations and provide corresponding strategies. Based on this mechatronic design, a three-layer control method is proposed. Preliminary experiments were carried out on a transfemoral amputee, demonstrating the features of low noise, high backdrivability and ability to reproduce natural walking gait.

Keywords: Learning from Demonstration, Visual Servoing, Perception for Grasping and Manipulation
Abstract: End-to-end visuomotor control is emerging as a compelling solution for robot manipulation tasks. However, imitation learning-based visuomotor control approaches tend to suffer from a common limitation, lacking the ability to recover from an out-of-distribution state caused by compounding errors. In this paper, instead of using tactile feedback or explicitly detecting the failure through vision, we investigate using the uncertainty of a policy neural network. We propose a novel uncertainty-based approach to detect and recover from failure cases. Our hypothesis is that policy uncertainties can implicitly indicate the potential failures in the visuomotor control task and that robot states with minimum uncertainty are more likely to lead to task success. To recover from high uncertainty cases, the robot monitors its uncertainty along a trajectory and explores possible actions in the state-action space to bring itself to a more certain state. Our experiments verify this hypothesis and show a significant improvement on task success rate: 12% in pushing, 15% in pick-and-reach and 22% in pick-and-place.

Keywords: Reinforcement Learning, Transfer Learning, Representation Learning
Abstract: In this study, we present a method to grasp diverse unseen real-world objects using an off-policy actor-critic deep reinforcement learning (RL) with the help of a simulation and the use of as little real-world data as possible. Actor-critic deep RL is unstable and difficult to tune when a raw image is given as an input. Therefore, we use state representation learning (SRL) to make actor-critic RL feasible for visual grasping tasks. Meanwhile, to reduce visual reality gap between simulation and reality, we also employ a typical pixel-level domain adaptation that can map simulated images to realistic ones. In our method, as the SRL model is a common preprocessing module for simulated and real-world data, we perform SRL using real and adapted images. This pixel-level domain adaptation enables the robot to learn grasping skills in a real environment using small amounts of real-world data. However, the controller trained in the simulation should adapt to the real world efficiently. Hence, we propose a method combining a typical pixel-level domain adaptation and the proposed SRL model, where we perform SRL based on a feature-level domain adaptation. In evaluations of vision-based robotics grasping tasks, we show that the proposed method achieves a substantial improvement over a method that only employs a pixel-level or domain adaptation.

Keywords: Perception-Action Coupling, Sensor-based Control
Abstract: Manipulation in a densely cluttered environment creates complex challenges in perception to close the control loop, many of which are due to the sophisticated physical interaction between the environment and the manipulator. Drawing from bio-inspiration, to handle the task in such a scenario, tactile sensing can be used to provide an additional dimension of the rich contact information from the interaction for decision making and action selection to manoeuvre towards a target. In this paper, a new tactile-based motion planning and control framework for a robot manipulator to manoeuvre in a cluttered environment is proposed and developed. An iterative two-stage machine learning approach is used in this framework: an autoencoder is used to extract important cues from tactile sensory readings and a reinforcement learning technique is used to generate optimal motion sequence to efficiently reach the given target. The framework is implemented on a KUKA LBR iiwa robot mounted with a SynTouch BioTac tactile sensor and tested with real-life experiments. The results show that the system is able to manipulate in the cluttered environment to reach the target effectively.

Keywords: Aerial Systems: Applications, Motion and Path Planning, Autonomous Vehicle Navigation
Abstract: Maintaining a map online is resource-consuming while a robust navigation system usually needs environment abstraction via a well-fused map. In this paper, we propose a mapless planner which directly conducts such abstraction on the unfused sensor data. A limited-memory data structure with a reliable proximity query algorithm is proposed for maintaining raw historical information. A sampling-based scheme is designed to extract the free-space skeleton. A smart waypoint selection strategy enables to generate high-quality trajectories within the resultant flight corridors. Our planner differs from other mapless ones in that it can abstract and exploit the environment information efficiently. The online replan consistency and success rate are both significantly improved against conventional mapless methods.

Keywords: Semantic Scene Understanding, Deep Learning for Visual Perception
Abstract: Robots are usually equipped with cameras to explore the indoor scene and it is expected that the robot can well describe the scene with natural language. Although some great success has been achieved in image and video captioning technology, especially on many public datasets, the caption generated from indoor scene video is still not informative and coherent enough. In this paper, we propose the problem of textit{Indoor Scene Captioning from Streaming Video}, which aims at generating a more accurate and informative caption from streaming video. To solve this problem, we firstly design an algorithm to organize the visual information of the indoor scene into a scene graph, and then implement a scene graph guided captioning method, which takes the scene graph and video frames as input to generate the caption from the video streaming. The proposed framework is evaluated both on the AI2THOR dataset and a real-world robotic platform, demonstrating the effectiveness of the framework.

Keywords: Deep Learning for Visual Perception, Visual Learning, Recognition
Abstract: Recently proposed DNN-based stereo matching methods that learn priors directly from data are known to suffer a drastic drop in accuracy in new environments. Although supervised approaches with ground truth disparity maps often work well, collecting them in each deployment environment is cumbersome and costly. For this reason, many unsupervised domain adaptation methods based on image-to-image translation have been proposed, but these methods do not preserve the geometric structure of a stereo image pair because the image-to-image translation is applied to each view separately. To address this problem, in this paper, we propose an attention mechanism that aggregates features in the left and right views, called Stereoscopic Cross Attention (SCA). Incorporating SCA to an image-to-image translation network makes it possible to preserve the geometric structure of a stereo image pair in the process of the image-to-image translation. We empirically demonstrate the effectiveness of the proposed unsupervised domain adaptation based on the image-to-image translation with SCA.

Keywords: Marine Robotics, Cognitive Modeling, Big Data in Robotics and Automation
Abstract: Justifying operational decisions for robots is a challenging task as the operator or the robot itself has to understand the underlying physical interaction between the robot and the environment to predict the potential outcome. It is desirable to understand how the decision influences the operational performance in the way of causal relationship for the purpose of explainable decision-making. Here we propose a novel causal inference framework for the discovery and inference on the reasoning of the operational decisions for robots. It unifies both domain knowledge integration and model-free causal inference, allowing a data-driven causal knowledge learning on time series data. The framework is evaluated in the experiments of an underwater robot with complex environmental interactions. The results show that the framework can learn the causal structure and inference model to accurately explain and predict the operation performance with integrated physics.

Keywords: Embodied Cognitive Science, Learning from Experience, Multi-Modal Perception for HRI
Abstract: We propose a promising neural network model with which to acquire a grounded representation of robot actions and the linguistic descriptions thereof. Properly responding to various linguistic expressions, including polysemous words, is an important ability for robots that interact with people via linguistic dialogue. Previous studies have shown that robots can use words that are not included in the action-description paired datasets by using pre-trained word embeddings. However, the word embeddings trained under the distributional hypothesis are not grounded, as they are derived purely from a text corpus. In this paper, we transform the pre-trained word embeddings to embodied ones by using the robot's sensory-motor experiences. We extend a bidirectional translation model for actions and descriptions by incorporating non-linear layers that retrofit the word embeddings. By training the retrofit layer and the bidirectional translation model alternately, our proposed model is able to transform the pre-trained word embeddings to adapt to a paired action-description dataset. Our results demonstrate that the embeddings of synonyms form a semantic cluster by reflecting the experiences (actions and environments) of a robot. These embeddings allow the robot to properly generate actions from unseen words that are not paired with actions in a dataset.

Keywords: Localization, Recognition, Object Detection, Segmentation and Categorization
Abstract: In this paper, HueCode, a meta-marker that robustly and simultaneously exposes the relative pose between a marker and a camera along with additional information, is proposed. It occupies the area of a single marker by overlaying multiple types of markers in different colored layers. Using perspective information from the first (most recognizable) type of element marker, the second or higher marker can be recognized with a better success rate. An experiment using a HueCode made from an ArUco marker and a QR code showed that the QR code within the HueCode is recognizable at an elevation angle of up to 15°, compared to 25° for a normal QR code. In addition, the versatility of HueCode is demonstrated using two robotic applications. The first is a lightweight estimation of absolute 6-DoF poses for mobile robots in a GNSS-denied environment, and the other is object annotation in two-dimensional image or three-dimensional space without any prior knowledge. The former realized pose estimation with a position error of less than 0.05 m, and the latter enabled annotation of a mirror and a transparent object, which are difficult for other sensors and machine learning to recognize.

Keywords: Deep Learning for Visual Perception, RGB-D Perception, Perception for Grasping and Manipulation
Abstract: Object 6D pose estimation is a fundamental task in many applications. Conventional methods solve the task by detecting and matching the keypoints, then estimating the pose. Recent efforts bringing deep learning into the problem mainly overcome the vulnerability of conventional methods to environmental variation due to the hand-crafted feature design. However, these methods cannot achieve end-to-end learning and good interpretability at the same time. In this paper, we propose REDE, a novel end-to-end object pose estimator using RGB-D data, which utilizes network for keypoint regression, and a differentiable geometric pose estimator for pose error back-propagation. Besides, to achieve better robustness when outlier keypoint prediction occurs, we further propose a differentiable outliers elimination method that regresses the candidate result and the confidence simultaneously. Via confidence weighted aggregation of multiple candidates, we can reduce the effect from the outliers in the final estimation. Finally, following the conventional method, we apply a learnable refinement process to further improve the estimation. The experimental results on three benchmark datasets show that REDE slightly outperforms the state-of-the-art approaches and is more robust to object occlusion. Our code is available at https://github.com/HuaWeitong/REDE.

Keywords: Transfer Learning, Deep Learning for Visual Perception, Computer Vision for Automation
Abstract: Utilizing the trained model under different conditions without data annotation is attractive for robot applications. Towards this goal, one class of methods is to translate the image style from another environment to the one on which models are trained. In this paper, we propose a weakly-paired setting for the style translation, where the content in the two images is aligned with errors in poses. These images could be acquired by different sensors in different conditions that share an overlapping region, e.g. with LiDAR or stereo cameras, from sunny days or foggy nights. We consider this setting to be more practical with: (i) easier labeling than the paired data; (ii) better interpretability and detail retrieval than the unpaired data. To translate across such images, we propose PREGAN to train a style translator by intentionally transforming the two images with a random pose, and to estimate the given random pose by differentiable non-trainable pose estimator given that the more aligned in style, the better the estimated result is. Such adversarial training enforces the network to learn the style translation, avoiding being entangled with other variations. Finally, PREGAN is validated on both simulated and real-world collected data to show the effectiveness. Results on down-stream tasks, classification, road segmentation, object detection, and feature matching show its potential for real applications.

Keywords: Localization, Deep Learning Methods
Abstract: Global localization and kidnapping are two challenging problems in robot localization. The popular method, Monte Carlo Localization (MCL) addresses the problem by iteratively updating a set of particles with a “sampling-weighting” loop. Sampling is decisive to the performance of MCL. However, traditional MCL can only sample from a uniform distribution over the state space. Variants of MCL fail to provide an accurate distribution or generalize across scenes. To better deal with these problems, we present a distribution proposal model named Deep Samplable Observation Model (DSOM). DSOM takes a map and a 2D laser scan as inputs and outputs a conditional multimodal probability distribution of the pose, making the samples more focusing on the regions with higher likelihood. With such samples, the convergence is expected to be more effective and efficient. Considering that the learning-based sampling model may fail to capture the accurate pose sometimes, we furthermore propose the Adaptive Mixture MCL (AdaM MCL), which deploys a trusty mechanism to adaptively select updating mode for each particle to tolerate this situation. Equipped with DSOM, AdaM MCL can achieve more accurate estimation, faster convergence and better scalability than previous methods in both synthetic and real scenes. Even in real environments with long-term changes, AdaM MCL is able to localize the robot using DSOM trained only by simulation observations from a SLAM map or a blueprint map.

Keywords: SLAM, Nonholonomic Mechanisms and Systems, Kinematics
Abstract: Purely vision-based localization and mapping is a cost-effective and thus attractive solution to localization and mapping on smart ground vehicles. However, the accuracy and especially robustness of vision-only solutions remain rivalled by more expensive, lidar-based multi-sensor alternatives. We show that a significant increase in robustness can be achieved if taking non-holonomic kinematic constraints on the vehicle motion into account. Rather than using approximate planar motion models or simple, pair-wise regularization terms, we demonstrate the use of B-splines for an exact imposition of smooth, non-holonomic trajectories inside the 6 DoF bundle adjustment. We introduce both hard and soft formulations and compare their computational efficiency and accuracy against traditional solutions. Through results on both simulated and real data, we demonstrate a significant improvement in robustness and accuracy in degrading visual conditions.

Keywords: Localization, Sensor Fusion, Autonomous Vehicle Navigation
Abstract: We present a tightly-coupled multi-sensor fusion architecture for autonomous vehicle applications, which achieves centimetre-level accuracy and high robustness in various scenarios. In order to realize robust and accurate point-cloud feature matching we propose a novel method for extracting structural, highly discriminative features from LiDAR point clouds. For high frequency motion prediction and noise propagation, we use incremental on-manifold IMU preintegration. We also adopt a multi-frame sliding window square root inverse filter, so that the system maintains numerically stable results under the premise of limited power consumption. To verify our methodology, we test the fusion algorithm in multiple applications and platforms equipped with a LiDAR-IMU system. Our results demonstrate that our fusion framework attains state-of-the-art localization accuracy, high robustness and a good generalization ability.

Keywords: SLAM, Mapping
Abstract: Straight line features have been increasingly utilized in visual SLAM and 3D reconstruction systems. The straight lines' parameterization, parallel constraint, and co-planar constraint are studied in many recent works. In this paper, we explore the novel intersection constraint of straight lines for structure reconstruction. First, a minimum parameterized representation of Ray-Point-Ray (RPR) structures is proposed to represent the intersection of two straight lines in the 3D space. Second, an efficient solver is designed for the camera pose estimation, which leverages the perpendicularity and intersection of straight lines. Third, we build a stereo visual odometry based on RPR features and evaluate it on the simulation and real datasets. The experimental results verify that the intersection constraints from RPR can effectively improve the accuracy and efficiency of line-based SLAM and reconstruction system.

Keywords: SLAM, Mapping, Factory Automation
Abstract: The LIght Detection And Ranging (LiDAR) sensor has become one of the most important perceptual devices due to its important role in simultaneous localization and mapping (SLAM). Existing SLAM methods are mainly developed for mechanical LiDAR sensors, which are often adopted by large scale robots. Recently, the solid-state LiDAR is introduced and becomes popular since it provides a cost-effective and lightweight solution for small scale robots. Compared to mechanical LiDAR, solid-state LiDAR sensors have higher update frequency and angular resolution, but also have smaller field of view (FoV), which is very challenging for existing LiDAR SLAM algorithms. Therefore, it is necessary to have a more robust and computationally efficient SLAM method for this new sensing device. To this end, we propose a new SLAM framework for solid-state LiDAR sensors. It involves feature extraction, odometry estimation, and probability map building. In the experiments, we demonstrate both the accuracy and efficiency of our method using an Intel L515 solid-state LiDAR. The results show that our method is able to provide precise localization at up to 30Hz on a warehouse robot and a handheld device. We made the source codes public at https://github.com/wh200720041/floam_ssl.

Keywords: Deep Learning in Grasping and Manipulation, Dexterous Manipulation, Reinforcement Learning
Abstract: Living objects are hard to grasp because they can actively dodge and struggle by writhing or deforming while or even prior to being contacted and modeling or predicting their responses to grasping is extremely difficult.This paper presents an algorithm based on reinforcement learning (RL) to attack this challenging problem. Considering the complexity of living object grasping, we divide the whole task into pre-grasp and in-hand stages and let the algorithm switch stages automatically. The pre-grasp stage is aimed at finding a good pose of a robot hand approaching a living object for performing a grasp. Dense reward functions are proposed for facilitating the learning of right hand actions based on the poses of both hand and object. Since an object held in hand may struggle to escape, the robot hand needs to adjust its configuration and respond correctly to the object's movement.Hence, the goal of the in-hand stage is to determine an appropriate adjustment of finger configuration in order for the robot hand to keep holding the object. At this stage, we treat the robot hand as a graph and use the graph convolutional network (GCN) to determine the hand action. We test our algorithm with both simulation and real experiments, which show its good performance in living object grasping. More results are available on our website: https://sites.google.com/view/graph-rl.

Keywords: Assembly, Compliance and Impedance Control, Reinforcement Learning
Abstract: Contact-rich tasks, wherein multiple contact transitions occur in a series of operations, are extensively studied for task automation. Precision assembly, one of the typical examples of contact-rich tasks, requires high time constants to cope with the change in contact state. This paper therefore proposes a local trajectory optimization method for precision assembly with high time constants. Since the non-diagonal components of the stiffness matrix is effective for inducing motion at high sampling frequency, we design a stiffness matrix to guide motion and propose a method to control a peg through the selection of the stiffness matrix. We introduced reinforcement learning (RL) for the selection of the stiffness matrix, because the relationship between the direction to be guided and the sensor response is difficult to model. The architecture with different sampling rates for RL and admittance control has an advantage of rapid response owing to the high time constant of the local trajectory optimization. The effectiveness of the method was verified via experiments involving two contact-rich tasks. The average total time of peg insertion was 1.64 s, which is less than the half the time reported by the best of the existing state of the art studies.

Keywords: Machine Learning for Robot Control, Reinforcement Learning, Probabilistic Inference
Abstract: This paper presents contact-safe Model-based Reinforcement Learning (MBRL) for robot applications that achieves contact-safe behaviors in the learning process. In typical MBRL, we cannot expect the data-driven model to generate accurate and reliable policies to the intended robotic tasks during the learning process due to data scarcity. Operating these unreliable policies in a contact-rich environment could cause damage to the robot and its surroundings. To alleviate the risk of causing damage through unexpected intensive physical contacts, we present the contact-safe MBRL that associates the probabilistic Model Predictive Control's (pMPC) control limits with the model uncertainty so that the allowed acceleration of controlled behavior is adjusted according to learning progress. Control planning with such uncertainty-aware control limits is formulated as a deterministic MPC problem using a computationally-efficient approximated GP dynamics and an approximated inference technique. Our approach's effectiveness is evaluated through bowl mixing tasks with simulated and real robots, scooping tasks with a real robot as examples of contact-rich manipulation skills.

Keywords: Machine Learning for Robot Control, Reinforcement Learning, AI-Based Methods
Abstract: Deep reinforcement learning (DRL) has been demonstrated to provide promising results in several challenging decision making and control tasks. However, the required inference costs of deep neural networks (DNNs) could prevent DRL from being applied to mobile robots which cannot afford high energy-consuming computations. To enable DRL methods to be affordable in such energy-limited platforms, we propose an asymmetric architecture that reduces the overall inference costs via switching between a computationally expensive policy and an economic one. The experimental results evaluated on a number of representative benchmark suites for robotic control tasks demonstrate that our method is able to reduce the inference costs while retaining the agent's overall performance.

Keywords: Reinforcement Learning, Deep Learning Methods, Probabilistic Inference
Abstract: Model-based deep reinforcement learning has achieved success in various domains that require high sample efficiencies, such as Go and robotics. However, there are some remaining issues, such as planning efficient explorations to learn more accurate dynamic models, evaluating the uncertainty of the learned models, and more rational utilization of models. To mitigate these issues, we present MEEE, a model-ensemble method that consists of optimistic exploration and weighted exploitation. During exploration, unlike prior methods directly selecting the optimal action that maximizes the expected accumulative return, our agent first generates a set of action candidates and then seeks out the optimal action that takes both expected return and future observation novelty into account. During exploitation, different discounted weights are assigned to imagined transition tuples according to their model uncertainty respectively, which will prevent model predictive error propagation in agent training. Experiments on several challenging continuous control benchmark tasks demonstrated that our approach outperforms other model-free and model-based state-of-the-art methods, especially in sample complexity.

Keywords: Representation Learning, Reinforcement Learning, Machine Learning for Robot Control
Abstract: In the present paper, we propose a decoder-free extension of Dreamer, a leading model-based reinforcement learning (MBRL) method from pixels. Dreamer is a sample- and cost-efficient solution to robot learning, as it is used to train latent state-space models based on a variational autoencoder and to conduct policy optimization by latent trajectory imagination. However, this autoencoding based approach often causes object vanishing, in which the autoencoder fails to perceives key objects for solving control tasks, and thus significantly limiting Dreamer's potential. This work aims to relieve this Dreamer's bottleneck and enhance its performance by means of removing the decoder. For this purpose, we firstly derive a likelihood-free and InfoMax objective of contrastive learning from the evidence lower bound of Dreamer. Secondly, we incorporate two components, (i) independent linear dynamics and (ii) the random crop data augmentation, to the learning scheme so as to improve the training performance. In comparison to Dreamer and other recent model-free reinforcement learning methods, our newly devised Dreamer with InfoMax and without generative decoder (Dreaming) achieves the best scores on 5 difficult simulated robotics tasks, in which Dreamer suffers from object vanishing.

Keywords: Model Learning for Control, Probabilistic Inference, Machine Learning for Robot Control
Abstract: Probabilistic regression techniques in control and robotics applications have to fulfill different criteria of data-driven adaptability, computational efficiency, scalability to high dimensions, and the capacity to deal with different modalities in the data. Classical regressors usually fulfill only a subset of these properties. In this work, we extend seminal work on Bayesian nonparametric mixtures and derive an efficient variational Bayes inference technique for infinite mixtures of probabilistic local polynomial models with well-calibrated certainty quantification. We highlight the model's power in combining data-driven complexity adaptation, fast prediction, and the ability to deal with discontinuous functions and heteroscedastic noise. We benchmark this technique on a range of large real-world inverse dynamics datasets, showing that the infinite mixture formulation is competitive with classical Local Learning methods and regularizes model complexity by adapting the number of components based on data and without relying on heuristics. Moreover, to showcase the practicality of the approach, we use the learned models for online inverse dynamics control of a Barrett-WAM manipulator, significantly improving the trajectory tracking performance.

Keywords: Model Learning for Control, Probabilistic Inference, Deep Learning Methods
Abstract: Domain randomization (DR) is a powerful tool to make a policy robust to the uncertainty of dynamics caused by unobservable environmental parameters. Conventional DR has adopted model-free reinforcement learning as a policy optimizer. However, the model-free methods in DR demand high time-complexity due to the randomization process where the environment is extremely changed. In this paper, we introduce model-based dynamics and policy learning for efficient DR. A Bayesian model of locally linear embedding is designed to fit the stochastic dynamics in DR. By virtue of locally linear dynamics, model-based optimal control is substituted for the policy optimization. Unlike previous works, our proposed Bayesian model with a MNIW prior allows the locally linear embedding to capture the dynamics in DR as a stochastic model. We show that a training method that combines variational and adversarial approaches is adequate for Bayesian embedding. Finally, a model-based controller is designed on our Bayesian locally linear embedding, and it shows better performance in DR environments compared with the non-Bayesian model of locally linear embedding.

Keywords: Autonomous Vehicle Navigation, Imitation Learning
Abstract: Navigation through dynamic pedestrian environments in a socially compliant manner is still a challenging task for autonomous vehicles. Classical methods usually lead to unnatural vehicle behaviours for pedestrian navigation due to the difficulty in modeling social conventions mathematically. This paper presents an end-to-end path planning system that achieves autonomous navigation in dynamic environments through imitation learning. The proposed system is based on a fully convolutional neural network that maps the raw sensory data into a confidence map for path extraction. Additionally, a classification network is introduced to reduce the unnecessary re-plannings and ensures that the vehicle goes back to the global path when re-planning is not needed. The imitation learning based path planner is implemented on an autonomous wheelchair and tested in a new real-world dynamic pedestrian environment. Experimental results show that the proposed system is able to generate paths for different driving tasks, such as pedestrian following, static and dynamic obstacles avoidance, etc. In comparison to the state-of-the-art method, our system is superior in terms of generating human-like trajectories.

Keywords: Reinforcement Learning, Learning from Demonstration, Motion and Path Planning
Abstract: State-of-the-art reinforcement learning (RL) algorithms suffer from high sample complexity, particularly in the sparse reward case. A popular strategy for mitigating this problem is to learn control policies by imitating a set of expert demonstrations. The drawback of such approaches is that an expert needs to produce demonstrations, which may be costly in practice. To address this shortcoming, we propose Probabilistic Planning for Demonstration Discovery (P2D2), a technique for automatically discovering demonstrations without access to an expert. We formulate discovering demonstrations as a search problem and leverage widely-used planning algorithms such as Rapidly-exploring Random Tree to find demonstration trajectories. These demonstrations are used to initialize a policy, then refined by a generic RL algorithm. We provide theoretical guarantees of P2D2 finding successful trajectories, as well as bounds for its sampling complexity. We experimentally demonstrate the method outperforms classic and intrinsic exploration RL techniques in a range of classic control and robotics tasks, requiring only a fraction of exploration samples and achieving better asymptotic performance.

Keywords: Cooperating Robots
Abstract: This paper is focused on the time-optimal Multi-Vehicle Trajectory Planning (MVTP) problem for multiple car-like robots when they travel in a tiny indoor scenario occupied by static obstacles. Herein, the complexity of the concerned MVTP task includes i) the non-convexity and narrowness of the environment, ii) the nonholonomy and nonlinearity of the vehicle kinematics, iii) the pursuit for a time-optimal solution, and iv) the absence of predefined homotopic routes for the vehicles. The aforementioned factors, when mixed together, are beyond the capability of the prevalent coupled or decoupled MVTP methods. This work proposes an adaptive-scaling constrained optimization (ASCO) approach, aiming to find the optimum of the nominally intractable MVTP problem in a decoupled way. Concretely, an iterative computation framework is built, wherein each intermediate subproblem contains only risky collision avoidance constraints within a certain range, thus being tractable in the scale. During the iteration, the constraint activation scale can change adaptively, thereby enabling to promote the convergence rate, to recover from an intermediate failure, and to get rid of a poor initial guess. ASCO is compared versus the state-of-the-art MVTP methods and is validated in real experiments conducted by a team of three car-like robots.

Keywords: Robotics and Automation in Construction, Motion and Path Planning, Optimization and Optimal Control
Abstract: In this paper, we present a novel optimization-based framework for autonomous excavator trajectory generation under task-specific constraints. Traditional excavation trajectory generators over-simplify the geometric trajectory parameterization thereby limiting the space for optimization. To expand the search space, we formulate a generic task specification for excavation by constraining the instantaneous motion of the bucket and adding a target-oriented constraint to control the amount of excavated soil. The trajectory is represented with a waypoint interpolating spline. Time intervals between waypoints are relaxed as variables to facilitate generating the time-optimal trajectory in one stage. Experiments on a real robot platform demonstrate that our method is adaptive to different terrain shapes and outperforms other optimal path planners in terms of the minimum joint length and minimum travel time.

Keywords: Legged Robots, Humanoid and Bipedal Locomotion, Whole-Body Motion Planning and Control
Abstract: This paper studies push recovery for humanoid robots based on a variable-height inverted pendulum (VHIP) model. We first develop an approach for treating zero-step capturability of the VHIP with a novel methodology based on Hamilton-Jacobi (HJ) reachability analysis. Such an approach uses the sub-zero level set of a value function to encode capturability of the VHIP, where the value function is obtained by numerically solving a HJ variational inequality offline. Based on this analysis, a simple and effective method for adjusting foothold locations is then devised for cases where the VHIP state is not zero-step capturable. In addition, the HJ reachability analysis naturally induces an optimal control law that allows for rapid planning with the VHIP during push recovery online. To enable use of the strategy with a position-controlled humanoid robot, an associated differential inverse kinematics based tracking controller is employed. The effectiveness of the overall framework is demonstrated with the UBTECH Walker robot in the MuJoCo simulator. Simulation validations show an approximately 20% improvement in push robustness as compared to the methods based on the classical linear inverted pendulum model.

Keywords: Underactuated Robots, Dynamics, Kinematics
Abstract: In this paper, we propose a meaningful definition of rotational centroidal orientation which is somewhat missed in the state-of-the-art centroidal momentum and dynamics theory for locomotion robots with one floating base. This centroidal instantaneous orientation rotates as the robot runs, and it is extracted from the total system angular inertia. The new centroidal frame is proposed to be parallel with the principal axes of the centroidal angular inertia, which can describe the whole-robot rotational motion. To avoid high fluctuations of centroidal frame orientation parameters between adjacent control loops, we develop one algorithm to enable the centroidal instantaneous frame to be smooth. The relationship between the centroidal angle rate and the centroidal angular velocity is derived, as well as the relationship in the acceleration level, which can be used for whole-body torque control. The new centroidal orientation or Euler angle is verified by two-scenario simulations, and another scenario is used to track and control the centroidal angular motion in the first-order kinematics level. The idea has considerable potential for system design, motion generation, and torque control in robotics communities with different research topics and theoretical backgrounds.

Keywords: Humanoid Robot Systems, SLAM, Computer Vision for Automation
Abstract: This paper proposes a framework for an autonomous humanoid robot, aimed at searching for a target object in an unknown environment using 3D-simultaneous localization and mapping (SLAM). The robot determines the next viewpoint in real-time from an environment map and object recognition results, and automatically finds and grasps the target object. Whereas most robot exploration studies require a static map, hints regarding object position, or area size limitations, our system can globally find an occluded object in an unknown environment, based only on the 3D target model. Notably, our robot can predict an unobserved area, and actively reveal it while avoiding obstacles. Our 3D-SLAM approach also estimates a self-camera location in a world coordinate system, so that our robot can re-plan its footsteps while walking, without pause. We validated the efficacy of this method through real experiments with an "HRP2-KAI" in several environments, and achieved fully automated searching and grasping.

Keywords: Soft Robot Materials and Design, Soft Robot Applications, Mechanism Design
Abstract: Soft growing robots, extending via tip eversion, have been attracting a

lot of interests due to their unique locomotion. Although having a visual feedback from the tip of this type of robot would greatly enhance the usefulness of these robots for exploring in the field, because the material at the tip continually moves as the robot grows, mounting a camera to the tip of the robot and keeping it stay at the tip during growth has been a major challenge. While previous designs seriously impeded the robots' intrinsic advantages, what remains still open is how to keep the camera stay at the tip without encumbering the robots' natural ability to morph its shape and grow along or over obstacle. This paper, for the first time, proposes a method to keep a camera stay

at the tip during growth while maintaining the compliant nature of the robot. Origami-based material feeding mechanism is designed for securing the camera's path from the base to the tip, which allows controlling the camera growing speed independent of the robot. We experimentally investigate the required control parameters, and present a vision-based control method for the camera position adjustment. Finally, we demonstrate the feasibility of the proposed design in simulated experimental scenarios including a narrow passage and a cluttered environment.

Keywords: Prosthetics and Exoskeletons, Human-Robot Collaboration, Human Factors and Human-in-the-Loop
Abstract: Human-prosthetic interfaces require their settings to be tuned to individual users. This can potentially be done autonomously while the prosthesis user performs a task by using online personalisation algorithms. These online personalisation algorithms adjust the interface parameters to optimise a given measure of performance. For convergence to be reached, both the human and the personalisation algorithm need to optimise towards the same objective. To date, task-oriented measures of performance have been utilised as the objective, requiring explicit feedback of the measure of performance to the prosthesis user, which is not practical. In this paper, the use of inherent human motor behaviour as the measure of performance for online personalisation algorithms is proposed and investigated. This allows the personalisation procedure to occur without the prosthesis user needing explicit knowledge of the measure of performance. The methodology for formulating inherent human motor behaviour within the framework of online personalisation of human-prosthetic interfaces is presented and validated through an experiment with nine able-bodied subjects. Experimental results demonstrate the efficacy of inherent human motor behaviour-based measures of performance in the design of an intuitive human-prosthetic interface specifically, applicable to human-robot interaction in general.

Keywords: Medical Robots and Systems, Haptics and Haptic Interfaces, Education Robotics
Abstract: To fulfill the need of reliable and consistent medical training of the neurological examination technique to assess ankle clonus, a series elastic actuator (SEA) based haptic training simulator was proposed and developed. The simulator’s mechanism and controller were designed to render a quiet and safe training environment. Benchtop tests demonstrated that the prototype simulator was able to accurately estimate the interaction torque from the trainee and closely track a chirp torque command up to 10 Hz. The high-level impedance controller could switch between different clinically encountered states based on trainee’s assessment technique. The simulator was evaluated by a group of 17 experienced physicians and physical therapists. Subjects were instructed to induce sustained clonus using their normal technique. The simulator was assessed in two common clinical positions (seated and supine). Subjects scored simulation realism on a variety of control features. To expedite controller design iteration, feedback from Day 1 was used to modify simulation parameters prior to testing on Day 2 with a new subject group. On average, all subjects could successfully trigger a sustained clonus response within 4-5 attempts in the first position and 2-3 in the second. Feedback on the fidelity of simulation realism improved between Day 1 and Day 2. Results suggest that this SEA-based simulator could be a viable training tool for healthcare trainees learning to assess ankle clonus.

Keywords: Compliance and Impedance Control, Force Control, Compliant Joints and Mechanisms
Abstract: Accurate and wide-range stiffness control is important for safe human-robot interaction. Accurate stiffness control can be better achieved using series elastic actuators (SEAs) than conventional rigid actuators. However, the stable range of virtual stiffness rendered by SEAs is limited by the stiffness of the actual spring, which cannot be too high in order to ensure good force control accuracy. Adding a virtual damper or derivative gain can increase the stable range of virtual stiffness, but the stable range would highly depend on the environmental stiffness. To relax the stiffness limitation in uncertain environments and explore more merits of SEAs, this paper proposes a hybrid impedance controller. This new controller linearly combines the spring force feedback and inertia force feedback. The stable range of virtual stiffness can be easily increased to ten times the actual spring stiffness with minimum effect on the force control accuracy. Unlike typical impedance controllers, the environmental stiffness can be used to raise the stable range of stiffness and hence the robustness of the controller can be ensured. Experiments will be provided to verify the hybrid impedance controller. We expect that the hybrid impedance controller can be used for SEAs in unstructured environments to provide a wide range of virtual stiffness.

Keywords: Assembly, Soft Robot Applications, Industrial Robots
Abstract: To design a general-purpose assembly robot system that can handle objects of various shapes, we propose a soft jig that fits to the shapes of assembly parts. The functionality of the soft jig is based on a jamming gripper developed in the field of soft robotics. The soft jig has a bag covered with a malleable silicone membrane, which has high friction, elongation, and contraction rates for keeping parts fixed. The bag is filled with glass beads to achieve a jamming transition. We propose a method to configure parts-fixing on the soft jig based on contact relations, reachable directions, and the center of gravity of the parts that are fixed on the jig. The usability of the soft jig was evaluated in terms of the fixing performance and versatility for various shapes and postures of parts.

Keywords: Perception for Grasping and Manipulation, Telerobotics and Teleoperation, Product Design, Development and Prototyping
Abstract: It is expected that the use of haptic information will improve the operability of remote machine systems. In this paper, for an object grasping task with the remotely operated arm, we compare three modal feedbacks based on either sound, vibration, and light as pseudo-haptic information of contact with the object. An experimental evaluation test was carried out by measuring the gripping force and brain waves for seven subjects. It was verified that the feedback with light can minimize the gripping force and suppress the processing load in the brain. This result supports the visual force tactile method, which superimposes haptic information on the fingertip contact point of a remote machine as an image to improve a remote machine operation system with high operability without the need of highly complex and expensive interface.

Keywords: Physical Human-Robot Interaction, Human Factors and Human-in-the-Loop, Neurorobotics
Abstract: In physical human-robot collaboration (pHRC), singularity avoidance strategies are often critical to obtain stable interaction dynamics. It is hypothesised a predictable singularity avoidance strategy is preferred in pHRC as humans tend to maximise predictability when using complex systems. By using an electroencephalogram (EEG), it is possible to assess the predictability of a task through a feature found in event-related potentials (ERP) and called prediction-error negativity (PEN). In this paper, two research questions are addressed. Can a complex pHRC singularity avoidance strategy generate a detectable PEN? Are PEN and human preferences related when comparing different control settings in a singularity avoidance strategy? Fourteen participants compared two different sets of parameters (modes) in a singularity avoidance strategy based on the exponentially damped least-squared (EDLS) method. ERP results are presented in terms of power spectral density (PSD). ERP results were then compared with human preferences to see whether they are related. Results show that the mode that causes PEN is also the one that participants did not like, suggesting that a lack of predictability might have an impact on human preference.

Keywords: Physical Human-Robot Interaction, Force and Tactile Sensing
Abstract: A human can socially interact in a non-verbal manner by understanding the intention behind a tactile stimulus. Patting on one's back is one of tactile communications, which is considered as a sign of encouragement in most cultures. The majority of such tactile communication is carried out by a dynamic tactile on large passive body parts and differently interpreted by how and where on the body is touched. Thus any robotic system that physically interacts with a human requires a dynamic tactile sensor for further social interaction. This paper presents a large dynamic tactile sensor that could cover a robot's passive body parts using a few sparsely distributed microphones to cover a large area in an efficient manner. A porous structured mesh, neoprene, and loop fabric are used to form a sensor's skin that could well generate and transfer a signal to distributed microphones when a touch is introduced. TDOA source localisation algorithms are implemented to find the touch point locating in between the distributed microphones, and a simple convolutional neural network is trained to classify a type of the touch. A localising performance is qualitatively achieved in a testbed of the sensor and applied to a mannequin's back to show the applicability, which classified a touch into six classes with an accuracy of 88 %.

Keywords: Autonomous Agents, Multi-Robot Systems, Deep Learning Methods
Abstract: Motion prediction of multiple agents in a dynamic scene is a crucial component in many real applications, including intelligent monitoring and autonomous driving. Due to the complex interactions among the agents and their interactions with the surrounding scene, accurate trajectory prediction is still a great challenge. In this paper, we propose a new method for robust trajectory prediction of multiple intelligent agents in a dynamic scene. The input of the method includes the observed trajectories of all agents, and optionally, the planning of the ego-agent and the surrounding high definition map at every time steps. Given observed trajectories, an efficient approach in a star computational topology is utilized to compute both the spatiotemporal interaction features and the current interaction features between the agents, where the time complexity scales linearly to the number of agents. Moreover, on an autonomous vehicle, the proposed prediction method can make use of the planning of ego-agent to improve the modeling of the interaction between surrounding agents. To increase the robustness to upstream perception noises, at the training stage, we randomly mask out the input data, a.k.a. the points on the observed trajectories of agents and the lane sequence. Experiments on autonomous driving and pedestrian-walking datasets demonstrate that the proposed method is not only effective when the planning of ego-agent and the high definition map are provided, but also achieves stat

Keywords: Robotics and Automation in Construction, Deep Learning in Grasping and Manipulation, Reinforcement Learning
Abstract: A method that enables an industrial robot to accomplish the peg-in-hole task for holes in concrete is proposed. The proposed method involves slightly detaching the peg from the wall, when moving between search positions, to avoid the negative influence of the concrete's high friction coefficient. It uses a deep neural network (DNN), trained via reinforcement learning, to effectively find holes with variable shape and surface finish (due to the brittle nature of concrete) without analytical modeling or control parameter tuning. The method uses displacement of the peg toward the wall surface, in addition to force and torque, as one of the inputs of the DNN. Since the displacement increases as the peg gets closer to the hole (due to the chamfered shape of holes in concrete), it is a useful parameter for inputting in the DNN. The proposed method was evaluated by training the DNN on a hole 500 times and attempting to find 12 unknown holes. The results of the evaluation show the DNN enabled a robot to find the unknown holes with average success rate of 96.1% and average execution time of 12.5 seconds. Additional evaluations with random initial positions and a different type of peg demonstrate the trained DNN can generalize well to different conditions. Analyses of the influence of the peg displacement input showed the success rate of the DNN is increased by utilizing this parameter. These results validate the proposed method in terms of its effectiveness in the construction field.

Keywords: Field Robots, Semantic Scene Understanding, Robotics and Automation in Construction
Abstract: Semantic maps are an important tool to provide robots with high-level knowledge about the environment, enabling them to better react to and interact with their surroundings. However, as a single measurement of the environment is solely a snapshot of a specific time, it does not necessarily reflect the underlying semantics. In this work, we propose a method to create a semantic map of a construction site by fusing multiple daily data. The construction site is measured by an unmanned aerial vehicle (UAV) equipped with a LiDAR. We extract clusters above ground level from the measurements and classify them using either a random forest or a deep learning based classifier. Furthermore, we combine the classification results of several measurements to generalize the classification of the single measurements and create a general semantic map of the working site. We measured two construction fields for our evaluation. The classification models can achieve an average intersection over union (IoU) score of 69.2% during classification on the Sanbongi field, which is used for training, validation and testing and an IoU score of 49.16% on a hold-out testing field. In a final step, we show how the semantic map can be employed to suggest a parking spot for a dump truck, and in addition, show that the semantic map can be utilized to improve path planning inside the construction site.

Keywords: Field Robots, AI-Based Methods, Task Planning
Abstract: We present a novel task planner - TaskNet for an autonomous excavator based on a data-driven method, which plans feasible task-level sequence by learning from demonstration data. Given a high-level excavation objective, our TaskNet planner can decompose it into sub-tasks, each of which can be further decomposed into task primitives with specifications. We train our TaskNet using an excavation trace generator and evaluate its performance using a 3D physically-based terrain and excavator simulator. As compared to imitation learning-based methods, the experimental results show that TaskNet can effectively learn task decomposition strategies. The resulting sequences of task primitives can be used as inputs by any excavator motion planner for generating feasible joint-level trajectories. We further validate TaskNet on a state-of-the-art autonomous excavator hardware and software system. The 49-ton autonomous excavator can successfully perform material loading tasks.

Keywords: Marine Robotics, Dynamics, Underactuated Robots
Abstract: A well known and well-studied feature of boats' dynamic is the effect of steering-induced roll. This property is used as a technique to stabilise ships in waves, called rudder roll stabilisation (RRS) to make the navigation safer and more pleasant. This technique is based on the generation of induced roll. Because of its specific application, studies have been limited to commercial vessels using a single propeller-rudder system (SPRS). This study not only broadens the technique to any propulsion and steering mechanisms that can be used with RRS by introducing the thrust asymmetry, but it also incorporates the effect of the centrifugal forces that were previously left off. To prove the capabilities of the new concept, a test is effectuated employing an RC demonstrator fitted with a differential jet pump system (DJPS), performing a turning test manoeuvre.

Keywords: Grasping, Grippers and Other End-Effectors
Abstract: This paper introduces a new method for simultaneously singulating and picking objects from clutter. The method can lead to effective robotic bin picking, which still remains elusive despite its importance in many industrial and domestic applications, especially for objects with a thin profile. We leverage planar quasistatic pushing manipulation as a way of standardized physical interaction between a robot and the object to pick. A gripper designed with digit asymmetry, realized as a two-fingered gripper with different finger lengths, is suggested as the key to successful singulating and picking through the controlled pushing maneuver. A detailed account of the manipulation process and design principles will be presented. An extensive set of experiments validate the effectiveness of our approach in three-dimensional bin picking tasks. Beyond picking, more complex manipulation capabilities such as autonomous pick-and-place/pack will also be presented.

Keywords: Deep Learning in Grasping and Manipulation, Grasping
Abstract: Food packing industry workers typically pick a target amount of food by hand from a food tray and place them in containers. Since menus are diverse and change frequently, robots must adapt and learn to handle new foods in a short time-span. Learning to grasp a specific amount of granular food requires a large training dataset, which is challenging to collect reasonably quickly. In this study, we propose ways to reduce the necessary amount of training data by augmenting a deep neural network with models that estimate its uncertainty through self-supervised learning. To further reduce human effort, we devise a data collection system that automatically generates labels. We build on the idea that we can grasp sufficiently well if there is at least one low-uncertainty (high-confidence) grasp point among the various grasp point candidates. We evaluate the methods we propose in this work on a variety of granular foods- coffee beans, rice, oatmeal and peanuts, each of which has a different size, shape and material properties such as volumetric mass density or friction. For these foods, we show significantly improved grasp accuracy of user-specified target masses using smaller datasets by incorporating uncertainty.

Keywords: Deep Learning in Grasping and Manipulation, Perception for Grasping and Manipulation
Abstract: Recently, tactile sensing has attracted great interest for robotic manipulation. Predicting if a grasp will be stable or not, i.e. if the grasped object will drop out of the gripper while being lifted, can aid robust robotic grasping. Previous methods paid equal attention to all regions of the tactile data matrix or all time-steps in the tactile sequence, which may include irrelevant or redundant information. In this paper, we propose to equip Convolution Neural Networks with spatial-channel and temporal attention mechanisms (SCT attention CNN) to predict future grasp stability. To the best of our knowledge, this is the first time to use attention mechanisms for predicting grasp stability only relying tactile information. We implement our experiments with 52 daily objects. Moreover, we compare different spatio-temporal models and attention mechanisms as an empirical study. We found a significant accuracy improvement of up to 5% when using SCT attention. We believe that attention mechanisms can also improve the performance of other tactile learning tasks in the future, such as slip detection and hardness perception.

Keywords: Force and Tactile Sensing, In-Hand Manipulation, Grippers and Other End-Effectors
Abstract: This paper studies the problem of controlling the sliding motion of an object held by a robot manipulator. We show how a parallel-jaw gripper can reliably control the motion of a rigid, prism-like object by 1) estimating the object's sliding velocity using measurements from tactile sensors at the gripper's fingertips and 2) controlling the grip strength to regulate the sliding velocity. We first train a neural network to estimate the sliding velocity from only tactile signals with data of tactile sensor measurements associated with various sliding velocities determined by an external motion capture system from repeated sliding trials of 28 different objects varying in size, shape, and surface texture. The velocity estimates from the neural network are then used as feedback for a closed-loop grip controller that maintains the desired sliding velocity. Experimental results show that our neural network estimates the object's sliding velocity with mean squared error under 0.5 (cm/s)^2, generalizes well to objects of new shapes and surface textures, and enables our closed loop grip controller to reliably slide objects at different target velocities.

Keywords: Simulation and Animation, Biologically-Inspired Robots
Abstract: Multiphysics simulation of magnetically actuated origami robots promises a range of applications such as synthetic data generation, design parameter optimization, predicting the robot’s performance, and implementing various control algorithms, but has rarely been explored. This paper presents a realistic multiphysics simulation of magnetically actuated origami robots implemented in Grasshopper3D using the Kangaroo plug-in. Due to the interaction between multiple magnets, the complex motion dynamics of a worm-like robot are generated and analyzed. We further show the possibility of accurately simulating origami structures made of different materials and permanent magnets of different shapes, sizes, and magnetic strength. The simulation’s unknown parameters are determined by conducting similar practical and simulated experiments and comparing their characteristics. Further, we show the close resemblance between the real and simulated behavior of the origami robot.

Keywords: Legged Robots, Mechanism Design, Robotics and Automation in Construction
Abstract: In this paper, we present a spherical magnetic joint for the inverted locomotion of a multi-legged robot. The permanent magnet's spherical shape allows the robot to attach its foot to a steel surface without energy consumption. However, the robot's inverted locomotion requires foot flexibility for placement and gait construction of the robot. Therefore, the spherical magnetic joint mechanism was designed and implemented for the robot feet to deal with angular placement. For decoupling the foot from the steel surface, the attractive force is adjusted by tilting the adjustable sleeve mechanism at an adequate angle between the surface and foot tip. Experimental results show that the spherical magnetic joint can maintain the attractive force at any angle, and the sleeve mechanism can reduce 20% of the reaction force for pulling the legs from the steel surfaces. Furthermore, the designed gait for inverted locomotion with a spherical magnetic joint was tested and compared to prove the concept of the spherical magnetic joint and sleeve mechanism.

Keywords: Multifingered Hands, Mechanism Design, Grasping
Abstract: This paper presents a new open-source mechanical design of a 6-DOF anthropomorphic ALARIS robotic hand that can serve as a low-cost design platform for further customization and utilization for research and educational purposes. The presented hand design employs linkage-based three-phalange finger and two-phalange adaptive thumb designs with non-backdrivable worm-and-rack transmission mechanisms. Combination of design improvements and solutions, discussed in the paper, are implemented in a functional robotic hand prototype with powerful grasping capabilities, which utilizes off-the-shelf inexpensive components and 3D printing technology ensuring the hand low manufacturing cost and replicability. The open-source mechanical design of the presented ALARIS robotic hand is freely available for downloading from the authors’ research lab web-site https://www.alaris.kz and https://github.com/alarisnu/alaris_hand.

Keywords: Tendon/Wire Mechanism, Modeling, Control, and Learning for Soft Robots, Biomimetics
Abstract: We have implemented a force-based operational space controller on a physical musculoskeletal humanoid robot arm. The controller calculates muscle activations based on a biomimetic Hill-type muscle model. We propose a method to include the joint torque nullspace in the optimization process, which enables the robot to exploit the nullspace to gradually lower its overall muscle activation. We have verified in experiments that it can react compliantly to external disturbances while retaining its operational space task.

Keywords: Micro/Nano Robots, Swarm Robotics, Biologically-Inspired Robots
Abstract: After years of development, various swarms of robots have been proposed for many complicated tasks, such as forming patterns, cooperative locomotion, and adapting to different environments. However, controlling microrobotic swarms is still a challenging task owing to the lacking of integrated devices on the small-scale agents, and actuation of multiple microrobotic swarms to different targets under the same global input will be even more difficult. In this work, we present a swarm of nickel nanorods and its diverse locomotion velocity compared with Fe3O4 nanoparticle swarms is implemented for actuating the two swarms to different targets under the same customized oscillating magnetic field. The effects of the magnetic anisotropy of agents on the macroscopic swarm behaviour are analysed theoretically. To prove the strategy, the speeds of the two swarms were characterized through experiments, and demonstrations were conducted to show the capability of driving the two swarms to different locations in the same environment. Furthermore, parallel locomotion of the two swarms towards opposite directions was also achieved on a tilted substrate. This work has proved the feasibility of simultaneously actuating two swarms to diverse targets and promoted fundamental understandings of microrobotic swarms.

Keywords: Automation at Micro-Nano Scales, Visual Servoing, Industrial Robots
Abstract: In the context of robotic high-precision soldering, we propose an image-based pin alignment control method based on active plastic deformation. The plastic deformation is a well-known failure mechanism in most situations, which includes a phenomenon that the objects do not return original state. Here, in contrast to this convention, we utilize the plastic deformation of the metal pin to do pin alignment for improving the quality of the solder joint. To address this, we embed the springback compensation into the image-based pin alignment controller. Lastly, the proposed strategy is successfully demonstrated and evaluated in a practical modified robotic manipulation system. The result shows that the alignment error is less than 20 μm, which is far less than pin alignment without considering plastic deformation and elastic recovery. This work considers active plastic deformation and spontaneous elastic recovery of soft object, which would greatly promote the use of robotics in micromanufacturing and microfabrication in lad and industry, especially for soft objects.

Keywords: Micro/Nano Robots, Automation at Micro-Nano Scales, Biological Cell Manipulation
Abstract: Three-dimensional multi-layer microfluidic device (MMD) fabricated by polydimethylsiloxane (PDMS) is a solution for on-chip high-complexity serial or parallel processes. In this paper, we propose and set up an automated 8-DOF alignment system assisted by computer vision, which is capable of automatic leveling, aligning, and in-situ bonding of multiple PDMS layers. A microscope is motorized by a Z stage with a grating ruler to optically focus the marks and record the Z positions. A 3-PRS mechanism with flexible hinges is proposed to achieve the leveling of the top layer. An XYZR platform is utilized for in-plane alignment of the top layer and moving the top layer down to bond with the bottom layer. The Z positions of marks, translational and rotational offsets obtained by image processing are used for the automated leveling and aligning of the PDMS layers. Experimental results showed that translational and rotational alignment errors are less than 5 μm and 0.1°, respectively. The whole bonding procedure, including the plasma treatment, took the time shorter than 3 mins. Finally, a fabricated 3-layer microfluidic device with a deformable microchannel is applied to the cell squeezing and proved that the proposed alignment system has great potential in developing the functional MMDs.

Keywords: Motion Control, Automation at Micro-Nano Scales, Grippers and Other End-Effectors
Abstract: Robotic handling of objects in the micro-world is a challenging problem. Due to the natural differences between micro-manipulation and robotic manipulators, it is difficult to produce such a micro-hand which can function as robotic hands in the physical world. In this paper, we propose a robotic handling approach for micro-objects using stochastic optically-actuated end-effector. An end-effector is first constructed by several optically-actuated micro-particles so as to perform an autonomous grasping task on a micro-object. Once being grasped by the end-effector, the object is maneuvered by using a robotic stage which acts as a robotic mobile base. This paper, therefore, offers a robotic control approach which is able to perform a ``pick and place" task in the micro-world. In the proposed approach, the Brownian effect on the optically-actuated end-effector is considered so as to reflect the real essence of optical tweezing. Besides, with the usage of the end-effector for stable grasping of the target object, the Brownian effect on the grasped object can also be reduced. In the proposed approach, a simple feedback control method is utilized for manipulation of the grasped object, and thus further enhancing the robustness of the control system in the presence of Brownian perturbations. Rigorous mathematical formulation and stability analysis is carried out, and experimental validation is performed to illustrate the feasibility of the robotic handling approach.

Keywords: Optimization and Optimal Control, Sensor Networks
Abstract: This paper considers a class of switching mixed data injection attacks using input derivatives in cyber-physical systems with linear quadratic(LQ) cost from the perspective of the attacker. The attacker injects data mixed with false data and its derivative into a healthy system, which destroys the performance of the original system. For this situations, the optimal mixed data injection attack strategy is designed to minimize the quadratic cost and most damage the performance of the system. After that, in order to increase the complexity, concealment and energy saving of the attack, we designed the optimal switching mixed data injection attack strategy. Finally, numerical results and comparative experiments are provided to illustrate the effectiveness of the proposed method.

Keywords: Reinforcement Learning, Performance Evaluation and Benchmarking, Simulation and Animation
Abstract: Recent advances in deep reinforcement learning (deep RL) enable researchers to solve challenging control problems, from simulated environments to real-world robotic tasks. However, deep RL algorithms are known to be sensitive to the problem formulation, including observation spaces, action spaces, and reward functions. There exist numerous choices for observation spaces but they are often designed solely based on prior knowledge due to the lack of established principles. In this work, we conduct benchmark experiments to verify common design choices for observation spaces, such as Cartesian transformation, binary contact flags, a short history, or global positions. Then we propose a search algorithm to find the optimal observation spaces, which examines various candidate observation spaces and removes unnecessary observation channels with a Dropout-Permutation test. We demonstrate that our algorithm significantly improves learning speed compared to manually designed observation spaces. We also analyze the proposed algorithm by evaluating different hyperparameters.

Keywords: AI-Based Methods, Agent-Based Systems
Abstract: The “Thinking, Fast and Slow” paradigm of Kahneman proposes that we use two different styles of thinking---a fast and intuitive System 1 for certain tasks, along with a slower but more analytical System 2 for others. While the idea of using this two-system style of thinking is gaining popularity in AI and robotics, our work considers how to interleave the two styles of decision-making, i.e., how System 1 and System 2 should be used together. For this, we propose a novel and general framework which includes a new System 0 to oversee Systems 1 and 2. At every point when a decision needs to be made, System 0 evaluates the situation and quickly hands over the decision-making process to either System 1 or System 2. We evaluate such a framework on a modified version of the classic Pac-Man game, with an already-trained RL algorithm for System 1, a Monte-Carlo tree search for System 2, and several different possible strategies for System 0. As expected, arbitrary switches between Systems 1 and 2 do not work, but certain strategies do well. With System 0, an agent is able to perform better than one that uses only System 1 or System 2.

Keywords: Probabilistic Inference, Probability and Statistical Methods, Model Learning for Control
Abstract: Learning the uncertain dynamical environments for online learning and prediction from noisy sensory measurement streams is essential for various tasks in robotics. Recently, Gaussian process (GP) online learning such as an infinite-horizon Gaussian process (IHGP) has shown effectiveness to cope with non-stationary dynamical random processes in learning hyperparameters online by reducing the computational cost. However, the IHGP was originally proposed to deal with only a single-output. Therefore, to tackle complex real-world problems, we propose a multi-output infinite-horizon Gaussian process (MOIHGP) that generalizes the single-output IHGP to deal with multiple outputs for better prediction. Our approach allows us to consider correlations between multiple outputs for better prediction, even with occlusions in a Bayesian way. Finally, we successfully demonstrate the effectiveness of our approach by benchmark and experimental results. For simulated benchmark experiments with high noise levels, our approach reduced 16.6% of the averaged RMSE value achieved by the single-output IHGP.

Keywords: Aerial Systems: Applications, Motion and Path Planning, Collision Avoidance
Abstract: It is ubiquitously accepted that during the autonomous navigation of the quadrotors, one of the most widely adopted unmanned aerial vehicles (UAVs), safety always has the highest priority. However, it is observed that the ego airflow disturbance can be a significant adverse factor during flights, causing potential safety issues, especially in narrow and confined indoor environments. Therefore, we propose a novel method to estimate and adapt indoor ego airflow disturbance of quadrotors, meanwhile applying it to trajectory planning. Firstly, the hover experiments for different quadrotors are conducted against the proximity effects. Then with the collected acceleration variance, the disturbances are modeled for the quadrotors according to the proposed formulation. The disturbance model is also verified under hover conditions in different reconstructed complex environments. Furthermore, the approximation of Hamilton-Jacobi reachability analysis is performed according to the estimated disturbances to facilitate the safe trajectory planning, which consists of kinodynamic path search as well as B-spline trajectory optimization. The whole planning framework is validated on multiple quadrotor platforms in different indoor environments.

Keywords: Aerial Systems: Perception and Autonomy, Motion and Path Planning
Abstract: This article proposes a new method that enables Unmanned Aerial Vehicles (UAVs) to actively find targets and shoot photographs of them in an unknown environment, while successfully avoiding surrounding obstacles and planning optimize routes. Owing to the limited computing ability on the UAVs, we obtained the point cloud data of surrounding objects, and selected the best segmentation method of the point cloud to perform real-time semantic segmentation on the collected point cloud data. The point cloud data with semantic attributes were merged into voxels. We reconstruct the real-time distance and angle between the surface of obstacles and the surrounding obstacles through Euclidean Signed Distance Fields (ESDFs), and adjust the gimbal angle and focal length of UAVs and use the two-dimensional image recognition to shoot the photographs of the target precisely. Considering the increasing scale of UAVs power inspections, we can improve the efficiency of fine inspections of power transmission lines by using the method we proposed.

Keywords: Aerial Systems: Applications, Collision Avoidance
Abstract: The quadrotor is popularly used in challenging environments due to its superior agility and flexibility. In these scenarios, trajectory planning plays a vital role in generating safe motions to avoid obstacles while ensuring flight smoothness. Although many works on quadrotor planning have been proposed, a research gap exists in incorporating self-adaptation into a planning framework to enable a drone to automatically fly slower in denser environments and increase its speed in a safer area. In this paper, we propose an environmental adaptive planner to adjust the flight aggressiveness effectively based on the obstacle distribution and quadrotor state. Firstly, we design an environmental adaptive safety aware method to assign the priority of the surrounding obstacles according to the environmental risk level and instantaneous motion tendency. Then, we apply it into a multi-layered model predictive contouring control (Multi-MPCC) framework to generate adaptive, safe, and dynamical feasible local trajectories. Extensive simulations and real-world experiments verify the efficiency and robustness of our planning framework. Benchmark comparison also shows superior performances of our method with another advanced environmental adaptive planning algorithm. Moreover, we release our planning framework as open-source ros-packages}.

Keywords: Motion and Path Planning, Autonomous Vehicle Navigation, Aerial Systems: Applications
Abstract: Gradient-based planners are widely used for quadrotor local planning, in which a Euclidean Signed Distance Field (ESDF) is crucial for evaluating gradient magnitude and direction. Nevertheless, computing such a field has much redundancy since the trajectory optimization procedure only covers a very limited subspace of the ESDF updating range. In this paper, an ESDF-free gradient-based planning framework is proposed, which significantly reduces computation time. The main improvement is that the collision term in penalty function is formulated by comparing the colliding trajectory with a collision-free guiding path. The resulting obstacle information will be stored only if the trajectory hits new obstacles, making the planner only extract necessary obstacle information. Then, we lengthen the time allocation if dynamical feasibility is violated. An anisotropic curve fitting algorithm is introduced to adjust higher order derivatives of the trajectory while maintaining the original shape. Benchmark comparisons and real-world experiments verify its robustness and high-performance. The source code is released as ros packages.

Keywords: Sensor Fusion, Engineering for Robotic Systems
Abstract: A novel 3-dimensional (3-D) alignment method for point-cloud registration is proposed where the time-differential information of the measured points is employed. The new problem turns out to be a novel multi-dimensional optimization. Analytical solution to this optimization is then obtained, which sets the ground of further correspondence matching using k-D trees. Finally, via many examples, we show that the new method owns better registration accuracy in real-world experiments.

Keywords: Probabilistic Inference, Motion and Path Planning, Robot Safety
Abstract: This paper addresses the safe path planning problem for autonomous mobility with multi-modal perception uncertainties. Specifically, we assume that different sensor inputs lead to different Gaussian process regulated perception uncertainties (named as multi-modal perception uncertainties). We implement a Bayesian inference algorithm which merges the multi-modal GP-regulated uncertainties into a unified one and translates the unified uncertainty into a dynamic risk map. With the safe path planner taking the risk map as input, we are able to plan a safe path for the autonomous vehicle to follow. Experimental results on an autonomous golf cart testbed validate the applicability and efficiency of the proposed algorithm.

Keywords: Social HRI, Human-Aware Motion Planning, Acceptability and Trust
Abstract: Moving in dynamic pedestrian environments is one of the important requirements for autonomous mobile robots. We present a model-based reinforcement learning approach for robots to navigate through crowded environments. The navigation policy is trained with both real interaction data from multi-agent simulation and virtual data from a deep transition model that predicts the evolution of surrounding dynamics of mobile robots. A reward function considering social conventions is designed to guide the training of the policy. Specifically, the policy model takes laser scan sequence and robot's own state as input and outputs steering command. The laser sequence is further transformed into stacked local obstacle maps disentangled from robot's ego motion to separate the static and dynamic obstacles, simplifying the model training. We observe that the policy using our method can be trained with significantly less real interaction data in simulator but achieve similar level of success rate in social navigation tasks compared with other methods. Experiments are conducted in multiple social scenarios both in simulation and on real robots, the learned policy can guide the robots to the final targets successfully in a socially compliant manner. Code is available at https://github.com/YuxiangCui/model-based-social-navigation.

Keywords: Human-Aware Motion Planning, Probabilistic Inference, Motion and Path Planning
Abstract: In this paper, we present a novel approach that predicts spatially and temporally crowd behaviour for robotic social navigation. Integrating mobile robots into human society involves the fundamental problem of navigation in crowds. A robot should attempt to navigate in a way that is minimally invasive to the humans in its environment. However, planning in a dynamic environment is difficult as the environment must be predicted into the future. This problem has been thoroughly studied considering the behaviour of pedestrians at the level of individuals. Instead, we represent a pedestrian crowd by its macroscopic properties over space, such as density and velocity. With this spatial representation, we propose to learn a convolutional recurrent model to predict these properties into the future. The key design of a probabilistic loss function capturing the crowd's macroscopic properties empowers the spatio-temporal crowd prediction. Using a social invasiveness metric defined on these properties predicted by our convolutional recurrent model, we develop a framework that produces globally-optimal plans in expectation. Extensive results using a realistic pedestrian simulator show the validity and performance of the proposed social navigation approach.

Keywords: Path Planning for Multiple Mobile Robots or Agents, AI-Based Methods, Distributed Robot Systems
Abstract: Multi-agent path finding (MAPF) is an indispensable component of large-scale robot deployments in numerous domains ranging from airport management to warehouse automation. In particular, this work addresses lifelong MAPF (LMAPF) for real-world warehouse operations, an online variant of the problem where agents are immediately assigned a new goal upon reaching their current one. Effectively solving LMAPF in such dense and highly structured environments requires expensive coordination between agents as well as frequent replanning abilities, a daunting task for existing coupled and decoupled approaches alike. With the purpose of achieving considerable agent coordination without any compromise on reactivity and scalability, we introduce PRIMAL2, a distributed RL framework for LMAPF where agents learn fully decentralized policies to reactively plan paths online in a partially observable world. We extend our previous work to dense and highly structured worlds by identifying behaviors and conventions which improve implicit agent coordination, and enable their learning through the construction of a novel local agent observation and various training aids. We present extensive results of PRIMAL2 in both MAPF and LMAPF environments and compare its performance to SotA planners in terms of makespan and throughput. We show that PRIMAL2 significantly surpasses our previous work and performs comparably to these baselines, while allowing real-time re-planning and scaling up to 2048 agents.

Keywords: Swarm Robotics, Multi-Robot Systems, Robot Safety
Abstract: In swarm control, many robots coordinate their actions in a distributed and decentralized way. We propose a consensus-based control barrier function (CCBF) for a swarm. CCBF restricts the states of the whole distributed system, not just those of the individual robots. The barrier function is approximated by a consensus filter. We prove that CCBF constrains the control inputs for holding the forward invariance of the safety set. Moreover, we applied CCBF to a practical problem and conducted an experiment with actual robots. The results showed that CCBF restricted the states of multiple robots to the safety set. To the best of our knowledge, this is the first CBF that can restrict the state of the whole distributed system with only local communication. CCBF has various applications such as monitoring with a swarm and maintaining the network between a swarm and a base station.

Keywords: Imitation Learning, Learning from Demonstration
Abstract: Scenarios requiring humans to choose from multiple seemingly optimal actions are commonplace, however standard imitation learning often fails to capture this behavior. Instead, an over-reliance on replicating expert actions induces inflexible and unstable policies, leading to poor generalizability in an application. To address the problem, this paper presents the first imitation learning framework that incorporates Bayesian variational inference for learning flexible non-parametric multi-action policies, while simultaneously robustifying the policies against sources of error, by introducing and optimizing disturbances to create a richer demonstration dataset. This combinatorial approach forces the policy to adapt to challenging situations, enabling stable multi-action policies to be learned efficiently. The effectiveness of our proposed method is evaluated through simulations and real-robot experiments for a table-sweep task using the UR3 6-DOF robotic arm. Results show that, through improved flexibility and robustness, the learning performance and control safety are better than comparison methods.

Keywords: Multi-Robot Systems, Path Planning for Multiple Mobile Robots or Agents, Autonomous Vehicle Navigation
Abstract: This paper presents a novel robot-environment interaction in navigation tasks such that robots have neither a representation of their working space nor planning function, instead, an active environment takes charge of these aspects. This is realized by spatially deploying computing units, called cells, and making cells manage traffic in their respective physical region. Different from stigmegic approaches, cells interact with each other to manage environmental information and to construct instructions on how robots move.

As a proof-of-concept, we present an architecture called AFADA and its prototype, consisting of modular cells and robots moving on the cells. The instructions from cells are based on a distributed routing algorithm and a reservation protocol. We demonstrate that AFADA enables a robot to move efficiently in a dynamic environment that stochastic changes its topology, comparing to self-navigation by a robot itself. This is followed by several demos, including multi-robot navigation, highlighting the power of offloading both representation and planning from robots to the environment. We expect that the concept of AFADA contributes to developing the infrastructure for multiple robots because it can engage online and lifelong planning and execution.

Keywords: Cooperating Robots, Multi-Robot Systems, Distributed Robot Systems
Abstract: In this paper, we explore a multi-agent reinforcement learning approach to address the design problem of communication and control strategies for multi-agent cooperative transport. Typical end-to-end deep neural network policies may be insufficient for covering communication and control; these methods cannot decide the timing of communication and can only work with fixed-rate communications. Therefore, our framework exploits event-triggered architecture, namely, a feedback controller that computes the communication input and a triggering mechanism that determines when the input has to be updated again. Such event-triggered control policies are efficiently optimized using a multi-agent deep deterministic policy gradient. We confirmed that our approach could balance the transport performance and communication savings through numerical simulations.

Keywords: Multi-Robot Systems, Distributed Robot Systems, Planning, Scheduling and Coordination
Abstract: We propose a game-theoretic multi-robot task allocation framework that enables a large team of robots to optimally allocate tasks in dynamically changing environments. As our main contribution, we design a decision-making algorithm that defines how the robots select tasks to perform and how they repeatedly revise their task selections in response to changes in the environment. Our convergence analysis establishes that the algorithm enables the robots to learn and asymptotically achieve the optimal stationary task allocation. Through experiments with a multi-robot trash collection application, we assess the algorithm’s responsiveness to changing environments and resilience to failure of individual robots.

Keywords: Multi-Robot Systems, Perception-Action Coupling, Aerial Systems: Applications
Abstract: Heterogeneous multi-robot systems are advantageous for operations in unknown environments because functionally specialised robots can gather environmental information, while other robots perform desired tasks. We define this decomposition as the scout-task robot architecture} and show how it avoids the need to balance between exploration and exploitation by allowing the system to perform both simultaneously. The challenge is how to guide exploration in a way that improves overall system performance for time-limited tasks. We derive a novel upper confidence bound for simultaneous exploration and exploitation based on mutual information and present a general solution for scout-task coordination using decentralised Monte Carlo tree search. We evaluate the performance of our algorithms in a multi-drone surveillance scenario where scout robots are equipped with low resolution, long range sensors and task robots observe more detailed information using higher resolution, short range sensors. These results introduce a new class of coordination problems for heterogeneous teams with many practical applications beyond surveillance.

Keywords: Multi-Robot Systems, Surveillance Robotic Systems, Planning, Scheduling and Coordination
Abstract: The Patrolling Problem is a crucial feature of the surveillance task in defense and other establishments. Most of the works in the literature concentrate on reducing the Idleness value at each location in the environment. However, there are often a few prioritized locations that cannot be left unvisited beyond a certain Time Period. In this paper, we study the problem of Prioritized patrolling - the task of patrolling the given environment using multiple agents while ensuring the prioritized locations are visited within the pre-specified Time Period. We present a novel algorithm, namely, Time Period Based Patrolling (TPBP) algorithm, to solve the prioritized patrolling problem. It determines a sequence of walks for each agent online that complies with the Time Period requirement of the Priority nodes while reducing the Idleness of all the other nodes. We have tested and validated the algorithm using SUMO - a realistic simulator developed for traffic management. Since the existing strategies are not designed for Prioritized Patrolling, we show through comparison that proposed algorithm is required to solve the problem.

Keywords: Big Data in Robotics and Automation, Autonomous Vehicle Navigation, Human-Aware Motion Planning
Abstract: Critical for the coexistence of humans and robots in dynamic environments is the capability for agents to understand each other's actions, and anticipate their movements. This paper presents Stochastic Process Anticipatory Navigation (SPAN), a framework that enables nonholonomic robots to navigate in environments with crowds, while anticipating and accounting for the motion patterns of pedestrians. To this end, we learn a predictive model to predict continuous-time stochastic processes to model future movement of pedestrians. Anticipated pedestrian positions are used to conduct chance constrained collision-checking, and are incorporated into a time-to-collision control problem. An occupancy map is also integrated to allow for probabilistic collision-checking with static obstacles. SPAN is novel in incorporating continuous-time stochastic processes, produced by a learned model, into a control problem for anticipatory navigation. We demonstrate the capability of SPAN in crowded simulation environments, as well as with a real-world pedestrian dataset.

Keywords: Human-Centered Automation, Rehabilitation Robotics, Human Factors and Human-in-the-Loop
Abstract: Real-time lower limbs motion or gait measurement is an important part in human-robotic interaction for the control of robotic walkers and rehabilitation devices. Laser range finder or infrared sensor that is mounted on the device has been widely used in applications. Although these sensors can provide accurate horizontal motion information of lower limbs during human walking, it is still difficult to measure the angular motion of lower limbs due to their functional principles. Using inertial measurement units (IMU) can measure the angular motion of lower limbs, but it requires a large amount of IMU units for measurements of all lower limb segments. In this study, a novel method is developed for real-time monitoring lower limbs (shanks and thighs) motion in human walking using just two shank-mounted IMUs. A pose prediction model based on multiple linear regression and Kalman filter is proposed. The root-mean-square error (RMSE) of the thigh orientation and the knee joint angle estimation in sagittal plane are 6.1 ± 1.3 and 6.8 ± 1.4 degs, respectively. The RMSE of the ankle, knee, and hip position estimation are 4.2 ± 1.3, 4.2 ± 1.1 and 3.5 ± 0.9 cm, respectively.

Keywords: Field Robots, Search and Rescue Robots, Mapping
Abstract: This paper presents an autonomous navigation system for ground robots traversing aggressive unstructured terrain through a cohesive arrangement of mapping, deliberative planning and reactive behaviour modules. All systems are aware of terrain slope, visibility and vehicle orientation, enabling robots to recognize, plan and react around unobserved areas and overcome negative obstacles, slopes, steps, overhangs and narrow passageways. This is one of pioneer works to explicitly and simultaneously couple mapping, planning and reactive components in dealing with negative obstacles. The system was deployed on three heterogeneous ground robots for the DARPA Subterranean Challenge, and we present results in Urban and Cave environments, along with simulated scenarios, that demonstrate this approach.

Keywords: Motion and Path Planning, Deep Learning in Grasping and Manipulation, Deep Learning Methods
Abstract: This paper presents c2g-HOF networks which learn to generate cost-to-go functions for manipulator motion planning. The c2g-HOF architecture consists of a cost-to-go function over the configuration space represented as a neural network (c2g-network) as well as a Higher Order Function (HOF) network which outputs the weights of the c2g-network for a given input workspace. Both networks are trained end-to-end in a supervised fashion using costs computed from traditional motion planners. Once trained, c2g-HOF can generate a smooth and continuous cost-to-go function directly from workspace sensor inputs (represented as a point cloud in 3D or an image in 2D). At inference time, the weights of the c2g-network are computed very efficiently and near-optimal trajectories are generated by simply following the gradient of the cost-to-go function. We compare c2g-HOF with traditional planning algorithms for various robots and planning scenarios. The experimental results indicate that planning with c2g-HOF is significantly faster than other motion planning algorithms, resulting in orders of magnitude improvement when including collision checking. Furthermore, despite being trained from sparsely sampled trajectories in configuration space, c2g-HOF generalizes to generate smoother, and often lower cost, trajectories. We demonstrate cost-to-go based planning on a 7 DoF manipulator arm where motion planning in a complex workspace requires only 0.13 seconds for the entire trajectory.

Keywords: Field Robots, Nonholonomic Motion Planning, Dynamics
Abstract: Nowadays, various algorithms based on the Rapidly-exploring Random Tree (RRT) methods are utilized to solve motion planning problems. Based on the RRT*, we developed a novel reconnection method that enables the planner to directly generate a smooth curved trajectory. Meanwhile, kinodynamic constraints of the robots are considered to generate the control input, which improves the feasibility of the algorithm. The trajectory planned by the Smooth-RRT* is significantly suitable for the non-holonomic robots. Planning tests are conducted in four scenarios to demonstrate performance of the proposed algorithm in comparison with the original RRT* and kinodynamic-RRT (Kino-RRT). Smooth-RRT* yields shorter and smoother planned path in all the scenarios compared with the Kino-RRT. It finds a solution with fewer expansion nodes than the RRT* under the same time consumption. The results demonstrate that the proposed algorithm can generate a smooth trajectory satisfied with the kinodynamic constraints and ensure the asymptotic optimality.

Keywords: Task and Motion Planning, Hybrid Logical/Dynamical Planning and Verification, Manipulation Planning
Abstract: We propose a new optimization-based task and motion planning (TAMP) with signal temporal logic (STL) specifications for robotic sequential manipulation such as pick-and-place tasks. Given a high-level task specification, the TAMP problem is to plan a trajectory that satisfies the specification. This is, however, a challenging problem due to the difficulty of combining continuous motion planning and discrete task specifications. The optimization-based TAMP with temporal logic specifications is a promising method, but existing works use mixed integer problems (MIP) and do not scale well. To address this issue, in our approach, a new hybrid system model without discrete variables is introduced and combined with smooth approximation methods for STL. This allows the TAMP to be formulated as a nonlinear programming problem whose computational cost is significantly less than that of MIP. Furthermore, it is also possible to deal with nonlinear dynamics and geometric constraints represented by nonlinear functions. The effectiveness of the proposed method is demonstrated with both numerical experiments and a real robot.

Keywords: Reinforcement Learning, Machine Learning for Robot Control, Deep Learning Methods
Abstract: The recent remarkable progress of deep reinforcement learning (DRL) stands on regularization of policy for stable and efficient learning. A popular method, named proximal policy optimization (PPO), has been introduced for this purpose. PPO clips density ratio of the latest and baseline policies with a threshold, while its minimization target is unclear. As another problem of PPO, the symmetric threshold is given numerically while the density ratio itself is in asymmetric domain, thereby causing unbalanced regularization of the policy. This paper therefore proposes a new variant of PPO by considering a regularization problem of relative Pearson (RPE) divergence, so-called PPO-RPE. This regularization yields the clear minimization target, which constrains the latest policy to the baseline one. Through its analysis, the intuitive threshold-based design consistent with the asymmetry of the threshold and the domain of density ratio can be derived. Through four benchmark tasks, PPO-RPE performed as well as or better than the conventional methods in terms of the task performance by the learned policy.

Keywords: Manipulation Planning
Abstract: This work considers the optimal non-repetitive coverage tasks with a single non-redundant manipulator for the case when the object can be positioned at a predefined set of locations within the workcell. The scenario is often encountered in typical industrial settings, for instance when the object presents itself along a conveyor belt and its surface can not be serviced at a single location - the object being large or complex for that endeavour. Given the non-bijective nature of manipulator kinematics between task and joint space, without an explicit consideration of joint-space continuity during its construction, a continuous coverage path designed in task-space may easily be truncated into intermittent segments where the manipulator needs to adopt a different configuration to continue the task, resulting in manipulator motions where the end-effector will need to lift off the surface, an altogether undesirable characteristic affecting the quality of the final product for smooth operations on objects such as polishing, painting or deburring. In this work, a novel algorithm to optimally partition the task-space whilst considering the various finite locations where the object may be stationed is proposed that ensures joint-space coverage continuity with minimal lift-offs. Results from the algorithm being challenged to achieve coverage of a number of objects, both in simulation and in real tests with an industrial manipulator.

Keywords: Field Robots, Imitation Learning, Deep Learning Methods
Abstract: Detection of road curbs is an essential capability for autonomous driving. It can be used for autonomous vehicles to determine drivable areas on roads. Usually, road curbs are detected on-line using vehicle-mounted sensors, such as video cameras and 3-D Lidars. However, on-line detection using video cameras may suffer from challenging illumination conditions, and Lidar-based approaches may be difficult to detect far-away road curbs due to the sparsity issue of point clouds. In recent years, aerial images are becoming more and more available worldwide, and we find that the visual appearances between road areas and off-road areas are usually different in aerial images, so we propose a novel solution to detect road curbs off-line using aerial images. The input to our method is an aerial image, and the output is directly a graph (i.e., vertices and edges) representing road curbs. To this end, we formulate the problem as an imitation learning problem, and design a novel network and an innovative training strategy to train an agent to iteratively find the road-curb graph. The experimental results on a pubic dataset confirm the effectiveness and superiority of our method. This work is accompanied with a demonstration video and a supplementary document at https://sites.google.com/view/icurb.

Keywords: Motion and Path Planning, Collision Avoidance, Autonomous Vehicle Navigation
Abstract: This paper presents a search-based partial motion planner to generate dynamically feasible trajectories for car-like robots in highly dynamic environments. The planner searches for smooth, safe, and near-time-optimal trajectories by exploring a state graph built on motion primitives, which are generated by discretizing the time dimension and the control space. To enable fast online planning, we first propose an efficient path searching algorithm based on the aggregation and pruning of motion primitives. We then propose a fast collision checking algorithm that takes into account the motions of moving obstacles. The algorithm linearizes relative motions between the robot and obstacles and then check collisions by comparing a point-line distance. Benefiting from the fast searching and collision checking algorithms, the planner can effectively and safely explore the state-time space to generate near-time-optimal solutions. The results through extensive experiments show that the proposed method can generate feasible trajectories within milliseconds while maintaining a higher success rate than up-to-date methods, which significantly demonstrates its advantages.

Keywords: Multi-Robot Systems, Motion and Path Planning, Manipulation Planning
Abstract: This paper introduces a novel task-space decomposed motion planning framework for multi-robot simultaneous locomotion and manipulation. When several manipulators hold an object, closed-chain kinematic constraints are formed, and it will make the motion planning problems challenging by inducing lower-dimensional singularities. Unfortunately, the constrained manifold will be even more complicated when the manipulators are equipped with mobile bases. We address the problem by introducing a dual-resolution motion planning framework which utilizes a convex task region decomposition method, with each resolution tuned to efficient computation for their respective roles. Concretely, this dual-resolution approach enables a global planner to explore the low-dimensional decomposed task-space regions toward the goal, then a local planner computes a path in high-dimensional constrained configuration space. We demonstrate the proposed method in several simulations, where the robot team transports the object toward the goal in the obstacle-rich environments.

Keywords: Planning, Scheduling and Coordination, Path Planning for Multiple Mobile Robots or Agents, Motion and Path Planning
Abstract: Efficient recharging is an essential requirement for autonomous mobile robots. In an indoor robotic application, charging stations can be installed offline. However, frequent trips to the charging stations cause inefficiency in the performance of the mobile robots. In an outdoor environment, a charging station cannot even be installed easily. We propose a framework and algorithms for enabling a group of mobile wireless rechargers to fulfill the energy requirement of autonomous mobile robots in a workspace efficiently. Our algorithm finds the optimal trajectory for the mobile rechargers in such a way that once there is a need for a recharge, the robots do not need to spend significant time and energy to get access to a recharger. Our algorithm is based on a reduction of the problems to Satisfiability Modulo Theory (SMT) solving problems. We present extensive experimental results to show that the optimal trajectories for mobile rechargers can be generated successfully for different types of robots and workspaces within a reasonable time. Moreover, a comparison with the performance of static charging stations establishes that mobile rechargers are more effective in terms of allowing the autonomous robot to continue their work for a longer time.

Keywords: Motion and Path Planning, Collision Avoidance, Wheeled Robots
Abstract: The path planning of mobile robots is an optimization problem that is difficult to solve directly owing to its nonlinear characteristics. This paper proposes the "global-local" Coupling Two-Stage Path Planning (CTSP) method. First, the globally optimal solution in the configuration space is given by the global planner. Then, in the local planning stage, the optimal solution of the local environment is constantly searched, guided by the prior information of the globally optimal solution. The strategy used in the global planning stage is the iterative optimization method based on an initial solution. The local planning stage adopts the sampling-evaluation strategy, that is, sampling the candidate paths and then using the evaluation function to perform path selection. The proposed method has two innovations: 1) a novel global iterative optimization method is proposed and 2) a new cost function for evaluating the sampled paths is constructed, which improves the coupling of the global and local paths. We implement and test this method in a simulation environment, where the experimental results verify the effectiveness of the proposed method.

Keywords: Machine Learning for Robot Control, AI-Enabled Robotics, Motion Control
Abstract: Deep reinforcement learning (DRL) demonstrates great potential in mapless navigation domain. However, such a navigation model is normally restricted to a fixed configuration of the range sensor because its input format is fixed. In this paper, we propose a DRL model that can address range data obtained from different range sensors with different installation positions. Our model first extracts the goal-directed features from each obstacle point. Subsequently, it chooses global obstacle features from all point-feature candidates and uses these features for the final decision. As only a few points are used to support the final decision, we refer to these points as support points and our approach as support point-based navigation (SPN). Our model can handle data from different LiDAR setups and demonstrates good performance in simulation and real-world experiments. Moreover, it shows great potential in crowded scenarios with small obstacles when using a high-resolution LiDAR.

Keywords: Motion and Path Planning, Optimization and Optimal Control
Abstract: In this paper, we proposed a novel path replanning algorithm on arbitrary graphs. To avoid computationally heavy preprocessing and to reduce required memory to store the information of the previous search and expanded vertices, we defined the feature vertices, which are extracted from the previous path by a simple algorithm to compare the costs between adjacent vertices along the path once. Proper additional heuristic functions are designed for these feature vertices to work as local attractors to guide the search toward the previous path's neighbors. To avoid unnecessary expansions and speed up the search, these additional heuristic functions are properly managed to stop intriguing or guiding search toward the feature vertices. The proposed algorithm of Fast Replanning Multi-Heuristic A* is a variation of Shared Multi-Heuristic A* while removing or deactivating the additional heuristic functions during the search. Fast Replanning Multi-Heuristic A* guarantees the bounded suboptimality while efficiently exploring the graph toward the goal vertex. The performance of the proposed algorithm was compared with weighted A* by simulating numerous path replanning problems in maze-like maps.

Keywords: Motion and Path Planning, Aerial Systems: Applications, Autonomous Vehicle Navigation
Abstract: For quadrotor trajectory planning, describing a polynomial trajectory through coefficients and end-derivatives both enjoy their own convenience in energy minimization. We name them double descriptions of polynomial trajectories. The transformation between them, causing most of the inefficiency and instability, is formally analyzed in this paper. Leveraging its analytic structure, we design a linear-complexity scheme for both jerk/snap minimization and parameter gradient evaluation, which possesses efficiency, stability, flexibility, and scalability. With the help of our scheme, generating an energy optimal (minimum snap) trajectory only costs 1 mu s per piece at the scale up to 1,000,000 pieces. Moreover, generating large-scale energy-time optimal trajectories is also accelerated by an order of magnitude against conventional methods.

Keywords: Biologically-Inspired Robots, Tendon/Wire Mechanism, Failure Detection and Recovery
Abstract: Self-healing function is a promising approach for damage management of high-load robot applications such as legged robots. Although the function is getting major in soft robotics, its application to life-sized “stiff” robots is of relatively minor interest. Although the authors have devised several self-healing tensile modules for tendon-driven robots, the design guideline to fulfil the large load endurance and large stroke are still unclear. The paper focuses on the parametric design for unleaked liquid-assisted healing of low melting point alloy structure. The method was validated with a benchtop module test. Moreover, the module enabled tendon-driven monopod testbed to perform squat motion three times after the landing impact fracture and the self-healing sequence, which was never accomplished.

Keywords: Tendon/Wire Mechanism, Parallel Robots, Mechanism Design
Abstract: We introduce an origami-reinforced parallel continuum robot which is capable of maintaining the orientation of the end effector regardless of the bending shape. The cross-routing tendons provide an effective actuation of the robot because the constant length of the backbones prevent the actuation of parallel arrangement of tendons. We utilise the arclength relationship of parallel curves to show that parallel backbones of equal lengths and constant spacing between the backbones enable the constant orientation of the end effector. The origami shell is introduced to increase the torsional stiffness of the continuum robot and minimise any twisting. This leads to planar and parallel bending of all backbones. Planar quintic Pythagorean Hodograph curves are utilised for shape reconstruction, which is more accurate as compared to the curves with piecewise constant curvatures. The concept of this continuum robot and the accuracy of the reconstruction are validated experimentally.

Keywords: Humanoid Robot Systems, Tendon/Wire Mechanism, Mechanism Design
Abstract: This paper proposes a coupled tendon-driven waist joint for humanoid robots. The waist joint was designed as a 3 degrees of freedom (DOF) structure to simulate the motion of a human waist. The power transmission was designed by adopting a 3-motor 3-DOF (3M3D) coupled tendon-driven mechanism, so that the torque on the joints was multiplied. We derived the torque transmission formula and the rotation angle formula of the 3M3D tendon-driven structures and designed the waist joint by adopting an appropriate structure according to their features. To evaluate the accuracy and load capacity of the waist joint, we performed a rotational accuracy experiment and a maximum torque experiment. The experiment results showed that the maximum error of joint rotation was below 1°, and the maximum torque of the pitch, roll, and yaw rotations were 87[Nm], 53[Nm], and 22.2[Nm], respectively.

Keywords: Compliant Joints and Mechanisms, Mechanism Design, Rehabilitation Robotics
Abstract: Our goal is to investigate different approaches to modulate stiffness and apply them to human-robot interaction. Here we report on our effort employing the concept of adjustable unsupported-length cantilever leaf spring, which has been previously applied to different designs of variable stiffness actuators. By transmitting the interaction force through the elastic component directly to the supporting structure instead of the actuation unit, this type of actuator requires low power to adjust and to maintain a desired stiffness. In the design of a 1-translational degree of freedom body weight support system of a rehabilitation robot, we used a leaf spring mechanism for stiffness modulation relying only on the spring deflection in combination with a non-backdrivable actuator for adjusting the vertical equilibrium position. This paper describes our approach in determining the spring parameters to attain a desired range of stiffness with a short traveling distance of the adjuster. To model the spring stiffness under deflection, the ideal cantilever support model cannot be assumed for a conventional design of dual roller-pairs slider, especially with a soft spring. A beam deflection model considering the non-zero slopes at the contact points between the rollers and the spring is presented, along with the validation experiments using different spring thicknesses on our prototype.

Keywords: Developmental Robotics, Tendon/Wire Mechanism, Flexible Robotics
Abstract: In this study, a new humanoid shoulder is developed using a coupled tendon-driven mechanism that consists of three motors to drive three degrees of freedom (DoFs). In the coupled tendon-driven mechanism, multiple motors simultaneously drive multiple joints with each joint driven by at least two motors coupled with tendons. The torque of a motor is redistributed to multiple joints in a certain proportion to improve the load capacity of the joints. The 3M3D (3-motor 3-DoF) coupled tendon-driven joint module is systematically analyzed and divided into four categories according to the torque transmission characteristics between the motors and the joints: fully routed motor-joint form, 1-unrouted motor-joint form, 2-unrouted motor-joint form, and 3-unrouted motor-joint form. The four categories were analyzed and compared with respect to the characteristics of the motor torque redistribution with different tendon routing forms. The 2-unrouted motor-joint form was adopted for the development of the 3-DoF shoulder joint. The problem of the coexistence of coupling and non-coupling in one joint is solved by introducing a 2-DoF rolling joint. The relationship between the angle and torque between the motors and joints was verified experimentally. The experimental results revealed that the 3M3D coupled tendon-driven shoulder realized torque enhancement effectively by tendon coupling, and the angle accuracy was sufficient as a humanoid shoulder.

Keywords: Mechanism Design, Wheeled Robots, Field Robots
Abstract: A vehicle is expected to exhibit omni-directional locomotion capability to provide improved rough terrain vehicle functionality. Generally, rough terrain vehicles are not holonomic and cannot travel in the lateral direction, whereas typical omni-directional vehicles have difficulty in traveling on rough terrain. This paper proposes installing a crank leg for a Mecanum-wheeled vehicle ("Mecanum crank") to enhance its rough-terrain locomotion. Compared with other holonomic vehicles designed for rough terrain locomotion, the proposed design exhibited superior capability in the longitudinal wheel direction. To avoid the conflict between the crank leg and Mecanum wheel, this paper proposes the use of a differential gear system. The simple structure of the crank leg enables the implementation of a swing equalizer and roller grousers, which further enhance its rough-terrain locomotion. In longitudinal locomotion experiments, the proposed mechanism could climb a 95 mm-high step, which is 95% of the Mecanum wheel diameter. Furthermore, Mecanum crank could climb miniature scale stairs and execute lateral locomotion on them.

Keywords: Actuation and Joint Mechanisms, Force Control, Mechanism Design
Abstract: The force required to drive a mechanism can be compensated by adding an equivalent load in the opposite direction. By reversing the input and output of the load compensation, we proposed the concept of a displacement-force converter that enables the deformation of the elastic element to be controlled steplessly by a minimal external force. Its principle was proved in our previous study, but challenges arose owing to the use of a wire and pulley. Here, we introduce a new compensation method using a noncircular cam that generates a compensation torque due to the contact force from the follower, which is split in the tangential direction of the cam by the pressure angle varying at rotation. Using a prototype for proof of concept, the maximum control force required for the extension of the spring was successfully reduced by 23.2%. Furthermore, uniform forces were obtained between extension and compression so that the difference between them decreased from 543% to 49% relative to compression. Thus, actuators and current supplies requiring less power could be selected. Moreover, the prototype model was incorporated into a variable stiffness mechanism of a soft robotic gripper as a wire tensioner to show the expandability of the displacement-force converter.

Keywords: Mechanism Design, Actuation and Joint Mechanisms, Parallel Robots
Abstract: Most articulated robots comprise multiple joints and links that control the position and posture of the end effector. The kinematic pair arrangement determines characteristics such as output force. The link configurations can be classified into serial link and parallel link mechanisms. A typical parallel link mechanism is the spherical parallel mechanism (SPM), designed to ensure the end effector has only rotational degrees of freedom.

However, the kinematic pair arrangement has not been sufficiently examined in two degrees of freedom (2-DOF) SPMs. Herein, we present a basic design method for the proposed 2-DOF SPM curved biaxial swing mechanism, with inputs comprising arc sliders. The swinging area of the passive link was small, and infinite rotation around a certain axis was achieved without collision or transfer to a singular posture. Using the kinematics of this mechanism, we clarified the linear roll output and non-linear pitch output. Moreover, we fabricated a prototype and measured its basic drive characteristics. The results revealed that the output performance was greatly dependent on the rotation angle, high movable range in the roll axis, and low movable range in the pitch axis.

Keywords: In-Hand Manipulation, Manipulation Planning, Dexterous Manipulation
Abstract: In this work, we propose an approach to manipulate objects by position-controlled robot hands: in-hand caging manipulation. In this method, an object is manipulated based on caging without force sensing or force control. An object is caged by a robot hand throughout manipulation, and we can locate the object around a goal by deformation of the cage without sensing the object configuration. In this paper, we considered 2-D polygonal objects as the targets of in-hand caging manipulation. We also considered some position-controlled hands as the devices to manipulate the objects. A motion planning algorithm for the hands was proposed and applied to planar in-hand caging manipulation. By moving the hands according to the result of the motion planning, it was possible to manipulate the objects without sensing. Our proposed method can be applied to a device such as a part feeder that aligns variously shaped parts in the same orientation.

Keywords: Contact Modeling, Dexterous Manipulation, Visual Servoing
Abstract: The assumption of fixed contact between robots and deformable objects (DOs) is widely used by previous DO manipulation (DOM) studies. However, the fixed contact setting is inapplicable to many real-life applications due to various factors, such as the end-effector's type and the DO's intrinsic material properties. In such cases, the non-fixed contact (NFC) configuration, which has not been well-investigated, usually demonstrates better applicability in terms of contact flexibility and payload capacity. In this paper, we investigate the problem of DOM with NFC, particularly the contact condition characterization and active contact adjustment strategy. To this end, we propose a versatile index and an optimization-based method for the contact description and adjustment. Then a systematic control framework combining both deformation control and active contact adjustment control is proposed for task-level DOM control. The feasibility and effectiveness of the proposed method were verified via the presented experiments.

Keywords: Humanoid and Bipedal Locomotion
Abstract: A novel control scheme for biped robots to manipulate the ZMP three-dimensionally apart from the actual ground profile is presented. It is shown that the linear inverted-pendulum-like dynamics with this scheme can represent a wider class of movements including variation of the body height. Moreover, this can also represent the motion in aerial phase. Based on this, the foot-guided controller proposed by the authors is enhanced to enable the robots to locomote on highly uneven terrains and also to seamlessly transition between walking and running without pre-planning the overall motion reference. The controller guarantees the capturability at landing and defines the motion by a time-variant state feedback, which is analytically derived from a model predictive optimization. It is verified through some computer simulations.

Keywords: Physical Human-Robot Interaction, Wearable Robotics, Force Control
Abstract: Overhead manipulation often needs collaboration of two operators, which is challenging in confined space such as in a compartment or on a ladder. Supernumerary Robotic Arm (SuperArm), as a promising wearable robotics solution for overhead tasks, can provide optimal assistance in terms of broader workspace, diverse manipulation functionalities, and labor-saving operations. However, the human-centered SuperArm interaction mechanism, taking into account human safety, is rarely studied to date, in particular, in the context of human standing balance. Motivated by this missing mechanism, our study proposes a novel method for the human-centered overhead tasks so that an individual operator can accomplish the overhead tasks with the assistance of SuperArm via tunable interaction force and support force regulation. The SuperArm-human interaction is modeled and a dynamics control method based on QR decomposition is adopted to decouple joint torques of the SuperArm and the interaction forces. As such, the supporting force can be regulated independently to guarantee the operator-SuperArm interaction forces in a safe region. Force plate is used for measuring the CoP position as an evaluation method of the standing balance. The critical horizontal push force is learned through experiment and used to guide the SuperArm balancing control. This method is implemented on a SuperArm prototype worn on the operator's back, providing necessary supporting forces for the overhead object while allowi

Keywords: Robot Audition, Localization
Abstract: Robotic audition is a basic sense that helps robots perceive the surroundings and interact with humans. Sound Source Localization (SSL) is an essential module for a robotic system. However, the performance of most sound source localization techniques degrades in noisy and reverberant environments due to inaccurate Time Difference of Arrival (TDoA) estimation. In robotic sound source localization, we are more interested in detecting the arrival of human speech than other sound sources. Ideally, we expect an effective TDoA estimation to respond only to speech signals, while masking off other interferences. In this paper, we propose a novel technique that learns to attend to speech fundamental frequency and harmonics while suppressing noise interference and reverberation. The novel TDoA feature is referred to as Generalized Cross Correlation with Phase Transform and Speech Mask (GCC-PHAT-SM). We perform sound source localization experiments on real-world data captured from a robotic platform. Experiments show that GCC-PHAT-SM feature significantly outperforms traditional Generalized CrossCorrelation (GCC) feature in noisy and reverberant acoustic environments.

Keywords: Localization, Autonomous Vehicle Navigation
Abstract: Global navigation satellite systems (GNSS) are one of the utterly popular sources for providing globally referenced positioning for autonomous systems. However, the performance of the GNSS positioning is significantly challenged in urban canyons, due to the signal reflection and blockage from buildings. Given the fact that the GNSS measurements are highly environmentally dependent and time-correlated, the conventional filtering-based method for GNSS positioning cannot simultaneously explore the time-correlation among historical measurements. As a result, the filtering-based estimator is sensitive to unexpected outlier measurements. In this paper, we present a factor graph-based formulation for GNSS positioning and real-time kinematic (RTK). The formulated factor graph framework effectively explores the time-correlation of pseudorange, carrier-phase, and doppler measurements, and leads to the non-minimal state estimation of the GNSS receiver. The feasibility of the proposed method is evaluated using datasets collected in challenging urban canyons of Hong Kong and significantly improved positioning accuracy is obtained, compared with the filtering-based estimator.

Keywords: Localization, Deep Learning Methods, Visual Learning
Abstract: Visual localization plays an indispensable role in robotics. Both learning and hand-crafted feature based methods for relocalization process keep their effectiveness and weakness. However, current algorithms seldom consider these two kinds of features under one framework. In this paper, focusing on this task, we propose a novel relocalization framework for RGB or RGB-D data source, which is composed of coarse localization process by learning features and pose refinement by hand-crafted features. In particular, coarse stage contains deep point cloud generation and registration. In this stage, instead of regressing camera pose directly, the paper novelly designs a neural network called PGNet to construct sparse point cloud with RGB or RGB-D as inputs. Further more, by means of training set, hand-crafted feature space is established. Based on the obtained camera pose in coarse stage, accurate point-topoint correspondences are set up through searching the space. Then accurate camera pose is obtained by applying RANSAC to correspondences or solving PnP. Finally, experiments on both outdoor and indoor benchmark datasets demonstrate state-ofthe-art performance over other existing methods.

Keywords: Localization, Multi-Robot SLAM, SLAM
Abstract: The core problem of visual multi-robot simultaneous localization and mapping (MR-SLAM) is how to efficiently and accurately perform multi-robot global localization (MR-GL). The difficulties are two-fold. The first is the difficulty of global localization for significant viewpoint difference. Appearance-based localization methods tend to fail under large viewpoint changes. Recently, semantic graphs have been utilized to overcome the viewpoint variation problem. However, the methods are highly time-consuming, especially in large-scale environments. This leads to the second difficulty, which is how to perform real-time global localization. In this paper, we propose a semantic histogram-based graph matching method that is robust to viewpoint variation and can achieve real-time global localization. Based on that, we develop a system that can accurately and efficiently perform MR-GL for both homogeneous and heterogeneous robots. The experimental results show that our approach is about 30 times faster than Random Walk based semantic descriptors. Moreover, it achieves an accuracy of 95% for global localization, while the accuracy of the state-of-the-art method is 85%.

Keywords: Localization
Abstract: A key challenge in visual place recognition (VPR) is recognizing places despite drastic visual appearance changes due to factors such as time of day, season, weather or lighting conditions. Numerous approaches based on deep-learnt image descriptors, sequence matching, domain translation, and probabilistic localization have had success in addressing this challenge, but most rely on the availability of carefully curated representative reference images of the possible places. In this paper, we propose a novel approach, dubbed Bayesian Selective Fusion, for actively selecting and fusing informative reference images to determine the best place match for a given query image. The selective element of our approach avoids the counterproductive fusion of every reference image and enables the dynamic selection of informative reference images in environments with changing visual conditions (such as indoors with flickering lights, outdoors during sunshowers or over the day-night cycle). The probabilistic element of our approach provides a means of fusing multiple reference images that accounts for their varying uncertainty via a novel training-free likelihood function for VPR. On difficult query images from two benchmark datasets, we demonstrate that our approach matches and exceeds the performance of alternative fusion approaches along with state-of-the-art techniques that are provided with prior (unfair) knowledge of the best reference images.

Keywords: Mapping, Simulation and Animation, Software, Middleware and Programming Environments
Abstract: Probabilistic volumetric mapping (PVM) represents a 3D environmental map for an autonomous robotic navigational task. A popular implementation such as Octomap is widely used in the robotics community for such a purpose. The Octomap relies on octree to represent a PVM and its main bottleneck lies in massive ray-shooting to determine the occupancy of the underlying volumetric voxel grids. In this paper, we propose GPU-based ray shooting to drastically improve the ray shooting performance in Octomap. Our main idea is based on the use of recent ray-tracing RTX GPU, mainly designed for real-time photo-realistic computer graphics and the accompanying graphics API, known as DXR. Our ray-shooting first maps leaf-level voxels in the given octree to a set of axis-aligned bounding boxes (AABBs) and employ massively parallel ray shooting on them using GPUs to find free and occupied voxels. These are fed back into CPU to update the voxel occupancy and restructure the octree. In our experiments, we have observed more than three-orders-of-magnitude performance improvement in terms of ray shooting using ray-tracing RTX GPU over a state-of-the-art Octomap CPU implementation, where the benchmarking environments consist of more than 77K points and 25K∼34K voxel grids.

Keywords: Mapping, Range Sensing
Abstract: Scan data of urban environments often include representations of dynamic objects, such as vehicles, pedestrians, and so forth. However, when it comes to constructing a 3D point cloud map with sequential accumulations of the scan data, the dynamic objects often leave unwanted traces in the map. These traces of dynamic objects act as obstacles and thus impede mobile vehicles from achieving good localization and navigation performances. To tackle the problem, this paper presents a novel static map building method called textit{ERASOR}, Egocentric RAtio of pSeudo Occupancy-based dynamic object Removal, which is fast and robust to motion ambiguity. Our approach directs its attention to the nature of most dynamic objects in urban environments being inevitably in contact with the ground. Accordingly, we propose the novel concept called textit{pseudo occupancy} to express the occupancy of unit space and then discriminate spaces of varying occupancy. Finally, Region-wise Ground Plane Fitting (R-GPF) is adopted to distinguish static points from dynamic points within the candidate bins that potentially contain dynamic points. As experimentally verified on SemanticKITTI, our proposed method yields promising performance against state-of-the-art methods overcoming the limitations of existing ray tracing-based and visibility-based methods.

Keywords: Sensor Fusion, Localization, Range Sensing
Abstract: Indoor positioning without GPS is a challenge task, especially, in complex scenes or when sensors fail. In this paper, we develop an ultra-wideband aided visual-inertial positioning system (UVIP) which aims to achieve accurate and robust positioning results in complex indoor environments. To this end, a point-line-based stereo visual-inertial odometry (PL-sVIO) is firstly designed to improve the positioning accuracy in structured or low-textured scenarios by making use of line features. Secondly, a loop closure method is proposed to suppress the drift of PL-sVIO based on image patch features described by a CNN for handing the situation of a large environment and viewpoint variation. Thirdly, an accurate relocalization approach is presented for the case when the visual sensor fails. In this scheme, a top-to-down matching strategy from image to point and line features is presented to improve relocalization performance. Finally, the UWB sensor is combined with the visual-inertial system to further improve the accuracy and robustness of the positioning system and provide the results in a fixed reference frame. Thus, desirable real-time positioning results are derived for complex indoor scenes. Evaluations on challenging public datasets and real-world experiments are conducted to demonstrate that the proposed UVIP can provide more accurate and robust positioning results in complex indoor environments, even in the case when the visual sensor fails or in the absence of UWB anchors.

Keywords: Intelligent Transportation Systems, SLAM, Localization
Abstract: Initial global localization is important to mobile robotics in terms of navigation initialization (or re-initialization) and loop closure in SLAM. 3D LiDARs are commonly used for mobile robotics, yet LiDAR-based initial global localization (especially at large scale such as in outdoor environments) is still challenging due to lack of salient features in LiDAR range data. Inspired by visual SLAM oriented initial global localization methods, we propose a method of LiDAR-based initial global localization using 2D submap projection image (SPI). For this, global descriptors from SPIs are extracted for place recognition; pose estimation is realized by feature point matching between the queried SPI and SPIs from a global map database. The proposed initial global localization module runs at 2.4 Hz with precision of 1.2 m and for translation and 1.2degree for rotation, which can serve as a suitable initial estimate for subsequent pose estimation refinement via existing mature point cloud registration methods.

Keywords: SLAM, Localization
Abstract: LiDAR odometry algorithms are complex and involve a number of hyper-parameters. The choice of hyper-parameters can substantively affect the performance of odometry estimation, and it is necessary to carefully fine-tune the hyper-parameters depending on the sensor, environment, and algorithm to achieve the best estimation results. While odometry estimation algorithms are often tuned manually, this is time-consuming and may also result in a sub-optimal parameter set. This paper presents an automatic hyper-parameter tuning approach for LiDAR odometry estimation. By taking advantage of the sequential model-based optimization (SMBO) approach, we automatically optimize the hyper-parameter set of a black-box odometry estimation algorithm without detailed knowledge of the algorithm. In addition, a LiDAR data augmentation approach is also proposed to prevent overfitting. Through evaluation, we show that the combination of SMBO-based parameter exploration and data augmentation enables us to efficiently and robustly optimize the hyper-parameter set for several different odometry estimation algorithms. We also demonstrate that the optimized parameter set exhibits superior performance with respect to KITTI dataset and in a real use scenario.

Keywords: Localization, Field Robots, Recognition
Abstract: Place Recognition enables the estimation of a globally consistent map and trajectory by providing non-local constraints in Simultaneous Localisation and Mapping (SLAM). This paper presents Locus, a novel place recognition method using 3D LiDAR point clouds in large-scale environments. We propose a method for extracting and encoding topological and temporal information related to components in a scene and demonstrate how the inclusion of this auxiliary information in place description leads to more robust and discriminative scene representations. Second-order pooling along with a non-linear transform is used to aggregate these multi-level features to generate a fixed-length global descriptor, which is invariant to the permutation of input features. The proposed method outperforms state-of-the-art methods on the KITTI dataset. Furthermore, Locus is demonstrated to be robust across several challenging situations such as occlusions and viewpoint changes in 3D LiDAR point clouds. The open-source implementation is available at: https://github.com/csiro-robotics/locus.

Keywords: Range Sensing, Calibration and Identification
Abstract: This paper proposes an automatic and accuracy-enhanced extrinsic calibration method for 3D LiDARs with a range offset correction, which needs only an arbitrarily-shaped single planar board. One of the most exhaustive parts of existing LiDAR calibration procedures is to manually find target objects from massive point clouds. To obviate user interventions, we propose an automated planar board detection from LiDAR range images. To extract a target completely, we suppress outliers and restore rejected inliers of the target board by introducing a target completion method. We empirically find that range measurements of various LiDARs are mainly skewed by constant offset values. To compensate for this, we suggest a range offset model for each laser channel in calibration procedures. The relative pose between LiDARs and range offsets are jointly estimated by minimizing bi-directional point-to-board distances within the iterative re-weighted least squares (IRLS) framework. To verify the suggested range offset model, we obtain and analyze extensive real-world measurements. By conducting experiments using the various sensor configurations and shapes of boards, we quantitatively and qualitatively confirm accuracy and versatility of the proposed method by comparing with the state-of-the-art LiDAR calibration methods. All the source code and data used in the paper are available at : https://github.com/JunhaAgu/AutoL2LCalib.

Keywords: Human-Centered Robotics, Human Detection and Tracking, Machine Learning for Robot Control
Abstract: Human–robot interaction (HRI) has been widely researched in diverse applications. A robot following a person is one such scenario investigated in the HRI field. However, human movements and actions are complex and can change dramatically. We herein demonstrate a machine learning-based system that allows a person-following robot to track in real-time the predicted future motion of a walking human, from a first-person perspective. We assume that a depth sensor that can detect the human skeleton is loaded on a mobile robot to provide data on the user’s motion from a first-person perspective. The system calculates the coordinates of the center of gravity (COG) and 25 body joints of the user. These coordinates of COG and 25 body joints are relative to the robot based on the position of the person tracked, and these are used for the input dataset of a neural network (NN) that predicts human motion. A five-layered NN estimates the relative vectors in real-time between the current person’s COG and the future position of the 25 body joints. Using a proportional–integral–derivative (PID) controller, the person-following robot can track the predicted position of a walking human 0.5 s in advance to increase the robustness of following and to avoid delays.

Keywords: Human-Centered Robotics, Cognitive Modeling, Human-Aware Motion Planning
Abstract: A human-centered robot needs to reason about the cognitive limitation and potential irrationality of its human partner to achieve seamless interactions. This paper proposes an anytime game-theoretic planner that integrates iterative reasoning models, a partially observable Markov decision process, and chance-constrained Monte-Carlo belief tree search for robot behavioral planning. Our planner enables a robot to safely and actively reason about its human partner's latent cognitive states (bounded intelligence and irrationality) in real-time to maximize its utility better}. We validate our approach in an autonomous driving domain where our behavioral planner and a low-level motion controller hierarchically control an autonomous car to negotiate traffic merges. Simulations and user studies are conducted to show our planner's effectiveness.

Keywords: Safety in HRI, Deep Learning Methods, Physical Human-Robot Interaction
Abstract: As robots begin to collaborate with people in real life, their applicability and practicality are continuously increasing. To reliably employ robots nearby, safety needs to be rigorously ensured. In addition to collision prevention algorithms, studies are being actively conducted on collision handling methods. Momentum Observer (MOB) was developed to estimate disturbance torque without using joint acceleration. However, the estimated disturbance from MOB contains not only the applied external torque but also model uncertainty such as friction and modeling error due to imprecise system identification. Our proposed method handles this problem by learning the model uncertainty with Long Short-Term Memory (LSTM) and thereby estimates the purely applied external torque with only proprioceptive sensors. The proposed method can be applied to the extreme modeling error case when there is no model on the dynamics of the robot at all. The experiments using a real robot show that the external torque can be estimated and collisions can be detected accordingly even in a limited situation where a precise dynamics model and friction model are not available.

Keywords: Telerobotics and Teleoperation, Virtual Reality and Interfaces, Human Performance Augmentation
Abstract: Teleoperation of robots can be challenging, especially for novice users with little to no experience at such tasks. The difficulty is largely due to the numerous degrees of freedom users must control and their limited perception bandwidth. To help mitigate these challenges, we propose in this paper a solution which relies on artificial intelligence to understand user intended motion and then on mixed reality to communicate the estimated trajectories to the users in an intuitive manner. User intended motion is estimated using a deep learning network trained on a dataset of motion primitives. During teleoperation, the estimated motions are augmented onto a first-person live video feed from the robot. Finally, if a suggested motion is accepted by the user, the robot is driven along that trajectory in an autonomous manner. We validate our proposed mixed reality teleoperation scheme with simulation experiments on a drone and demonstrate, through subjective and objective evaluation, its advantages over other teleoperation methods.

Keywords: Deep Learning for Visual Perception
Abstract: Bottom-up approaches for image-based multi-person pose estimation consist of two stages: (1) keypoint detection and (2) grouping of the detected keypoints to form person instances. Current grouping approaches rely on learned embedding from only visual features that completely ignore the spatial configuration of human poses. In this work, we formulate the grouping task as a graph partitioning problem, where we learn the affinity matrix with a Graph Neural Network (GNN). More specifically, we design a Geometry-aware Association GNN that utilizes spatial information of the keypoints and learns local affinity from the global context. The learned geometry-based affinity is further fused with appearance-based affinity to achieve robust keypoint association. Spectral clustering is used to partition the graph for the formation of the pose instances. Experimental results on two benchmark datasets show that our proposed method outperforms existing appearance-only grouping frameworks, which shows the effectiveness of utilizing spatial context for robust grouping.

Keywords: Deep Learning for Visual Perception, Data Sets for Robotic Vision, Recognition
Abstract: The purpose of this paper is the robotic hanging manipulation of an object of various shapes that is not limited to a specific category. To achieve this, we propose a method that allows the estimator to learn many different shapes with hanging points without any manual annotation. A random shape generator using GAN solves the limitation of the number of 3D models and can handle objects of various shapes. In addition, hanging is repeated in the dynamics simulation, and hanging points are automatically generated. A large amount of training data is generated by rendering random-textured objects with hanging points in the random simulation environment. A deep neural network trained with these data was able to estimate hanging points of an unknown category object in the real world and achieved hanging manipulation by a robot.

Keywords: Imitation Learning, Deep Learning in Grasping and Manipulation, Bioinspired Robot Learning
Abstract: A high-precision manipulation task, such as needle threading, is challenging. Physiological studies have proposed connecting low-resolution peripheral vision and fast movement to transport the hand into the vicinity of an object, and using high-resolution foveated vision to achieve the accurate homing of the hand to the object. The results of this study demonstrate that a deep imitation learning based method, inspired by the gaze-based dual resolution visuomotor control system in humans, can solve the needle threading task. First, we recorded the gaze movements of a human operator who was teleoperating a robot. Then, we used only a high-resolution image around the gaze to precisely control the thread position when it was close to the target. We used a low-resolution peripheral image to reach the vicinity of the target. The experimental results obtained in this study demonstrate that the proposed method enables precise manipulation tasks using a general-purpose robot manipulator and improves computational efficiency.

Keywords: Cellular and Modular Robots, Localization
Abstract: Modular self-reconfigurable robotic (MSRR) systems are potentially more robust and more adaptive than conventional systems. Following our previous work where we proposed a freeform MSRR module called FreeBOT, this paper presents a novel configuration detection system for FreeBOT using a magnetic sensor array. A FreeBOT module can be connected by up to 11 modules, and the proposed configuration detection system can locate a variable number of connection points accurately in real-time. By equipping FreeBOT with 24 magnetic sensors, the magnetic field density produced by magnets and steel spherical shells can be monitored. The connectable area is split into 199 non-uniform regions, including 84 uniform regions. Using a Graph Convolutional Network (GCN) based algorithm, the connection points can be located accurately under ferromagnetic environments. The system can locate a variable number of connection points for such a region division with only single connection point training data. Finally, the localization algorithm can run faster than 40 Hz on FreeBOT. With the real-time configuration detection system, the FreeBOT system has the potential to reconfigure automatically and accurately.

Keywords: Reinforcement Learning, AI-Based Methods, Representation Learning
Abstract: Policy optimization in reinforcement learning requires the selection of numerous hyperparameters across different environments. Fixing them incorrectly may negatively impact optimization performance leading notably to insufficient or redundant learning. Insufficient learning (due to convergence to local optima) results in under-performing policies whilst redundant learning wastes time and resources. The effects are further exacerbated when using single policies to solve multi-task learning problems. In this paper, we study how the Evidence Lower Bound (ELBO) used in Variational Auto-Encoders (VAEs) is affected by the diversity of image samples. Different tasks or setups in visual reinforcement learning incur varying diversity. We exploit the ELBO to create an auto-tuning technique in self-supervised reinforcement learning. Our approach can auto-tune three hyperparameters: the replay buffer size, the number of policy gradient updates during each epoch, and the number of exploration steps during each epoch. We use the state-of-the-art self-supervised robotic learning framework (Reinforcement Learning with Imagined Goals (RIG) using Soft Actor-Critic) as baseline for experimental verification. Experiments show that our method can auto-tune online and yields the best performance at a fraction of the time and computational resources. Code, video, and appendix for simulated and real-robot experiments can be found at url{www.JuanRojas.net/autotune}.

Keywords: Art and Entertainment Robotics, Data Sets for Robot Learning, Dual Arm Manipulation
Abstract: In this paper, we present a model of a diabolo that can be used for training agents in simulation to play diabolo, as well as running it on a real dual robot arm system. We first derive an analytical model of the diabolo-string system and compare its accuracy using data recorded via motion capture, which we release as a public dataset of skilled play with diabolos of different dynamics. We show that our model outperforms a deep-learning-based predictor, both in terms of precision and physically consistent behavior. Next, we describe a method based on optimal control to generate robot trajectories that produce the desired diabolo trajectory, as well as a system to transform higher-level actions into robot motions. Finally, we test our method on a real robot system playing the diabolo, and throw it to and catch it from a human player.

Keywords: Big Data in Robotics and Automation, Multi-Robot Systems, Machine Learning for Robot Control
Abstract: A technological revolution is occurring in the field of robotics with the data-driven deep learning technology. However, building datasets for each local robot is laborious. Meanwhile, data islands between local robots make data unable to be utilized collaboratively. To address this issue, the work presents Peer-Assisted Robotic Learning (PARL) in robotics, which is inspired by the peer-assisted learning in cognitive psychology and pedagogy. PARL implements data collaboration with the framework of cloud robotic systems. Both data and models are shared by robots to the cloud after semantic computing and training locally. The cloud converges the data and performs augmentation, integration, and transferring. Finally, fine tune this larger shared dataset in the cloud to local robots. Furthermore, we propose the DAT Network (Data Augmentation and Transferring Network) to implement the data processing in PARL. DAT Network can realize the augmentation of data from multi-local robots. We conduct experiments on a simplified self-driving task for robots (cars). DAT Network has a significant improvement in the augmentation in self-driving scenarios. Along with this, the self-driving experimental results also demonstrate that PARL is capable of improving learning effects with data collaboration of local robots.

Keywords: Imitation Learning, Motion and Path Planning, Vision-Based Navigation
Abstract: One of the challenges to reduce the gap between the machine and the human level driving is how to endow the system with the learning capacity to deal with the coupled complexity of environments, intentions, and dynamics. In this paper, we propose a hierarchical driving model with explicit models of continuous intention and continuous dynamics, which decouples the complexity in the observation-to-action reasoning in the human driving data. Specifically, the continuous intention module takes perception to generate a potential map encoded with obstacles and intentions. Then, the potential map is regarded as a condition, together with the current dynamics, to generate a continuous trajectory as output by a continuous function approximator network, whose derivatives can be used for supervision without additional parameters. Finally, our method is validated by both datasets and stimulation, demonstrating that our method has higher prediction accuracy of displacement and velocity and generates smoother trajectories. Our method is also deployed on the real vehicle with loop latency, validating its effectiveness. To the best of our knowledge, this is the first work to produce the driving trajectory using a continuous function approximator network.

Keywords: Legged Robots, Mechanism Design, Wheeled Robots
Abstract: This paper introduces a leg-wheel transformable quadruped robot named Lywal which can switch to the leg-mode and the wheel-mode for locomotion, and the claw-mode for picking up and transport functions. First, the mechanical structure of Lywal is designed by using an innovative 2-DoF transformable mechanism. Second, the calculation of kinematics is analyzed in detail. Then, the switching-mode strategy and the mobile control strategies in different modes are designed. Finally, the prototype of Lywal is built. The properties of the mobile modes are analyzed, and the picking-up and transport functions of the claw-mode are verified through physical experiments.

Keywords: Legged Robots, Hydraulic/Pneumatic Actuators, Embedded Systems for Robotic and Automation
Abstract: This paper proposes a design method of a compact embedded hydraulic power unit (HPU) for a bipedal robot and a controller to regulate the supply pressure. The HPU consists of an integrated pump-motor unit. The unit is immersed in hydraulic oil for efficient space utilization and heat dissipation from the motor. This HPU design is analyzed to establish a relationship between the thermal variables and the motor design parameters such as gap radius and wire radius via its thermal and electrical modeling. Through this analysis, the design parameters of suitable pump-driving motor are chosen. This paper also proposes a control method for the HPU to regulate the supply pressure while minimizing the energy loss caused by the bypass through a pressure-regulating valve. The HPU is mounted on top of the bipedal robot platform, LIGHT, with twelve degrees of freedom actuated by the proposed HPU. Finally, the durability of the designed HPU is demonstrated through a long-term driving test at a high pressure. Furthermore, air-walking and squat motion experiments are conducted with the bipedal robot to demonstrate the capabilities of the HPU and its controller.

Keywords: Legged Robots, Mechanism Design, Simulation and Animation
Abstract: Staircase is a typical obstacle for the legged robot to overcome in buildings. This paper studies the stair climbing capability-based dimensional synthesis for a hexapod legged robot, i.e., exploring how to determine the leg length and the longitudinal body length concerning the target staircase in the mechanical design stage. In climbing a staircase, leg-staircase interference is one of the predominant issues. The three possible interference cases are illustrated in detail with a 2-DOF (degree of freedom) leg mechanism and the staircase size, based on the predefined tripod gait sequence. The mathematical relationships between the leg length, longitudinal body length, and the target staircase size are derived. The leg length and the body length are finally determined with the target staircase size. The virtual simulations and prototype experiments verify the effectiveness of the dimensional synthesis for the hexapod robot.

Keywords: Humanoid and Bipedal Locomotion, Legged Robots, Motion and Path Planning
Abstract: Stable walking in real-world environments is a challenging task for humanoid robots, especially when considering the dynamic disturbances, e.g., caused by external perturbations that may be encountered during locomotion. The varying nature of disturbance necessitates high adaptability. In this paper, we propose an enhanced Nonlinear Model Predictive Control (NMPC) approach for robust and adaptable walking – we term it versatile locomotion, by limiting both the Center of Pressure (CoP) and Divergent Component of Motion (DCM) movements. Due to utilization of the Nonlinear Inverted Pendulum plus Flywheel model, the robot is endowed with the capabilities of CoP manipulation (if equipped with finite-sized feet), step location adjustment, upper body rotation, and vertical height variation. Considering the feasibility constraints, especially the usage of relaxed CoP constraints, the NMPC scheme is established as a Quadratically Constrained Quadratic Programming problem, which is solved efficiently by Sequential Quadratic Programming with enhanced solvability. Simulation experiments demonstrate the effectiveness of our method to recruit optimal hybrid strategies in order to realize versatile locomotion, for the robot with finite-sized or point feet.

Keywords: Human Factors and Human-in-the-Loop, Rehabilitation Robotics, Wearable Robotics
Abstract: External feedback in the form of visual, auditory and tactile cues has been used to assist patients to overcome mobility challenges. However, these cues can become less effective over time. There is limited research on adapting cues to account for inter and intra-personal variations in cue responsiveness. We propose a cue-provision framework that consists of a gait performance monitoring algorithm and an adaptive cueing strategy to improve gait performance. The proposed approach learns a model of the person's response to cues using Gaussian Process regression. The model is then used within an on-line optimization algorithm to generate cues to improve gait performance. We conduct a study with healthy participants to evaluate the ability of the adaptive cueing strategy to influence human gait, and compare its effectiveness to two other cueing approaches: the standard fixed cue approach and a proportional cue approach. The results show that adaptive cueing is more effective in changing the person's gait state once the response model is learned compared to the other methods.

Keywords: Mapping, Learning from Experience
Abstract: This paper presents a method for LiDAR sensor-based traversability analysis for autonomous mobile robots in urban environments. Although urban environments are structured environments, a typical terrain comprises hazardous regions for mobile robots. Therefore, a reliable method for detecting traversable regions is required to prevent robots from getting stuck in the middle of the road. Conventional approaches require considerable efforts to obtain a model for traversability analysis for a specific robot or environment. In particular, learning-based methods require explicit training data. This paper introduces a method for traversability mapping based on a self-training algorithm to eliminate the hand labeling process. A neural network was applied to the underlying classifier of the self-training algorithm. With our approach, the model can be learned with even weakly labeled data obtained from robot-specific parameters and the robot’s footprint. In practical experiments, the self-trained model performed better performance than the existing supervised learning method. Moreover, as the fraction of unlabeled data increased, the performance also increased. Therefore, the demonstrations in the urban environments indicate the effectiveness of the proposed method for traversability mapping.

Keywords: Mapping, Perception for Grasping and Manipulation
Abstract: In this letter, we present an interactive probabilistic mapping framework for a mobile manipulator picking objects from a pile. The aim is to map the scene, actively decide where to go next and which object to pick, make changes to the scene by picking the chosen object, and then map these changes alongside. The proposed framework uses a novel dynamic Gaussian Process (GP) Implicit Surface method to incrementally build and update the scene map that reflects environment changes. Actively the framework computes the next-best-view, balancing the terms of object reachability for picking and map information gain (IG) for fidelity and coverage. To enforce a priority of visiting boundary segments over unknown regions, the IG formulation includes an uncertainty-gradient based frontier score by exploiting the GP kernel derivative. This leads to an efficient strategy that addresses the often conflicting requirement of unknown environment exploration and object picking exploitation given a limited execution horizon. We demonstrate the effectiveness of our framework with software simulation and real-life experiments.

Keywords: Social HRI, Emotional Robotics, Intention Recognition
Abstract: Existing approaches of proactive HRI are either based on multi-stage decision processes or based on end-to-end decision models. However, the rule-based approaches require sedulous expert efforts and only handle minimal pre-defined scenarios. Besides, existing works with end-to-end models are limited to very general greetings or few behavior patterns (< 10). To address those challenges, we propose a new end-to-end framework, the TransFormer with Visual Tokens for Human-Robot Interaction (TFVT-HRI). The proposed framework extracts visual tokens of relative objects from an RGB camera first. To ensure the correct interpretation of the scenario, a transformer decision model is then employed to process the visual tokens, which is augmented with the temporal and spatial information. It predicts the appropriate action to take in each scenario and identifies the right target. Our data is collected from an in-service receptionist robot in an office building, which is then annotated by experts for appropriate proactive behavior. The action set includes 1000+ diverse patterns by combining language, emoji expression, and body motions. We compare our model with other SOTA end-to-end models on both offline test sets and online user experiments in realistic office building environments to validate this framework. It is demonstrated that the decision model achieves SOTA performance, resulting in more humanness and intelligence when compared with the previous reactive reception policies.

Keywords: Biologically-Inspired Robots, Redundant Robots, Reinforcement Learning
Abstract: Reinforcement Learning (RL) usually needs thousands of episodes, leading its applications on physical robots expensive and challenging. Little research has been reported about snake robot control using RL due to additional difficulty of high redundancy of freedom. We propose a coach-based deep learning method for snake robot control, which can effectively save convergence time with much less episodes. The main contributions include: 1) a unified graph-based Bayesian framework integrating a coach module to guide the RL agent; 2) an explicit stochastic formulation of robot-environment interaction with uncertainty; 3) an efficient and robust training process for snake robot control to achieve both path planning and obstacle avoidance simultaneously. The performance has been demonstrated on both simulation and real-world data in comparison with state-of-the art, showing promising results.

Keywords: Marine Robotics, Environment Monitoring and Management, Probability and Statistical Methods
Abstract: We present a method to estimate two-dimensional, time-invariant oceanic flow fields based on data from both ensemble forecasts and online measurements. Our method produces a realistic estimate in a computationally efficient manner suitable for use in marine robotics for path planning and related applications. We use kernel methods and singular value decomposition to find a compact model of the ensemble data that is represented as a linear combination of basis flow fields and that preserves the spatial correlations present in the data. Online measurements of ocean current, taken for example by marine robots, can then be incorporated using recursive Bayesian estimation. We provide computational analysis, performance comparisons with related methods, and demonstration with real-world ensemble data to show the computational efficiency and validity of our method. Possible applications in addition to path planning include active perception for model improvement through deliberate choice of measurement locations.

Keywords: Legged Robots, Deep Learning Methods, Data Sets for Robot Learning
Abstract: Legged robots are popular candidates for missions in challenging terrains due to their versatile locomotion strategies. Terrain classification is a key enabling technology for autonomous legged robots, allowing them to harness their innate flexibility to adapt to the demands of their operating environment. We show how highly capable machine learning techniques, namely gated recurrent neural networks, allow our target legged robot to correctly classify the terrain it traverses in both supervised and semi-supervised fashions. Tests on a benchmark dataset shows that our time-domain classifiers are well capable of handling raw and variable-length data with small amount of labels and outperform frequency-domain classifiers. The classification results on our own extended dataset opens up a range of high-performance behaviours that are specific to those environments. Furthermore, we show how raw unlabelled data is used to improve significantly the classification results in a semi-supervised model.