Keywords: Model Learning for Control, Learning and Adaptive Systems
Abstract: Multi-robot transfer learning allows a robot to use data generated by a second, similar robot to improve its own behavior. The potential advantages are reducing the time of training and the unavoidable risks that exist during the training phase. Transfer learning algorithms aim to find an optimal transfer map between different robots. In this paper, we investigate, through a theoretical study of single-input single-output (SISO) systems, the properties of such optimal transfer maps. We first show that the optimal transfer learning map is, in general, a dynamic system. The main contribution of the paper is to provide an algorithm for determining the properties of this optimal dynamic map including its order and regressors (i.e., the variables it depends on). The proposed algorithm does not require detailed knowledge of the robots' dynamics, but relies on basic system properties easily obtainable through simple experimental tests. We validate the proposed algorithm experimentally through an example of transfer learning between two different quadrotor platforms. Experimental results show that an optimal dynamic map, with correct properties obtained from our proposed algorithm, achieves 60-70% reduction of transfer learning error compared to the cases when the data is directly transferred or transferred using an optimal static map.

Keywords: Model Learning for Control, Learning and Adaptive Systems, Motion Control
Abstract: In this work, we propose a model-based and data efficient approach for reinforcement learning. The main idea of our algorithm is to combine simulated and real rollouts to efficiently find an optimal control policy. While performing rollouts on the robot, we exploit sensory data to learn a probabilistic model of the residual difference between the measured state and the state predicted by a simplified model. The simplified model can be any dynamical system, from a very accurate system to a simple, linear one. The residual difference is learned with Gaussian processes. Hence, we assume that the difference between real and simplified model is Gaussian distributed, which is less strict than assuming that the real system is Gaussian distributed. The combination of the partial model and the learned residuals is exploited to predict the real system behavior and to search for an optimal policy. Simulations and experiments show that our approach significantly reduces the number of rollouts needed to find an optimal control policy for the real system.

Keywords: Model Learning for Control, Learning and Adaptive Systems
Abstract: Inverse dynamics models are applied to a plethora of robot control tasks such as computed-torque control, which are essential for trajectory execution. The analytical derivation of such dynamics models for robotic manipulators can be challenging and depends on their physical characteristics. This paper proposes a machine learning approach for modeling inverse dynamics and provides information about its implementation on a physical robotic system. The proposed algorithm can perform online multi-target learning, thus allowing efficient implementations on real robots. Our approach has been tested both offline, on datasets captured from three different robotic systems and online, on a physical system. The proposed algorithm exhibits state-of-the-art performance in terms of generalization ability and convergence. Furthermore, it has been implemented within ROS for controlling a Baxter robot. Evaluation results show that its performance is comparable to the built-in inverse dynamics model of the robot.

Keywords: Model Learning for Control, Learning and Adaptive Systems
Abstract: Modern robotics is gravitating toward increasingly collaborative human robot interaction. Tools such as acceleration policies can naturally support the realization of reactive, adaptive, and compliant robots. These tools require us to model the system dynamics accurately -- a difficult task. The fundamental problem remains that simulation and reality diverge--we do not know how to accurately change a robot's state. Thus, recent research on improving inverse dynamics models has been focused on making use of machine learning techniques. Traditional learning techniques train on the actual realized accelerations, instead of the policy's desired accelerations, which is an indirect data source. Here we show how an additional training signal -- measured at the desired accelerations -- can be derived from a feedback control signal. This effectively creates a second data source for learning inverse dynamics models. Furthermore, we show how both the traditional and this new data source, can be used to train task-specific models of the inverse dynamics, when used independently or combined. We analyze the use of both data sources in simulation and demonstrate its effectiveness on a real-world robotic platform. We show that our system incrementally improves the learned inverse dynamics model, and when using both data sources combined converges more consistently and faster.

Keywords: Model Learning for Control, Learning and Adaptive Systems, Autonomous Vehicle Navigation
Abstract: Motion planning with predictable timing and velocity will enable a number of interesting applications such as autonomous intersection management (AIM). These planning algorithms depend on an accurate model of the performance of the vehicular controllers, which can be highly non-linear. Au et al. proposed a motion planning algorithm to satisfy the arrival requirements in AIM. However, they assumed that the performance models are given for every road and did not discuss how to learn these models. In this paper, we propose an instance-based learning approach to learn the performance models automatically, and argue that instance-based learning is suitable for this learning task because performance models for different roads can have a high correlation with each other. Moreover, an exploration strategy based on the principle of least effort is given to speed up the learning process. Our experiments showed that the instance-based learning method with distance-based exploration strategy offers a faster learning rate than the artificial neural network methods.

Keywords: Autonomous Agents, Autonomous Vehicle Navigation, Sensor Fusion
Abstract: We consider the problem of developing robots that navigate like pedestrians on sidewalks through city centers for performing various tasks including delivery and surveillance. One particular challenge for such robots is crossing streets without pedestrian traffic lights. To solve this task the robot has to decide based on its sensory input if the road is clear. In this work, we propose a novel multi-modal learning approach for the problem of autonomous street crossing. Our approach solely relies on laser and radar data and learns a classifier based on Random Forests to predict when it is safe to cross the road. We present extensive experimental evaluations using real-world data collected from multiple street crossing situations which demonstrate that our approach yields a safe and accurate street crossing behavior and generalizes well over different types of situations. A comparison to alternative methods demonstrates the advantages of our approach.

Keywords: Compliant Assembly, Industrial Robots
Abstract: Automated assembly, especially peg-in-hole insertion, is a common task in manufacturing. In particular, the high-precision assembly is achieved by high-precision manipulator and sensing system. However, uncertainty and various parts for assembly are still challenges for robotic assembly, especially for low-precision robot and sensors. It is noteworthy that human can implement assembly tasks although the precision of the arm and hand is not comparable with a common industrial robot, in which process compliance is the key characteristic of their motion. In this paper, we present a human-inspired compliant strategy for peg-in-hole assembly task using the environmental constraint and coarse force information. In the proposed strategy, a constraint region is designed for motion planning and utilized for eliminating the uncertainty of the initial positioning error of the peg. Force sensor is applied to sense the contact force of which the direction is used to adjust the movement of the peg. Therefore, high-precision sensor is not necessarily required. Inspired by human compliant assembly, a from coarse to fine adjustment strategy is executed. The contribution of our strategy is that high precision assembly task can be solved by low precision system. The constraint region and force guided directional adjustment have increased the robustness of the system. The strategy is carried out in simulation for round peg-in-hole assembly task. The experimental results show that the assembly task can be successfully completed and demonstrate the effectiveness of our strategy.

Keywords: Compliant Joint/Mechanism, Biologically-Inspired Robots, Physical Human-Robot Interaction
Abstract: The closer distance between robots and human partners in the same space makes compliant robots essential in these days. While torque sensor based impedance control scheme suffers insufficient bandwidth, musculoskeletal systems can achieve similar ability with smaller resources by using mechanically guaranteed dynamics. In this study, inspired by active stiffness mechanism of biological muscle, we developed a variable stiffness actuator which can realize precise joint stiffness without torque sensor based feedback control. The actuator consists of two antagonistically allocated DC motors with a variable radius gear transmission mechanism. The equilibrium position and joint stiffness can be independently controlled adjusting the activation level of motors by feedforward control. Theoretically predicted stiffness is realized well in both static and dynamic condition. In this paper, the design, functional principle and control of the actuator are introduced with analytical investigation.

Keywords: Compliant Joint/Mechanism, Compliance and Impedance Control
Abstract: The paper presents a control method to catch a moving object with a joint actuated by means of a variable stiffness actuator. The controller is designed such that the variable stiffness joint acts as a virtual damper that absorbs the kinetic energy of the moving object. The virtual damping and the output stiffness of the variable stiffness actuator are the control variables. To obtain a critically damped system, the damping coefficient is scheduled on both the output stiffness and the inertia of the system. Experiments on the rotational variable stiffness actuator vsaUT-II validate the control method.

Keywords: Compliant Joint/Mechanism, Force Control, Compliance and Impedance Control
Abstract: The paper proposes a method to assess the feasible torque bandwidth for electrically driven torque controllable actuators over its entire torque amplitude range. The method solely relies on the knowledge of hardware parameters and thereby determines the physically feasible torque control bandwidth at a given torque amplitude, independent of a controller. The method yields torque-frequency diagrams that are suitable to benchmark torque controllers, formulate actuator design specifications and compare as well as select actuators for a specific torque control application. The paper exemplifies the method on a WALK-MAN leg actuator with locked actuator output and the more practical case of a varying load inertia.

Keywords: Compliant Joint/Mechanism, Force Control
Abstract: Series-elastic actuators are quickly becoming the core component of robots operating in real world environments, allowing for robust, safe, torque controlled robots. This paper investigates the influence of the selected stiffness and control parameters. By consolidating several analyses, it is shown two opposing criteria exist for the choice of stiffness: lowering stiffness increases actuator transparency, while increasing stiffness increases torque tracking bandwidth. This fact is used in the context of impedance rendering to propose an optimal stiffness selection criterion that maximises the renderable impedance frequency. This provides a concrete method for choosing stiffness based on the application of the joint. Extensive simulation and experimental results are presented that validate the analyses.

Keywords: Compliant Joint/Mechanism, Mechanism Design, Product Design, Development and Prototyping
Abstract: This paper is concerned with the analysis, design, and prototyping of rotary planar springs for robotics applications such as rotary series elastic actuators, or mechanical couplings. The key contribution is the development of a mathematical model, based on curved beam theory, that allows rapid design, analysis, and optimization of rotary springs that have arbitrary arm shape. The paper also introduces methods to reduce the spring mass via composite arm structures, or arm cutouts. A prototype is designed, analyzed and tested to demonstrate the validity of the model.

Keywords: Dual Arm Manipulation, Compliant Assembly, Force Control
Abstract: In this work a constraint-based control approach is proposed in order to perform a cap rotation task with a dual- arm robot. The method relies on the introduction of a robust specification for the constraint on the interaction force arising during the task, accounting for robot-environment contact model uncertainties, in addition to force measurement noise and surface uncertainties. Experiments have been performed on an ABB dual-arm prototype robot to validate the proposed approach in a cap assembly task, employing a model-based sensorless observer of the interaction forces.

Keywords: Dual Arm Manipulation, Dynamics, Motion Control of Manipulators
Abstract: In this paper, the well-known problem of codependence between inverse dynamics torque and contact force in bimanual object manipulation is addressed. The common contact constraint, namely rigid grasping, is exploited to decompose the set of dynamics equations into two orthogonally decoupled sets. Subsequently, the inverse dynamics control is formulated in a sub-manifold that is independent of the contact force, leading to analytically correct solutions that do not need to resort to common approximations for the aforementioned codependence problem. The contact force is also analytically computed and, therefore, can be optimally distributed using the torque redundancy. Relying on this prediction is most significant in situations where a force sensor at the end-effector is not present or is faulty. Even in the availability of sensory data, the predicted force may be used to correct typically noisy or delayed when filtered measurements, resulting in improved robustness. Simulation experiments on a planar bimanual manipulation model are presented.

Keywords: Dual Arm Manipulation, Intelligent and Flexible Manufacturing, Learning from Demonstration
Abstract: This paper presents a vision-based learning-by-demonstration approach for multi-robot manipulation. With this method, a vision system is involved in both the task demonstration and reproduction stages, and the speed and accuracy of the task reproduction are adapted according to the context of the demonstration. An expert first demonstrates how to use tools to perform a task, while the tool motion is observed using a vision system. The demonstrations are then encoded using a statistical model to generate a reference motion trajectory. Equipped with the same tools and the learned model, the robot is guided by vision to reproduce the task. The task performance was evaluated in terms of both accuracy and speed. However, simply increasing the robot's speed could decrease the reproduction accuracy. To this end, a dual-rate Kalman filter is employed to compensate for latency between the robot and vision system. More importantly, the robot speed is adapted according to the learned motion model. We demonstrate the effectiveness of our approach by performing two tasks: a trajectory reproduction task and a bimanual sewing task. We show that using our vision-based approach, the robots can conduct effective learning by demonstrations and perform accurate and fast task reproduction. The proposed approach is generalisable to other manipulation tasks, where bimanual or multi-robot cooperation is required.

Keywords: Dual Arm Manipulation, Learning and Adaptive Systems
Abstract: Motor synergies are an important concept in human motor control. Through the co-activation of multiple muscles, complex motion involving many degrees-of-freedom can be generated. However, leveraging this concept in robotics typically entails using human data that may be incompatible for the kinematics of the robot. In this paper, our goal is to enable a robot to identify synergies for low-dimensional control using trial-and-error only. We discuss how synergies can be learned through latent space policy search and introduce an extension of the algorithm for the re-use of previously learned synergies for exploration. The application of the algorithm on a bimanual manipulation task for the Baxter robot shows that performance can be increased by reusing learned synergies intra-task when learning to lift objects. But the reuse of synergies between two tasks with different objects did not lead to a significant improvement.

Keywords: Force and Tactile Sensing, Dexterous Manipulation, Grippers and Other End-Effectors
Abstract: The use of tactile feedback for precision manipulation in robotics still lags far behind human capabilities. This study has two principal aims: 1) To demonstrate in-hand reorientation of grasped objects through active tactile manipulation. 2) To present the development of a novel TacTip sensor and GR2 gripper platform for tactile manipulation. Through the use of Bayesian active perception algorithms, the system successfully achieved in-hand reorientation of cylinders of different diameters (20, 25, 30 and 35,mm) using tactile feedback. Average orientation errors along manipulation trajectories were below 5 deg. for all cylinders with reorientation ranges varying from 42 deg. to 67 deg. We also demonstrated an improvement in active tactile manipulation accuracy when using additional training data. Our methods for active tactile manipulation with the GR2 TacTip gripper are model-free, can be used to investigate principles of dexterous manipulation and could lead to essential advances in the areas of robotic tactile manipulation and tele-operated robots.

Keywords: Visual Learning, Deep Learning in Robotics and Automation, Learning from Demonstration
Abstract: Manipulation tasks requiring high precision are difficult for reasons such as imprecise calibration and perceptual inaccuracies. We present a method for visual task outcome verification that provides an assessment of the task status as well as information for the robot to improve this status. The final status of the task is assessed as success, failure or in progress. We propose a deep learning strategy to learn the task with a small number of training episodes and without requiring the robot. A probabilistic, appearance-based pose estimation method is used to learn the demonstrated task. For real-data efficiency, synthetic training images are created around the trajectory of the demonstrated task. We show that our method can estimate the task status with high accuracy in several instances of different tasks, and demonstrate the accuracy of a high-precision task on a real robot.

Keywords: Optimal Control, Motion and Trajectory Generation, Motion and Path Planning
Abstract: A landmark based heuristic is investigated for reducing query phase run-time of the probabilistic roadmap (PRM) motion planning method. The heuristic is generated by storing minimum spanning trees from a small number of vertices within the PRM graph and using these trees to approximate the cost of a shortest path between any two vertices of the graph. The intermediate step of preprocessing the graph increases the time and memory requirements of the classical motion planning technique in exchange for speeding up individual queries making the method advantageous in multi-query applications. This paper investigates these trade-offs on PRM graphs constructed in randomized environments as well as a practical manipulator simulation. We conclude that the method is preferable to Dijkstra's algorithm or the A* algorithm with conventional heuristics in multi-query applications.

Keywords: Optimization and Optimal Control, Aerial Systems: Mechanics and Control, Control Architectures and Programming
Abstract: This paper presents a unified control framework for controlling a quadrotor tail-sitter UAV. The most salient feature of this framework is its capability of uniformly treating the hovering and forward flight, and enabling continuous transition between these two modes, depending on the commanded velocity. The key part of this framework is a nonlinear solver that solves for the proper attitude and thrust that produces the required acceleration set by the position controller in an online fashion. The planned attitude and thrust are then achieved by an inner attitude controller that is global asymptotically stable. To characterize the aircraft aerodynamics, a full envelope wind tunnel test is performed on the full-scale quadrotor tail-sitter UAV. In addition to planning the attitude and thrust required by the position controller, this framework can also be used to analyze the UAV’s equilibrium state (trimmed condition), especially when wind gust is present. Finally, simulation results are presented to verify the controller’s capacity, and experiments are conducted to show the attitude controller’s performance.

Keywords: Optimization and Optimal Control, Distributed Robot Systems, Multi-Robot Systems
Abstract: In this paper, we introduce a novel approach of reducing the communication load in distributed model predictive control (DMPC) for mobile robots. The key idea is to project the predicted state trajectory onto a grid resulting in an occupancy grid prediction. This approach has the advantage of utilizing continuous optimization methods while only quantized information is exchanged. We consider non-holonomic mobile robots to numerically and experimentally investigate the proposed method.

Keywords: Optimization and Optimal Control, Humanoid and Bipedal Locomotion, Motion and Path Planning
Abstract: Trajectory optimization with direct collocation are widely used in various bipedal walking studies, from dynamic simulation in biomechanics to efficient bipedal walking motion generation with multiple contact domains. Although the latter has gained popularity, most of the approaches in this field in general rely on pre-determined contact sequence (domains). This motivates us to use trajectory optimization through contact for generating an efficient and human-like walking gait, because this approach can automatically generate the contact sequence by solving a nonlinear program (NLP) with complementary constraints. However, in this approach the initial guess affects the result significantly, and the direct collocation with Euler method may not be accurate enough for the system dynamics. Therefore, we propose a modified framework and constraints to improve the generated results. We used a zero moment point (ZMP)-based flat-feet walking gait as an initial guess. We also show how to add virtual components like springs at ankle joints to alter the behavior of the resultant walking gait. In addition, considering the one-sided springs at the passive toe joints of the bipedal robot AMBER 3, additional complementary constraints are introduced for a better match of the full dynamics. The results of our modified approach with different constraint conditions are presented and discussed.

Keywords: Optimization and Optimal Control, Learning and Adaptive Systems, Motion and Path Planning
Abstract: Nonlinear optimal control problems are challenging to solve due to the prevalence of local minima that prevent convergence and/or optimality. This paper describes nearest-neighbors optimal control (NNOC), a data-driven framework for nonlinear optimal control using indirect methods. It determines initial guesses for new problems with the help of precomputed solutions to similar problems, retrieved using k-nearest neighbors. A sensitivity analysis technique is introduced to linearly approximate the variation of solutions between new and precomputed problems based on their variation of parameters. Experiments show that NNOC can obtain the global optimal solution orders of magnitude faster than standard random restart methods, and sensitivity analysis can further reduce the solving time almost by half. Examples are shown on two optimal control problems in vehicle control.

Keywords: Optimization and Optimal Control, Motion and Path Planning, Collision Avoidance
Abstract: In this paper, we apply lossless and successive convexification techniques and employ real-time on-board convex optimization to perform constrained motion planning for quad-rotors. In general, this problem is challenging for real-time on-board applications due to its non-convex nature. The ability to generate feasible trajectories quickly and reliably is central to operating high-performance aerial robots in populated spaces. Motivated by our earlier research on convexification of non-convex optimal control problems, we use these convexification techniques to cast the problems into one (or a sequence of) Second-Order Cone Programming (SOCP) problem(s). In doing so, we are able to attain our solutions by leveraging modern advances in Interior Point Method (IPM) algorithms. Here, we focus on 3-degree-of-freedom trajectory generation, whereby closed-loop control is used to track a translational trajectory computed on-board at the onset of the maneuver. To the best of our knowledge, this is the first demonstration of convexification techniques used in real-time on-board trajectory generation for high-performance quad-rotor flight. We present two example scenarios: (1) a case where lossless convexification is used to increase the available control envelope, thus enabling an agile flip maneuver, and (2) a case where both convexification techniques are used to compute a path through a flight space containing ellipsoidal keep-out zones. We present flight demonstration results obtained using the Autonomous Control Laboratory's (ACL's) custom quad-rotor platforms and SOCP optimization software. Additionally, computation timing statistics for the example scenarios obtained using a series of mobile ARM and Intel processors show a minimum mean computation time of 36.5 and 122.2 milliseconds, respectively.

Keywords: Climbing Robots, Legged Robots, Robotics in Hazardous Fields
Abstract: In this paper, a new experimental legged robot for inspection and disaster response in social infrastructures designed for humans, such as factories and power plants is presented. The robot has distinctive features to climb up and down ladders with a narrow cage, to pass through narrow environments, to move upon scattered debris at disaster sites and to deal with sudden and unexpected contact. The robot also satisfies environmental requirements related to dust, water and temperature. To implement environmental resistance to a slim body for movement in a narrow environment, cooling of heat sources such as CPUs is a serious problem. Therefore a new cooling structure for humanoid robots is proposed. The proposed robot experiments on the above features are also shown in this paper.

Keywords: Legged Robots, Mechanism Design, Motion Control
Abstract: This paper introduces a leg design method aiming at speed maximization for quadruped robots with two-segment compliant legs, in trotting and bounding. The proposed method is an effort to address the leg design challenge in a holistic way, exploiting the coupling between gait parameters, leg design parameters and hardware constraints, while remaining control scheme independent. Optimal body trajectories and footfalls are derived using a simplified centroidal dynamics model, whereas joint trajectories and torques are computed by a more complex dynamic model, incorporating actuation parameters and constraints. The method is applied using real robot parameters to yield an optimal leg design, validated through a realistic trotting simulation experiment. In this experiment, the robot achieves accelerating motion from stance towards the maximum speed predicted by the method.

Keywords: Legged Robots, Motion Control, Humanoid and Bipedal Locomotion
Abstract: The main motivation of this paper is to understand the role of foot placement and ankle push-off control in stabilizing bipedal gaits. We modify the simplest walker (heavy torso, light legs) by incorporating a hip spring, a hip actuator, and a telescopic linear actuator. We consider two stability criteria: one-step dead-beat stabilization for full correction of disturbance in a single step and exponential orbital stabilization using discrete control Lyapunov function. Our findings are as follows: (1) Both control strategies have almost similar robustness as measured by the number of steps walked on stochastic terrain before failure, but both strategies are more robust for changing terrain with step down and less robust for step up. (2) One step dead-beat stabilization is more energy-efficient than exponential stabilization. (3) Control strategy for step up is to decrease foot placement or maintain push-off and for step down is to increase foot placement or decrease the push-off. However, it is most energy-efficient to use foot placement control for step up and push-off control for step down.

Keywords: Legged Robots, Robust/Adaptive Control of Robotic Systems, Model Learning for Control
Abstract: The dynamic parameters of the leg links are not frequently altered during expeditions and therefore are best suitable to be identified offline. On the other hand, the trunk parameters depend on the different modules that will be mounted on the robot, like a motor to provide the hydraulic power, or different sets of cameras for perception. Based on this motivation, in this paper we propose two recursive approaches to online identify the position of the Center of Mass (CoM) of the robot trunk, when a payload change occurs. We will demonstrate the effectiveness of the proposed approaches analysing their robustness in simulation. Furthermore, experimental trials on our 80 kg quadruped robot HyQ, showed that the proposed strategies are applicable during locomotion to cope with big payload changes. These can create shifts on the CoM that can possibly hamper locomotion stability. In these cases our online identification strategy is able to promptly update the trunk model parameters in the planner/controller, enhancing the locomotion stability.

Keywords: Legged Robots, Motion Control, Motion and Path Planning
Abstract: Various control strategies using trajectory planning, object recognition and learning are researched for avoiding obstacles when legged robot is walking. In this paper, designed trajectory by using Non Uniform Basis Spline (NUBS) curve and control strategy, by using proposed trajectory, to effectively overcome obstacles are presented. The trajectory designed by NUBS curve has several advantages: 1) local modification, 2) tracking velocity control for each domain, and 3) low degree trajectory with a large number of control points. The robot gets remarkable effectiveness when these advantages are used to generate the trajectory for walking and overcoming obstacles. By implementation of the proposed control strategy, quadruped robot can walk over obstacles while keeping its gait, speed and balance without collision although adjusting relatively preferable state which is more suitable position or posture of the walking robot shortly before encountering obstacles is not required. The proposed trajectory and control strategy are discussed, and performance is validated through experimental evaluations.

Keywords: Legged Robots, Sensor-based Control, Robust/Adaptive Control of Robotic Systems
Abstract: Animal locomotion exhibits all the features of complex non linear systems such as multi-stability, critical fluctuation, limit cycle behavior and chaos. Studying these aspects on real robots has been proved difficult and therefore results mostly rely on the use of computer simulation. Simple control approaches - based on phase oscillators - have been proposed and exhibit several of these features. In this work, we compare two types of controllers: (a) an open loop control approach based on phase oscillators and (b) the Tegotae-based closed loop extension of this controller. The first controller has been shown to exhibit synchronization features between the body and the controller when applied to a quadruped robot with compliant leg structures. In this contribution, we apply both controllers to the locomotion of a stiff quadruped structure. We show that the Tegotae-controller exhibits self-organizing behavior, such as spontaneous gait transition and critical fluctuation. Moreover, it exhibits features such as the ability to stabilize both asymmetric and symmetric morphological changes, despite the lack of compliance in the leg.

Keywords: RGB-D Perception, Object detection, segmentation, categorization, Deep Learning in Robotics and Automation
Abstract: Fusing RGB and depth data is compelling in boosting performance for various robotic and computer vision tasks. Typically, the streams of RGB and depth information are merged into a single fusion point in an early or late stage to generate combined features or decisions. The single fusion point also means single fusion path, which is congested and inflexible to fuse all the information from different modalities. As a result, the fusion process is brute-force and consequently insufficient. To address this problem, we propose a multi-scale multi-path multi-modal fusion network (M³Net), in which the fusion path is scattered to diversify the contributions of each modality from global and local perspectives. Specially, the CNN streams of each modality are fused with a global understanding branch and meanwhile a local capturing branch. By filtering and regulating information flow in a multi-path way, the M3Net are equipped with more adaptive and flexible fusion mechanism, thus easing the gradient-based learning process, improving the directness and transparency of fusion process and simultaneously facilitating the fusion process with multi-scale perspectives. Comprehensive experiments on three benchmark datasets demonstrate the significant and consistent improvements of the proposed approach over state-of-the-art methods.

Keywords: RGB-D Perception, Object detection, segmentation, categorization, Semantic Scene Understanding
Abstract: We propose a new method for the propagation of semantic labels in RGB-D video of indoor scenes given a set of ground truth keyframes. Manual labeling of all pixels in every frame of a video sequence is labor intensive and costly, yet required for training and testing of semantic segmentation methods. The availability of video enables propagation of labels between the frames for obtaining large amounts of annotated pixels. While previous methods commonly used optical flow motion cues for label propagation, we present a novel approach using the camera poses and 3D point clouds for propagating the labels in superpixels computed on the unannotated frames of the sequence. The propagation task is formulated as an energy minimization problem in a Conditional Random Field (CRF). We performed experiments on the 8 video sequences from SUN3D dataset and showed superior performance to an optical flow based label propagation approach. Furthermore, we demonstrated that the propagated labels can be used to learn better models using data hungry deep convolutional neural network (DCNN) based approaches for the task of semantic segmentation. The approach demonstrates an increase in performance when the ground truth keyframes are combined with the propagated labels during training.

Keywords: RGB-D Perception, Domestic Robots, Service Robots
Abstract: Robots performing everyday tasks such as cooking in a kitchen need to be able to deal with variations in the household tools that may be available. Given a particular task and a set of tools available, the robot needs to be able to assess which would be the best tool for the task, and also where to grasp that tool and how to orient it. This requires an understanding of what is important in a tool for a given task, and how the grasping and orientation relate to performance in the task. A robot can learn this by trying out many examples. This learning can be faster if these trials are done in simulation using tool models acquired from the Web. We provide a semi-automatic pipeline to process 3D models from the Web, allowing us to train from many different tools and their uses in simulation. We represent a tool object and its grasp and orientation using 21 parameters which capture the shapes and sizes of principal parts and the relationships among them. We then learn a ‘task function’ that maps this 21 parameter vector to a value describing how effective it is for a particular task. Our trained system can then process the unsegmented point cloud of a new tool and output a score and a way of using the tool for a particular task. We compare our approach with the closest one in the literature and show that we achieve significantly better results.

Keywords: RGB-D Perception, Visual Tracking, Learning and Adaptive Systems
Abstract: This paper presents a self-supervised online learning approach for 3D object tracking that requires no pretraining of appearance. Our method focuses on selecting the most relevant parts of the RGBD input by continuously updating appearance classifiers in conjunction with the spatial occupancy of the target. Fine-grained regions selected via the learned bottom-up saliency, together with spatial cues of the 3D shape model, are used to identify and localize the target via shape registration. The subsequent 3-D pose estimate along with positive and negative labels from the registration are used for online learning appearance. The proposed method outperforms competing model-based tracking algorithms on public datasets as well as on a new motion scene dataset that we have collected.

Keywords: RGB-D Perception, Object detection, segmentation, categorization, Computer Vision for Other Robotic Applications
Abstract: We propose a new saliency-guided method for generating supervoxels in 3D space. Rather than using an evenly distributed spatial seeding procedure, our method uses visual saliency to guide the process of supervoxel generation. This results in densely distributed, small, and precise supervoxels in salient regions which often contain objects, and larger supervoxels in less salient regions that often correspond to background. Our approach largely improves the quality of the resulting supervoxel segmentation in terms of boundary recall and undersegmentation error on publicly available benchmarks.

Keywords: RGB-D Perception, SLAM, Mapping
Abstract: The paper exploits weak Manhattan constraints to parse the structure of indoor environments from RGB-D video sequences in an online setting. We extend the previous approach for single view parsing of indoor scenes to video sequences and formulate the problem of recovering the floor plan of the environment as an optimal labeling problem solved using dynamic programming. The temporal continuity is enforced in a recursive setting, where labeling from previous frames is used as a prior term in the objective function. In addition to recovery of piecewise planar weak Manhattan structure of the extended environment, the orthogonality constraints are also exploited by visual odometry and pose graph optimization. This yields reliable estimates in the presence of large motions and absence of distinctive features to track. We evaluate our method on several challenging indoors sequences demonstrating accurate SLAM and dense mapping of low texture environments. On existing TUM benchmark we achieve competitive results with the alternative approaches which fail in our environments.

Keywords: Biomimetics, Hydraulic/Pneumatic Actuators, Marine Robotics
Abstract: Sub-seafloor exploration is an important area of biological and geological research, and often requires samples of seafloor mud to be recovered. The minerals and marine sediments trapped in this mud are also potentially valuable resources. However, seabed exploration has been limited by the complexity and cost of the traditional equipment used. Wide area exploration can be made possible by deploying multiple robots to autonomously search beneath the ocean floor. In this study, we prototyped a sub-seafloor excavation robot for use in seabed exploration. The key system comprised a propulsion unit and an excavation unit that allow the robot to move freely within the seabed. We experimentally demonstrated the ability of the excavation robot to produce curved boreholes with a diameter of 160 mm, a turning radius of 1.67 m, and a depth of 613 mm. Our experimental results confirm that nonlinear excavation is possible using the excavation robot developed.

Keywords: Biomimetics, Humanoid Robots, Tendon/Wire Mechanisms
Abstract: The human forearm is composed of two long, thin bones called the radius and the ulna, and rotates using two axle joints. We aimed to develop a forearm based on the body proportion, weight ratio, muscle arrangement, and joint performance of the human body in order to bring out its benefits. For this, we need to miniaturize the muscle modules. To approach this task, we arranged two muscle motors inside one muscle module, and used the space effectively by utilizing common parts. In addition, we enabled the muscle module to also be used as the bone structure. Moreover, we used miniature motors and developed a way to dissipate the motor heat to the bone structure. Through these approaches, we succeeded in developing a forearm with a radioulnar joint based on the body proportion, weight ratio, muscle arrangement, and joint performance of the human body, while keeping maintainability and reliability. Also, we performed some motions such as soldering, opening a book, turning a screw, and badminton swinging using the benefits of the radioulnar structure, which have not been discussed before, and verified that Kengoro can realize skillful motions using the radioulnar joint like a human.

Keywords: Biomimetics, Recognition, Deep Learning in Robotics and Automation
Abstract: Underwater electrosense is bio-inspired by weakly electric fishes that use an electric field to see the objects in the water. Current studies on engineering electrosense focus on designing sophisticated sensors and algorithms for emulating biological functions including localization and identification. This work aimed to develop a planar sensor equipped with a dense electrode array that is capable of providing accurate and dense data for identifying objects of similar volume and aspect ratio, which has been a challenge in underwater sensing. After sensor design and implementation were presented, convolutional neural networks (CNN), which are widely used in digital image recognition, was trained using both simulation and experimental data. In simulation, the overall success rate on identifying the sphere, cube, and rod is 92.6% by a 28x28 electrode array. In the preliminary experimental tests, a sensor with 16x16 electrode array achieved an overall success rate of 90.4% on identifying a sphere and a rod.

Keywords: Passive Walking, Kinematics, Legged Robots
Abstract: The planar rimless wheel (PRW) is a classic and simple passive dynamic mechanism to simulate biped walking, different simplified PRW models have respective descriptions and limited applications. This paper focus on constructing the general PRW model, and analyzing the intrinsic and mathematical relation between different PRW models. Based that, the limit cycle of symmetric PRW and asymmetric PRW are proposed separately. Moreover, the stability in universal domain are investigated in detail. Furthermore, simulation and experiment results show that the 2-period limit cycle motion of the asymmetric PRW is more effective, flexible and self-adaptive compared with regular rimless wheel, and more actual applications would be achieved by adopting the general model.

Keywords: Climbing Robots, Space Robotics and Automation, Underactuated Robots
Abstract: This paper presents a new teleoperated spherical tensegrity robot capable of performing locomotion on steep inclined surfaces. With a novel control scheme centered around the simultaneous actuation of multiple cables, the robot demonstrates robust climbing on inclined surfaces in hardware experiments and speeds significantly faster than previous spherical tensegrity models. This robot is an improvement over other iterations in the TT-series and the first tensegrity to achieve reliable locomotion on inclined surfaces of up to 24 degrees. We analyze locomotion in simulation and hardware under single and multi-cable actuation and introduce two novel multi-cable actuation policies, suited for steep incline climbing and speed, respectively. We propose compelling justifications for the increased dynamic ability of the robot and motivate development of optimization algorithms able to take advantage of the robot’s increased control authority.

Keywords: Performance Evaluation and Benchmarking, Social Human-Robot Interaction, Physical Human-Robot Interaction
Abstract: Handshakes are common greetings, and humans therefore have strong priors of what a handshake should feel like. This makes it challenging to create compelling and realistic human-robot handshakes, necessitating the consideration of human haptic perception in the design of robot hands. At its most basic level, haptic perception is encoded by contact points and contact pressure distributions on the skin.

This motivates our work on measuring the contact area and contact pressure in human handshaking interactions. We present two benchmarking experiments in this regard, measuring the contact locations in human-human/human-robot handshaking and the contact pressure distribution for handshakes with a sensorized palm. We present results from human studies with the benchmarking experiments, providing a baseline for comparison with robot hands as well as presenting new insights into human handshaking. We also show initial work in using these results for the evaluation of robot hands, and progressing towards iterative design of robot hands optimized for social hand interactions.

Keywords: Humanoid Robots, Humanoid and Bipedal Locomotion, Contact Modelling
Abstract: Recent studies have explored humanoid robots in contact with the environment in various ways. However, many of them assumed static rather than sliding contacts.Studies on humanoid shuffle motion planning have realized sliding motions, such as turning, but relied on quasi-static balance control. In this paper, we propose a dynamic balance control method for sliding contact motions.The proposed method consists of the Distributed Force Contact Constraint (D.F.C.C.), which describes rotational sliding contact constraints, and the Slide Friction Control (S.F.C.), which controls humanoid dynamic balance based on the Model Predictive Control by using the D.F.C.C. The D.F.C.C. segments a contact face into a grid of contact points and optimize the vertical component of the contact forces. This enables us to calculate the sliding friction forces at each contact point. The S.F.C. is the Model Predictive Control for distributing contact forces to each contact face considering sliding frictional dynamics. The D.F.C.C. is simple and easy to apply to the S.F.C. In our online stabilizer, we control not only a ZMP but also contact forces for realizing the contact force distributions planned in the S.F.C. Finally, we show our method's validity through the experiment using life-sized humanoid robot JAXON.

Keywords: Humanoid Robots, Humanoid and Bipedal Locomotion, Legged Robots
Abstract: In this paper, we introduce a control strategy aimed at generating a stable walking pattern for a humanoid pushing a heavy load on a cart. In contrast to previous approaches that rely on force/torque sensors to measure the interaction between the robot and the pushed object, we present a simple model-based controller that can be implemented on most robots due to its computationally efficient design. Every aspect of the controller design is covered, from the formulation and validation of the dynamic model, to the implementation and validation on an actual humanoid robot. The experimental results show that the controller can efficiently make a NAO humanoid transport, in a stable way, the equivalent of its own weight on a rolling cart.

Keywords: Humanoid and Bipedal Locomotion, Humanoid Robots, Optimization and Optimal Control
Abstract: This paper presents a COM trajectory generation method for 3D walking and skating motion by nonlinear optimization. In our method, we solve the following problems: (1) dealing with both walking and skating motion in the same framework, (2) generating center of mass (COM) trajectory faster than execution time, (3) executing motion with large acceleration. For solving (1) and (2), we calculate the COM trajectory at every step and introduce frictional constraints to the Divergent Component of Motion as terminal conditions. By changing the terminal condition, we can generate both skating and walking motion. Besides, the nonlinear constrained optimization using Gauss Pseudospectral Method is introduced for solving (2) and (3). Thanks to this method, we generate the 3D COM trajectory considering contact constraints and kinematic constraints faster than execution time. Finally, the walking and skating experiment were carried out to confirm the effectiveness of our method using life-sized humanoid HRP-2. Applying the proposed method, HRP-2 could successfully walk at 0.4 [m/s] and skate at 1.0 [m/s].

Keywords: Humanoid and Bipedal Locomotion, Humanoid Robots
Abstract: The traction force of any moving object on a floor is the frictional force between the object and the floor. Therefore, investigation of the frictional constraint is very important for such moving object on a floor as well as a biped walking robot. Conventionally, the study of frictional constraint of a biped robot was only constrained in translational slip, without considering rotational slip, which is actually important in actual biped walking. This is one of the causes the walking speed of biped robots lower than the one of human beings. In this paper, an evaluation method of frictional constraints for biped robots is proposed. The translational friction force and the twist frictional torque acting on the biped robot’s sole are both deduced and verified for single support phase. First, based on a biped robot model, an approach to calculate the frictional force and torque is deduced as the quantitative expression of frictional constraint. Then, the experimental verification method is designed and the proposed calculation approach is confirmed. Finally, the consideration of the result of experiments is discussed. This approach is expected to be used for helping the realization of fast biped walkings.

Keywords: Humanoid and Bipedal Locomotion
Abstract: We present a real-time pattern generator for dynamic walking over rough terrains. Our method automatically finds step durations, a critical issue over rough terrains where they depend on terrain topology. To achieve this level of generality, we consider a Floating-base Inverted Pendulum (FIP) model where the center of mass can translate freely and the zero-tilting moment point is allowed to leave the contact surface. This model is equivalent to a linear inverted pendulum with variable center-of-mass height, but its equations of motion remain linear. Our solution then follows three steps: (i) we characterize the FIP contact-stability condition; (ii) we compute feedforward controls by solving a nonlinear optimization over receding-horizon FIP trajectories. Despite running at 30~Hz in a model-predictive fashion, simulations show that the latter is too slow to stabilize dynamic motions. To remedy this, we (iii) linearize FIP feedback control into a constrained linear-quadratic regulator that runs at 300~Hz. We finally demonstrate our solution in simulations with a model of the HRP-4 humanoid robot, including noise and delays over state estimation and foot force control.

Keywords: Humanoid and Bipedal Locomotion, Legged Robots
Abstract: This paper presents a method for navigating 3D dynamically walking bipedal robots amidst obstacles. Our framework relies on composing gait primitives corresponding to limit-cycle locomotion behaviors and it produces nominal motion plans that are compatible with the system's dynamics and can be tracked with high fidelity. The low-level controllers of the biped are designed within the Hybrid Zero Dynamics (HZD) framework. Exploiting the dimensional reduction afforded by HZD and properties of invariant sets of switching systems among multiple equilibria, we obtain polynomial approximations of a reduced order Poincare map and of the net change of the center of mass location over a stride. These polynomials are then incorporated in a high-level Rapidly Exploring Random Tree (RRT) planner to generate nominal plans which are tracked by the biped with drastically low drifting errors, without adversely affecting the time for computation.

Keywords: Automation in Life Sciences: Biotechnology, Pharmaceutical and Health Care, Physically Assistive Devices, Service Robots
Abstract: Elderly people have particular needs in performing bathing activities, since these tasks require body flexibility. Our aim is to build an assistive robotic bath system, in order to increase the independence and safety of this procedure. Towards this end, the expertise of professional carers for bathing sequences and appropriate motions have to be adopted, in order to achieve natural, physical human - robot interaction. In this paper, a real-time end-effector motion planning method for an assistive bath robot, using on-line Point-Cloud information, is proposed. The visual feedback obtained from Kinect depth sensor is employed to adapt suitable washing paths to the user’s body part motion and deformable surface. We make use of a navigation function-based controller, with guarantied globally uniformly asymptotic stability, and bijective transformations for the adaptation of the paths. Experiments were conducted with a rigid rectangular object for validation purposes, while a male and a female subject took part to the experiment in order to evaluate and demonstrate the basic concepts of the proposed methodology.

Keywords: Automation in Life Sciences: Biotechnology, Pharmaceutical and Health Care, Physically Assistive Devices, Medical Robots and Systems
Abstract: When designing a user-friendly Mobility Assistive Device (MAD) for mobility constrained people, it is important to take into account the diverse spectrum of disabilities, which results into completely different needs to be covered by the MAD for each specific user. An intelligent adaptive behavior is necessary for the deployment of a MAD. In this work we present experimental results, using an in house developed methodology for estimating the gait of users with different mobility status while interacting with a robotic MAD. Our methodology uses a laser scanner, mounted on the MAD to track the legs positions using Particle Filters and Probabilistic Data Association (PDA-PF). The legs' states are fed to an HMM-based pathological gait cycle recognition system to compute in real-time the gait parameters that are crucial for the mobility status characterization of the user. We aim to show that a gait assessment system would be an important feedback for an intelligent MAD. Thus, we use this system to compare the gaits of the subjects using two different control settings of the MAD and we experimentally validate the ability of our system to recognize the impact of the control designs on the users' walking performance. The results demonstrate that a generic control scheme does not meet every patient's needs, and therefore, an Adaptive Context-Aware MAD (ACA MAD), that can understand the specific needs of the user, is important for enhancing the human-robot physical interaction.

Keywords: Automation in Life Sciences: Biotechnology, Pharmaceutical and Health Care, Surgical Robotics: Laparoscopy
Abstract: This paper presents an approach to supervisory control of a DaVinci surgical robot. At present, such robots are controlled by teleoperation, with dissimilar kinematics of the operator interface vs. the robot. As a result, it can be difficult for the operator to visualize the kinematic restrictions on the robot, particularly for desired extended, precise trajectories, such as circular needle driving. The interface presented here constitutes a means to elevate the operator from teleoperation mode to supervisory mode. The operator interacts directly with a point-cloud display, allowing selection of task specifications from which the system automatically computes and executes precise trajectories to achieve the task goals. The intent is to allow the operator to focus on task specifications and rely on automation to achieve faster and more precise execution.

Keywords: Automation in Life Sciences: Biotechnology, Pharmaceutical and Health Care, Health Care Management, Sensor Fusion
Abstract: Poor balance control and falls are big issues for older adults that due to aging decline have a lower postural balance and directional control in balance performance than younger age groups. The four square step test (FSST) was developed to evaluate rapid stepping that is often required when changing direction and avoiding obstacles while walking. However, previous researchers used only the total time as the assessment in the test. The aim of this paper is to objectively quantify the sequence and direction of the steps in FSST, by using two inertial sensors placed on both feet. An algorithm was developed to automatically segment the steps performed during the test, and calculate the stepping direction from the linear velocity of the foot. Experiments were conducted with 100 Japanese healthy older adults, where sensor data and video of 20 subjects were randomly subtracted for algorithm verification. The results showed that the algorithm succeeded for 71.7% trials in recognizing both the step sequence and step direction in FSST, while 90.2% of the detection failure could be excluded with an auto verification method.

Keywords: Robotics in Construction, Computer Vision for Other Robotic Applications, Visual Learning
Abstract: Trip hazards are a significant contributor to accidents on construction and manufacturing sites. Current safety inspections are labour intensive and limited by human fallibility,making automation of trip hazard detection appealing from both a safety and economic perspective. Trip hazards present an interesting challenge to modern learning techniques because they are defined as much by affordance as by object type; for example wires on a table are not a trip hazard, but can be if lying on the ground. To address these challenges, we conduct a comprehensive investigation into the performance characteristics of 11 different colour and depth fusion approaches,including 4 fusion and one non fusion approach; using colour and two types of depth images. Trained and tested on over 600 labelled trip hazards over 4 floors and 2000m² in an active construction site, this approach was able to differentiate between identical objects in different physical configurations.Outperforming a colour-only detector, our multi-modal trip detector fuses colour and depth information to achieve a 4% absolute improvement in F1-score. These investigative results and the extensive publicly available dataset moves us one step closer to assistive or fully automated safety inspection systems on construction sites.

Keywords: Brain Machine Interfaces
Abstract: Human Swarm Interaction (HSI) is a new field which relates to the effective control of robotic swarms by human operators and has received increased attention lately. Based on the approaches that have been proposed, it is evident that the control of swarms can become quite complicated. On the other hand, Brain Machine Interfaces (BMI) can offer intuitive control in a plethora of applications where other interfaces alone (e.g. joysticks) are inadequate or impractical, e.g. for people with motor disabilities. There are multiple types of BMI, but most of them rely on the analysis of ElectroEncephaloGraphic (EEG) signals. The authors have previously shown that swarm behaviors elicit specific brain activity on human subjects that observe them. Motivated by this result, in this work, we present preliminary results of a hybrid BMI that combines information from the brain and an external device. An algorithm for extracting information from the frequency domain of EEG signals that allows integration with the manual task of using a joystick is presented. The hybrid interface shows high accuracy and robustness when used as a brain-robot interface. Moreover, it allows for continuous control variables extracted from the EEG signals. Finally, its efficacy is proven across multiple subjects, while its performance is also demonstrated in the real-time control of a swarm of quadrotors.

Keywords: Object detection, segmentation, categorization, Autonomous Agents, Deep Learning in Robotics and Automation
Abstract: We present a system for creating object models from RGB-D views acquired autonomously by a mobile robot. We create high-quality textured meshes of the objects by approximating the underlying geometry with a Poisson surface. Our system employs two optimization steps, first registering the views spatially based on image features, and second aligning the RGB images to maximize photometric consistency with respect to the reconstructed mesh. We show that the resulting models can be used robustly for recognition by training a Convolutional Neural Network (CNN) on images rendered from the reconstructed meshes. We perform experiments on data collected autonomously by a mobile robot both in controlled and uncontrolled scenarios. We compare quantitatively and qualitatively to previous work to validate our approach.

Keywords: Object detection, segmentation, categorization, Mapping, Semantic Scene Understanding
Abstract: For intelligent robots to interact in meaningful ways with their environment, they must understand both the geometric and semantic properties of the scene surrounding them. The majority of research to date has addressed these mapping challenges separately, focusing on either geometric or semantic mapping. In this paper we address the problem of building environmental maps that include both semantically meaningful, object-level entities and point- or mesh-based geometrical representations. We simultaneously build geometric point cloud models of previously unseen instances of known object classes and create a map that contains these object models as central entities. Our system leverages sparse, feature-based RGB-D SLAM, image-based deep-learning object detection and 3D unsupervised segmentation.

Keywords: Object detection, segmentation, categorization, RGB-D Perception, Recognition
Abstract: We present a novel method for classifying 3D objects that is particularly tailored for the requirements in robotic applications. The major challenges here are the comparably small amount of available training data and the fact that often data is perceived in streams and not in fixed-size pools. Traditional state-of-the-art learning methods, however, require a large amount of training data, and their online learning capabilities are usually limited. Therefore, we propose a modality-specific selection of convolutional neural networks (CNN), pre-trained or fine-tuned, in combination with a classifier that is designed particularly for online learning from data streams, namely the Mondrian Forest (MF). We show that this combination of trained features obtained from a CNN can be improved further if a feature selection algorithm is applied. In our experiments, we use the resulting features both with a MF and a linear Support Vector Machine (SVM). With SVM we beat the state of the art on an RGB-D dataset, while with MF a strong result for active learning is achieved.

Keywords: Object detection, segmentation, categorization, RGB-D Perception, Perception for Grasping and Manipulation
Abstract: Probabilistic 3D map has been used for object segmentation with multiple camera viewpoints, however, con- ventional methods lack of real-time efficiency or functionality of multilabel object mapping. In this paper, we propose a method to generate three-dimensional map with multilabel occupancy in real-time. Extending our previous work in which only target label occupancy is mapped, we achieve multilabel object segmentation in a single looking around action. We evaluate our method by testing segmentation accuracy with 39 different ob- jects, and applying to robotic manipulation of multiple objects in the experiments. Our mapping-based method outperforms the conventional registration-based method: 40 - 96% relative to 12.6 mean IU3d, and the robot successfuly recognizes (86.9%) and manipulates multiple objects (60.7%) in an environment with heavy occlusions.

Keywords: Object detection, segmentation, categorization, Visual Learning, Computer Vision for Transportation
Abstract: This paper is about enabling robots to improve their perceptual performance through repeated use in their operating environment, creating local expert detectors fitted to the places through which a robot moves. We leverage the concept of 'experiences' in visual perception for robotics, accounting for bias in the data a robot sees by fitting object detector models to a particular 'place'. The key question we seek to answer in this paper is simply: how do we define a place? We build bespoke pedestrian detector models for autonomous driving, highlighting the necessary trade off between generalisation and model capacity as we vary the extent of the 'place' we fit to. We demonstrate a sizeable performance gain over a current state-of-the-art detector when using computationally lightweight bespoke place-fitted detector models.

Keywords: Object detection, segmentation, categorization, Computer Vision for Transportation, Autonomous Vehicle Navigation
Abstract: This work addresses the semantic segmentation of images of street scenes for autonomous vehicles based on a new RGB-Thermal dataset, which is also introduced in this paper. An increasing interest in self-driving vehicles has necessitated the adaptation of semantic segmentation for self-driving systems. However, recent research relating to semantic segmentation is mainly based on RGB images acquired during times of poor visibility at night and under adverse weather conditions. Furthermore, most of these methods only focused on improving performance while ignoring time consumption. The aforementioned problems prompted us to propose new convolutional neural network architecture for multi-spectral image segmentation that enables the segmentation accuracy to be retained during real-time operation. We benchmarked our method by creating an RGB-Thermal dataset in which thermal and RGB images are combined. We showed that the segmentation accuracy was significantly increased by adding thermal infrared information.

Keywords: Aerial Systems: Mechanics and Control, Underactuated Robots
Abstract: A special type of rotary-wing Unmanned Aerial Vehicles (UAV), called Quadcopter have prevailed to the civilian use for the past decade. They have gained significant amount of attention within the UAV community for their redundancy and ease of control, despite the fact that they fall under an under-actuated system category. They come in a variety of configurations. The "+" and "x" configurations were introduced first. Literature pertinent to these two configurations is vast. However, in this paper, we define 6 additional possible configurations for a Quadcopter that can be built under either "+" or "x" setup. These configurations can be achieved by changing the angle that the axis of rotation for rotors make with the main body, i.e., fuselage. This would also change the location of the COM with respect to the propellers which can add to the overall stability. A comprehensive dynamic model for all these configurations is developed for the first time. The overall stability for these configurations are addressed. In particular, it is shown that one configuration can lead to the most statically-stable platform by adopting damping motion in Roll/Pitch/Yaw, which is described for the first time to the best of our knowledge.

Keywords: Aerial Systems: Mechanics and Control, Aerial Systems: Applications, Robotics in Construction
Abstract: This paper describes a octo-rotor unmanned aerial vehicle (UAV) with one degree of freedom (DoF) manipulator for bridge inspection. The UAV has an ability to contact bridge pier to conduct hammering test. To control contact force applied by the UAV to the bridge pier, one DoF is enough for the manipulator implemented on the UAV. A preliminary experiment shows that the force is proportional to the pitch angle of the UAV when it contacts the bridge pier. A bridge pier contact experiment shows that our developed UAV can control contact force between its end-effector and the bridge pier; mean error of the contact force is 4.27[N] (S.D. is 3.64 [N]) against 20[N] desired force.

Keywords: Aerial Systems: Mechanics and Control, Motion Control of Manipulators, Redundant Robots
Abstract: In this paper, a tracking control approach for robot assisted landing of UAV rotorcraft, such as multicopters and helicopters, is presented. The aim is to safely land these type of flying vehicles under side wind conditions and on moving surfaces using a robot manipulator arm mounted on the ground. In our previous work, we have considered light UAVs compared to the robot's payload capabilities. In this work, we present a method that enables to realize robot assisted landing for arbitrary combinations of aerial vehicles and robot arms. We show that the combined system is overactuated. This allows to distribute the workload and enables to assist heavy rotorcraft, whose weight would otherwise exceed the nominal payload of the robot arm. The performance of the controller is evaluated in numerical simulations as well as in experiments using a custom-built hexacopter and a KUKA/DLR light-weight robot.

Keywords: Aerial Systems: Mechanics and Control, Control Architectures and Programming, Dynamics
Abstract: We present a hierarchical control approach that can be used to fulfill autonomous flight, including vertical takeoff, landing, hovering, transition, and level flight, of a quadrotor tail-sitter vertical takeoff and landing unmanned aerial vehicle (VTOL UAV). A unified attitude controller, together with a moment allocation scheme between elevons and motor differential thrust, is developed for all flight modes. A comparison study via real flight tests is performed to verify the effectiveness of using elevons in addition to motor differential thrust. With the well-designed switch scheme proposed in this paper, the aircraft can transit between different flight modes with negligible altitude drop or gain. Intensive flight tests have been performed to verify the effectiveness of the proposed control approach in both manual and fully autonomous flight mode.

Keywords: Aerial Systems: Mechanics and Control, Aerial Systems: Applications
Abstract: Given the wide variety of flight conditions typically encountered by fixed-wing aerial vehicles, the flight performance of a solar unmanned aerial vehicle (SUAV) depends on many factors. Predicting the performance for a given application requires characterization of both system and environmental components. Furthermore, a UAV with a transformable airframe increases the number of characterizable states. Proposed in [1] is a SUAV capable of transforming between quad-rotor and fixed-wing configurations. Each state has a unique set of capabilities which result in significant differences with respect to power consumption and solar panel orientation, increasing the design limitations on various system components. This paper characterizes the solar radiation, propulsion system, and power electronics for both quad-rotor and fixed-wing states. Clear-sky and solar panel models are used to predict the power available for a given location and time. The propulsion system is modeled to predict power consumption for a given mass and considers the optimum propeller pitch. Custom power electronics are modeled and simulated to determine conversion efficiencies and battery sizes required to lift a given payload. The overall system model is discussed for specific applications and is compared with experimental flight data.

Keywords: Aerial Systems: Mechanics and Control, Aerial Systems: Applications
Abstract: Small robots benefit from the ability to go places where humans cannot and are attractive for numerous practical reasons such as portability and manufacturing simplicity. However, with smaller scale comes more difficulty traversing rough terrain, especially for robots which use wheel-based locomotion. Previous approaches to overcome this drawback have included auxiliary mechanisms such as jumping, transformations of the robot or its appendages, and alternative forms of locomotion such as aerial flight capability.

This paper presents a small scale robot that is capable of both ground travel and aerial flight. These locomotion types, in combination, allow for efficient ground-based movement as well as the ability to overcome obstacles or explore otherwise unreachable locations through air travel. The novel aspect of this approach is a transformation between ground and air configurations which allows flight components to be stowed safely within the robot frame while in ground mode. This feature offers advantages over previous approaches such as a highly compact ground configuration and protection of delicate flight hardware when not in use. In this paper the concept is compared to other approaches to address ground robot mobility drawbacks and a detailed design is presented with a focus on the transformation between these modes. Lastly, a fully functional prototype is presented which is capable of ground and air locomotion and the transformation between these configurations.

Keywords: Robotics in Agriculture and Forestry, Field Robots
Abstract: Precision farming robots offer a great potential for reducing the amount of agro-chemicals that is required in the fields through a targeted, per-plant intervention. To achieve this, robots must be able to reliably distinguish crops from weeds on different fields and across growth stages. In this paper, we tackle the problem of separating crops from weeds reliably while requiring only a minimal amount of training data through a semi-supervised approach. We exploit the fact that most crops are planted in rows with a similar spacing along the row, which in turn can be used to initialize a vision-based classifier requiring only minimal user efforts to adapt it to a new field. We implemented our approach using C++ and ROS and thoroughly tested it on real farm robots operating in different countries. The experiments presented in this paper show that with around 1 min of labeling time, we can achieve classification results with an accuracy of more than 95% in real sugar beet fields in Germany and Switzerland.

Keywords: Robotics in Agriculture and Forestry, Field Robots, Object detection, segmentation, categorization
Abstract: This paper describes an algorithm for visually detecting crop stalks in-situ. Field-based crop stalk detection is a challenging computer vision problem due to occlusion by leaves, similarity of color and texture between stalks and surrounding foliage, and high stalk density within rows. Detecting stalks for grasping adds further challenges such as computation time and computing hardware. The proposed algorithm utilizes multiple stereo images taken from an unmanned ground vehicle and exploits geometric features specific to stalks such as vertical continuity, height, and the direction of surface normals. The described hardware and software pipeline is capable of detecting stalks for a manipulator grasp in approximately 12 seconds, and preliminary results show that hardware improvements can reduce detection time to 4 seconds. The algorithm is tested on 378 point clouds generated from 916 images of Sorghum bicolor, a grain crop. Performance is evaluated according to two methods, demonstrating precision greater than 93%.

Keywords: Autonomous Agents, Path Planning for Multiple Mobile Robots or Agents, Agricultural Automation
Abstract: In this work, we address the problem of centralized scheduling and motion planning for multiple grain carts with finite capacity that serve a group of combine harvesters engaged in a harvesting operation. First, we solve a facility location problem of finding the optimal depot position to minimize the distance traveled by the grain carts. Next, we formulate the scheduling problem as a constraint satisfaction problem. For a rectangular field, we present two schemes which allow the grain carts to unload all the combine harvesters without interrupting the harvesting activity. Finally, for both schemes, we present a relation between the physical parameters of the vehicles that need to be satisfied in order to meet the scheduling constraints.

Keywords: Computer Vision for Automation, Robotics in Agriculture and Forestry
Abstract: This work presents a rapidly deployable system for automated precision weeding with minimal human labeling time. This overcomes a limiting factor in robotic precision weeding related to the use of vision-based classification systems trained for species that may not be relevant to specific farms. We present a novel approach to overcome this problem by employing unsupervised weed scouting, weed-group labeling, and finally, weed classification that is trained on the labeled scouting data. This work demonstrates a novel labeling approach designed to maximize labeling accuracy whilst needing to label as few images as possible. The labeling approach is able to provide the best classification results of any of the examined exemplar-based labeling approaches whilst needing to label over seven times fewer images than full data labeling.

Keywords: Robotics in Agriculture and Forestry, Agricultural Automation, Computer Vision for Automation
Abstract: Automatically correlating plant observable characteristics to their underlying genetics will streamline selection methods in plant breeding. Measurement of plant observable characteristics is called phenotyping, and knowing plant phenotypes accurately and throughout a plant's growth is central to making breeding decisions. In-field plant phenotyping in an automated and noninvasive manner is hence crucial to accelerating plant breeding methods. However, most of the existing methods on plant phenotyping using visual imaging are confined to controlled greenhouse environments.

This paper presents an automated method of mapping 2D images collected in an outdoor sorghum field to segmented 3D plant units that are of interest for phenotyping. This method leverages multiple horizontal and vertical viewpoints while capturing 2D images from a robotic platform so as to generate in-field 3D reconstructions of the sorghum plant. We develop and quantitatively evaluate segmentation methods on these 3D reconstructions and also compare against reconstructions obtained from a controlled greenhouse environment. We present analysis that contrasts the role of purely local geometric features and the effect of addition of global context in both datasets. This work furthers capabilities of in-field phenotyping which paves the way forward for plant biologists to study the coupled effect of genetics and environment on improving crop yields.

Keywords: Robotics in Agriculture and Forestry, Agricultural Automation, Computer Vision for Other Robotic Applications
Abstract: Selective weeding is one of the key challenges in the field of agriculture robotics. To accomplish this task, a farm robot should be able to accurately detect plants and to distinguish them between crop and weeds. Most of the promising state-of-the-art approaches make use of appearance-based models trained on large annotated datasets. Unfortunately, creating large agricultural datasets with pixel-level annotations is an extremely time consuming task, actually penalizing the usage of data-driven techniques. In this paper, we face this problem by proposing a novel and effective approach that aims to dramatically minimize the human intervention needed to train the detection and classification algorithms. The idea is to procedurally generate large synthetic training datasets randomizing the key features of the target environment (i.e., crop and weed species, type of soil, light conditions). More specifically, by tuning these model parameters, and exploiting a few real-world textures, it is possible to render a large amount of realistic views of an artificial agricultural scenario with no effort.

The generated data can be directly used to train the model or to supplement real-world images. We validate the proposed methodology by using as testbed a modern deep learning based image segmentation architecture. We compare the classification results obtained using both real and synthetic images as training data. The reported results confirm the effectiveness and the potentiality of our approach.

Keywords: Hybrid Logical/Dynamical Planning and Verification, Optimization and Optimal Control, Motion and Path Planning
Abstract: We present a multi-query recovery policy for a hybrid system with goal limit cycle. The sample trajectories and the hybrid limit cycle of the dynamical system are stabilized using locally valid Time Varying LQR controller policies which probabilistically cover a bounded region of state space. The original LQR Tree algorithm builds such trees for non-linear static and non-hybrid systems like a pendulum or a cart-pole. We leverage the idea of LQR trees to plan with a continuous control set, unlike methods that rely on discretization like dynamic programming to plan for hybrid dynamical systems where it is hard to capture the exact event of discrete transition. We test the algorithm on a compass gait model by stabilizing a dynamic walking hybrid limit cycle with point foot contact from random initial conditions. We show results from the simulation where the system comes back to a stable behavior with initial position or velocity perturbation and noise.

Keywords: Motion and Path Planning, Multi-Robot Systems, Optimization and Optimal Control
Abstract: In search and surveillance applications in robotics, it is intuitive to spatially distribute robot trajectories with respect to the probability of locating targets in the domain. Ergodic coverage is one such approach to trajectory planning in which a robot is directed such that the percentage of time spent in a region is in proportion to the probability of locating targets in that region. In this work, we extend the ergodic coverage algorithm to robots operating in constrained environments and present a formulation that can capture sensor footprint and avoid obstacles and restricted areas in the domain. We demonstrate that our formulation easily extends to coordination of multiple robots equipped with different sensing capabilities to perform ergodic coverage of a domain.

Keywords: Motion and Path Planning, Task Planning, Planning, Scheduling and Coordination
Abstract: This paper presents a novel sampling-based motion planning method for shape inspection applications with a redundant robotic system. In this paper, a 7-Degree-of-Freedom (DOF) redundant robotic system consisting of a 6-DOF manipulator and a 1-DOF turntable is used for the industrial inspection problem. A Set Covering Problem (SCP) is formulated to select suitable viewpoints that satisfy the inspection requirements, and a Generalized Travelling Salesman Problem (GTSP) is formulated to determine both the robot poses and the visiting sequences. While previous studies solve the two problems separately, we formulate the SCP and GTSP problems as a combined sequencing SC-GTSP problem. A Random-Key Genetic Algorithm (RKGA) is then used to solve the combined SC-GTSP problem in a one-step optimization process. To validate the effectiveness of our method, we applied the proposed method to several motion planning cases. The results show that the proposed method outperforms the previous approaches by requiring up to 28.1% less total inspection time.

Keywords: Space Robotics and Automation, Motion Control, Legged Robots
Abstract: This paper presents the control strategies used to adapt the actively articulated suspension system of the rover SherpaTT to irregular terrain. Experimental validation of the approach with the physical system is conducted and presented. The coordinated control of the legs constituting the suspension system is encapsulated in a Ground Adaption Process (GAP) that operates independently from high level motion commands. The GAP makes use of force and orientation measurements to control the suspension system with 20 active degrees of freedom. The active suspension is used to achieve multi-objective terrain adaption encompassing (i) active force distribution at the wheel-ground contact points, (ii) keeping all wheels in permanent ground contact, and (iii) body orientation wrt gravity.

Keywords: Learning from Demonstration
Abstract: Movement prioritization is a common approach to combine controllers of different tasks for redundant robots, where each task is assigned a priority. The priorities of the tasks are often hand-tuned or the result of an optimization, but seldomly learned from data. This paper combines Bayesian task prioritization with probabilistic movement primitives to prioritize full motion sequences that are learned from demonstrations. Probabilistic movement primitives (ProMPs) can encode distributions of movements over full motion sequences and provide control laws to exactly follow these distributions. The probabilistic formulation allows for a natural application of Bayesian task prioritization. We extend the ProMP controllers with an additional feedback component that accounts inaccuracies in following the distribution and allows for a more robust prioritization of primitives. We demonstrate how the task priorities can be obtained from imitation learning and how different primitives can be combined to solve even unseen task-combinations. Due to the prioritization, our approach can efficiently learn a combination of tasks without requiring individual models per task combination. Further, our approach can adapt an existing primitive library by prioritizing additional controllers, for example, for implementing obstacle avoidance. Hence, the need of retraining the whole library is avoided in many cases. We evaluate our approach on reaching movements under constraints with redundant simulated planar robots and two physical robot platforms, humanoid robot ``iCub'' and a humanoid robot arm.

Keywords: Energy and Environment-Aware Automation, Motion and Path Planning, Multi-Robot Systems
Abstract: This paper presents a mission planning problem for a cooperative team of unmanned ground vehicles (UGVs), which includes multiple rovers and a solar-powered mobile charging station. The team is required to start at an initial point and visit a series of objective points before arriving at the final point selected from the set of objective points, where the UGVs will be charged from the solar-powered mobile charging station. This mission is represented as a multi-Hamiltonian Path Problem (mHPP). In order to effectively coordinate the team, an understanding of the mission environment is first obtained by generating a scalar field representation of the solar insolation of the environment from a visual-spectrum image. Then, a cascaded heuristic optimization algorithm, using modified genetic algorithm and particle swarm optimization, is used to generate a time-optimized mission plan for the team of UGVs, which guides each UGV to its assigned objective points and then rendezvous at the final charging location while guaranteeing compliance with the net energy gain constraint. The feasibility and efficiency of the proposed algorithm are verified using an experimental testbed and constructed indoor simulation environments.

Keywords: Humanoid Robots, Force Control, Legged Robots
Abstract: This paper explains low-latency joint torque feedback control based on torque estimation on each joint for gear-driven humanoid robots with harmonic drives. Force control of gear-driven robots has an advantage in its fully variable stiffness in comparison with elastic robots. However, feedback latency makes gear-driven robots vulnerable to impact rising in several milliseconds. It would be resolved by low-latency torque feedback loop in a single joint, but torque sensors are too large for life-sized humanoid robots. We estimate joint torque from motor current and rotation observed in each joint, and give artificial elasticity to joints using compliance control and shock absorption control. Our controller performance is demonstrated by landing experiments.

Keywords: Humanoid and Bipedal Locomotion, Reactive and Sensor-Based Planning, Humanoid Robots
Abstract: We consider a pursuit-evasion problem between humanoids in the presence of obstacles. In our scenario, the pursuer enters the safety area of the evader headed for collision, while the latter executes a fast evasive motion. Control schemes are designed for both the pursuer and the evader. They are structurally identical, although the objectives are different: the pursuer tries to align its direction of motion with the line-of-sight to the evader, whereas the evader tries to move in a direction orthogonal to the line-of-sight to the pursuer. At the core of the control architecture is a Model Predictive Control scheme for generating a stable gait. This allows for the inclusion of workspace obstacles, which we take into account at two levels: during the determination of the footsteps orientation and as an explicit MPC constraint. We illustrate the results with simulations on NAO humanoids.

Keywords: Humanoid and Bipedal Locomotion, Physical Human-Robot Interaction, Legged Robots
Abstract: This paper presents a control method for steering three dimensional (3D) dynamically walking bipeds that are engaged in cooperative tasks such as object transportation. Towards achieving safe interaction with a leading human (or robot) collaborator, the walking biped is required to exhibit compliance at the port of interaction, while simultaneously adapting its walking pattern in response to the perceived interaction forces. To address these issues, we propose a method that fuses impedance control of the biped's arm with position control of its legs in a way that the biped adaptively modifies its stepping pattern according to the collaborator's intentions. The method is applied on a 3D bipedal robot that is driven in the workspace by a collaborator, whose intention is communicated to the biped through the interaction force.

Keywords: Humanoid and Bipedal Locomotion, Humanoid Robots, Motion Control
Abstract: This paper addresses an omnidirectional locomotion control to make a biped robot walk back and forth, turn to any direction, move sideways and carry out them seamlessly. This is achieved by integrating smooth-path-tracking and lateral walking controllers that have been proposed by the authors, which is not straightforward because the path tracking is designed to converge to a referential path while the lateral walking aims to go away from the path. It can be avoided by extending the path-tracking controller to consecutively redraw the orbit that passes the middle of the feet. The proposed controller is designed completely in the robot-centric frame, namely, all the references and the manipulated variables are represented from the robot's viewpoint so that the operator can issue the referential values in the first-person view. Additionally, as the robot is ruled by neither a plan of footsteps nor time-dependent trajectories, it can flexibly respond to various perturbations. The validity of the proposed controller is investigated through computer simulations and an application to navigation of the robot is also shown.

Keywords: Humanoid Robots, Legged Robots, Sensor-based Control
Abstract: In this paper we introduce a biped foot placement controller based on capture point (CP) feedback control. This capture point feedback controller generates desired Zero Moment Point (ZMP) according to current capture point error. We call this desired ZMP as Control-ZMP (cZMP). Using this cZMP, we constructed (i) a disturbance adapting walking pattern generator, (ii) an ankle torque reference generator, (iii) a landing foot position adjustor and, (iv) a step time adjustor. By applying these controllers, the biped system becomes robust over uneven terrain and external pushing disturbances. We considered cZMP as a key indicator of bipedal walking stability. If the cZMP is within the support polygon, the robot will track the cZMP by using (i) and (ii). If the cZMP is outside of the support polygon, the robot will change footstep position and time by using (iii) and (iv) (COM pattern will be changed by (i) too). Capture point dynamics is described based on a linear inverted pendulum (LIP) model. The performance of the suggested control algorithm is validated in the Choreonoid simulator with a model of DRC-HUBO+ [11], [12].

Keywords: Passive Walking, Legged Robots, Dynamics
Abstract: This paper discusses the condition necessary for achieving stable passive-dynamic walking of a compass-like biped robot with semicircular feet on a slippery downhill, and analyzes the fundamental gait properties through numerical simulations. First, we introduce the semicircular-footed compass model and develop the equation of motion, holonomic constraint condition, and equation of collision considering the effect of impulsive frictional force. The paradox in transformation of the impulsive frictional force effect is also discussed. Second, we numerically show that short- and long-period sliding passive compass gaits can be generated under the same condition except the initial condition, and the sliding directions of the fore foot immediately after impact in the two gaits are different from each other. Furthermore, we conduct parametric studies to understand the change tendency of the fundamental gait properties with respect to the system parameters such as the foot radius and frictional coefficient. The simulation results show that the effect of semicircular feet makes it possible for the passive biped to walk on slippery downhill stably in the absence of the hip-damper.

Keywords: SLAM, Localization, Field Robots
Abstract: This paper presents an effective solution for 3D urban map generation using aerial images as prior information for Simultaneous Localization and Mapping (SLAM). The research targets urban canyons where Global Positioning System (GPS) signals are highly sporadic and erroneous. As a solution for urban canyon mapping, this study proposes aerial imagebased heading correction and kinematics considered odometry modeling. Aerial images often suffer from being significantly askew due to the height of structures and viewpoint changes, preventing direct integration with SLAM. However, we found that the direction of the structural edges were still valid, despite slanted building images. Making use of this property, this paper proposes a heading correction method using aerial images. As heading error is a critical factor in SLAM when mapping large areas, the proposed heading correction method substantially reduces estimation errors. Aiming for large urban area mapping, this paper also focuses on sensor modeling for odometry and GPS data. In odometry generation, mobile robot kinematics is considered by incorporating wheel diameter and base distance uncertainties into odometry covariance modeling. Results are presented from various types of urban areas over a path 28 km in length. For thorough validation, we ran the algorithm on three urban areas with differing degrees of GPS availability and structural complexity: a downtown area with medium complexity, a building complex, and the downtown area of a large city.

Keywords: Localization, Computer Vision for Other Robotic Applications, Sensor Fusion
Abstract: This paper proposes an image-based localization method that enables to estimate a bounded domain of the pose of an unmanned aerial vehicle (UAV) from uncertain measurements of known landmarks in the image. The approach computes a domain that should contain the actual robot pose, assuming bounded image measurement errors and landmark position uncertainty. It relies on interval analysis and constraint propagation techniques to rigorously back-propagate the errors through the non-linear observation model. Attitude information from onboard sensors is merged with image observations to reduce the pose uncertainty domain, along with prediction based on velocity measurements. As tracking landmarks in the image is prone to errors, the proposed method also enable fault detection from measurement inconsistencies. This method is tested using a quadcopter UAV with an onboard camera.

Keywords: Localization, Aerial Systems: Perception and Autonomy, Sensor Fusion
Abstract: This paper presents an algorithm for localizing an unmanned aerial vehicle (UAV) in GPS denied environments. Localization is performed with respect to a pre-built map of the environment represented using the distance function of a binary mosaic, avoiding the need for extraction and explicit matching of visual features. Edges extracted from images acquired by an onboard camera are projected to the map to compute an error metric that indicates the misalignment between the predicted and true pose of the UAV. A constrained extended Kalman filter (EKF) framework is used to generate an estimate of the full 6-DOF location of the UAV by enforcing the condition that the distance function values are zero when there is no misalignment. Use of an EKF also makes it possible to seamlessly incorporate information from any other system on the UAV, for example, from its auto-pilot, a height sensor or an optical flow sensor. Experiments using a hexarotor UAV both in a simulation environment and in the field are presented to demonstrate the effectiveness of the proposed algorithm.

Keywords: Multi-Robot Systems, Aerial Systems: Mechanics and Control, Cooperating Robots
Abstract: In this paper we propose an algorithm for stabilizing circular formations of fixed-wing UAVs with constant speeds. The algorithm is based on the idea of tracking circles with different radii in order to control the inter-vehicle phases with respect to a target circumference. We prove that the origin of the error dynamics of the inter-vehicle phases is exponentially stable. Furthermore, thanks to the stability properties of the guidance vector field that guides the vehicles, the algorithm can be extended to other closed trajectories. One of the main advantages of this approach is that the algorithm guarantees the confinement of the team in a specific area, even when communications or sensing among vehicles are lost. We show the effectiveness of the algorithm with an actual formation flight of three aircraft. The algorithm is ready to use for the general public in the open-source Paparazzi autopilot.

Keywords: Autonomous Vehicle Navigation, AI-Based Methods, Autonomous Agents
Abstract: Search and rescue missions and surveillance require finding targets in a large area. These tasks often use unmanned aerial vehicles (UAVs) with cameras to detect and move towards a target. However, common UAV approaches make two simplifying assumptions. First, they assume that observations made from different heights are deterministically correct. In practice, observations are noisy, with the noise increasing as the height used for observations increases. Second, they assume that a motion command executes correctly, which may not happen due to wind and other environmental factors. To address these, we propose a sequential algorithm that determines actions in real time based on observations, using partially observable Markov decision processes (POMDPs). Our formulation handles both observations and motion uncertainty and errors. We run offline simulations and learn a policy. This policy is run on a UAV to find the target efficiently. We employ a novel compact formulation to represent the coordinates of the drone relative to the target coordinates. Our POMDP policy finds the target up to 3.4 times faster when compared to a heuristic policy.

Keywords: Aerial Systems: Perception and Autonomy, Motion and Path Planning, Visual Tracking
Abstract: We propose a novel approach for safe tracking of a moving target in cluttered environments using a quadrotor. The key contribution of our work is a formulation that enables the generation of safe and dynamical feasible tracking trajectories that satisfy arbitrary relative motion patterns (circling, parallel tracking, undirectional tracking, etc.) with respect to the target. In our framework, forming the desired relative motion pattern between the quadrotor and the target only requires a generative function that specifies relative positions at different time instants. Our method generates samples to fit a piecewise-polynomial representation of the desired relative motion pattern and embeds it into a cost function for solving valid tracking trajectories via quadratic programming. Collision avoidance is achieved by squeezing the trajectory into a collision-free flight corridor, and dynamical feasibility is achieved by enforcing bounds on corresponding derivatives. Both of which can be written as linear constraints for the quadratic programming. Our approach is lightweight and can be implemented for real-time target tracking. We use a simulated cluttered environment and multiple desired relative motion patterns to demonstrate the performance of the proposed approach.

Keywords: Reactive and Sensor-Based Planning, Motion and Path Planning, Autonomous Vehicle Navigation
Abstract: We characterize and propose advances in the technique of Belief Roadmap Search (BRMS), the process of searching a roadmap in belief space for robot motion planning under localization uncertainty. We discuss the conditions required for optimal substructure in the single-source search of a roadmap in belief space, demonstrating that there are several desirable cost functions for which this property cannot be achieved. Practical performance issues of BRMS are discussed, including the implications of a commonly-used anti-cycling rule, and the computational complexity realized in practical applications of the technique. We propose a best-first implementation of BRMS, in contrast to the standard breadth-first implementation, which we show to improve the computational cost of search by up to 49% by eliminating unnecessary node expansions - the mechanics of both approaches are compared in detail. A variety of motion planning examples are explored.

Keywords: Formal Methods for Robotics, Reactive and Sensor-Based Planning, Task Planning
Abstract: There are many applications where robots have to operate in environments that other agents can change. In such cases, it is desirable for the robot to achieve a given highlevel task despite interference. Ideally, the robot must decide its next action as it observes the changes in the world, i.e. act reactively. In this paper, we consider a reactive planning problem for finite robotic tasks with resource constraints. The task is represented using a temporal logic for finite behaviors and the robot must achieve the task using limited resources under all possible finite sequences of moves of other agents. We present a formulation for this problem and an approach based on quantitative games. The efficacy of the approach is demonstrated through a manipulation case study.

Keywords: Robot Audition, Gesture, Posture, Social Spaces and Facial Expressions, Humanoid Robots
Abstract: We propose a probabilistic model that generates nod motions based on utterance categories estimated from the speech input. The model comprises two main blocks. In the first block, dialogue act-related categories are estimated from the input speech. Considering the correlations between dialogue acts and head motions, the utterances are classified into three categories having distinct nod distributions. Linguistic information extracted from the input speech is fed to a cluster of classifiers which are combined to estimate the utterance categories. In the second block, nod motion parameters are generated based on the categories estimated by the classifiers. The nod motion parameters are represented as probability distribution functions (PDFs) inferred from human motion data. By using speech energy features, the parameters are sampled from the PDFs belonging to the estimated categories. Subjective experiment results indicate that the motions generated by our proposed approach are considered more natural than those of a previous model using fixed nod shapes and hand-labeled utterance categories.

Keywords: Visual-Based Navigation, Deep Learning in Robotics and Automation, Autonomous Vehicle Navigation
Abstract: In this paper, we propose a system for autonomous vehicle following without a line of sight. From monocular camera images, the leading vehicle extracts scene descriptors which it transmits to the following vehicle by means of vehicle-to-vehicle (V2V) communication. The follower is able to recognize the scenes using its own camera and follow autonomously. A particle filter framework is employed for jump-free localization on the driven path of the leading vehicle. We compare the performance of different place features for accurate localization on a custom application-oriented dataset and evaluate methods to reduce the feature size for low-bandwidth V2V communication, while maintaining and even improving the recognition performance. Real-world results demonstrate the applicability of our system.

Keywords: Domestic Robots, Robot Companions
Abstract: We introduce RoomSeg, a novel method for segmenting occupancy grid maps into regions that represent rooms and corridors in the real world. The segmentation is utilized for systematic room-by-room cleaning on autonomous vacuum cleaners running in private homes. RoomSeg is based on automated clutter removal and watershed segmentation on grid maps. Segmented regions are merged into rooms by semantic decision rules. Presented experimental results clearly indicate the efficiency and accuracy of the approach when compared with state of the art methods. When deployed on cleaning robots, a substantial decrease in mission time can be achieved.

Keywords: Reactive and Sensor-Based Planning, Legged Robots, Underactuated Robots
Abstract: We propose a reactive locomotion strategy, called height reflex, that is useful to address big elevation changes in the terrain (e.g. when a quadruped robot has to step down from a high platform). In these cases the swing leg can lose mobility creating issues in the subsequent steps. The height reflex is a foot trajectory replanning strategy that redistributes the swing motion (in a smart way) to the stance legs to ”lower” the whole trunk and to aid the foothold searching motion. To spread the motion we exploit a massless link model of the robot with virtual dampers at the joints, which is used to replan the feet trajectories. The proposed approach is able to incorporate kinematic limits, it is easy-to-tune, computationally efficient and suitable for real-time implementations. The reflex is implemented and experimentally evaluated on the 80 kg hydraulic quadruped HyQ. With our approach we were able to address high steps, up to 24 cm which is 30% of HyQ leg length and 53% of its retractable leg range.

Keywords: Redundant Robots, Parallel Robots, Kinematics
Abstract: We consider a class of robotic systems composed of high elongation linear actuators connected at universal joints. We derive the differential kinematics of such robots, and formalize concepts of controllability based on graph rigidity. Control methods are then developed for two separate applications: locomotion and shape morphing. The control algorithm in both cases solves a series of linearly constrained quadratic programs at each time step to minimize an objective function while ensuring physical feasibility. We present simulation results for locomotion along a prescribed path, and morphing to a target shape.

Keywords: Redundant Robots, Physical Human-Robot Interaction, Motion and Path Planning
Abstract: Kinematic redundancy is a characteristic and beneficial property in collaborative robots nowadays as it enhances the flexibility and dexterity of the system. While the robot is manipulating an object, it is often necessary to kinematically reconfigure the robot, for example, when it obstructs the human. For this, internal or so-called null space motions can be carried out which do not affect the main task. In general, it is desirable that the human coworker can anticipate how the robot will move at any time. However, for null space motions this is typically not the case as they are non-intuitive and not suitable for interaction. In this work, we develop intuitive null space interaction behaviors for redundant manipulators, where the human can easily guide the robot. We want to provide users with a tool, that is straightforward to implement and solves real-world problems effectively. Two practical applications for an eight- and ten-DOF robot demonstrate the performance of the proposed method.

Keywords: Redundant Robots, Parallel Robots, Flexible Robots
Abstract: Precision is an important feature in modern robotics and manipulation technology. Elevation of the robot's end-effector precision depends on different features as stiffness, tolerance etc. which should be considered during the synthesis and design phase of the manipulator. The focus of this study is on the structural stiffness of parallel kinematic manipulators (PKM) exhibiting actuation redundancy. It is determined how the stiffness depends on physical and geometrical characteristics of the manipulator and how it is possible to take advantage of redundancy to enhance the stiffness of the manipulator. It is shown that redundantly actuated PKM (RA-PKM) with components made of softer and accordingly lighter materials demonstrate comparable stiffness to the non-redundant manipulators which consequently decrease the required energy for moving the dead-load in operation. Based on the model built in this study, optimizations are conducted and manipulators with optimal morphologies for different tasks are introduced. As a case study, RA-PKM of type n-RRPaRs are considered.

Keywords: Simulation and Animation, Formal Methods for Robotics, Dynamics
Abstract: We propose a novel integrator for implementing bilateral constraints in multibody simulation using variational integrator methods. We first construct a variational penalty method, which is used to enforce a constraint. The penalty term is simulated using an asynchronous variational integrator, allowing the penalty part of the system to be simulated using a smaller time step. We compute the Discrete Euler-Lagrange (DEL) equations for an equivalent penalty term with a larger time step and then use this rescaled system in the aforementioned variational penalty method, thereby enforcing the constraints. This enables us to incorporate some of the behavior of a very stiff system, which would only be stable on the small time scale, into the system on the large time scale. The effect is better adherence to the constraints, at a larger time step. We demonstrate the method with a simulation of a chain of rigid bodies. We then discuss the potential applications of the integrator and highlight how the work can be used to better interpret the tuned values of the coefficients used in penalty formulations.

Keywords: Robot Safety, Physical Human-Robot Interaction, Redundant Robots
Abstract: Ensuring safety is a primary goal in physical human-robot interaction. In various collision experiments it was found that the robot's effective mass, velocity, and geometry are the key parameters which influence the human injury severity during an impact. Recently, a velocity controller was proposed that limits the robot speed to a biomechanically safe value, taking into account the mass and the curvature in the direction of movement for a given point of interest. The mass and the geometry depend on the mechanical design, however, the effective mass also depends on the robot configuration. In this paper, we exploit the redundant degree(s) of freedom of a joint torque controlled seven- and eight-DOF robot to minimize the effective mass without affecting the desired Cartesian end-effector trajectory and with the goal to improve the performance of the safe velocity controller at the same time. Given recent results in robotics injury analysis, we analyze when such a redundancy resolution scheme actually improves safety. For the considered robots, we find reflected mass extrema that can be obtained by null space motions, and propose a real-time, torque-based redundancy resolution scheme, which is finally verified in experiments.

Keywords: Legged Robots, Search and Rescue Robots, Robotics in Hazardous Fields
Abstract: This paper describes a novel four-limbed robot having high mobility capability in extreme environments. At disaster sites, there are various types of environments where a robot must move such as rough terrain with possibility of collapse, narrow places, stairs, vertical ladders and so forth. In this paper, first we categorized extreme environments based on three indexes: unevenness, narrowness, and inclination. To move in such extreme environments, we proposed a four-limbed robot having various locomotion styles such as bipedal/quadrupedal walking, crawling and ladder climbing. The main contribution of this paper is the concept and hardware design of the four-limbed robot. We developed a prototype of the four-limbed robot having commonly structured limbs. The number of DoF for the whole body is 29, with 7-DoFs in each limb and 1-DoF in the trunk. The robot weight is 110 kg, and the height is 1,290 mm when standing on two legs. The end-effector has hook-like shape. Verification of the prototype robot is conducted through simulations and experiments.

Keywords: Failure Detection and Recovery, Learning and Adaptive Systems, Service Robots
Abstract: Activities of daily living (ADLs) are important for quality of life. Robotic assistance offers the opportunity for people with disabilities to perform ADLs on their own. However, when a complex semi-autonomous system provides real-world assistance, occasional anomalies are likely to occur. Robots that can detect, classify and respond appropriately to common anomalies have the potential to provide more effective and safer assistance. We introduce a multimodal execution monitor to detect and classify anomalous executions when robots operate near humans. Our system builds on our past work on multimodal anomaly detection. Our new monitor classifies the type and cause of common anomalies using an artificial neural network. We implemented and evaluated our execution monitor in the context of robot-assisted feeding with a general-purpose mobile manipulator. In our evaluations, our monitor outperformed baseline methods from the literature. It succeeded in detecting 12 common anomalies from 8 able-bodied participants with 83% accuracy and classifying the types and causes of the detected anomalies with 90% and 81% accuracies, respectively. We then performed an in-home evaluation with Henry Evans, a person with severe quadriplegia. With our system, Henry successfully fed himself while the monitor detected, classified the types, and classified the causes of anomalies with 86%, 90%, and 54% accuracy, respectively.

Keywords: Tendon/Wire Mechanism, Failure Detection and Recovery, Motion Control
Abstract: Tendons in tendon-driven mechanisms are relatively weak components and sometimes break. This paper proposes a detection method of tendon breakage and a switching control system for tendon-driven mechanisms with redundant tendons. The detection method immediately detects the tendon breakage from the kinematic information of the tendons. The switching control system compensates the control performance by only using the remaining tendons. We experimentally confirm that virtual tendon stretch in a breakage case is adequently larger than that in a normal case to detect the breakage from the kinematic information and the switching controller improves the tracking performance when a tendon is broken.

Keywords: Redundant Robots, Kinematics, Failure Detection and Recovery
Abstract: A measure of local fault tolerance for kinemati- cally redundant robots has previously been defined based on the properties of the singular values of the Jacobian matrix. Based on these measures, one can determine a Jacobian that is optimal. Because these measures are solely based on the singular values of the Jacobian, permutation of the columns does not affect the optimality. Therefore, when one generates a kinematic robot design from this optimal Jacobian, there will be 7! robot designs with the same locally optimal fault tolerant property. The work described here shows how to analyze and organize the kinematic structure of these 7! designs in terms of their Denvavit and Hartenberg (DH) parameters. Furthermore, global fault tolerant measures are defined in order to evaluate the different designs. It is shown that robot designs that are very similar in terms of DH parameters, e.g., robots generated from Jacobians where the columns are in reverse order, can have very different global properties. Finally, a computationally efficient approach to calculate the global pre- and post-failure dexterity measures is presented and used to identify two Pareto optimal robot designs. The workspaces for these optimal designs are also shown.

Keywords: Assembly, Failure Detection and Recovery, Recognition
Abstract: Robotic failure is all too common in unstructured robot tasks. Despite well-designed controllers, robots often fail due to unexpected events. Robots under a sense-plan-act paradigm do not have an additional loop to check their actions. In this work, we present a principled methodology to bootstrap online robot introspection for contact tasks. In effect, we seek to enable the robot to recognize and expect its behavior, else detect anomalies. We postulated that noisy wrench data inherently contains patterns that can be effectively represented by a vocabulary. The vocabulary is obtained by segmenting and encoding data. And when wrench information represents a sequence of sub-tasks, the vocabulary represents a set of words or sentence and provides a unique identifier. The grammar, which can also include unexpected events, was classified both offline and online for simulated and real robot experiments. Multi-class Support Vector Machines (SVMs) were used offline, while online probabilistic SVMs were used to give temporal confidence to the introspection result. Our work's contribution is the presentation of a generalizable online semantic scheme that enables a robot to understand its high-level state whether nominal or anomalous. It is shown to work in offline and online scenarios for a particularly challenging contact task: snap assemblies. We perform the snap assembly in one-arm simulated and real one-arm experiments and a simulated two-arm experiment. The data set itself is also fully available online and provides a valuable resource by itself for this type of contact task. Our verification mechanism can be used by high-level planners or reasoning systems to enable intelligent failure recovery or determine the next most optimal manipulation skill to be used. Supplemental information, code, data, and other supporting documentation can be found at [1].

Keywords: Failure Detection and Recovery, Computer Vision for Other Robotic Applications
Abstract: The visual difference between outcomes in many robotics tasks is often subtle, such as the tip of a screw being near a hole versus in the hole. Furthermore, these small differences are often only observable from certain viewpoints or may even require information from multiple viewpoints to fully verify. We introduce and compare three approaches to selecting viewpoints for verifying successful execution of tasks: (1) a random forest-based method that discovers highly informative fine-grained visual features, (2) SVM models trained on features extracted from pre-trained convolutional neural networks, and (3) an active, hybrid approach that uses the above methods for two-stage multi-viewpoint classification. These approaches are experimentally validated on an IKEA furniture assembly task and a quadrotor surveillance domain.

Keywords: Failure Detection and Recovery, Learning and Adaptive Systems, Planning, Scheduling and Coordination
Abstract: Modern robotic systems often consist of a growing set of information-producing components that need to be appropriately connected for the system to function properly. This is commonly done manually or through relatively simple scripts by specifying explicitly which components to connect. However, this process is cumbersome and error-prone, does not scale well as more components are introduced, and lacks flexibility and robustness at run-time. This paper presents an algorithm for setting up and maintaining implicit subscriptions to information through its semantics rather than its source, which we call semantic subscriptions. The proposed algorithm automatically reconfigures the system when necessary in response to changes at run-time, making the semantic subscriptions adaptive to changing circumstances. To illustrate the effectiveness of adaptive semantic subscriptions, we present a case study with two SoftBank Robotics NAO robots for handling the cases when a component stops working and when new components, in this case a second robot, become available. The solution has been implemented as part of a stream reasoning framework integrated with the Robot Operating System (ROS).

Keywords: Control Architectures and Programming, Collision Avoidance, Autonomous Vehicle Navigation
Abstract: We introduce Evo-ROS, a framework combining evolutionary search and ROS/Gazebo-based simulation. The framework enables researchers and developers to take advantage of evolutionary search during design. Evolutionary algorithms can be applied to many aspects of robot development, including optimal configuration and placement of sensors and actuators, generation of compensatory behavior in case of failed/faulty components, and detection of “unlikely-but-possible” situations that might cause system failure.

Keywords: Semantic Scene Understanding, Cognitive Human-Robot Interaction, Marine Robotics
Abstract: Many applications of robots in space and underwater exploration can benefit from real-time human telepresence. Such environments, however, impose drastic constraints on data transmission, limiting the utility of both the human and the robot. Semantic maps can provide useful context to an operator, but previous methods in semantic mapping rely on supervised learning methods that require a fixed and known set of classes. We make use of topic modeling to learn a spatial-semantic map in an unsupervised way, transmitting only anomalous images and representative topic images. This allows a human operator to interpret the learned topics, and therefore map content, without requiring a trained model, so that the operator may aid in decision-making or exploration.

Keywords: Distributed Robot Systems, Object detection, segmentation, categorization, SLAM
Abstract: Since its emergence in 2010, cloud robotics has been a major trend in today’s robotics, however, there are some real-time robotics applications for mobile robots that are not able to use this paradigm due to their sensitive latency requirements. In this paper, hybrid cloud robotics is proposed as a processing model utilizing both edge and cloud computing in robotics. This model is implemented for two computation intensive algorithms, object recognition and SLAM, for a robot with a light single-board computer on-board.

Keywords: AI-Based Methods, Kinematics
Abstract: Symbolic inverse kinematics analysis solves the problem of how to control the robot joints to achieve desired end effector location. It is a non-trivial task critical for any robot design. While there are many numerical packages for inverse kinematics, they share common flaws including, finding only one of the multiple solutions, requirement of a starting value, dependence on the starting value, and problems with convergence near singular configurations. Up till now, closed-form symbolic inverse kinematics analysis was generally handled by human experts, because of the high-level mathematical reasoning needed.

Our major contributions to automate closed-form kinematics solving are:

1. We built an expert system -called "IKBT"- that solves inverse kinematics automatically without human supervision or input, using Behavior Tree, a control frame work used in game AI and robotics control.

2. We incorporated knowledge that frequently used (by human experts) when solving inverse kinematics into leaf nodes of the behavior tree. These knowledge nodes are independent of degree-of-freedoms (DOFs), applicable to any robot.

3. Our expert system can solve complicated robots, such as 6-DOF commercial robot manipulator PUMA. IKBT achieved a general success rate of 80% on all 20+ test robots.

4. On average, IKBT generates solutions for one robot in a few seconds - the same work load usually takes human experts 30 minutes to hours to solve.

5. IKBT generates a dependency graph of joint variables after solving, includes all possible solutions. Contrary to the common belief, we discovered that solution tree is often insufficient to represent the intricate dependencies among joint variables - a graph is more suitable to fulfill this role.

6. IKBT has several defining advantages including applicability to any robot, extensible toolbox, up-to-date and easy to implement language (Python), and limited dependency limited only to a few libraries. These characteristics will spur the wild adaption of IKBT into robotics research community, as well as initiate changes in curriculum design.

Keywords: Aerial Systems: Mechanics and Control, Force Control, Aerial Systems: Applications
Abstract: This work proposes a feedback controller for regulating the thrust produced by Brushless Direct Control (BLDC) motors using feedback from a load-cell attached to the base of the motor. The direct control of mechanical thrust provides stable control under external disturbances from wind and objects in the surroundings. Experiments have shown that the controller is able to track reference thrust trajectories accurately. This controller can also be used with different motors, propellers and battery voltages without additional calibration.

Keywords: Robotics in Construction, Big Data in Robotics and Automation, Field Robots
Abstract: Industrial rework is a huge problem in large scale capital projects, as it is a waste of time, money and resources. Veerum’s mission is to eliminate rework using autonomous robots and AI technology to create a “Digital Twin” of the project. This can be used to keep a project on schedule and on budget, by identifying and resolving issues during design, fabrication, logistics, and construction phases in capital projects. Veerum’s long term vision is an autonomous construction procedure with an AI deploying & managing robot to build the world’s critical infrastructure.

Keywords: Collision Avoidance, Reactive and Sensor-Based Planning, Robot Safety
Abstract: Viability theory provides a set of concepts and algorithms to study uncertain continuous dynamic systems under viability (or state) constraints. Interval computation is about guaranteed numerical methods for approximating sets. The main tool to be used is based on the idea of enclosing real numbers in intervals and real vectors in boxes. Refined interval techniques as contractor programming and guaranteed integration allow to implement the viability kernel and the capture basin algorithms. Results are provided considering the kinematics of the game of two cars. Viability kernel and capture basin algorithms are used to compute backward reachable sets which are differential game capture zones assuming predefined maximum time horizons. Then, collision avoidance between two non cooperative ground mobile robots is performed based on the backward reachable sets mentioned above. Keywords: Non Cooperative Differential Games, Pursuit-Evasion Games, Backward Reachable Sets, Level Sets of the Value Function, Viability Theory, Viability Kernel, Capture Basin, Interval Computing, Contractor Programming, Collision Avoidance, Game of Two Identical Cars, Robotics.

Keywords: Distributed Robot Systems, Sensor Fusion, Deep Learning in Robotics and Automation
Abstract: Abstract— It is no secret that robotic systems are expanding into many human roles or to augmentation of human roles. The Robot Operating System (ROS) is an open source standard for the robotic industry, that enables locomotion, manipulation, navigation, and recognition tasks by integrating sensors, motors and controllers into reusable modules over a distributed messaging architecture. As reliance on robotic systems increases, these systems become high value targets, for example in autonomous vehicles where human life is at risk. The original ROS implementations were not designed to mitigate the security risks associated with hostile actors. ROS 2, the next generation of the Robot Operating System, addresses this shortcoming, leveraging Data Distributed Services (DDS) for its messaging architecture and DDS security extension for its data protection in motion. This paper provides a systematic review of ROS 2 and identifies potential risks for this new robotic system paradigm. A ROS 2 robotic system is layered from the hardware which includes sensors, motors and controllers to the software layers which includes the Operating System (OS), security services, protocols, messaging and the cognitive layer for observation, learning and action. Since ROS 2 and security are new considerations for robotics systems as they move from the laboratory into the mainstream, many questions emerge regarding what needs to be secure vs non secure, and is this an all or nothing model. Another concern is the performance vs security models, since latency and throughput are critical to real-time robotic functionality. The ROS 2 security model protects a number of the robotic layers, but concerns are identified when compared to a holistic security model.

Keywords: Mapping, RGB-D Perception, Range Sensing
Abstract: We propose a novel registration method of 3D point cloud maps. Navigation based on 3D point cloud maps is the one of the most important technologies in unmanned vehicle field. To combine and extend the 3D maps, ICP method has been used widely, but this method takes long computation time when aligning large scale maps which include a lot of points. Therefore, a point cloud map registration method which precisely register two maps using low-cost cameras is proposed. By combining odometry information derived from the mapping procedure and the 3D position of the image feature points, location of the coexisting places in both maps are extracted. Then, the transform between the origin of each map can be estimated.

Keywords: Soft Material Robotics, Wheeled Robots, Mechanism Design
Abstract: In this paper, we propose a novel design for a variable-diameter wheel using an origami-based soft robotics design approach.

Keywords: Computer Vision for Other Robotic Applications, Field Robots
Abstract: Limp darkening is an optical phenomenon that causes the center of a celestial star like the sun to appear brighter than the overall circular disk. This leads to, so called, the black sun effect where the CMOS pixel imaging the sun center becomes oversaturated and darkened instead due to the electron overspill under saturation. This paper presents a novel sun sensor extracting the sun vector accurately and robustly based on a micro size, 1mmx1mmx1.74mm, of CMOS camera. The black sun effect allows the sun center to be accurately extracted even under the sun image appearing irregular and noisy due to glares. The sun center is localized precisely by the iterative extraction of corners within the sun glare while removing out any spurious corners. The proposed sun sensor is implemented in the following two application modes: 1) a stationary mode targeted for tracking the sun for heliostats or solar panels with a single image sensor configuration and 2) a non-stationary mode targeted for determining the orientation of space rovers with six sun sensors configured in the icosahedron geometry of 23mmx23mmx12mm in size. For the stationary mode, we obtained the accuracy of 0.1° by combining the sun sensor measurement with the sun vector prediction in a Kalman filter framework. For the non-stationary mode, we obtained the accuracy of 0.3° by fusing the measurements from three sun sensors available at any instant of time. Experiments indicate that the sun vector extraction takes an average of 85msec. This paper shows that a compact size, highly cost-effective, yet highly accurate and robust sun sensor can be made available for various applications.

Keywords: Soft Material Robotics, Gripper and Other End-Effectors, Grasping
Abstract: Locking two surfaces with minimum normal force may result in safe grasping of objects in robotic hands. This paper presents a preliminary approach on design and analysis of a bio-inspired soft pad that enhances the adhesion with the environment by morphological optimization of its surface at micro-scale. The design principle is originated from the biological wet attachment of a tree-frog toes and surrounding environment caused by capillary force and surface tension of a secretion film between the toe and the surface. We fabricated soft pad that was deposited with a similar morphological network by molding process on a silicon substrate with etching technique. With wet adhesion was considered in robotic grasping, this research is expected to be applied in wet and high-moisture environment.

Keywords: Deep Learning in Robotics and Automation, Motion and Path Planning
Abstract: A planning task can be thought of as a loop process that contains a two-step problem. In the first step, the agent perceives its environment. In the second step, the agent executes an action based on the perceived environment. Such emph{perception-action} loops can be successfully learned by using tools from the area of deep learning

However, in planning problems, the optimal actions are sequential in nature. Purely convolutional architectures (CNNs) perform poorly when applied to planning problems due to the reactive nature of the policies learned by them. The complexity of this problem is compounded when the environment is only partially observable as is the case with most real world tasks. It has been shown that when learning how to plan in such environments, it becomes necessary to use memory to retain information about states visited. Using recurrent networks to store past information and learn optimal control has been explored before. Recent advances in memory augmented networks have shown that it is beneficial to use external memory with write operators that can be learned by a neural network over. We are specifically interested in the differentiable neural computer architecture. Using this as motivation we split our planning problem into two levels. At a lower level, we use an approximation of the value iteration algorithm within a convolutional networks to learn optimal plans on the partially observed environments. At a higher level, another network controller interfaced with an external memory takes in these locally optimal plans and a sparse representation of the environment as its input and produces plans that are optimal globally. We show impressive results with this network when trained with supervised labels on grid worlds. We also test our hypothesis that the network learns the structure of the environment by testing it on maps with cul-de-sacs or tunnels with a dead end where the agent must explore it all the way to the end when it sees the dead end and retraces its steps.

Generating supervised labels is not always easy and are often not consistent (two pilots flying the same course with a drone can have vastly different control policies). Reinforcement learning can be a good alternative to allowing the agent to learn the control policy on its own. This does not work in environments that have a sparse reward structure. In reinforcement learning the agents uses the reward signal from the environment to update its policy. Environments where the reward is only received when the agent reaches the goal could be several time steps away. To solve this we introduce deep reward shaping that gives the agents small rewards when its uncertainty is high forcing it to explore the environment. By adding this exploration module, we show our agent is able to learn how to navigate simple 2D grid worlds, graphs as well as complex 3D environments with sparse rewards.

Keywords: Deep Learning in Robotics and Automation, Robot Audition, AI-Based Methods
Abstract: For restoration or regression tasks, deep learning is helpful, and thus, we propose a restoration method based on LSTM (long-short term memory), which can consider a temporal sequence of acoustic signals.

Keywords: Prosthetics and Exoskeletons, Force and Tactile Sensing, AI-Based Methods
Abstract: We present a large area tactile sensor module that integrates sensors with processing circuits. This system comprises a piezoresistive fabric material as the sensor that emulates the skin, an interface that produces spike patterns that emulate the neural signals from the mechanoreceptors in the skin, and an extreme learning Machine (ELM) chip for computing and pattern recognition. We demonstrate recognition capability of our system in several tactile pattern recognition tasks, including object recognition and texture categorization. Benefiting from intrinsic advantages of biologically-inspired event-driven systems and massively parallel processing capabilities of the ELM chip, the proposed architecture offers a fast and energy-efficient alternative for processing tactile information. Moreover, it provides the opportunity for development of low-cost tactile modules for prosthetic and robotic applications.

Keywords: Distributed Robot Systems, Big Data in Robotics and Automation, Software, Middleware and Programming Environments
Abstract: We introduce a novel, decentralized architecture facilitating consensual, blockchain-secured computation and verification of data / knowledge. Through the integration of (i) a decentralized content-addressable storage system, (ii) a decentralized communication and time stamping server, and (iii) a decentralized computation module, it enables a scalable, transparent, and semantically interoperable cloud robotics ecosystem, capable of powering the emerging internet of robots.

Keywords: Manipulation Planning, Learning and Adaptive Systems, Grasping
Abstract: Researches in cognitive science reveal that human central nervous system internally simulates dynamic behavior of the motor system using internal models. Being inspired, in this research, how to better develop the internal models for robot arm motion control in a human-inspired manner is discussed. Developing the internal models is essentially establishing mappings between motor command and body state, where the mapping from motor onto state is called forward model (FM) and the reverse one is named inverse model (IM). Considering that human proprioception plays an important role for the development of internal models, we propose to use autoencoder neural network to establish the proprioception of robots, and the internal models are then learned based on the proprioception. To evaluate the proprioception based internal models, two types of FM and three different structures of IM are compared respectively. Acting like human, all the models are trained in a developmental manner along with the self-produced arm babbling (unconscious arm movements). Experimental results indicate that borrowing mechanism from human proprioception does help to improve the performance of both FM and IM. Additionally, an integrated framework involving the FM and IM is further proposed for several arm motion control tasks, i.e. reaching, grasping and placing, and the results on our PKU-HR6.0 humanoid robot consistently demonstrate the effectiveness.

Keywords: Mechanism Design, Prosthetics and Exoskeletons, Rehabilitation Robotics
Abstract: The current generation of commercially available exoskeletons used in lower limb rehabilitation or as walking assistants provide power at both hips and knees, although not at the ankles. Due to unactuated ankles, existing exoskeletons produce a compensated walking pattern, limiting their walking speeds and efficiency of gait. This work presents the design of a reconfigurable modular actuator design, where the same mechanism is used to efficiently power the hip, knee, and ankle in the sagittal plane. A novel ball screw support assembly was designed allowing an in-line motor to be used in a compact and robust design.

To reduce hardware cost, the actuator design is modular, so that the same assembly can be reconfigured to the hip, knee, and ankle. Although biomechanical requirements while walking (speed & torque) vary considerably between each lower-limb joint, the reconfigurable design takes advantage of the variance in the range of motions to provide for more speed or torque as appropriate. This is achieved by changing the anchor point of the driving link in our offset slider-crank mechanism.

The linear actuation is achieved with a ball screw, and is directly coupled to a 100W brushless DC motor. Managing the axial load generated in a compact and lightweight manner required a novel custom solution.

A compact ball screw linear assembly was achieved through the design of a novel mechanical coupling between the motor shaft and ball screw. This allows for placing angular contact bearings on the outer diameter of the coupling, thereby making the total length of the linear assembly shorter while able to withstand high loads.

A single joint prototype has been fabricated and tested. The full assembly is currently under fabrication. This lower-limb exoskeleton will be used to test a self-aligning joint length adjustment method, and investigate novel control strategies on this six active degrees of freedom system.

Keywords: Mining Robotics, Field Robots, Robotics in Hazardous Fields
Abstract: The development of mining technologies has grown significantly in recent years. For mining companies, the adoption of new technologies on automation and remote operation represents a significant opportunity for gains in productivity and safety. This paper describes the development of an integrated solution for autonomous explosive loading applied to mining processes. This method can reduce or eliminate the risks to which operators are exposed in environments such as rock blast areas. The proposed solution was validated with simulated and real tests. Tests were carried out in a laboratory, using a mobile platform with a robotic arm.

Keywords: Tendon/Wire Mechanisms, Underactuated Robots, Multifingered Hands
Abstract: Multiple grasping applications based on the Hardware Embedded Reduced Intricacy (HERI) Hand [1], which demonstrates balanced dexterous finger manipulation as well as powerful grasping of common objects, will be presented in this track.

Keywords: Wearable Robots, Human-Centered Robotics, Virtual Reality and Interfaces
Abstract: The "Third Arm", a new type of wearable robot arm is proposed to help people who wants to perform two concurrent tasks in daily life. To be useful in various, complex, and unpredictable tasks such as occurred in housework at homes or displaying goods at shops, the Third Arm should be not only controlled autonomously by its artificial intelligence, but also operative by humans voluntarily and intuitively. However, it is difficult to operate the Third Arm because humans could not use their both hands and arms for operation under dual-task situations. Therefore, we proposed a "face vector" interface as the original solution for operating the Third Arm. A face vector is a modality to instruct the target object voluntarily and intuitively. The three-dimensional coordinates of target object from humans could be get by measuring the direction of a human’s face and the distance between the object surface and a human’s face, using gyro sensors and distance sensors. Combining the object instruction by a face vector with the motion instruction by a voice command or machine learning systems, people can easily operate the Third Arm without using their hands and arms, and their eyes can be free to move between the main work and the other work. However, it is considered that there is the error distance of object instruction because a face vector is invisible. In addition, it is considered that the efficiency of a main task is affected by the cognitive load of a face vector interface because humans must pay attention for two different tasks simultaneously. Therefore, we evaluate the error distance and cognitive load of a proposed face vector interface for voluntariness and intuitiveness. The experiment was conducted in the environment of a real world and a virtual reality. To easy recognize of a point that humans are now instructing by their face vector, we made a visual biofeedback of a face vector using laser pointer. To build a dual-task situation, the wooden blocks discrimination task by touching their shapes as a main task, and pointing at objects by their face vector as a sub task. The required time of main tasks and the error distance of object instructions in sub tasks are recorded in conditions of with and without a visual biofeedback and single-task and dual-task situations. The results show that the error distance could be significantly decreased by showing a visual biofeedback in both single-task and dual-task situations. On the other hand, the results also show that the efficiency of a main work could be significantly decreased only when humans are showed a visual biofeedback. These results indicate that a visual biofeedback function could increase the accuracy of object instructions but decrease the work efficiency of dual-task situations. Therefore, a face vector could be useful as the interface of voluntary and intuitive object instructions for the Third Arm if a visual biofeedback function could be adjustable depending on users or situations.

Keywords: Aerial Systems: Applications, Autonomous Vehicle Navigation, Mapping
Abstract: In this proposal, we deal with the problem of generating 3D magnetic maps using cooperative and autonomous aerial robots. We propose a novel approach for autonomous mapping using a custom fluxgate magnetometer and multiple commercial cameras embedded on the aerial platforms. We also present a novel method for generating a Digital Magnetic Elevation Model (DEM) from the gathered data. Our method was validated in simulation and real experiments. Results have shown high accuracy in the measurements generated from the DEM model and an accurate estimated magnetic representation of metal objects placed in a test scenario.

Keywords: Agricultural Automation, Autonomous Vehicle Navigation, Optimization and Optimal Control
Abstract: This paper presents the development, application, and experimental results of a real-time receding horizon estimation and control (RHEC) framework applied to the 3D printed field robot to increase its steering accuracy.

Keywords: Prosthetics and Exoskeletons, Legged Robots, Rehabilitation Robotics
Abstract: In this work, we show that a control paradigm based on a special combination of holonomic and nonholonomic virtual constraints results in stable walking with an active transfemoral prosthesis and enables the subject to comfortably manage rapid changes in speed and other walking attributes.

Keywords: Marine Robotics, Biologically-Inspired Robots
Abstract: The long-term goal of this project is to provide a complete, laboratory-scale testbed that significantly lowers the barrier to entry for this area of research, which would serve as a stepping-stone for real world implementations. This testbed will consist of a robot that has all the required sensing, communication, and locomotion capabilities to operate in a collective, but is low-cost and small enough that large-scale collectives can be achieved with only modest robotic and infrastructure cost. The current research phase focuses on the physical design and locomotion system of such a robot, although some sensing capabilities have been added to validate the locomotion capabilities. The robot is 100 mm in the longest dimension and has a component cost of 100. The robot is highly maneuverable in 3D, being capable of forward and backward swimming, in-place rotation, and vertical ascent and descent.

Keywords: Aerial Systems: Applications, Networked Robots, Learning and Adaptive Systems
Abstract: This poster focuses on the use of Gaussian Process (GP) to predict air-to-ground communication strength for UAV relay in urban environments. In particular, Gaussian Process is used to learn inaccuracies in communication model and effects of line of sight obstructions for support of a trajectory planner. Two implementations are discussed. First where UAV performs back and forth search pattern to learn communication model which is then used for trajectory planning. Second where data collection and trajectory planner is run simultaneously. It is shown that Second implementation benefits from being able to start relay mission immediately, while second offers higher communication improvements.

Keywords: Biologically-Inspired Robots, Motion and Path Planning, Legged Robots
Abstract: Recreating complex, coordinated movements exhibited by humans is not a well-understood process. There are methods present in the literature for coordinated movement based on the relative distance between multiple limbs. A method is presented for generating different trajectories through configuration space between an initial and a final pose. This method is based on a concept from Laban/Bartenieff Movement Studies that describes large, body-level coordinated movement strategies, called Patterns of Body Organization. This set of patterns is used as a basis of human movements, meaning that a different pattern choice between two fixed poses gives different movements. An example pattern in humans, Upper-Lower, is used when diving in a pool where the swimmer orients its limbs to make two distinct limb groups divided at the waist and coordinates them for a successful dive.

Keywords: Social Human-Robot Interaction, Aerial Systems: Applications
Abstract: Nowadays, it can be observed an extensive progress in the field of human robot interaction. Both aerial and ground robots are working on a large range of applications, for instances, human robot assistance, robot acting as a companion, or ageing care. Despite this progress, the interaction between autonomous aerial robots and non-trained humans is one of the most difficult challenges which researchers are starting to investigate.

Thanks to modern systems of perception and control, the research on aerial robots has also grown substantially. This research paves the way for a near future wherein aerial and ground robots interact with people in houses, streets, or retail environments in order to perform specific tasks. One of the first works to address human drone interaction (HDI) was presented in [1]. In this research, the aerial robot is used to extend human abilities, for example, by enhancing the human’s field of view to report accidents or anomalies. Moreover, in [2] the authors proposed a flying drone acting as a personal companion to support people in emergency situations. To enable reliable human-drone interaction, a communication channel is required a phone, or gestural control [3]. Nevertheless, there are few feedback techniques for HDI. Some recent works studied head movements and propeller noise in order to present emotional states [4].

We presented previously in [5] a navigation model suitable for accompanying people in a comfortable manner, whereby aerial robots could navigate with people in indoor and outdoor environments alike. To accomplish this goal, we have seen that the interaction between robots and humans plays a key role, because robots should act in accordance with human motion and observe the interactive forces, which enable the robot navigate side-by-side with a human.

The side-by-side navigation is a subject of study where efforts are mainly focused on the development of autonomous companion robots capable of performing human-robot interactions in a more natural way [6]. Nevertheless, these works were developed for ground robots. Similarly, other works for robot companion introduce mediating factors, such as person’s individual experience with robots [13], however the experiments were conducted on controlled areas with no dynamic obstacles nor people.

With regards to aerial robots, the research on robots accompanying people is relatively minimal, [1], [3]. Furthermore, the works have been mainly devoted to simple robotics tasks with little interaction between the robot and the person. Unlike previous works for ground robots, in [5], we propose a new robot companion approach for aerial robots in urban environments.

Our current efforts are focused on study the impact of robots navigating and accompanying people in urban areas, looking for the optimal configuration of the pair human-drone and the study of the personal space the robot should consider to develop the task as comfortable as possible.

Keywords: Telerobotics and Teleoperation, Humanoid Robots
Abstract: This work presents a system that enables a humanoid robot to imitate complex human motions real-time while walking. To capture human motions, a Microsoft Kinect Depth sensor is used. Unlike the cumbersome motion capture suits, the sensor makes the system more comfortable to interact with. The user does not have to wear any special suit. The skeleton data is extracted from the Kinect which is processed to generate the robot's joint angles. Robot Operating System (ROS) is used for the communication between the various parts of the code to achieve minimal latency. Thus the robot imitates the human in real-time with negligible lag. Unlike most of the human motion imitation systems, this system is not stationary. The robot can walk while imitating human motions. The lower body motions of the user are also captured by the Kinect and processed, and modified slightly to attain maximum stability while placing steps. When the user is in a single support phase in between steps, the foot trajectories of the robot that satisfy the ZMP criterion are obtained by cubic interpolation and the gait is generated. At any given point in time, the robot's feet are placed in such a way that the center of mass lies over the support polygon.

Keywords: Human-Centered Robotics, Telerobotics and Teleoperation, Gesture, Posture, Social Spaces and Facial Expressions
Abstract: In many human-robot-interaction and robot teleoperation scenarios, it is difficult to convey spatial commands to the robot. In a model-driven approach, there are generic parameterization approaches, such as Unified Robot Description Format (URDF), for platforms to define different poses. Another example is DH parameters for the class of robotic manipulators. Here, we present a representation framework of high-level, spatial commands that specifies many joint angles with relatively few parameters based on a spatial architecture. This is inspired from the Body and Space categories in Laban/ Bartenieff Movement Studies (LBMS). Moreover, this framework can be extended to any platform.

Keywords: Space Robotics and Automation, Mechanism Design, Field Robots
Abstract: This paper describes the design method of the shoe plate of the hopping mechanism for the small hopping rover which travels on the moon, Mars, and asteroid surfaces. For efficient hopping, the design of a shoe plate with low slip is important. To design a push plate with a small amount of slip on the granular media for a stable and efficient hopping, it is necessary to know the reaction force received from the ground. In this paper, we describe the force estimation method of shoe plate design based on Resistive Force Theory, and calculate the reaction force when the shoe plate pressing the soil. Finally, the force acts on the shoe plate is verified using 3D printed experimental equipment.

Keywords: Telerobotics and Teleoperation, Human-Centered Automation, Industrial Robots
Abstract: During the latest years the use of robotic systems for dangerous intervention for replacing humans is becoming of global interest. Several situations require intervening in the minimum amount of time in a dangerous environment (presence of high risks, high temperatures, high radiation, oxygen deficiency etc) and robots can prevent the exposure of humans to such hazards. Therefore several robots have been developed for various scenarios such as search and rescue, bomb disposal, inspection and maintenance in hazardous environments, surveillance etc. For the development of such robots there are various aspects to take in to account: the reliability of the robot, the communication with it, the mechanical parts, the sensors, the control and the interface between the operator and the robot. In the research field, a lot of effort has been put on control, both of robotic platforms and manipulators, sensors, and communication. The industries instead, focused more on the mechanical aspects (robustness, payload etc.) and the reliability of the system. A key-point in the development of a robotic system is the interface between the robot and the operator. Industrial robots usually come with a specific control panel which allow to operate that specific robot and none else. The use of these robots is usually complicated and requires a lot of training from the operator. On the research side, a lot of effort have been put on how to provide a proper feedback to the operator while controlling the robot, for example by using haptic devices. Nevertheless, a complete study on a Human-Robot Interaction system was never done. A Human-Robot Interaction system should not only take into account the control methods, the communication with the robot and the display of the information, but it should be developed operator-centered, considering his/her possible inexperience, mistakes, confusion, stress and tiredness. This work aims to apply techniques used for the development of Graphical User Interfaces for the creation of a Human-Robot Interface (HRI) which is multimodal, adaptable, usable and learnable and which provides a complete environment for inspection, telemanipulation, data collection and training. The HRI is currently developed, used and tested at CERN, the European Center for Nuclear Research, for robotic interventions in the accelerator complex which is composed by more than 50 km of underground tunnels presenting several hazards such as high radiation, high temperature, oxygen deficiencies but also low luminosity and contamination risks.

Keywords: Learning from Demonstration, Learning and Adaptive Systems, Motion and Path Planning
Abstract: An algorithm called GMM-CDSP that learns the dynamics of point-to-point motions from demonstrated data is presented. The motion trajectories are modeled using an autonomous nonlinear system with partially contracting behavior. To accommodate the variability in demonstration data, Gaussian mixture models (GMM) are used to represent the motion dynamics. The parameters of the GMM are learned subject to the constraints derived from partial contraction analysis, which yields state-dependent constraints. A novel relaxation approach, inspired by sum of square (SOS) decomposition, is proposed to eliminate the state dependence on the constraints. The GMM-CDSP algorithm is evaluated on a shape-learning-and-drawing task using the Baxter robot platform and also compared with two state-of-the-art learning-from-demonstrations algorithms.

Keywords: Telerobotics and Teleoperation, Virtual Reality and Interfaces, Mapping
Abstract: This paper presents a current development of an enhancing teleoperation method by integrating augmented-reality for robot manipulators for tasks in hazardous and safety critical environment. Also a key to success of such integration is a new model reconstruction algorithm with high resolution dynamic tracking capability. The proposed innovation will allow deploying simple and robust robots in remote sites performing precise and dexterous manipulation under teleoperation.

Keywords: Biologically-Inspired Robots, Robust/Adaptive Control of Robotic Systems, Dynamics
Abstract: This poster will provide an overview of the recent and ongoing research in the Virginia Tech Robotics and Mechatronics Lab related to serpentine robotic tails. Three tail designs that utilizing different mechanisms for distributing actuation with a tail’s actuated segment will be presented and contrasted. The dynamic models utilized to simulate these tails will be presented, along with the controllers utilized to operate these tails. Case studies showing the tail’s ability to stabilize and maneuver bipedal and quadrupedal mobile robots will also be presented.

Keywords: Education Robotics, Multi-Robot Systems, Swarms
Abstract: In this work, we present a novel platform for swarm robotics that is low cost, easy to assemble using off the shelf components and is deeply integrated with the most used robotic framework available today: ROS (Robot Operating System). The robotic platform is completely open, composed of a 3D printed body and open source software. We describe its architecture, present its main features and evaluate its functionalities executing real experiments using a couple of robots. We concluded that the proposed swarm platform is a feasible and usable system for education and research, given its reduced cost, easy to use, and small size.

Keywords: Humanoid and Bipedal Locomotion, Model Learning for Control, Calibration and Identification
Abstract: Humanoid balance controllers are influenced by many deviations from the humanoid system itself and from the environment too. This work presents an experimental method to improve the Linear Inverted Pendulum Model (LIPM) used in balance control. Measuring the Zero Moment Point (ZMP) in the real system and comparing the results with theoretical values helps to model the ZMP deviation. These diferences are introduced in the control system by means of a 'scheduling map' for each working point, increasing the accuracy of the balance control.

Keywords: Search and Rescue Robots, SLAM, Mapping
Abstract: While using robots in search and rescue missions would help ensure the safety of first responders, a key issue is the time needed by the robot to operate. Even though SLAM is faster and faster, it might still be too slow to enable the use of robots in critical situations. One way to speed up operation time is to use prior information.

We aim at integrating emergency-maps into SLAM to complete the SLAM map with information about not yet explored part of the environment. By integrating prior information, we can speed up exploration time or provide valuable prior information for navigation, for example, in case of sensor blackout/failure. However, while extensively used by firemen in their operations, emergency maps are not easy to integrate in SLAM since they are often not up to date or with non consistent scales.

The main challenge we are tackling is in dealing with the imperfect scale of the rough emergency maps and integrate it with the online SLAM map in addition to challenges due to incorrect matches between these two types of map. We developed a formulation of graph-based SLAM incorporating information from an emergency map into SLAM, and propose a novel optimization process adapted to this formulation.

We extract corners from the emergency map and the SLAM map, in between which we find correspondences using a distance measure. We then build a graph representation associating information from the emergency map and the SLAM map. Corners in the emergency map, corners in the robot map, and robot poses are added as nodes in the graph, while odometry, corner observations, walls in the emergency map, and corner associations are added as edges. To conserve the topology of the emergency map, but correct its possible errors in scale, edges representing the emergency map's walls are given a covariance so that they are easy to extend or shrink but hard to rotate. Correspondences between corners represent a zero transformation for the optimization to match them as close as possible. The graph optimization is done by using a combination robust kernels. We first use the Huber kernel, to converge toward a good solution, followed by Dynamic Covariance Scaling, to handle the remaining errors.

We demonstrate our system in an office environment. We run the SLAM online during the exploration. Using the map enhanced by information from the emergency map, the robot was able to plan the shortest path toward a place it has not yet explored. This capability can be a real asset in complex buildings where exploration can take up a long time. It can also reduce exploration time by avoiding exploration of dead-ends, or search of specific places since the robot knows where it is in the emergency map.

Keywords: Aerial Systems: Applications, Path Planning for Multiple Mobile Robots or Agents
Abstract: In this work, we propose a new geometric based deployment approach for a fleet of quadrotors. Our task is to perform a distributed sensing in order to accomplish a coverage mission in cluttered environment where each vehicle plays the role of a flying tunable sensor. The proposed algorithm yields an optimal coverage path for each aerial robot while avoiding the obstacles, such that the union of all paths generates the full coverage of the 2D damaged target area. This latter is covered if the robots cross the overall existing scarred points, called Points Of Interest (POIs) that are automatically generated using a mesh generation algorithm. The problem is solved considering K-means Multi-directional optimal Fixed Node Rapidly-exploring Random Tree (RRT*FN) based allocation. By means of the pair-wise costs between points, the overall shortest path for each quadrotor is obtained by solving a Traveling Salesman Problem (TSP) using Genetic Algorithms (GA).

Keywords: Computer Vision for Transportation, Intelligent Transportation Systems, Autonomous Vehicle Navigation
Abstract: Autonomous detection of lane markers improves road safety, and purely visual tracking is desirable for widespread vehicle compatibility and reducing sensor intrusion, cost, and energy consumption. However, visual approaches are often ineffective because of a number of factors, including but not limited to occlusion, poor weather conditions, and paint wear-off. We present an approach towards robust lane tracking for assisted and autonomous driving, particularly under poor visibility. Our method, named SafeDrive, attempts to improve visual lane detection approaches in drastically degraded visual conditions without relying on additional active sensors other than vision and location data, sensors that are readily available on a standard smartphone.

Keywords: Humanoid and Bipedal Locomotion, Humanoid Robots, Legged Robots
Abstract: Inspired by the motion of juggling a ball from one hand to the other, humanoid hopping motion actually can be treated as juggling its center of mass (CoM) from one leg to the other. In both cases, stance phase and flight phase are involved. During the stance phase, the CoM trajectories of humanoid hopping can be generated based on the juggling motion characteristics. During the flight phase, the CoM follows ballistic dynamics. However, the relative position of feet with respect to CoM can still be controlled for the incoming landing. The feet position can be calculated by a simple heuristic foot placement algorithm considering the current CoM state. Afterwards, the CoM and feet tracking tasks are accomplished by a torque controller based on the full dynamics of the robot with respect to kinematic and dynamic constraints. The feasibility of the proposed control method is fully addressed by different simulations.

Keywords: Planning, Scheduling and Coordination, Path Planning for Multiple Mobile Robots or Agents
Abstract: The persistent operation of robotic missions is contingent on the smart use of energy. This work provides a solution in the intersection of trajectory optimization and energy autonomy for robotic area coverage missions. In this paper, a mission planning method is proposed that generates trajectories for mobile robots and locations of static charging stations for repeated battery replenishment through the mission. The method takes the working robots specifications and environmental constraints into consideration and can adapt to covering regions with different priorities. Systematic simulation experimentation has proven the effectiveness of the developed algorithm in area coverage and handling environmental constraints. In-lab aerial coverage scenario experiments illustrate the application of the obtained trajectories.

Keywords: SLAM, Sensor Fusion, Visual-Based Navigation
Abstract: This paper presents a real-time simultaneous localization and mapping (SLAM) algorithm for underwater cave combining visual data from a stereo camera with the angular velocity and linear acceleration data from Inertial Measurement Unit (IMU) and sonar data.

Keywords: Medical Robots and Systems, Mechanism Design, Model Learning for Control
Abstract: Magnetic resonance imaging (MRI)-guided, manipulator-assisted interventions have the potential to improve patient outcomes. This work presents a force transmission mechanism, called solid-media transmission (SMT), for actuating manipulators inside MRI scanners. The SMT mechanism is based on conduits filled with spheres and spacers made of a nonmagnetic, nonconductive material that forms a backbone for bidirectional transmission. Early modeling and experimental studies assessed SMT and identified limitations and improvements. Simulations demonstrated the detrimental role of friction, which can be alleviated with a choice of low friction material and long spacers. However, the length of the spacer is limited by the desired bending of the conduit. A closed-loop control law was implemented to drive the SMT. The 3rd order system fit ratio is 92.3%. A 1-m long SMT was experimentally tested under this closed-loop controller with heuristically set parameters using a customized benchtop setup. For commanded displacements of 1 to 50 mm, the SMT-actuated 1 degree of freedom stage exhibited sub-millimeter accuracy, which ranged from 0.109±0.057 mm to 0.045±0.029 mm depending on the commanded displacement. However, such accuracy required long control times inversely proportional to displacement ranging 7.56±1.85 s to 2.53±0.11 s. This was attributed to friction as well as backlash which is due to the suboptimal packing of the media. In MR studies, a 4-m long SMT-actuated 1 DoF manipulator was powered by a servo motor located inside the scanner room but outside the 5 Gauss line of the magnet. With shielding and filtering, the SNR of MR images during the operation of the servo motor and SMT-actuation was found to be 89±9% of the control case.

Keywords: Mining Robotics, Telerobotics and Teleoperation, Haptics and Haptic Interfaces
Abstract: In the mining industry, achieving a consistent and efficient production process has always been crucial. Therefore, the utilization of teleoperation techniques rises as a relevant solution to the problem. However, besides the development of teleoperation interfaces, it is also important to evaluate its suitability to the user. Hence, this paper presents the application of a methodology for evaluating such interfaces. A simplified experiment was conducted in a controlled environment using a mobile platform driven through obstacles with a teleoperation interface consisting of video and haptic devices. Results indicate that the utilization of such methodology can significantly contribute to the development of robust teleoperation interfaces.

Keywords: Perception for Grasping and Manipulation, RGB-D Perception, Service Robots
Abstract: We present an ongoing investigation into an interactive perception model for sorting objects observed in a workcell scene into bins based on Gaussian Process (GP) classification, which is capable of recognising objects categories from point cloud data.

Keywords: Grasping, Deep Learning in Robotics and Automation, Perception for Grasping and Manipulation
Abstract: We want to build robots that are useful in unstructured real world applications, such as doing work in the household. Grasping in particular is an important skill in this domain, yet it remains a challenge. One of the key hurdles is handling unexpected changes or motion in the objects being grasped and kinematic noise or other errors in the robot. This paper proposes an approach to learning a closed-loop controller for robotic grasping that dynamically guides the gripper to the object. We use a wrist-mounted sensor to acquire depth images in front of the gripper and train a convolutional neural network to learn a distance function to true grasps for grasp configurations over an image. The training sensor data is generated in simulation, a major advantage over previous work that uses real robot experience, which is costly to obtain. Despite being trained in simulation, our approach works well on real noisy sensor images. We compare our controller in simulated and real robot experiments to a strong baseline for grasp pose detection, and find that our approach significantly outperforms the baseline in the presence of kinematic noise, perceptual errors and disturbances of the object during grasping.

Keywords: Perception for Grasping and Manipulation, Deep Learning in Robotics and Automation
Abstract: In this work we address the problem of robotic pick and place, where the robot is given a point cloud and an object category identification (e.g. bottles, mugs), and where the robot must place the object in a desired way. For example the robot must pick a bottle out of the clutter and place it upright, collision-free, and not so high that it falls over after being released.

Our approach to the problem is to formulate it as a Markov decision process (MDP) and apply reinforcement learning (RL) methods to train the robot in simulation. We utilize our previous grasp detection work to provide the robot with a sampling of grasps to choose from for the pick action. The place action is chosen from a set of fixed place poses. As in DQN, an artificial neural network (ANN) is used to parameterize an approximation of the action-value function (Q-function). In training, the robot updates its Q-function through trial and error, essentially learning which grasps and places to choose. An off-the-shelf motion planner is used to plan the motions to the grasp and place poses.

Our approach has several advantages over previous attempts. First, our method can handle novel object instances within a category, whereas others require a model of the manipulated object. Second, it works well in clutter, which is a challenging scenario for perception. And finally, the MDP/RL framework allows for easy extension to more complex problems, such as where re-grasping and re-placing steps are required. (For example, a mug is initially upside-down and must be flipped before it can be grasped from the mouth -- a precondition for placing it upright in a deep box.)

The robot was able to quickly learn to place bottles and mugs in simulation. We also evaluated the trained ANN on a real robot with novel test objects.

Keywords: Aerial Systems: Applications, Environment Monitoring and Management, Localization
Abstract: Civil infrastructures are essential to ensuring the ongoing functionality of human living environments. In order to make sure that aging infrastructures do not experience catastrophic failures, however, they require continuous monitoring and maintenance. Traditionally, visual inspection has been carried out to monitor the health of structures. However, assessments conducted by trained inspectors and monitoring methods using sensing systems based on contact sensors on the structures are costly and inefficient due to the number of inspectors and sensors required. To date, data acquisition using unmanned aerial vehicles (UAVs) equipped with cameras has been popular, and a great deal of research has been done concerning the use of UAVs for the visual inspection of infrastructures. However, UAVs require skilled pilots or a global positioning system (GPS) for autonomous flight. Unfortunately, for some parts of the structure, GPS cannot be used for the autonomous flight of the UAV. For example, the GPS signal beneath a bridge is unreliable, but this is one of the critical locations that should be inspected to monitor and ensure structural health. In order to explore a solution to this difficulty, this study examines an autonomous UAV method using ultra wide band beacons. As an exemplar of structural damage, concrete cracks were detected successfully using a deep convolutional neural network by processing the video data collected from the autonomous UAV.

Keywords: Deep Learning in Robotics and Automation, Gesture, Posture, Social Spaces and Facial Expressions, Human Detection and Tracking
Abstract: We present a robust real-time systen for detecting hands and faces in RGB images. Starting with the YOLO Deep Convolutional Neural Network architecture and re-training with a mixture of third party datasets and our own original labelled data, we obtain qualitatively good results at 60Hz on commodity GPU. We designed the detector to be a useful component for Human-Robot Interaction systems.

Keywords: Haptics and Haptic Interfaces, Soft Material Robotics, Wearable Robots
Abstract: We propose a soft pneumatic actuator (SPA) based thin, stretchable and wearable device that can be safely placed on the skin to provide a modulable and high fidelity haptic feedback. The compliant `SPA-skin' device is capable of producing high frequency (0- 100 Hz) vibrations with modulable amplitude (0- 4 mm) controlled through integrated sensing of vibrations using low profile PZT sesors.

Keywords: Legged Robots, Underactuated Robots, Kinematics
Abstract: Robotic legged locomotion has seen tremendous development in recent years. However, very few robots have shown dynamic locomotive capabilities and operational robustness while executing gaits at high-speed, such as the MIT Cheetah, the ETH StarlETH, and the IIT HYQ. These impressive successes are the result of complex structural designs and elaborate control schemes. This poster presents the recent and ongoing research undertaken at the Virginia Tech Robotics and Mechatronics lab with the intent to reduce the complexity of robotic legged systems for high-speed dynamic locomotion. The details of the novel leg mechanism will be highlighted, along with the design evolution of the leg through optimization. Experiments and its results obtained through the prototype showing that the leg is capable of performing dynamic locomotive gaits are presented.

Keywords: Medical Robots and Systems, Robot Safety, Surgical Robotics: Planning
Abstract: Autonomy is a fundamental attribute of robotics, yet it is one of the hardest terms to define, assess and regulate. Autonomy probably poses the greatest safety challenge within the medical field. AI methods and decision support systems are playing an increasing role in all domains of healthcare, and their breakthrough in medical robotics is just around the corner, assuming a safe regulatory framework can be developed. Supporting these trends, a new generation of robotics standards is emerging both in the IEEE and in the ISO/IEC domain, already focusing on the big challenge of the future medical systems: autonomy. This work provides a brief overview of the brand new standards targeting this complex area. The importance of standardization had become paramount in the medical domain, since that is believed to be the best way to increase safety systematically. Harmonized global standards and regulatory frameworks are offering ways to product certification through standardized testing requirements and protocols. However, until very recently, a huge gap existed, since the traditional robotics standards excluded service robots and medical robots, particularly. In the past 7 years, a joint working group of robotic experts was addressing this problem, and this year, the very first ISO/IEC joint Technical Report (TR) on the problem of autonomy for medical electrical systems (MES) (including robots) appeared. This is a first step towards the standardized assessment of robot capabilities, primarily focusing on their autonomous functions, while practical guidelines on methods for robot categorization and certification are also on the horizon. The new TR offers an unambiguous solution to describe and assess the autonomous capabilities of an MES. Relying on some earlier work in the field of industrial automation and service robotics, the TR recommends the parametrization of Degree of Autonomy (DoA) along four cognition-related functions of a system, which are affecting capabilities of an MES to Generate, Execute, Monitor and Select an option related to a robot task. Each of these functions can be driven by a human or a computed, which would then lead to the objective assessment of the DoA of the full system. DoA can vary from low to high, with zero meaning “no autonomy”, and the other end of the scale meaning a “full autonomy” system. DoA can be classified at different granularity levels, depending on where and how the above safety functions are implemented. Basic safety & essential performance for surgical robots The IEC TC 62/SC 62D joint committee worked to provide a practical degree of safety for surgical robots, resulting in a brand new standard to be published later this year. It defines the basic types of surgical robots and tools, and identifies integrated components. The standard collects all relevant mechanical and thermal hazards, along with the fault conditions of the equipment and the required usability trials.

Keywords: Mobile Manipulation, Service Robots, Virtual Reality and Interfaces
Abstract: This work shows the development of three smarphone-based interfaces to teleoperate a Barrett WAM arm and hand to pick-and-place an object in a particular location. Visual feedback is provided by streaming video of the robot's workspace to the smartphone. Based on our designs and experimental evaluation, we develop intuitions about the advantages of smartphone-based interfaces.

Keywords: Deep Learning in Robotics and Automation, Grasping, Perception for Grasping and Manipulation
Abstract: Deep learning-based grasp planning has made strong progress the past several years due to algorithmic improvements and increased data availability. However, state-of-the-art models are often trained on only hundreds or thousands of unique object instances, and as a result frequently do not generalize perfectly to previously unseen objects at test time.

In this work, we explore a novel data generation pipeline for training a deep neural network to perform grasp planning that applies the idea of domain randomization to object synthesis. We generate millions of unique, non-realistic randomly generated objects with a range of physics and appearance properties, and train a deep neural network entirely in simulation to perform grasp planning on these objects.

Since the distribution of robust grasps for a given object can be highly multimodal, we propose an autoregressive grasp planning model that maps an image of an object to a probability distribution p_{theta} where p_{theta}(g) corresponds to the probability that the grasp g is the most robust for that object. Our model allows us to sample grasps efficiently at test time (or avoid sampling entirely), a critical design criterion for scaling deep-learning based grasp planning to higher dimensional human-like end effectors that can have 20 or more degrees of freedom.

We currently have preliminary results exploring these ideas in simulation, and plan to also test them on hardware.

Keywords: Computer Vision for Automation, Health Care Management, Human-Centered Automation
Abstract: In previous work, an automatic assessment system was presented for the evaluation of patient progress and effectiveness of the therapy, that is based on the Box and Blocks Test (BBT) of manual dexterity. In this article, a study of the feasibility to use in Parkinson treatment is presented. The results advocates the use of automated methods in physical rehabilitation assessment.

Keywords: AI-Based Methods, Haptics and Haptic Interfaces, Task Planning
Abstract: Contact forces of the hand are visually unobservable, but they play a crucial role in understanding hand-object interactions. We propose an unsupervised learning framework to learn a hierarchical model, Temporal And-Or graph (T-AOG), for hand-object interactions. The interactions are captured using a tactile glove that measures both hand pose and contact forces. Force vectors are extracted as features from the raw sensory input of poses and forces, and segmented by clustering. The T-AOG is induced in an unsupervised manner. Experiments show that our method achieves a high accuracy in terms of segmentation and labeling.

Keywords: Gripper and Other End-Effectors, Force and Tactile Sensing, Haptics and Haptic Interfaces
Abstract: We show that to obtain tactile information about and recognize a textile, a robot gripper's fingertips together can be rubbed together in a human-inspired exploratory motion. Our method not only recognizes 18 different materials with over 94% accuracy, but also distinguishes between one and multiple layers of the same material.

Keywords: Swarms, Networked Robots, Aerial Systems: Applications
Abstract: ROSBuzz provides fast script-based programming of complex behaviors, Over-The-Air update and most importantly, coherent performance from simulation to field deployment.

Keywords: Biologically-Inspired Robots, Computer Vision for Other Robotic Applications, Autonomous Vehicle Navigation
Abstract: The abstract describes a biologically inspired neuromorphic vision based approach for tracking and following an object of interest in the scene.

Keywords: Contact Modelling, Simulation and Animation
Abstract: We present an interactive and stable multi-contact dynamic simulation algorithm for rigid bodies. Our approach is based on fast frictional dynamics (FFD) [14], which is designed for large sets of non-convex rigid bodies. We use a new friction model that performs velocity-level multi-contact simulation using impulse decomposition. Moreover, we accurately handle friction at each contact point using contact distribution and frictional impulse solvers, which also account for relative motion. We evaluate our algorithm’s performance on many complex multi-body benchmarks with thousands of contacts.In practice, our dynamics simulation algorithm takes a few milliseconds per timestep and exhibits more stable behaviors.

Keywords: Soft Material Robotics, Surgical Robotics: Laparoscopy, Medical Robots and Systems
Abstract: This paper presents a soft robotic tool that prevents tissue damage during surgery while grasping and retracting soft tissue. The manipulator design, fabrication, and functional grasping and retracting forces are reported.

Keywords: Soft Material Robotics, Cellular and Modular Robots, Gripper and Other End-Effectors
Abstract: Abstract— In previous research, “SoBL”, a fast-build modularized design block for soft robotics, was introduced to overcome maintenance problems and simplify rapidly prototyping of soft robots. In this paper, a smaller size SoBL unit based on previous research was introduced. A three fingered soft gripper was assembled with the smaller size SoBL units and tested to pick up off-the-shelf mechanical components. This test showed the feasibility of design variation of SoBL units in terms of scale. In future, we hope that diversity of SoBL would be expanded in terms of both scale and unit designs for the bottom-up design concept of soft robotics.

I. INTRODUCTION Field of soft robotics changes robots from relatively hard structures to soft and flexible structures. Most of the soft robots were fabricated with molding and casting process with elastomers [1], [2]. Comparing to traditional hard material based robots, soft robots fabricated with these methods have difficulties in maintenance and prototyping. To overcome these problems, “SoBL”, modular units for soft robotics, was introduced in our previous research [3]. In this paper, the smaller size of the SoBL unit was introduced. With the small SoBL units, the three fingered soft gripper was assembled and tested for check feasibility on a different scale. The gripper could securely pick up small mechanical components and shift their locations.

II. DESIGN AND FABRICATION Based on the square shaped bending unit of SoBL in our previous research, the size of the unit was reduced by half. Connecting between units was maintained as screw-thread but redesigned for the changed size of the unit. A three fingered soft gripper was assembled with these size reduced units as shown in Figure 1 (a). Each finger was assembled with three units and a fingertip was assembled at the end of fingers. The gripper attached to the soft manipulator which was assembled with independent bending units.

III. EXPERIMENTS The three fingered soft gripper was tested to grasp small objects. The gripper was actuated with around 30 kPa of air pressure. All three fingers shared an air channel, therefore, the fingers of the gripper were actuated simultaneously. At first, the gripper was tested with an off-the-shelf M3 nut (Figure 1(b)). Thanks to the fingertips, the gripper safely picked up and move the nut. In addition, the gripper also tested to pick up both an off-the-shelf M3 nut and a 10 mm long M3 bolt. As shown in Figure 1 (c), the gripper also could safely pick up and move both nut and bolt, simultaneously.

IV. CONCLUSION In this paper, the small size modular unit for soft robotics was introduced and tested by assembling the three fingered soft gripper. This shows the feasibility of design variation of SoBL units in terms of scales. In future, we hope SoBL set enlarged in terms of scale and unit designs for bottom-up design concept of soft robotics.

Keywords: Flexible Robots, Medical Robots and Systems, Surgical Robotics: Steerable Catheters/Needles
Abstract: Access inside the human body is a challenge for minimally invasive surgery. For endovascular surgery (ES), push- catheters are inserted via an artery in the groin and pushed through the vasculature to gain access to distal arteries, including the arteries of the brain. Push-catheters, however, have limited ability to make sharp turns, and therefore may inadvertently damage the vessels. Here we describe a novel tip-extending catheter that grows through vessels in a manner analogous to how plants grow, termed Vascular Internal Navigation by Extension (VINE). The VINE catheter efficiently navigates tight turns and extreme angles due to its low bending stiffness and lack of sliding friction with vessel walls. Further, the soft, growing construction of VINE catheters results in reduced force on the leading edge, potentially reducing the risk of arterial injury compared to standard push-catheters. Our work helps advance the accessibility of ES tools and the speed and safety of ES.

Keywords: Human-Centered Robotics, Autonomous Vehicle Navigation, Deep Learning in Robotics and Automation
Abstract: Drivers and pedestrians engage in non-verbal and social cues to signal their intent, which is crucial to their interactions in traffic scenarios. We propose to learn such cues and model a pedestrian's intent with a Generative Adversarial Network. The learnt model then predicts actions likely to be performed 400-600 ms in the future. Responding to adverse actions in advance, we tread towards full autonomy.

Keywords: Visual Tracking
Abstract: Visual object tracking, which tracks a specified target in a changing video sequence automatically, is a fundamental problem in many aspects such as visual analytics, automatic driving, pose estimation and et al. Recently, the correlation filter-based tracking methods have been paid more attention to solve the problem of high computational costs. Correlation operations can be performed by element-wise multiplications using discrete Fourier transform in frequency domain, and discrete Fourier transform can be computed by the efficient fast Fourier transform in practice. This property of correlation filters makes it efficient for fast training and detection in visual tracking. However, most of correlation filters only use appearance features, ignoring motion information. We derive the back-propagation of correlation filters and combine the FlowNet into the network. Specifically, the correlation filters subnetwork explains for one-order information, and the flow subnetwork encodes the second-order information. The overall network are training end-to-end to ensure top tracking performance.

Keywords: Deep Learning in Robotics and Automation, Localization, SLAM
Abstract: Our proposed method makes SLAM in underwater more reliable and efficient by focusing on hard immobile coral species for visual features. The proposed method also eliminates features from unreliable objects. In order to provide such prior knowledge, we developed a convolutional neural network based classifier. We have collected coral reef footage from Barbados and the videos were annotated by experts. The CNN classifier was trained on the annotated data. This classifier can provide class activation map which can be converted to binary mask. The mask will help limiting extraction of visual features to image areas containing hard coral species providing the much needed prior knowledge to SLAM for localizing AUV.

Keywords: Mining Robotics, Mechanism Design, Simulation and Animation
Abstract: One of the great motivations in the development of robots is to use them in dangerous environments. For mobile devices, it is empirical to be familiar with the locomotion system in use.This allows to understand its capabilities and limitations in order to guarantee the mission success and the devices integrity. The robot used in this paper has an interchangeable locomotion system that allows the device to move with legs, wheels, tracks or hybrid configurations. This proposal analyses mobility characteristics of divers locomotion modes employed by the same robot and then compares their performances. Tests were performed to collect locomotion data using a simulator and a real prototype. Many concepts predicted in the simulation were validated with the real robot. Finally, a comparison of the different locomotion characteristics is obtained.

Keywords: Learning and Adaptive Systems, Cooperating Robots, Social Human-Robot Interaction
Abstract: We used an evolutionary algorithm to optimize the parameters for an artificial agent model which can perform a cooperative herding task with either a human partner or another artificial agent. The algorithm evolved agents that could herd simulated sheep in a randomly-generated dynamical environment. We utilized different graded fitness functions to vary the amount of herding “success” to determine whether the agents would adopt similar strategies to human learners as they progress through the experiment. Results indicated that, like humans, the artificial agent was able to evolve from one mode of fixed-point behavior to an oscillatory regime when the model was limited to how quickly it could move and the demands for success increased. We tested one of these agents alongside human novices. A majority (80%) of participants learned the more optimal, oscillatory behavior. Further, a majority of participants (70%) were mistaken to believing they were playing with a human instead of the evolved agent.

Keywords: Cognitive Human-Robot Interaction, Human-Centered Robotics
Abstract: This paper investigates subtle communicative cues in human reaching motions to be utilized in collaborative human-robot interaction scenarios. An experiment with 14 participants was performed in which each participant is asked to reach to grasp one of two cylindrical objects which are placed in four different configurations. The experiment had two conditions; (1) control condition where the participant reaches an object with no observer and (2) test condition where the participant is encouraged to exhibit communicative motion with an observer present. The experiments were only video recorded and no other sensors were placed in order to create an as natural environment as possible for the participants. The video recordings were annotated and analyzed to measure total action duration, time to maximum hand-aperture, and the relative time to maximum hand-aperture. The results show that some of these measures differ significantly between conditions. Some participants also exhibit grasp modality differences and slight exaggeration between the conditions. Furthermore, a post-experiment survey suggests that some differences might be unconscious. These findings show that subtle cues may be important in communicative motion generation and should be considered in human-robot interaction scenarios. The paper concludes by suggesting the use of the findings in developing algorithms and robotic systems.

Keywords: Robust/Adaptive Control of Robotic Systems, Aerial Systems: Mechanics and Control, Neural and Fuzzy Control
Abstract: robust adaptive nonlinear control system consisting of a neural network approximation and a modified reference model is considered for small quadrotors. Quadrotor systems, like any dynamical system, suffer from the presence of uncertainties in their model and parameters which may lead the system to instability or inaccuracy. In addition, the presence of input constraints in quadrotor systems violates the assumption of the control system to be affine in its control signal. Consequently, these problems add additional challenges and complicate designing the control system. The proposed control algorithm combined an adaptive neural network with the reference model modification techniques to overcome the mentioned difficulties. The proposed control system allows the quadrotor system to follow a designed path despite the system’s uncertainties, disturbances, and actuator saturation. In addition, the stability of the proposed controller is proven within the region of interest and validated using simulation and experimental setup.

Keywords: Model Learning for Control, Deep Learning in Robotics and Automation, Motion and Path Planning
Abstract: We investigate incremental transfer learning in the context of an exploration for mapping task. We refer to this procedure as the ‘Training Wheels’ method. We present initial results in a mapping and exploration task.

Keywords: Visual Tracking, Robust/Adaptive Control of Robotic Systems, Recognition
Abstract: In this paper, we propose an event-based feature extraction and tracking method for high dynamic-ranged environments. Unlike other Visual Odometry (VO) methods that utilize frame-wise image data from the complementary metal- oxide-semiconductor (CMOS) sensors, we use data from event-based vision sensors. While only providing pixel-wise temporal intensity changes, the Dynamic Vision Sensor (DVS) robustly performs in higher dynamic ranges than a conventional charge- coupled device (CCD) sensor. This characteristic enables the DVS to achieve nearly no loss of pixel information even under drastic light condition changes. To exploit the obtained events in a VO, we introduce an event-based feature-extraction method and its descriptor, to robustly track features and lower the position error of VO in high dynamic-range environments.

Keywords: Gesture, Posture, Social Spaces and Facial Expressions, RGB-D Perception, Social Human-Robot Interaction
Abstract: Abstract—We propose to exploit a state-of-the-art Deep CNN hand and face detector for RGB images, then use the depth channel of RGB-D images to obtain full 3D pointing vectors at 60Hz on commodity hardware.

Keywords: Humanoid Robots, Space Robotics and Automation
Abstract: In this work we present design and implementation of a robust state machine that provides full autonomy over known tasks to Valkyrie R5 robot along with the ability for operator to override state machine commands.

Keywords: SLAM, Multi-Robot Systems, Mapping
Abstract: This work presents a novel method currently being developed to efficiently factorize the combination of multiple factor graphs having common variables of estimation. Recent factor graph formulation for Simultaneous Localization and Mapping (SLAM) like Incremental Smoothing and Mapping using the Bayes tree (ISAM2) has been very successful and garnered much attention. Variable ordering, a well-known technique in linear algebra is employed for efficiently solving the factor graph. Our primary contribution in this work is to reuse the variable ordering of the graphs being combined to find the ordering of the fused graph called fusion ordering. In the case of mapping, multiple robots provide a great advantage over single robot by providing a faster map coverage and better estimation quality. This coupled with an inevitable increase in the number of robots around us produce a demand for faster algorithms. For example, a city full of self-driving cars could exhibit a "friendly" gesture by pooling their observation measurements rapidly to plan a traffic free navigation. By reusing the variable ordering of the parent graphs we were able to produce an order-of-magnitude difference in the time required for solving the fused graph. A formal verification is attempted to show that the proposed strategy does not violate any of the relevant standards. The fusion ordering is experimented on the standard dataset used in the sparse linear algebra community called SuiteSparse and a real-world multi-robot AP Hill dataset.

Keywords: Localization, Object detection, segmentation, categorization, AI-Based Methods
Abstract: In this paper we propose a method to classify glass and non-glass objects for mobile robots navigating in indoor environments using laser rangefinders. The classification results are used to build a glass grid map. Specifically, the proposed classifier, which is based on a 4-layer neural network, employs the LR's received intensity, the measured distance and the incident angle to classify glass and non-glass objects in the mapped environment. The validity of the proposed method has been tested experimentally. Preliminary experimental results shown the accuracy of the proposed solution to accurately classify glass and no-glass objects.

Keywords: Biologically-Inspired Robots, Visual Tracking
Abstract: We newly propose a high-speed robotic system that can investigate running insects such as ants on a 2D space.

Keywords: Aerial Systems: Applications, Social Human-Robot Interaction, Visual Servoing
Abstract: We demonstrate a very simple touchscreen user interface whereby from a sparse series of user touch-screen inputs, a robot performs visual servo ‘hops’ from target to target and can reliably arrive at a distant, small, initially invisible target from hundreds of meters away. We demonstrate the method on a real robot.

Keywords: Flexible Robots, Compliant Joint/Mechanism, Industrial Robots
Abstract: The robots interaction has been studied since the creation of them, because they share the environment with other objects or humans. The solutions have to face both the safety and the efficiency (in terms of speed and accuracy) because the robot will be even more used for helping or substitute the repetitive jobs in industrial plants. These two characteristics often are apparently in contrast, because the high speed or heavy payload look as the dangerous in case of impact. In this work a compliant approach is presented, but the flexibility is not implemented into the control, but into the links. The robot is built in elastic material to permit the deformation during the contact with other bodies. At same time the control deals with this compliance and then both the accuracy and speed is still guarantee.

Keywords: Visual Learning, Deep Learning in Robotics and Automation, Visual-Based Navigation
Abstract: Many model-based Visual Odometry (VO) algo- rithms have been proposed in the past decade, often restricted to the type of camera optics, or the underlying motion manifold observed. We envision robots to be able to learn and perform these tasks, in a minimally supervised setting, as they gain more experience. To this end, we propose a fully trainable solution to visual ego-motion estimation for varied camera optics. We propose a visual ego-motion learning architecture that maps observed optical flow vectors to an ego-motion density estimate via a Mixture Density Network (MDN). By modeling the architecture as a Conditional Variational Autoencoder (C-VAE), our model is able to provide introspective reasoning and prediction for ego-motion induced scene-flow. Additionally, our proposed model is especially amenable to bootstrapped ego-motion learning in robots where the supervision in ego-motion estimation for a particular camera sensor can be obtained from standard navigation-based sensor fusion strategies (GPS/INS and wheel-odometry fusion). Through experiments, we show the utility of our proposed approach in enabling the concept of self-supervised learning for visual ego-motion estimation in autonomous robots.

Keywords: Robot Safety, RGB-D Perception, Cognitive Human-Robot Interaction
Abstract: To ensure system integrity, robots need to proactively avoid any unwanted physical perturbation that may cause damage to the underlying hardware. In this paper, we investigate a machine learning approach that allows robots to anticipate impending physical perturbations from perceptual cues. In contrast to other approaches that require knowledge about sources of perturbation to be encoded before deployment, our method is based on experiential learning. Robots learn to associate visual cues with subsequent physical perturbations and contacts. In turn, these extracted visual cues are then used to predict potential future perturbations acting on the robot. To this end, we introduce a novel deep network architecture which combines multiple sub-networks for dealing with robot dynamics and perceptual input from the environment. We present a self-supervised approach for training the system that does not require any labeling of training data. Extensive experiments in a human-robot interaction task show that a robot can learn to predict physical contact by a human interaction partner without any prior information or labeling.

Keywords: Deep Learning in Robotics and Automation, Aerial Systems: Mechanics and Control, AI-Based Methods
Abstract: In this paper, we present a method to control a quadrotor with a neural network trained using reinforcement learning techniques. The proposed network directly maps state to thrust forces making any predefined control structure obso- lete for training. Moreover, we present a new learning algorithm which differs from the existing ones in certain aspects. Our algorithm is conservative but stable for complicated tasks. Therefore, it is more applicable to controlling a quadrotor than existing algorithms. We demonstrate the performance of the trained policy both in simulation and with a real quadrotor. Experiments show that our policy network can react to step response relatively accurately. With the same policy, we also demonstrate that we can stabilize the quadrotor in the air even under very harsh initialization (manually throwing it upside- down in the air with an initial velocity of 5 m/s). Computation time of evaluating the policy is only 7 μs per time step which is two orders of magnitude less than common trajectory optimization algorithms with an approximated model.

Keywords: Deep Learning in Robotics and Automation, Visual Learning, Sensor Fusion
Abstract: This paper demonstrates a feasible method for using a deep neural network as a sensor to estimate the attitude of a flying vehicle using only flight video. A dataset of still images and associated gravity vectors was collected and used to perform supervised learning. The network builds on a previously trained network and was trained to be able to approximate the attitude of the camera with an average error of about 8 degrees. Flight test video was recorded and processed with a relatively simple visual odometry method. The aircraft attitude is then estimated with the visual odometry as the state propagation and network providing the attitude measurement in an extended Kalman filter. Results show that the proposed method of having the neural network provide a gravity vector attitude measurement from the flight imagery reduces the standard deviation of the attitude error by approximately 12 times compared to a baseline approach.

Keywords: Recognition, Object detection, segmentation, categorization, Deep Learning in Robotics and Automation
Abstract: Deep networks thrive when trained on large scale data collections. This has given ImageNet a central role in the development of deep architectures for visual object classification. However, ImageNet was created during a specific period in time, and as such it is prone to aging, as well as dataset bias issues. Moving beyond fixed training datasets will lead to more robust visual systems, especially when deployed on robots in new environments which must train on the objects they encounter there. To make this possible, it is important to break free from the need for manual annotators. Recent work has begun to investigate how to use the massive amount of images available on the Web in place of manual image annotations. We contribute to this research thread with two findings: (1) a study correlating a given level of noisily labels to the expected drop in accuracy, for two deep architectures, on two different types of noise, that clearly identifies GoogLeNet as a suitable architecture for learning from Web data; (2) a recipe for the creation of Web datasets with minimal noise and maximum visual variability, based on a visual and natural language processing concept expansion strategy. By combining these two results, we obtain a method for learning powerful deep object models automatically from the Web. We confirm the effectiveness of our approach through object categorization experiments using our Web-derived version of ImageNet on a popular robot vision benchmark database, and on a lifelong object discovery task on a mobile robot.

Keywords: Recognition, Visual Learning, Semantic Scene Understanding
Abstract: This paper presents an approach for semantic place categorization using data obtained from RGB cameras. Previous studies on visual place recognition and classification have shown that, by considering features derived from pre-trained Convolutional Neural Networks (CNNs) in combination with part-based classification models, high recognition accuracy can be achieved, even in presence of occlusions and severe viewpoint changes. Inspired by these works, we propose to exploit local deep representations, representing images as set of regions applying a Na"{i}ve Bayes Nearest Neighbor (NBNN) model for image classification. As opposed to previous methods where CNNs are merely used as feature extractors, our approach seamlessly integrates the NBNN model into a fully-convolutional neural network.