Keywords: Calibration and Identification
Abstract: Extrinsic calibration is a necessary step when using heterogeneous sensors for robotics applications. Most existing methods work under the assumption that the prior data correspondence is known. Considering data loss and false measurements, the correspondence may not be accessible in practice. To solve this problem without knowing the correspondence, several probabilistic methods have been proposed. However, an implicit restriction on input data limits their application. Therefore, in this paper, we propose a more stable correspondence-free method with two improvements that can relax the restrictions on inputs and improve the calibration accuracy. The first improvement finds consistent sets from raw inputs using screw invariants, which significantly improve the robustness in case of outliers and data loss. A new optimization method on matrix Lie group is proposed as the second improvement, which demonstrates better accuracy. The experimental results on both numerical and real data show the superiority and robustness of the proposed method.

Keywords: Calibration and Identification
Abstract: In order to fuse measurements from multiple sensors mounted on a mobile robot, it is needed to express them in a common reference system through their relative spatial transformations. In this paper, we present a method to estimate the full 6DoF extrinsic calibration parameters of multiple heterogeneous sensors (Lidars, Depth and RGB cameras) suitable for automatic execution on a mobile robot. Our method computes the 2D calibration parameters (x, y, yaw) through a motion-based approach, while for the remaining 3 parameters (z, pitch, roll) it requires the observation of the ground plane for a short period of time. What set this proposal apart from others is that: i) all calibration parameters are initialized in closed form, and ii) the scale ambiguity inherent to motion estimation from a monocular camera is explicitly handled, enabling the combination of these sensors and metric ones (Lidars, stereo rigs, etc.) within the same optimization framework. We provide a formal definition of the problem, as well as of the contributed method, for which a C++ implementation has been made publicly available. The suitability of the method has been assessed in simulation and with real data from indoor and outdoor scenarios. Finally, improvements over state-of-the-art motion-based calibration proposals are shown through experimental evaluation.

Keywords: Calibration and Identification, Computer Vision for Transportation, Intelligent Transportation Systems
Abstract: Multiple LiDARs have progressively emerged on autonomous vehicles for rendering a rich view and dense measurements. However, the lack of precise calibration negatively affects their potential applications. In this paper, we propose a novel system that enables automatic multi-LiDAR calibration without any calibration target, prior environment information, and initial values of the extrinsic parameters. Our approach starts with a hand-eye calibration for automatic initialization by aligning the motions of each sensor. The initial results are then refined with an appearance-based method by minimizing a cost function constructed by point-plane distance. Experimental results on simulated and real-world data sets demonstrate the reliability and accuracy of our calibration approach. The proposed approach can calibrate a multi-LiDAR system with the rotation and translation errors less than 0.04rad and 0.1m respectively for a mobile platform.

Keywords: Calibration and Identification, Formal Methods in Robotics and Automation, Computer Vision for Medical Robotics
Abstract: In the eye-in-hand robot configuration, hand-eye calibration plays a vital role in completing the link between the robot and camera coordinate systems. Calibration algorithms are mature and provide accurate transformation estimations for an effective camera-robot link but rely on a sufficiently wide range of calibration data to avoid errors and degenerate configurations. This can be difficult in the context of keyhole surgical robots because they are mechanically constrained to move around a remote centre of motion (RCM) which is located at the trocar port. The trocar limits the range of feasible calibration poses that can be obtained and results in ill-conditioned hand-eye constraints. In this paper, we propose a new approach to deal with this problem by incorporating the RCM constraints into the hand-eye formulation. We show that this not only avoids ill-conditioned constraints but is also more accurate than classic hand-eye calibration with a free 6DoF motion, due to solving simpler equations that take advantage of the reduced DoF. We validate our method using simulation to test numerical stability and a physical implementation on an RCM constrained KUKA LBR iiwa 14 R820 equipped with a NanEye stereo camera.

Keywords: Calibration and Identification, Kinematics, Wheeled Robots
Abstract: Robotic calibration allows for the fusion of data from multiple sensors such as odometers, cameras, etc., by providing appropriate relationships between the corresponding reference frames. For wheeled robots equipped with camera/lidar along with wheel encoders, calibration entails learning the motion model of the sensor or the robot in terms of the data from the encoders and generally carried out before performing tasks such as simultaneous localization and mapping (SLAM). This work puts forward a novel Gaussian Process-based non-parametric approach for calibrating wheeled robots with arbitrary or unknown drive configurations. The procedure is more general as it learns the entire sensor/robot motion model in terms of odometry measurements. Different from existing non-parametric approaches, our method relies on measurements from the onboard sensors and hence does not require the ground truth information from external motion capture systems. Alternatively, we propose a computationally efficient approach that relies on the linear approximation of the sensor motion model. Finally, we perform experiments to calibrate robots with un-modelled effects to demonstrate the accuracy, usefulness, and flexibility of the proposed approach.

Keywords: Calibration and Identification, Sensor Fusion, Autonomous Vehicle Navigation
Abstract: Accurate extrinsic sensor calibration is essential for both autonomous vehicles and robots. Traditionally this is an involved process requiring calibration targets, known fiducial markers and is generally performed in a lab. Moreover, even a small change in the sensor layout requires recalibration. With the anticipated arrival of consumer autonomous vehicles, there is demand for a system which can do this automatically, after deployment and without specialist human expertise.

To solve these limitations, we propose a flexible framework which can estimate extrinsic parameters without an explicit calibration stage, even for sensors with unknown scale. Our first contribution builds upon standard hand-eye calibration by jointly recovering scale. Our second contribution is that our system is made robust to imperfect and degenerate sensor data, by collecting independent sets of poses and automatically selecting those which are most ideal.

We show that our approach's robustness is essential for the target scenario. Unlike previous approaches, ours runs in real time and constantly estimates the extrinsic transform. For both an ideal experimental setup and a real use case, comparison against these approaches shows that we outperform the state-of-the-art. Furthermore, we demonstrate that the recovered scale may be applied to the full trajectory, circumventing the need for scale estimation via sensor fusion.

Keywords: Deep Learning in Robotics and Automation, Aerial Systems: Mechanics and Control, Optimization and Optimal Control
Abstract: Synthesis of a feedback controller for nonlinear dynamical systems like a quadrotor requires to deal with the trade-off between performance and online computation requirement of the controller. Model predictive controllers (MPC) provide excellent control performance, but at the cost of high online computation. In this paper, we present our experience in approximating the behavior of an MPC for a quadrotor with a feed-forward neural network. To facilitate the collection of training data, we create a faithful model of the quadrotor and use Gazebo simulator to collect sufficient training data. The deep neural network (DNN) controller learned from the training data has been tested on various trajectories to compare its performance with a model-predictive controller. Our experimental results show that our DNN controller can provide almost similar trajectory tracking performance at a lower control computation cost, which helps in increasing the flight time of the quadrotor. Moreover, the hardware requirement for our DNN controller is significantly less than that for the MPC controller. Thus, the use of DNN based controller also helps in reducing the overall price of a quadrotor.

Keywords: Deep Learning in Robotics and Automation, Object Detection, Segmentation and Categorization, Aerial Systems: Perception and Autonomy
Abstract: Deep Learning-based object detectors enhance the capabilities of remote sensing platforms, such as Unmanned Aerial Vehicles (UAVs), in a wide spectrum of machine vision applications. However, the integration of deep learning introduces heavy computational requirements, preventing the deployment of such algorithms in scenarios that impose low-latency constraints during inference, in order to make mission-critical decisions in real-time. In this paper, we address the challenge of efficient deployment of region-based object detectors in aerial imagery, by introducing an informed methodology for extracting candidate detection regions (proposals). Our approach considers information from the UAV on-board sensors, such as flying altitude and light-weight computer vision filters, along with prior domain knowledge to intelligently decrease the number of region proposals by eliminating false-positives at an early stage of the computation, reducing significantly the computational workload while sustaining the detection accuracy. We apply and evaluate the proposed approach on the task of vehicle detection. Our experiments demonstrate that state-of-the-art detection models can achieve up to 2.6x faster inference by employing our altitude-aware data-driven methodology. Alongside, we introduce and provide to the community a novel vehicle-annotated and altitude-stamped dataset of real UAV imagery, captured at numerous flying heights under a wide span of traffic scenarios.

Keywords: Deep Learning in Robotics and Automation, Aerial Systems: Mechanics and Control, Learning and Adaptive Systems
Abstract: Quadrotor stabilizing controllers often require careful, model-specific tuning for safe operation. We use reinforcement learning to train policies in simulation that transfer remarkably well to multiple different physical quadrotors. Our policies are low-level, i.e., we map the rotorcrafts' state directly to the motor outputs. The trained control policies are very robust to external disturbances and can withstand harsh initial conditions such as throws. We show how different training methodologies (change of the cost function, modeling of noise, use of domain randomization) might affect flight performance. To the best of our knowledge, this is the first work that demonstrates that a simple neural network can learn a robust stabilizing low-level quadrotor controller (without the use of a stabilizing PD controller) that is shown to generalize to multiple quadrotors.

Keywords: Deep Learning in Robotics and Automation, Aerial Systems: Mechanics and Control, Model Learning for Control
Abstract: Designing effective low-level robot controllers often entail platform-specific implementations that require manual heuristic parameter tuning, significant system knowledge, or long design times. With the rising number of robotic and mecha- tronic systems deployed across areas ranging from industrial automation to intelligent toys, the need for a general approach to generating low-level controllers is increasing. To address the challenge of rapidly generating low-level controllers, we argue for using model-based reinforcement learning (MBRL) trained on relatively small amounts of automatically generated (i.e., without system simulation) data. In this paper, we explore the capabilities of MBRL on a Crazyflie centimeter-scale quadrotor with rapid dynamics to predict and control at <50Hz. To our knowledge, this is the first use of MBRL for controlled hover of a quadrotor using only on-board sensors, direct motor input signals, and no initial dynamics knowledge. Our controller leverages rapid simulation of a neural network forward dynamics model on a GPU-enabled base station, which then transmits the best current action to the quadrotor firmware via radio. In our experiments, the quadrotor achieved hovering capability of up to 6 seconds with 3 minutes of experimental training data.

Keywords: Deep Learning in Robotics and Automation, Visual-Based Navigation, Aerial Systems: Perception and Autonomy
Abstract: Object detection, extended to recognize and localize indoor structural features, is used to enable a quadrotor drone to autonomously navigate through indoor environments. The video stream from a monocular front-facing camera on-board a quadrotor drone is fed to an off-board system that runs a Convolutional Neural Network (CNN) object detection algorithm to identify specific features such as dead-ends, doors, and intersections in hallways. Using pixel-scale dimensions of the bounding boxes around the recognized objects, the distance to intersections, dead-ends and doorways can be estimated accurately using a Support Vector Regression (SVR) model to generate flight control commands for consistent real-time autonomous navigation at flight speeds approaching 2 m/s.

Keywords: Deep Learning in Robotics and Automation
Abstract: Learned Neural Network based policies have shown promising results for robot navigation. However, most of these approaches fall short of being used on a real robot due to the extensive simulated training they require. These simulations lack the visuals and dynamics of the real world, which makes it infeasible to deploy on a real robot. We present a novel Neural Net based policy, NavNet, which allows for easy deployment on a real robot. It consists of two sub policies – a high level policy which can understand real images and perform long range planning expressed in high level commands; a low level policy that can translate the long range plan into low level commands on a specific platform in a safe and robust manner. For every new deployment, the high level policy is trained on an easily obtainable scan of the environment modeling its visuals and layout. We detail the design of such an environment and how one can use it for training a final navigation policy. Further, we demonstrate a learned low-level policy. We deploy the model in a large office building and test it extensively, achieving 0.80 success rate over long navigation runs and outperforming SLAM-based models in the same settings.

Keywords: Learning from Demonstration, Learning and Adaptive Systems
Abstract: During the past few years, probabilistic approaches to imitation learning have earned a relevant place in the literature. One of their most prominent features, in addition to extracting a mean trajectory from task demonstrations, is that they provide a variance estimation. The intuitive meaning of this variance, however, changes across different techniques, indicating either variability or uncertainty. In this paper we leverage kernelized movement primitives (KMP) to provide a new perspective on imitation learning by predicting variability, correlations and uncertainty about robot actions. This rich set of information is used in combination with optimal controller fusion to learn actions from data, with two main advantages: i) robots become safe when uncertain about their actions and ii) they are able to leverage partial demonstrations, given as elementary sub-tasks, to optimally perform a higher level, more complex task. We showcase our approach in a painting task, where a human user and a KUKA robot collaborate to paint a wooden board. The task is divided into two sub-tasks and we show that using our approach the robot becomes compliant (hence safe) outside the training regions and executes the two sub-tasks with optimal gains.

Keywords: Learning from Demonstration, Learning and Adaptive Systems
Abstract: A common approach to learn robotic skills is to imitate a policy demonstrated by a supervisor. One of the existing problems is that, due to the compounding of small errors and perturbations, the robot may leave the states where demonstrations were given. If no strategy is employed to provide a guarantee on how the robot will behave when facing unknown states, catastrophic outcomes can happen. An appealing approach is to use Bayesian methods, which offer a quantification of the action uncertainty given the state. Bayesian methods are usually more computationally demanding and require more complex design choices than their non-Bayesian alternatives, which limits their application. In this work, we present a Bayesian method that is both simple to set up, computationally efficient and that can adapt to a wide range of problems. These advantages make this method very convenient for imitation of robotic manipulation tasks in the continuous domain. We exploit the provided uncertainty to fuse the imitation policy with other policies. The approach is validated on a Panda robot with three tasks using different control input/state pairs.

Keywords: Probability and Statistical Methods, Learning and Adaptive Systems, Deep Learning in Robotics and Automation
Abstract: Humans perceive continuous high-dimensional information by dividing it into significant segments such as words and units of motion. We believe that such unsupervised segmentation is also important for robots to learn topics such as language and motion. To this end, we previously proposed a hierarchical Dirichlet process--Gaussian process--hidden semi-Markov model (HDP-GP-HSMM). However, an important drawback to this model is that it cannot divide high-dimensional time-series data. Further, low-dimensional features must be extracted in advance. Segmentation largely depends on the design of features, and it is difficult to design effective features, especially in the case of high-dimensional data. To overcome this problem, this paper proposes a hierarchical Dirichlet process--variational autoencoder--Gaussian process--hidden semi-Markov model (HVGH). The parameters of the proposed HVGH are estimated through a mutual learning loop of the variational autoencoder and our previously proposed HDP-GP-HSMM. Hence, HVGH can extract features from high-dimensional time-series data, while simultaneously dividing it into segments in an unsupervised manner. In an experiment, we used various motion-capture data to show that our proposed model estimates the correct number of classes and more accurate segments than baseline methods. Moreover, we show that the proposed method can learn latent space suitable for segmentation.

Keywords: Learning from Demonstration, Learning and Adaptive Systems, Motion and Path Planning
Abstract: Stable dynamical systems are a flexible tool to plan robotic motions in real-time. In the robotic literature, dynamical system motions are typically planned without considering possible limitations in the robot’s workspace. This work presents a novel approach to learn workspace constraints from human demonstrations and to generate motion trajectories for the robot that lie in the constrained workspace. Training data are incrementally clustered into different linear subspaces and used to fit a low dimensional representation of each subspace. By considering the learned constraint subspaces as zeroing barrier functions, we are able to design a control input that keeps the system trajectory within the learned bounds. This control input is effectively combined with the original system dynamics preserving eventual asymptotic properties of the unconstrained system. Simulations and experiments on a real robot show the effectiveness of the proposed approach.

Keywords: Learning and Adaptive Systems, Optimization and Optimal Control, Probability and Statistical Methods
Abstract: Enabling robots to act according to human preferences across diverse environments is a crucial task, extensively studied by both roboticists and machine learning researchers. To achieve it, human preferences are often encoded by a reward function which the robot optimizes for. This reward function is generally static in the sense that it does not vary with time or the interactions. Unfortunately, such static reward functions do not always adequately capture human preferences, especially, in non-stationary environments: Human preferences change in response to the emergent behaviors of the other agents in the environment. In this work, we propose learning reward dynamics that can adapt in non-stationary environments with several interacting agents. We define reward dynamics as a tuple of reward functions, one for each mode of interaction, and mode-utility functions governing transitions between the modes. Reward dynamics thereby encodes not only different human preferences but also how the preferences change. Our contribution is in the way we adapt preference-based learning into a hierarchical approach that aims at learning not only reward functions but also how they evolve based on interactions. We derive a probabilistic observation model of how people will respond to the hierarchical queries. Our algorithm leverages this model to actively select hierarchical queries that will maximize the volume removed from a continuous hypothesis space of reward dynamics. We empirically demonstrate reward dynamics can match human preferences accurately.

Keywords: Deep Learning in Robotics and Automation, Neural and Fuzzy Control
Abstract: Fast and precise motion control is important for industrial robots in manufacturing applications. However, some collaborative robots sacrifice precision for safety, particular for high motion speed. The performance degradation is caused by the inability of the joint servo controller to address the uncertain nonlinear dynamics of the robot arm, e.g., due to joint flexibility. We consider two approaches to improve the trajectory tracking performance through feedforward compensation. The first approach uses iterative learning control, with the gradient-based iterative update generated from the robot forward dynamics model. The second approach uses dynamic inversion to directly compensate for the robot forward dynamics. If the forward dynamics is strictly proper or is non-minimum-phase (e.g., due to time delays), its stable inverse would be non-causal. Both approaches require dynamical models. This paper presents results of using recurrent neural networks (RNNs) to approximate these dynamical models – forward dynamics in the first case, inverse dynamics (possibly non-causal) in the second case. We use the bi-directional RNN to capture the noncausality. The RNNs are trained based on a collection of commanded trajectories and the actual robot responses. We use a Baxter robot to evaluate the two approaches. The Baxter robot exhibits significant joint flexibility due to the series-elastic joint actuators. Both approaches achieve sizable improvement over the uncompensated robot motion, for both random joint trajectories and Cartesian motion. The inverse dynamics method is particularly attractive as it may be used to more accurately track a user input as in teleoperation.

Keywords: Manipulation Planning
Abstract: When robots perform manipulation tasks, they need to determine their own movement, as well as how to make and break contact with objects in their environment. Reasoning about the motions of robots and objects simultaneously leads to a constrained planning problem in a high-dimensional state-space. Additionally, when environments change dynamically motions must be computed in real-time.

To this end, we propose a feedback planner for manipulation. We model manipulation as constrained motion and use this model to automatically derive a set of constraint-based controllers. These controllers are used in a switching-control scheme, where the active controller is chosen by a reinforcement learning agent. Our approach is capable of addressing tasks with second-order dynamics, closed kinematic chains, and time-variant environments. We validated our approach in simulation and on a real, dual-arm robot. Extensive simulation of three distinct robots and tasks show a significant increase in robustness compared to a previous approach.

Keywords: Optimization and Optimal Control, AI-Based Methods
Abstract: We present a feedback motion planning algorithm, Bounded-Error LQR-Trees, that leverages reinforcement learning theory to find a policy with a bounded amount of error. The algorithm composes locally valid linear-quadratic regulators (LQR) into a nonlinear controller, similar to how LQR-Trees constructs its policy, but minimizes the cost of the constructed policy by minimizing the Bellman Residual, which is estimated in the overlapping regions of LQR controllers. We prove a sample-based upper bound on the true Bellman Residual, and demonstrate a five-fold reduction in cost over previous methods on a simple underactuated nonlinear system.

Keywords: Intelligent Transportation Systems, Autonomous Agents, Deep Learning in Robotics and Automation
Abstract: In order to drive safely and efficiently under merging scenarios, autonomous vehicles should be aware of their surroundings and make decisions by interacting with other road participants. Moreover, different strategies should be made when the autonomous vehicle is interacting with drivers having different level of cooperativeness. Whether the vehicle is on the merge-lane or main-lane will also influence the driving maneuvers since drivers will behave differently when they have the right-of-way than otherwise. Many traditional methods have been proposed to solve decision making problems under merging scenarios. However, these works either are incapable of modeling complicated interactions or require implementing hand-designed rules which cannot properly handle the uncertainties in real-world scenarios. In this paper, we proposed an interaction-aware decision making with adaptive strategies (IDAS) approach that can let the autonomous vehicle negotiate the road with other drivers by leveraging their cooperativeness under merging scenarios. A single policy is learned under the multi-agent reinforcement learning (MARL) setting via the curriculum learning strategy, which enables the agent to automatically infer other drivers' various behaviors and make decisions strategically. A masking mechanism is also proposed to prevent the agent from exploring states that violate common sense of human judgment and increase the learning efficiency. An exemplar merging scenario was used to implement and examine the proposed method.

Keywords: Micro/Nano Robots, Biologically-Inspired Robots, Aerial Systems: Mechanics and Control
Abstract: We introduce Bee⁺, a 95-mg four-winged microrobot with improved controllability and open-loop-response characteristics with respect to those exhibited by state-of-the-art two-winged microrobots with the same size and similar weight (i.e., the 75-mg Harvard RoboBee). The key innovation that made possible the development of Bee⁺ is the introduction of an extremely light (28-mg) pair of twinned unimorph actuators, which enabled the design of a new microrobotic mechanism that flaps four wings independently. A first main advantage of the proposed design, compared to those of two-winged flyers, is that by increasing the number of actuators from two to four, the number of direct control inputs increases from three to four when simple sinusoidal excitations are employed. A second advantage of Bee⁺ is that its four-wing configuration and flapping mode naturally damp the rotational disturbances that commonly affect the yaw degree of freedom of two-winged microrobots. In addition, the proposed design greatly reduces the complexity of the associated fabrication process compared to those of other microrobots, as the unimorph actuators are fairly easy to build. Lastly, we hypothesize that given the relatively low wing-loading affecting their flapping mechanisms, the life expectancy of Bee⁺s must be considerably higher than those of the two-winged counterparts. The functionality and basic capabilities of the robot are demonstrated through a set of simple control experiments.

Keywords: Robot Safety, Autonomous Vehicle Navigation, Intelligent Transportation Systems
Abstract: In a context of autonomous robots, one of the most important tasks is to ensure the safety of the robot and its surrounding. The risk of navigation is usually said to be the probability of collision. This notion of risk is not well defined in the literature, especially when dealing with occupancy grids. The Bayesian occupancy grid is the most used method to deal with complex environments. However, this is not fitted to compute the risk along a path by its discrete nature. In this article, we present a new way to store the occupancy of the environment that allows the computation of risk along a given path. We then define the risk as the force of collision that would occur for a given obstacle. Using this framework, we are able to generate navigation paths ensuring the safety of the robot.

Keywords: Robot Safety, Industrial Robots, Optimization and Optimal Control
Abstract: Future manufacturing environments will see an increased need for cooperation between humans and machines. In this paper we propose a method that allows industrial manipulators to safely operate around humans. This approach guarantees that the manipulator will never collide with human operators while performing its normal tasks. This is done in an near-optimal way by considering how forward reachable sets of human operators grow with time, and by continuously updating these reachable sets based on current position estimates of the operators near the robot. An implicit active set invariance filter is then used to constrain the system---in a minimally invasive way---to stay in the complement of that forward reachable set. We demonstrate this approach in simulation on an industrial robotic arm: the ABB IRB 6640.

Keywords: Robot Safety, Networked Robots, Software, Middleware and Programming Environments
Abstract: Event data recording is crucial in robotics research, providing prolonged insights into a robot's situational understanding, progression of behavioral state, and resulting outcomes. Such recordings are invaluable when debugging complex robotic applications or profiling experiments ex post facto. As robotic developments mature into production, both the roles and requirements of event logging will broaden, to include serving as evidence for auditors and regulators investigating accidents or fraud. Given the growing number of high profile public incidents involving self-driving automotives resulting in fatality and regulatory policy making, it is paramount that the integrity, authenticity and non-repudiation of such event logs are maintained to ensure accountability. Being mobile cyber-physical systems, robots present new threats and vulnerabilities beyond traditional IT: unsupervised physical system access or postmortem collusion between robot and OEM could result in the truncation or alteration of prior records. In this work, we address immutablization of log records via integrity proofs and distributed ledgers with special considerations for mobile and public service robot deployments.

Keywords: Robot Safety, Performance Evaluation and Benchmarking, Computer Vision for Other Robotic Applications
Abstract: In this paper, we present the first large-scale synthetic dataset for visual perception in disaster scenarios, and analyze state-of-the-art methods for multiple computer vision tasks with reference baselines. We simulated before and after disaster scenarios such as fire and building collapse for fifteen different locations in realistic virtual worlds. The dataset consists of more than 300K high-resolution stereo image pairs, all annotated with ground-truth data for semantic segmentation, depth, optical flow, surface normal estimation and camera pose estimation. To create realistic disaster scenes, we manually augmented the effects with 3D models using physical-based graphics tools. We use our dataset to train state-of-the-art methods and evaluate how well these methods can recognize the disaster situations and produce reliable results on virtual scenes as well as real-world images. The results obtained from each task are then used as inputs to the proposed visual odometry network for generating 3D maps of buildings on fire. Finally, we discuss challenges for future research.

Keywords: Robot Safety, Physical Human-Robot Interaction, Gripper and Other End-Effectors
Abstract: In the practical application of robots, usually certain tools are attached to the robot to manipulate objects and carry out various tasks. Besides adding gravitational load to the robot that results in large joint torques, such payloads may influence the collision safety characteristics through changing surface curvature properties, effective mass and robot speed along a given direction of motion. In this paper, we evaluate the contribution of a known, unactuated payload that is attached to the robot end effector to the robot inertial parameters and maximum task velocity. The proposed mass update approach relies on the analysis of the kinetic energy matrices. The velocity maximization is tackled by formulating static optimization problems with different constraints on angular motion of the end-effector. Finally, we present simulation results with a PUMA 560 robot that has an exemplary payload attached to its end-effector, which enables us to check the validity and accuracy of the developed methods. We further show the application for updating the effective mass and obtaining velocity optimization solutions in the context of Safety Maps and also via different geometrical visualizations.

Keywords: Robot Safety, Product Design, Development and Prototyping, Performance Evaluation and Benchmarking
Abstract: In production line, the conventional industrial robots and automatic machines require machinery safety protection to guarantee the safety of human operators. A scalable and easy-to-configure safety system concept called “Real-time Safety Virtual Positioning” (RSVP) is proposed, which could act as potentially key enabler for agile production systems by eliminating fixed safety installation and thus increasing productivity and flexibility. The RSVP provides easy access to robots of automatic assembly lines in plants, e.g., automotive OEM, and supports virtualization and transparent to fully automatic and semi-automatic assembly line by knowing the position of persons and relevant objects (e.g., tools, finished and/or semi-finished goods, and materials). The focus of the paper will discuss the functional safety certification realization (concept approved by TÜV (Technical Inspection Association)) and validation details of indoor localization-based safety system developments. The detailed strategy of the functional safety requirements, danger diagnosis and reaction approach, communication among safety controller, robot and machine, and safe failure reaction are listed and analyzed. The physical hardware framework and software architecture of the safety system are built and developed with the 1oo2-architecture according to the Performance Levels (PL d) of ISO 13849 and the Safety Integrity Levels (SIL 2) of IEC 61508. The implicated algorithms and data process of the UWB-based (Ultra-Wide Band) indoor localization system are introduced. The safety system concept is validated and verified in an ABS (Anti-lock Braking System) production line with human-robot co-existence environment.

Keywords: Aerial Systems: Mechanics and Control, Aerial Systems: Applications
Abstract: Winged aerial robots represent an evolution of aerial manipulation robots, replacing the multirotor vehicles by fixed or flapping wing platforms. The development of this morphology is motivated in terms of efficiency, endurance and safety in some inspection operations where multirotor platforms may not be suitable. This paper presents a first prototype of compliant dual arm as preliminary step towards the realization of a winged aerial robot capable of perching and manipulating with the wings folded. The dual arm provides 6 DOF (degrees of freedom) for end effector positioning in a human-like kinematic configuration, with a reach of 25 cm (half-scale w.r.t. the human arm), and 0.2 kg weight. The prototype is built with micro metal gear motors, measuring the joint angles and the deflection with small potentiometers. The paper covers the design, electronics, modeling and control of the arms. Experimental results in test-bench validate the developed prototype and its functionalities, including joint position and torque control, bimanual grasping, the dynamic equilibrium with the tail, and the generation of 3D maps with laser sensors attached at the arms.

Keywords: Aerial Systems: Applications, Aerial Systems: Mechanics and Control, Contact Modelling
Abstract: Glass curtain walls have been widely used in modern architecture. This makes it urgent to inspect and clean these glasses at regular intervals. Up to now, most of these work is performed by workers, which is expensive and inefficient. Therefore, a novel robot—the contact aerial manipulator system—is developed. The new designed system presents priorities in the aspects of high flexibility and easy operation. In this paper, the system mechanical structure is first introduced. Subsequently, the hybrid force/motion control framework is utilized to realize the precise and steady motion on the two-dimensional plane and maintain a certain sustained contact force, simultaneously. Finally, two flight experiments (including continuous square-wave trajectory tracking and aerial drawing task) are performed and the results indicate that the developed contact aerial manipulator works and presents good performance.

Keywords: Aerial Systems: Applications, Aerial Systems: Mechanics and Control
Abstract: Transformable aerial robots are favorable in aerial manipulation tasks for their flexible ability to change configuration during the flight. By assuming robot keeping in the mild motion, the previous researches sacrifice aerial agility to simplify the complex non-linear system into a single rigid body with a linear controller. In this paper, we present a framework towards agile swing motion for the transformable multi-links aerial robot. We introduce a computational-efficient non-linear model predictive controller and joints motion primitive framework to achieve agile transforming motions and validate with a novel robot named HYRURS-X. Finally, we implement our framework under a table tennis task to validate the online and agile performance.

Keywords: Aerial Systems: Applications, Aerial Systems: Perception and Autonomy
Abstract: The use of drones for aerial cinematography has revolutionized several applications and industries that require live and dynamic camera viewpoints such as entertainment, sports, and security. However, safely controlling a drone while filming a moving target usually requires multiple expert human operators; hence the need for an autonomous cinematographer. Current approaches have severe real-life limitations such as requiring fully scripted scenes, high-precision motion-capture systems or GPS tags to localize targets, and prior maps of the environment to avoid obstacles and plan for occlusion.

In this work, we overcome such limitations and propose a complete system for aerial cinematography that combines: (1) a vision-based algorithm for target localization; (2) a real-time incremental 3D signed-distance map algorithm for occlusion and safety computation; and (3) a real-time camera motion planner that optimizes smoothness, collisions, occlusions and artistic guidelines. We evaluate robustness and real-time performance in series of field experiments and simulations by tracking dynamic targets moving through unknown, unstructured environments. Finally, we verify that despite removing previous limitations, our system achieves state-of-the-art performance.

Keywords: Aerial Systems: Applications, Agricultural Automation, Robotics in Agriculture and Forestry
Abstract: This paper describes a computationally-enhanced M100 UAV platform with an onboard deep learning inference system for integrated computer vision and navigation. The system is able to autonomously find and visually identify by coat pattern individual Holstein Friesian cattle in freely moving herds. We propose an approach that utilises three deep convolutional neural network architectures running live onboard the aircraft: (1) a YOLOv2-based species detector, (2) a dual-stream deep network delivering exploratory agency, and (3) an InceptionV3-based biometric long-term recurrent convolutional network for individual animal identification. We evaluate the performance of each of the components offline, and also online via real-world field tests comprising 147 minutes of autonomous low altitude flight in a farm environment over a dispersed herd of 17 heifer dairy cows. We report error free identification performance on this online experiment. The presented proof-of-concept system is the first of its kind. It represents a practical step towards autonomous biometric identification of individual animals from the air in open pasture environments for tag-less AI support in farming and ecology.

Keywords: Aerial Systems: Applications, Computer Vision for Other Robotic Applications, Recognition
Abstract: State-of-the-art methods for counting people in crowded scenes rely on deep networks to estimate crowd density in the image plane. While useful for this purpose, this image- plane density has no immediate physical meaning because it is subject to perspective distortion. This is a concern in sequences acquired by drones because the viewpoint changes often. This distortion is usually handled implicitly by either learning scale- invariant features or estimating density in patches of different sizes, neither of which accounts for the fact that scale changes must be consistent over the whole scene. In this paper, we explicitly model the scale changes and reason in terms of people per square-meter. We show that feeding the perspective model to the network allows us to enforce global scale consistency and that this model can be obtained on the fly from the drone sensors. In addition, it also enables us to enforce physically-inspired temporal consistency constraints that do not have to be learned. This yields an algorithm that outperforms state-of-the-art methods in inferring crowd density from a moving drone camera especially when perspective effects are strong.

Keywords: Computer Vision for Other Robotic Applications, Object detection, segmentation, categorization, Autonomous Vehicle Navigation
Abstract: Current perception systems mostly require direct line of sight to anticipate and ultimately prevent potential collisions at intersections with other road users. We present a fully integrated autonomous system capable of detecting shadows or weak illumination changes on the ground caused by a dynamic obstacle in NLoS scenarios. This additional virtual sensor ``ShadowCam'' extends the signal range utilized so far by computer-vision ADASs. We show that (1) our algorithm maintains the mean classification accuracy of around 70% even when it doesn't rely on infrastructure -- such as AprilTags -- as an image registration method. We validate (2) in real-world experiments that our autonomous car driving in night time conditions detects a hidden approaching car earlier with our virtual sensor than with the front facing 2-D LiDAR.

Keywords: Computer Vision for Other Robotic Applications, RGB-D Perception, Cognitive Human-Robot Interaction
Abstract: Action recognition has wide applications in assisted living, health monitoring, surveillance, and human-computer interaction. In traditional action recognition methods, RGB video-based ones are effective but computationally inefficient, while skeleton-based ones are computationally efficient but do not make use of low-level detail information. This work considers action recognition based on a multimodal fusion between the 3D skeleton and the RGB image. We design a neural network that uses a 3D skeleton sequence and a single middle frame from an RGB video as input. Specifically, our method picks up one frame in a video and extracts spatial features from it using two attention modules, a self-attention module, and a skeleton-attention module. Further, temporal features are extracted from the skeleton sequence via a BI-LSTM subnetwork. Finally, the spatial features and the temporal features are combined via a feature fusion network for action classification. A distinct feature of our method is that it uses only a single RGB frame rather than an RGB video. Accordingly, it has a light-weighted architecture and is more efficient than RGB video-based methods. Comparative evaluation on two public datasets, NTU-RGBD and SYSU, demonstrates that our method can achieve competitive performance compared with state-of-the-art methods.

Keywords: Computer Vision for Other Robotic Applications, Visual Learning, Aerial Systems: Perception and Autonomy
Abstract: We present an end-to-end deep learning approach for performing metric scale-sensitive regression tasks such visual odometry with a single camera and no additional sensors. We propose a novel 3D convolutional architecture, 3DC-VO, that can leverage temporal relationships over a short moving window of images to estimate linear and angular velocities. The network makes local predictions on stacks of images that can be integrated to form a full trajectory. We apply 3DC-VO to the KITTI visual odometry benchmark and the task of estimating a pilot's control inputs from a first-person video of a quadrotor flight. Our method exhibits increased accuracy relative to comparable learning-based algorithms trained on monocular images. We also show promising results for quadrotor control input prediction when trained on a new dataset collected with a UAV simulator.

Keywords: Computer Vision for Transportation, Computer Vision for Automation
Abstract: Recognizing abnormal events such as traffic violations and accidents in natural driving scenes is essential for successful autonomous driving and advanced driver assistance systems. However, most work on video anomaly detection suffers from two crucial drawbacks. First, they assume cameras are fixed and videos have static backgrounds, which is reasonable for surveillance applications but not for vehicle-mounted cameras. Second, they pose the problem as one-class classification, relying on arduously hand-labeled training datasets that limit recognition to anomaly categories that have been explicitly trained. This paper proposes an unsupervised approach for traffic accident detection in first-person (dashboard-mounted camera) videos. Our major novelty is to detect anomalies by predicting the future locations of traffic participants and then monitoring the prediction accuracy and consistency metrics with three different strategies. We evaluate our approach using a new dataset of diverse traffic accidents, AnAn Accident Detection (A3D), as well as another publicly-available dataset. Experimental results show that our approach outperforms the state-of-the-art. Code and the dataset developed in this work are available at: https://github.com/MoonBlvd/tad-IROS2019

Keywords: Computer Vision for Other Robotic Applications, Computer Vision for Automation, Deep Learning in Robotics and Automation
Abstract: In this paper, we present a multi-camera sensor system along with its control algorithm for automated visual inspection from a moving vehicle. To accomplish this task, we propose a unique hardware configuration consisting of a frontal stereo vision system, six lateral cameras motorized to tilt, and a GPS/IMU sensor mounted on the roof of a car. From the frontal stereo system, we detect electric poles and estimate their corresponding 3D positions. Based on this 3D estimation, the tilt angles of the motorized lateral cameras are controlled in real-time to capture high resolution images of the equipment - typically installed a few meters above the road surface. In addition, inertial odometry information from the GPS/IMU module is utilized for pose estimation, object localization, and re-identification among cameras. Experimental results demonstrate the efficiency and robustness of our system for automated electric equipment maintenance, which can reduce human effort significantly.

Keywords: Computer Vision for Other Robotic Applications, Deep Learning in Robotics and Automation, Computational Geometry
Abstract: Three-dimensional (3D) point cloud processing has attracted a great deal of attention in computer vision, robotics, and the machine learning community because of significant progress in deep neural networks on 3D data. Another trend in the community is learning of generative models based on generative adversarial networks. In this paper, we propose a framework for 3D point cloud generation based on a combination of auto-encoders and generative adversarial networks. The framework first trains auto-encoders to learn latent representations, and then trains generative adversarial networks in the learned latent space. We focus on improving the training method for auto-encoders in order to generate 3D point clouds with higher fidelity and coverage. We add parallel sub-decoders that reconstruct subsets of the input point cloud. In order to construct these subsets, we introduce a point sampling algorithm that imposes a method to sample spatially localized point sets. These local subsets are utilized to measure local reconstruction losses. We train auto-encoders to learn an effective latent representation for both global and local shape reconstruction based on the multi-task learning strategy. Furthermore, we add global and local adversarial losses to generate more plausible point clouds. Quantitative and qualitative evaluations demonstrate that the proposed method outperforms state-of-the-art method on the task of 3D point cloud generation.

Keywords: Autonomous Vehicle Navigation, Deep Learning in Robotics and Automation, AI-Based Methods

Keywords: Social Human-Robot Interaction, Agent-Based Systems
Abstract: When disagreements arise in hierarchical relationships, relationship members sometimes prefer conflict management strategies that avoid or quickly end the overt conflict even if the relationship is left in a state of dissatisfaction. Our lab has proposed that a peripheral robotic agent may be able to support these types of relationships during conflict. In this paper, we present the results of an IRB- approved human-robot interaction study that examines how the members of a hierarchical relationship involved in conflict respond to the presence of an unengaged robot. This study serves as a baseline for additional studies. The unengaged robot appears to have a minimal influence on the interaction. The observed conflicts followed the patterns typically described in mediation literature. Our lab previously proposed a computational model to identify weakness and alienation in these relationships. We discuss a partial implementation of this model, and its ability to recognize problems in certain relationships within the data collected. Based on our observations, and the performance of the model’s partial implementation, we suggest considerations that need to be made for an intervening robotic agent.

Keywords: Social Human-Robot Interaction, Cognitive Human-Robot Interaction, Learning and Adaptive Systems
Abstract: In socially assistive robotics, an important research area is the development of adaptation techniques and their effect on human-robot interaction. We present a meta-learning based policy gradient method for addressing the problem of adaptation in human-robot interaction and also investigate its role as a mechanism for trust modelling. By building an escape room scenario in mixed reality with a robot, we test our hypothesis that bi-directional trust can be influenced by different adaptation algorithms. We found that our proposed model increased the perceived trustworthiness of the robot and influenced the dynamics of gaining human's trust. Additionally, participants evaluated that the robot perceived them as more trustworthy during the interactions with the meta-learning based adaptation compared to the previously studied statistical adaptation model.

Keywords: Social Human-Robot Interaction, Cognitive Human-Robot Interaction, Semantic Scene Understanding
Abstract: The increasing use of autonomous mobile robots in different parts of society, and not restricted only to industrial environments, makes it important to propose techniques that will allow them to behave in the most socially acceptable way as possible. In most real-world scenarios, individuals in the environment are interacting with each other and are arranged into groups. Therefore, it is paramount the proposition of techniques to efficiently and correctly identify and represent such groups. This information can be useful in different tasks such as approaching and initiating an interaction, escorting, and the navigation itself. In this work, we propose a novel graph-based approach to evaluate the possible association of individuals in the environment based on their position and body orientation. Next, based on this association, we propose a representation of the combined social space of individuals in the same group. The methodology was evaluated using synthetic and real-world datasets, showing that it achieves results comparable to or better than the state-of-the-art.

Keywords: Social Human-Robot Interaction, Cognitive Human-Robot Interaction, Service Robots
Abstract: Human has an ability to adjust utterance depending on the state of interlocutor. In this study, we explore the verbal behaviors of human through interaction with two smart speakers that have different level of task competence. We analyzed (1) linguistic behaviors appeared in user’s utterance, (2) length of the uttered speech, and (3) required pragmatics skills to understand the user’s intent. As a result, there were no significant difference in linguistic behaviors and length of the speech while user interacts with speakers with different task competence. In addition, various pragmatics elements were equally utilized and especially, implied intentions were frequently observed in user’s short utterance even under simple interaction scenarios.

Keywords: Social Human-Robot Interaction, Entertainment Robotics, Cognitive Human-Robot Interaction
Abstract: This paper consists of a failure analysis of two robot performance productions. Both included three-week rehearsal periods, and culminated in live performances including both robots and humans. To develop these productions, a theater artist collaborated with a robotics lab to develop, (1) a narrative dance performed live on stage, and, (2) an improvisational performance in a public space. While the interdisciplinary team did not set out to explore robot failures, during the eighteen rehearsals and two live performances, failures played an ever-present role. This paper presents the technical and choreographic failures encountered, and details strategies for addressing, planning for, and rehearsing responses to robot failures on stage. In addition to scaffolding future robot theater performances, we discuss how these strategies apply to other customer- and audience-facing robots, including sponsor demos. The on-stage exploration of robot chairs and human performers also suggests that humans can conceptualize minimal robots as both characters and props, moving fluidly from one to the other. We hope these insights ensure that future audiences will *want* the robot shows to go on, as well as expand ideas about the types of robots that can be cast in future human-robot productions.

Keywords: Social Human-Robot Interaction
Abstract: We expand on previous work for negotiating human-robot navigation contention around doorways to produce a more socially-compliant autonomous robot behaviour. Our goal is to improve the integration of robots navigating in human environments by eliciting human recognition of the robot's right of way. This is achieved by incorporating feedback from a user study of our previous system to create a more communicative, reciprocal, and assertive behaviour. Our contribution includes both the updated behaviour and a new user study that evaluates and compares the system to its predecessor. Results show that participants are more likely to respect the robot's right of way given the new robot behaviour, but their responses also highlight the challenges of socially integrating robots into human spaces.

Keywords: SLAM, Autonomous Vehicle Navigation
Abstract: Mobile robots can be considered completely autonomous if they embed active algorithms for Simultaneous Localization And Mapping (SLAM). This means that the robot is able to autonomously, or actively, explore and create a reliable map of the environment, while simultaneously estimating its pose. In this paper, we propose a novel framework to robustly solve the active SLAM problem, in scenarios in which some prior information about the environment is available in the form of a topo-metric graph. This information is typically available or can be easily developed in industrial environments, but it is usually affected by uncertainties. In particular, the distinguishing features of our approach are: the inclusion of prior information for solving the active SLAM problem; the exploitation of this information to pursue active loop closure; the on-line correction of the inconsistencies in the provided data. We present some experiments, that are performed in different simulated environments: the results suggest that our method improves on state-of-the-art approaches, as it is able to deal with a wide variety of possibly large uncertainties.

Keywords: SLAM, Localization, Autonomous Vehicle Navigation
Abstract: Localization is a key capability for autonomous vehicles. High-Definition maps are a popular method to represent the environment and to enable precise localization. However, the creation is very demanding and it is not always guaranteed to receive accurate map information especially for unstructured areas. In this paper, we introduce a novel probabilistic localization and mapping framework that brings together the advantages of sparse feature maps, multi-target tracking for landmark detection, probabilistic global vehicle localization and a graph-based formulation to achieve a consistent map. The front-end of our Simultaneous Localization and Mapping (SLAM) framework is based on Monte Carlo Localization (MCL). Our novel measurement model integrates a virtual topological Path-Map with sparse map features to obtain global localization. The graph-based back-end optimizes online the vehicle trajectory and the landmarks’ configuration to create a globally aligned map. Furthermore, our method allows weaker requirements in terms of accuracy of the sparse feature map as we represent the degree of uncertainty by means of probabilistic distribution. Additionally, the sparse map representation needs substantially less memory than other approaches, which is an advantage for autonomous vehicles. The framework has been tested and evaluated in real experiments for several autonomous runs. The results demonstrate the robustness of our system.

Keywords: SLAM, Localization, Mapping
Abstract: In this paper, we present the RISE-SLAM algorithm for performing visual-inertial simultaneous localization and mapping (SLAM), while improving estimation consistency. Specifically, in order to achieve real-time operation, existing approaches often assume previously-estimated states to be perfectly known, which leads to inconsistent estimates. Instead, based on the idea of the Schmidt-Kalman filter, which has processing cost linear in the size of the state vector but quadratic memory requirements, we derive a new consistent approximate method in the information domain, which has linear memory requirements and adjustable (constant to linear) processing cost. In particular, this method, the resource-aware inverse Schmidt estimator (RISE), allows trading estimation accuracy for computational efficiency. Furthermore, and in order to better address the requirements of a SLAM system during an exploration vs. a relocalization phase, we employ different configurations of RISE (in terms of the number and order of states updated) to maximize accuracy while preserving efficiency. Lastly, we evaluate the proposed RISE-SLAM algorithm on publicly-available datasets and demonstrate its superiority, both in terms of accuracy and efficiency, as compared to alternative visual-inertial SLAM systems.

Keywords: SLAM, Localization, Mapping
Abstract: A Simultaneous Localization and Mapping (SLAM) system is a complex program consisting of several interconnected components with different functionalities such as optimization, tracking or loop detection. Whereas the literature addresses in detail how enhancing the algorithmic aspects of the individual components improves SLAM performance, the modal aspects, such as when to localize, relocalize or close a loop, are usually left aside. In this paper, we address the modal aspects of a SLAM system and show that the design of the modal controller has a strong impact on SLAM performance in particular in terms of robustness against unforeseen events such as sensor failures, perceptual aliasing or kidnapping. We preset a novel taxonomy for the components of a modern SLAM system, investigate their interplay and propose a highly modular architecture of a generic SLAM system using the Unified Modeling Language (UML) state machine formalism. The result, called SLAM state machine, is compared to the modal controller of several state-of-the-art SLAM systems and evaluated in two experiments. We demonstrate that our state machine handles unforeseen events much more robustly than the state-of-the-art systems.

Keywords: SLAM, Localization, Mapping
Abstract: Simultaneous Localization and Mapping (SLAM) is a fundamental task to mobile and aerial robotics. LiDAR based systems have proven to be superior compared to vision based systems due to its accuracy and robustness. In spite of its superiority, pure LiDAR based systems fail in certain degenerate cases like traveling through a tunnel. We propose Stereo Visual Inertial LiDAR (VIL) SLAM that performs better on these degenerate cases and has comparable performance on all other cases. VIL-SLAM accomplishes this by incorporating tightly-coupled stereo visual inertial odometry (VIO) with LiDAR mapping and LiDAR enhanced visual loop closure. The system generates loop-closure corrected 6-DOF LiDAR poses in real-time and 1cm voxel dense maps near real-time. VIL-SLAM demonstrates improved accuracy and robustness compared to state-of-the-art LiDAR methods.

Keywords: SLAM, Localization, Recognition
Abstract: Visual loop closure detection, which can be considered as an image retrieval task, is an important problem in SLAM (Simultaneous Localization and Mapping) systems. The frequently used bag-of-words (BoW) models can achieve high precision and moderate recall. However, the requirement for lower time costs and fewer memory costs for mobile robot applications is not well satisfied. In this paper, we propose a novel loop closure detection framework titled "FILD" (Fast and Incremental Loop closure Detection), which focuses on an on-line and incremental graph vocabulary construction for fast loop closure detection. The global and local features of frames are extracted using the Convolutional Neural Networks (CNN) and SURF on the GPU, which guarantee extremely fast extraction speeds. The graph vocabulary construction is based on one type of proximity graph, named Hierarchical Navigable Small World (HNSW) graphs, which is modified to adapt to this specific application. In addition, this process is coupled with a novel strategy for real-time geometrical verification, which only keeps binary hash codes and significantly saves on memory usage. Extensive experiments on several publicly available datasets show that the proposed approach can achieve fairly good recall at 100% precision compared to other state-of-the-art methods. The source code can be downloaded at https://github.com/AnshanTJU/FILD for further studies.

Keywords: Medical Robots and Systems, Telerobotics and Teleoperation, Mechanism Design
Abstract: This paper describes the design, manufacture, and performance of a highly dexterous, low-profile, 7 Degree-of-Freedom (DOF) robotic arm for CT-guided percutaneous needle biopsy. Direct CT guidance allows physicians to localize tumours quickly; however, needle insertion is still performed by hand. This system is mounted to a fully active gantry superior to the patient's head and teleoperated by a radiologist. Unlike other similar robots, this robot's fully serial-link approach uses a unique combination of belt and cable drives for high-transparency and minimal-backlash, allowing for an expansive working area and numerous approach angles to targets all while maintaining a small in-bore cross-section less than 16cm^2. Simulations verified the system's expansive collision free work-space and ability to hit targets across the entire chest, as required for lung cancer biopsy. Targeting error is on average <1mm on a teleoperated accuracy task, illustrating the system's sufficient accuracy to perform biopsy procedures. The system is designed for lung biopsies due to the large working volume that is required for reaching peripheral lung lesions, though, with its large working volume and small in-bore cross-sectional area, the robotic system is effectively a general-purpose CT-compatible manipulation device for percutaneous procedures. Finally, with the considerable development time undertaken in designing a precise and flexible-use system and with the desire to reduce the burden of other researchers in developing algorithms for image-guided surgery, this system provides open-access, and to the best of our knowledge, is the first open-hardware image-guided biopsy robot of its kind.

Keywords: Medical Robots and Systems, Telerobotics and Teleoperation
Abstract: Accurate master-slave control is important for Robot-Assisted Microsurgery (RAMS). This paper presents a handheld master controller for the operation and training of RAMS. A 9-axis Inertial Measure Unit (IMU) and a micro camera are utilized to form the sensing system for the handheld controller. A new hybrid marker pattern is designed to achieve reliable visual tracking, which integrated QR codes, Aruco markers, and chessboard vertices. Real-time multi-sensor fusion is implemented to further improve the tracking accuracy. The proposed handheld controller has been verified on an in-house microsurgical robot to assess its usability and robustness. User studies were conducted based on a trajectory following task, which indicated that the proposed handheld controller had comparable performance with the Phantom Omni, demonstrating its potential applications in microsurgical robot control and training.

Keywords: Surgical Robotics: Laparoscopy, Haptics and Haptic Interfaces, Optimization and Optimal Control
Abstract: In Minimally Invasive Surgery (MIS), the incision point acts as a fulcrum about which the surgical instrument pivots. Most robotic MIS systems foresee procedures to carefully align the robot with the incision in the patient. Such procedures disrupt the normal workflow. This hampers the clinical translation of such assistive technology. In contrast, Backdrivable, Wristed (BW) robots, especially when operated as comanipulation devices, have a minimal impact on the traditional surgical workflow in MIS, as they are readily equipped with algorithms that automate fulcrum point estimation.

To improve safety, accuracy and/or task completion time, virtual fixtures can be implemented at the distal instrument tip, using an assistive BW system. This letter formalizes the problems related to such distal virtual fixtures. It then develops a hybrid active/passive wrist control strategy to improve the performance of virtual fixtures with BW systems while minimizing stability issues. Aside from an analytical example, experimental validation is added to show that the new algorithm increases the achievable stiffness of distal virtual fixtures.

Keywords: Surgical Robotics: Laparoscopy, Mechanism Design, Robot Safety
Abstract: For safety enhancement reasons, passive gravity compensation is widely applied in robotic systems used in minimally invasive surgery (MIS). MIS robotic systems have a remote center of motion (RCM) in which a surgical instrument conducts a fulcrum motion around a point of invasion. RCM mechanisms include three-degrees-of-freedom (3-DoF): roll, pitch, and translation. Existing studies to date have focused on multi-degrees-of-freedom (MDoF) gravity compensation mechanisms by installing springs and wires in a robot. However, a gravity compensation mechanism with 3-DoF that simultaneously uses all three directional movements (roll, pitch, and translation) has not yet been researched. Here, we propose a novel gravity compensation mechanism applicable to a 3-DoF MIS robotic arm with an RCM mechanism. When a translational motion is exerted, the proposed gravity compensator can adjust the roll-pitch-directional compensating torque by utilizing a reduction gear box and wire cable. To verify the 3-DoF gravity compensation, a gravity-compensated robotic arm for MIS and customized torque sensors were manufactured and calibrated. Results showed the proposed static balancing mechanism can compensate for the gravitational torque with respect to roll, pitch, and translation. The total torque error along the roll and pitch axis was less than 0.38 Nm. In particular, the torque variation due to the translational motion was less than 0.13 Nm.

Keywords: Surgical Robotics: Laparoscopy, Medical Robots and Systems
Abstract: Master manipulators represent a key component of Robot-Assisted Minimally Invasive Surgery (RAMIS). In this paper, an Analytic Hierarchy Process (AHP) method is used to construct an effective workspace analysis framework, which can assist the configuration selection and design evaluation of a portable master manipulator for surgical robot control and training. The proposed framework is designed based on three criteria: 1) compactness, 2) workspace quality, and 3) mapping efficiency. A hardware prototype, called the Hamlyn Compact Robotic Master (Hamlyn CRM), is constructed following the proposed framework. Experimental verification of the platform is conducted on the da Vinci Research Kit (dVRK) with which a da Vinci robot is controlled as a slave. The proposed CRM is compared with Phantom Omni, a commercial portable master device, with results demonstrating the relative merits of the new platform in terms of task completion time, average control speed and number of clutching.

Keywords: Surgical Robotics: Laparoscopy, Product Design, Development and Prototyping, Multi-Robot Systems
Abstract: Minimally invasive surgery is now a well established field in surgery but continuous efforts are made to reduce invasiveness even further. This paper proposes a novel concept of small-diameter multi-arm instrument for Single-Port Access Surgery. The concept introduces a novel combination of backbone and actuation principles in a macro-micro fashion to achieve an excellent decoupling of the triangulation platform (macro) and of the end-effectors (micro). Concentric tube robots are used for the triangulation platform, while compliant fluidic-actuated bending segments are used as end-effectors. The fluidic actuation is advantageous as it minimally interferes with the triangulation platform. The triangulation platform on the other hand provides a stable base for the end-effectors such that large distal actuation bandwidth can be achieved. A specific embodiment for Spina Bifida repair is developed and proposed. The surgical and technical requirements as well as the mechanical design are presented in details. A first prototype is built and characterization experiments are conducted to evaluate its performance.

Keywords: Human Detection and Tracking, Human Factors and Human-in-the-Loop, Task Planning
Abstract: In collaborative robotics applications, the human behaviour is a major source of uncertainty. Predicting the evolution of the current human activity might be beneficial to the effectiveness of task planning, as it enables a higher level of coordination of robot and human activities. This paper addresses the problem of monitoring the advancement of human tasks in real-time giving an estimate of their expected duration. The proposed method relies on dynamic time warping to align the current activity with a reference template. No training phase is required, as the prototypical execution is learnt online from previous instances of the same activity. The applicability and performance of the method within an industrial context have been verified on a realistic assembly task.

Keywords: Human Detection and Tracking, RGB-D Perception, Deep Learning in Robotics and Automation
Abstract: An essential feature for navigating socially with a mobile robot is the upper body orientation of persons in its vicinity. For example, in a supermarket orientation indicates whether a person is looking at goods on the shelves or where a person is likely to go. However, given limited computing and battery capabilities, it is not possible to rely on high- performance graphics cards to run large, computationally expensive deep neural networks for orientation estimation in real time. Nevertheless, deep learning performs quite well for regression problems. Therefore, we tackle the problem of upper body orientation estimation with small yet efficient deep neural networks on a mobile robot in this paper. We employ a fast person detection approach as preprocessing that outputs fixed size person images before the actual estimation of the orientation is done. The combination with lightweight networks allows us to estimate a continuous angle in real time, even using a CPU only. We experimentally evaluate the performance of our system on a new, self-recorded data set consisting of more than 100,000 RGB-D samples from 37 persons, which is made publicly available. We also do an extensive comparison of different network architectures and output encodings for their applicability in estimating orientations. Furthermore, we show that depth images are more suitable for the task of orientation estimation than RGB images or the combination of both.

Keywords: Human Detection and Tracking, Robot Safety, Human-Centered Robotics
Abstract: In this work, we present a novel non-visual HAR system that achieves state-of-the-art performance on realistic SCE tasks via a single wearable sensor. We leverage surface electromyography and inertial data from a low-profile wearable sensor to attain performant robot perception while remaining unobtrusive and user-friendly. By capturing both convolutional and temporal features with a hybrid CNN-LSTM classifier, our system is able to robustly and effectively classify complex, full-body human activities with only this single sensor. We perform a rigorous analysis of our method on two datasets representative of SCE tasks, and compare performance with several prominent HAR algorithms. Results show our system substantially outperforms rival algorithms in identifying complex human tasks from minimal sensing hardware, achieving F1-scores up to 84% over 31 strenuous activity classes. To our knowledge, we are the first to robustly identify complex full-body tasks using a single, unobtrusive sensor feasible for real-world use in SCEs. Using our approach, robots will be able to more reliably understand human activity, enabling them to safely navigate sensitive, crowded spaces.

Keywords: Human Detection and Tracking, Sensor Fusion, Intelligent Transportation Systems
Abstract: In this paper, a novel normal distribution mixture matching based model free object tracking algorithm using 2D LIDAR is proposed. Each target object is modeled as a normal distribution mixture that captures the distribution of the points scanned from the surface of the object. This novel representation enables normal distribution transform (NDT) to accurately estimate the motion of objects, even if the shape of the points differs depending on where it is observed. Our evaluation of the proposed algorithm shows good performance in practical applications. In addition, we provides an alternative way of segmentation and data association using occupancy grid map to avoid a problem that defines a distance metric between the mixture and the point cloud. As a result, the proposed algorithm works in real time in our experiments.

Keywords: Human Detection and Tracking, Surveillance Systems, Computer Vision for Automation
Abstract: As smart cameras are becoming ubiquitous in mobile robot systems, there is an increasing concern in camera devices invading people's privacy by recording unwanted images. We want to fundamentally protect privacy by blurring unwanted blocks in images, such as faces, yet ensure that the robots can understand the video for their perception. In this paper, we propose a novel mobile robot framework with a deep learning-based privacy-preserving camera system. The proposed camera system detects privacy-sensitive blocks, i.e., human face, from extreme low resolution (LR) images, and then dynamically enhances the resolution of only privacy-insensitive blocks, e.g., backgrounds. Keeping all the face blocks to be extreme LR of 15x15 pixels, we can guarantee that human faces are never at high resolution (HR) in any of processing or memory, thus yielding strong privacy protection even from cracking or backdoors. Our camera system produces an image on a real-time basis, the human faces of which are in extreme LR while the backgrounds are in HR. We experimentally confirm that our proposed face detection camera system outperforms the state-of-the-art small face detection algorithm, while the robot performs ORB-SLAM2 well even with videos of extreme LR faces. Therefore, with the proposed system, we do not too much sacrifice robot perception performance to protect privacy.

Keywords: Human Detection and Tracking
Abstract: We present a pedestrian tracking algorithm, DensePeds, that tracks individuals in highly dense crowds > 2 pedestrians per square meter. Our approach is designed for videos captured from front-facing or elevated cameras. We present a new motion model called Front RVO for predicting pedestrian movements in dense situations using collision avoidance constraints and combine it with state-of-the-art Mask R-CNN to compute sparse feature vectors that reduce the loss of pedestrian tracks (false negatives). We evaluate DensePeds on the standard MOT benchmarks as well as a new dense crowd dataset. In practice, our approach is 4.5X faster than prior tracking algorithms on the MOT benchmark and we are state-of-the-art in dense crowd videos by over 2.6% on the absolute scale on average.

Keywords: Humanoid and Bipedal Locomotion, Biologically-Inspired Robots, Legged Robots
Abstract: Balancing or postural stability is one of important locomotor subfunctions in bipedal gaits. The inverted pendulum and virtual pivot point (VPP) are common modeling approaches to analyze balance control in human and robot walking. In this paper, we employ the VPP concept to investigate posture control in the frontal plane. The outcomes demonstrate that unlike posture control in the sagittal plane, the VPP in the frontal plane is placed below center of mass. This finding explains a novel hybrid strategy for lateral stability in human walking. The here proposed model shows that switching between unstable inverted virtual pendulums generates stable posture control in the frontal plane. This outcome is consistent within a group of seven human subjects walking at normal and slow speeds.

Keywords: Humanoid and Bipedal Locomotion, Computer Vision for Other Robotic Applications, Robotics in Hazardous Fields
Abstract: This paper introduces an obstacle-avoiding algorithm for bipedal robots, especially in push recovery situations. Typically, There are many algorithms that plan footstep to avoid obstacles based on vision recognition data. However, if the robot is pushed, the planned footprint will change, and thus, there is no guarantee that it will avoid obstacles. Although modified stepping positions can be limited, the robot’s stability is not assured. Our proposed algorithm focuses on avoiding obstacles through vision recognition in push recovery situations and generating compensation actions for instability by restricting modified footsteps. We fuse vision feedback with our previous push recovery algorithm, which optimizes the ankle, hip, and stepping strategies. We build simple grid data using vision recognition and apply it to the inequality constraint of the stepping position. We validate the effectiveness of our algorithm using the bipedal platform GAZELLE with the Kinect V2 RGBD sensor.

Keywords: Humanoid and Bipedal Locomotion, Humanoid Robots, Collision Avoidance
Abstract: We control a biped robot moving in a crowd with a Model Predictive Control (MPC) scheme that generates stable walking motions, with automatic footstep placement. Most walking strategies propose to re-plan the walking motion to adapt to changing environments only once at every footstep. This is because a footstep is planted on the ground, it usually stays there at a constant position until the next footstep is initiated, what naturally constrains the capacity for the robot to react and adapt its motion in between footsteps. The objective of this paper is to measure if re-planning the walking motion more often than once at every footstep can lead to an improvement in collision avoidance when navigating in a crowd. Our result is that re-planning twice (or more) during each footstep leads to a significant reduction of the number of collisions when walking in a crowd, but depends on the density of the crowd.

Keywords: Humanoid and Bipedal Locomotion, Humanoid Robots, Legged Robots
Abstract: In this paper, we propose walking balance control based on Caputurability. The proposed method consists of five strategies: (i) moving Zero Moment Point (ZMP) in the support polygon (ii) landing position modification (iii) landing timing modification (iv) angular momentum control (v) falling detection and fall control. Walking pattern generation calculates the ZMP so that the Capture Point (CP) reaches the position of the supporting foot at the end of the double support phase. Owing to the asymmetry of the reachable landing region, landing timing modification is different in the sagittal and lateral planes, and the step time is extended in the lateral plane depending on the direction of disturbances. The torque around the center of gravity to avoid falling is realized through whole-body inverse kinematics with constraints on the angular momentum. In addition, we propose falling detection considering the reachable landing region. We verified the effectiveness of the proposed method through experiments in which the biped robot was disturbed by pushing during tether-free walking. The robot could prevent breakdown by detecting possible falling and performed knee bending motions to suppress damage.

Keywords: Humanoid and Bipedal Locomotion, Humanoid Robots
Abstract: Estimating the center of mass position and the angular momentum derivative of legged systems is essential for both controlling legged robots and analyzing human motion. In this paper, a novel recursive approach to concurrently and accurately estimate these two quantities together is introduced. The proposed method employs kinetic and kinematic measurements from classic sensors available in robotics and biomechanics, to effectively exploits the accuracy of each measurement in the spectral domain. The soundness of the proposed approach is first validated on a simulated humanoid robot, where ground truth data is available, against an Extend Kalman Filter. The results demonstrate that the proposed method reduces the estimation error on the center of mass position with regard to kinematic estimation alone, whereas at the same time, it provides an accurate estimation of the derivative of angular momentum. Finally, the effectiveness of the proposed method is illustrated on real measurements, obtained from walking experiments with the HRP-2 humanoid robot.

Keywords: Humanoid and Bipedal Locomotion, Legged Robots, Motion and Path Planning
Abstract: We present a unified control framework that generates dynamic walking motions for biped and quadruped robots with online relative footstep optimization. The footstep optimization is formulated as a discrete-time Model Predictive Control problem which determines future footstep locations. The framework has a hierarchical structure consisting of three layers: footstep planner, trajectory generator and whole-body controller. The footstep planner plans next footstep position based on Linear Inverted Pendulum (LIP) model. Relative footstep optimization is proposed to enable automatic footstep planning without the use of any predefined footstep sequences. The trajectory generator will generate CoM and feet trajectory given the next footstep placement. In order to generalize to quadruped robots, ``virtual leg'' concept has been used to coordinate leg pair movement. The whole-body inverse dynamic controller calculates joint torques to track given Cartesian reference trajectories. To include under-actuation into consideration, contact vertices formulation of ground reaction forces (GRFs) has been adopted. Generalized whole-body controller can handle biped robot with line feet as well as quadruped robots with point feet walking with dynamic gaits. Several simulations have been performed to demonstrate the robustness and generality of the proposed framework.

Keywords: Space Robotics and Automation, Industrial Robots, Motion Control
Abstract: In this work we present a novel approach which compensates the destabilising effects of the time delay intrinsic in the control loop of an admittance-controlled robot employed for satellite dynamics simulation. The method is based on an energy storing element, the tank, which is exploited by the controller to preserve the passivity of the system and to avoid instability. Furthermore, we compare the performance of the proposed method with existing energy-based approaches, namely time-domain-passivity and wave variable transformation. The performance comparison and robustness of the methods are analysed in a Montecarlo simulation and validated experimentally.

Keywords: Space Robotics and Automation, Mining Robotics
Abstract: Successful sampling on the surface of asteroids is difficult because of their weightless environment and unknown material mechanical property. This work presents an asteroid sampler based on sweeping and grinding methods to improve the success rate of sampling. The sampler uses two brushes rotating clockwise and counterclockwise to collect sample particles on the surface of asteroids. When encountering the hard rock or sample particles with large cohesion, the sampler adopts a drill bit to grind them to loose samples suitable for collecting by the brushes. The interaction between the brushes and the regolith is modeled and the sweeping mechanism is designed. A simple grinding mechanism is also designed. Numerical simulation and prototype experiments, at different parameters including blades number, rotational speed, and feeding speed of the brushes, mechanical property of the sample, and gravity, were conducted for validating the proposed methods. The 280g sampler prototype with 8 blades of brushes could collect about 19g regolith simulant in 25s in earth environment. The drill bits could work together with the brushes to improve the sampling efficiency through DEM simulation in case of large cohesion among sample particles. The sampler will be a good choice for installing into an asteroid rover in exploration.

Keywords: Space Robotics and Automation, Motion and Path Planning
Abstract: Remote sensing can provide crucial information for planetary rovers. However, they must validate these orbital observations with in situ measurements. Typically, this involves validating hyperspectral data using a spectrometer on-board the field robot. In order to achieve this, the robot must visit sampling locations that jointly improve a model of the environment while satisfying sampling constraints. However, current planners follow sub-optimal greedy strategies that are not scalable to larger regions. We demonstrate how the problem can be effectively defined in an MDP framework and propose a planning algorithm based on Monte Carlo Tree Search, which is devoid of the common drawbacks of existing planners and also provides superior performance. We evaluate our approach using hyperspectral imagery of a well-studied geologic site in Cuprite, Nevada.

Keywords: Space Robotics and Automation, Wheeled Robots, Deep Learning in Robotics and Automation
Abstract: This paper presents a prediction algorithm of driving energy for future Mars rover missions. The majority of future Mars rovers would be solar-powered, which would require energy-optimal driving to maximize the range with limited energy. The essential and arguably the most challenging technology for realizing energy-optimal driving is the capability to predict the driving energy, which is needed to construct an energy-aware cost function for path planning. In this paper, we propose vision-based algorithms to remotely predict the driving energy consumption using machine learning. Specifically, we develop and compare two machine-learning models in this paper, namely VeeGer-EnergyNet and Veeger-TerramechanicsNet, respectively. The former is trained directly using recorded power, while the latter estimates terrain parameters from the images using a simplified-terramechanics model, and calculate the power based on the model. The two approaches are fully automated self-supervised learning algorithms. To combine RGB and depth images efficiently with high accuracy, we propose a new network architecture called Two-PNASNet-5, which is based on PNASNet-5. We collected a new dataset to verify the effectiveness of the proposed approaches. Comparison of the two approaches showed that Veeger-TerramechanicsNet had better performance than VeeGer-EnergyNet.

Keywords: Space Robotics and Automation, Wheeled Robots, Nonholonomic Mechanisms and Systems
Abstract: We present an approach to enhance wheeled planetary rover dead-reckoning localization performance by leveraging the use of zero-type constraint equations in the navigation filter. Without external aiding, inertial navigation solutions inherently exhibit cubic error growth. Furthermore, for planetary rovers that are traversing diverse types of terrain, wheel odometry is often unreliable for use in localization, due to wheel slippage. For current Mars rovers, computer vision-based approaches are generally used whenever there is a high possibility of positioning error; however, these strategies require additional computational power, energy resources, adequate features in the environment, and significantly slow down the rover traverse speed. To this end, we propose a navigation approach that compensates for the high likelihood of odometry errors by providing a reliable navigation solution that leverages non-holonomic vehicle constraints as well as state-aware pseudo-measurements (e.g., zero velocity and zero angular rate) updates during periodic stops. By using this, computationally expensive visual-based corrections could be performed less often. Experimental tests that compare against GPS-based localization are used to demonstrate the accuracy of the proposed approach. The source code, post-processing scripts, and example datasets associated with the paper are published in a public repository.

Keywords: Force Control, Contact Modelling, Space Robotics and Automation
Abstract: The German Aerospace Center (DLR) has developed the Terramechanics Robotics Locomotion Lab (TROLL) to provide a feasible testing facility for developing planetary exploration rovers, as well as validating terramechanical models. A robotic manipulator is used to provide the required degrees of freedom to the mounted wheel or subsystem, making it necessary to actively control the interaction force of the wheel-soil contact. This paper is concerned with the development of a feasible force control strategy for the testbench wheel-soil contact during single-wheel experiments. For this purpose, a terramechanical model has been developed to accurately map the dynamic processes relevant for the force control design, which is later used in a testbench simulation framework to predict and evaluate the performance of control strategies. The Adaptive Admittance Control (AAC) scheme developed is adapting the gain based on the current control deviation, the rotational velocity of the wheel and an estimated soil stiffness during the experiment. The AAC is evaluated using a benchmark single-wheel experiment and shows superior performance compared to standard admittance control.

Keywords: Kinematics, Motion and Path Planning, Optimization and Optimal Control
Abstract: This paper presents an inverse kinematics (IK) method which can control future velocities and accelerations for multi-body systems. The proposed IK method is formulated as a quadratic programing (QP) that optimizes future joint trajectories. The features of the proposed IK are: (1) the evaluation of accelerations at future time instances, (2) the trajectory representation that can implicitly integrate the time integral formula into QP, (3) the computation of future Jacobian matrices based on the comprehensive theory of differential kinematics proposed in our previous work. Those features enable a stable and fast IK computation while evaluating the future accelerations. We also conducted thorough numerical studies to show the efficiency of the proposed method.

Keywords: Motion and Path Planning, Computational Geometry, Simulation and Animation
Abstract: Visibility integrity (VI) is a measurement of similarity between the visibilities of regions. It can be used to approximate the visibility of coherently moving targets, called group visibility. It has been shown that computing visibility integrity using agglomerative clustering takes O(n^4*logn) for n samples. Here, we present a method that speeds up the computation of visibility integrity and reduces the time complexity from O(n^4*logn) to O(n^2). Based on the idea of visibility integrity, we show for the first time that the visibility-integrity roadmap (VIR), a data structure that partitions a space into zones, can be calculated efficiently in 3D. More specifically, we offer two different offline approaches, a naive one and a kernel-based one, to compute a VIR. In addition, we demonstrate how to apply a VIR to solve group visibility and group following problems in 3D. We propose a planning algorithm for the camera to maintain visibility of group targets by querying the VIR. We evaluate our approach in different 3D simulation environments and compare it to other planning methods.

Keywords: Motion and Path Planning, Recognition, Computer Vision for Other Robotic Applications
Abstract: Camera systems in fast motion suffer from the effects of motion blur, which degrades image quality and can have a large impact on the performance of visual tasks. The degradation in image quality can be mitigated through the use of image reconstruction. Blur effects and the resulting reconstruction performance are highly dependent on the point-spread function resulting from camera motion. This work focuses on the motion planning problem for a camera system with boundary conditions on time and position, with the objective of improving the performance of optical character recognition. Tuned edge-preserving trajectories are shown to result in higher recognition accuracy when compared to inverse error and linear trajectories. Simulation and experimental results provide quantitative measures to verify edge-preservation and greater recognition performance.

Keywords: Path Planning for Multiple Mobile Robots or Agents, Swarms, Autonomous Agents
Abstract: This paper introduces an arbitrary-region based shape-control methodology for a swarm-robotics framework to conquer the traditional obstacle-avoidance problem. In this control approach, during the movement of a team of robots through an unknown occluded environment, a virtual boundary for the robotic crowd that could pass through even a very narrow passage, as a swarm, has been created based on the agents’/robots’ sensing information. The primary challenge is to force each and every agent to fit into the arbitrary boundary, thus created, so that they can form a swarm and eventually progress towards the goal while avoiding obstacles. Consequently, the process maximizes the inter-agent cohesion. Furthermore, the virtual region is subdivided into so-called cells designated by a non-overlapping circular geometry. The utmost endeavor is to make an agent place itself at the center of a circular cell and a mechanism has been devised, where each agent is duly attracted and eventually is resident at the center of each cell completely blanketing the entire virtual region. To place each agent inside the stipulated contour, a novel circle-packing algorithm has been proposed such that the contour is fitted with a number of identical circles. Finally, the simulation results along with hardware implementation demonstrate the effectiveness of the proposed control technique.

Keywords: Path Planning for Multiple Mobile Robots or Agents, Swarms, Collision Avoidance
Abstract: This paper presents a new trajectory planning method for multiple quadrotors in obstacle-dense environments. We suggest a relative safe flight corridor (RSFC) to model safe region between a pair of agents, and it is used to generate linear constraints for inter-collision avoidance by utilizing the convex hull property of relative Bernstein polynomial. Our approach employs a graph-based multi-agent pathfinding algorithm to generate an initial trajectory, which is used to construct a safe flight corridor (SFC) and RSFC. We express the trajectory as a piecewise Bernstein polynomial and formulate the trajectory planning problem into one quadratic programming problem using linear constraints from SFC and RSFC. The proposed method can compute collision-free trajectory for 16 agents within a second and for 64 agents less than a minute, and it is validated both through simulation and indoor flight test.

Keywords: Motion and Path Planning, Kinematics, Optimization and Optimal Control
Abstract: The paper presents a fast online predictive method to solve the task-priority differential inverse kinematics of redundant manipulator under kinematic constraints. It implements a task-scaling technique to preserve the desired geometrical task, when the trajectory is infeasible for the robot capabilities. Simulation results demonstrate the effectiveness of the methodology.

Keywords: Grasping, AI-Based Methods, Perception for Grasping and Manipulation
Abstract: Robotic grasping in cluttered environments is often infeasible due to obstacles preventing possible grasps. Then, pre-grasping manipulation like shifting or pushing an object becomes necessary. We developed a robotic system that can learn, in addition to grasping, to shift objects in such a way that their grasp probability increases. Our research contribution is threefold: First, we present an algorithm for learning the optimal pose of manipulation primitives like clamping or shifting. Second, we learn non-prehensible actions that explicitly increase the grasping probability. Making one skill (shifting) directly dependent on another (grasping) removes the need of sparse rewards, leading to more data-efficient learning. Third, we apply a real-world solution to the industrial task of bin picking, resulting in the ability to empty bins completely. The system is trained in a self-supervised manner with around 25000 grasp and 2500 shift actions. Our robot is able to grasp and file objects with 274 picks per hour. Furthermore, we demonstrate the system's ability to generalize to novel objects.

Keywords: Grasping, AI-Based Methods
Abstract: In this paper, a novel robotic grasping system is established to automatically pick up objects in cluttered scenes. A composite robotic hand composed of a suction cup and a gripper is designed for grasping the object stably. The suction cup is used for lifting the object from the clutter first and the gripper for grasping the object accordingly. We utilize the affordance map to provide pixel-wise lifting point candidates for the suction cup. To obtain a good affordance map, the active exploration mechanism is introduced to the system. An effective metric is designed to calculate the reward for the current affordance map, and a deep Q-Network (DQN) is employed to guide the robotic hand to actively explore the environment until the generated affordance map is suitable for grasping. Experimental results have demonstrated that the proposed robotic grasping system is able to greatly increase the success rate of the robotic grasping in cluttered scenes.

Keywords: Grasping, Computer Vision for Automation, Multifingered Hands
Abstract: Assistive robotic manipulators (ARMs) play an important role for people with upper-limb disabilities and the elderly to assist them in fulfilling Activities of Daily Living (ADLs). However, as the objects to be handled in ADLs differ in size, shape and manipulation constraints, many two or three fingered end-effectors of ARMs have difficulty robustly interacting with these objects. In this paper, we proposed vision-based control of a 5-fingered manipulator (Schunk SVH), allowing it to automatically change its shape based on object classification using computer vision combined with deep learning. The control method is tested in a simulated environment and achieves a more robust grasp with the properly shaped five-fingered hand than with a comparable three-fingered gripper (Barrett Hand) using the same control sequence. In addition, the final optimal grasp pose (x, y, and theta) is learned through a deep regressor in the penultimate stage of the grasp. This method correctly identifies the optimal grasp pose in 78.35% of the test cases when considering all three parameters.

Keywords: Grasping, Computer Vision for Automation, Visual Learning
Abstract: Robots are required to support people’s work. In order to alleviate the burden on people, it is desirable that robot can automatically generate and execute complicated motions according to simple directions from people. However, there are multiple grasping methods for one object. In order to select a motion suitable for the direction, it is important to estimate candidates of grasping method for the object. In this research, we purpose to recall candidates of grasping position and hand shape from an object image. In learning, a network that outputs a plurality of grasping method candidates for one object image to each channel of a multi-channel image is used. At this time, a plurality of grasping methods are not learned at same time, learned one by one. The similar grasping method for the similar object shape is automatically clustered to each output channel in the learning process, and a grasping method having a characteristic difference is presented as a candidate. We show the usefulness of this method using experimental examples.

Keywords: Grasping, Computer Vision for Other Robotic Applications, Factory Automation
Abstract: One of the main challenges in the vision-based grasping is the selection of feasible grasp regions while interacting with novel objects. Recent approaches exploit the power of convolutional neural network (CNN) to achieve accurate grasping at the cost of high computational power and time. In this paper, we present a novel unsupervised learning-based algorithm for the selection of feasible grasp regions. Unsupervised learning infers the pattern in dataset without any external labels. We applied k-means clustering at every sampling stage on image plane to identify the grasp regions, followed by axes assignment method. We define a novel concept of Grasp Decide Index (GDI) to select the best grasp pose in the image plane. We have conducted several experiments in clutter or isolated environment on standard objects of Amazon Robotics Challenge 2017 and Amazon Picking Challenge 2016. We compared the results with prior learning-based approaches to validate the robustness and adaptive nature of our algorithm for a variety of novel objects in different domains.

Keywords: Grasping, Human Factors and Human-in-the-Loop
Abstract: Grasping a simple object from the side is easy --- unless the object is almost as big as the hand or space constraints require positioning the robot hand awkwardly with respect to the object. We show that humans --- when faced with this challenge --- adopt coordinated finger movements which enable them to successfully grasp objects even from these awkward poses. We also show that it is relatively straight forward to implement these strategies autonomously. Our human-studies approach asks participants to perform grasping task by either ``puppetteering'' a robotic manipulator that is identical~(geometrically and kinematically) to a popular underactuated robotic manipulator~(the Barrett hand), or using sliders to control the original Barrett hand. Unlike previous studies, this enables us to directly capture and compare human manipulation strategies with robotic ones. Our observation is that, while humans employ underactuation, how they use it is fundamentally different (and more effective) than that found in existing hardware.

Keywords: Biological Cell Manipulation, Automation at Micro-Nano Scales, Computer Vision for Automation
Abstract: Extensive research efforts have been made toward automating the microinjection of biological cells by leveraging micro-robotic technologies. However, to best knowledge of the authors, there is no report on the automation of the time- consuming process: moving the injection tools (a micropipette, a grippers, etc.) and cells into the field of view (FOV) of micro- scope from the macro FOV(outside the microscopic FOV). This paper presents a novel macro-micro conversion strategy, and a grid detection and positioning algorithm to automate the time-consuming step of moving the injection tools and cells to the microscopic FOV. The proposed solution can free the techni -cian from the laborious hand-eye coordination operations for moving the injection tools and cells to the target position within the microscopic FOV. Furthermore, this paper proposes an auto-focusing algorithm to automate the operation step: moving down the gripper from the air outside the culture media and then precisely clamping a cell in the liquid environment for injection. In the proposed solution, the active window-based auto-focusing algorithm is developed to solve the challenging problem: the image information is lost due to the “viscous effect” taking place when the gripper jaw touches the water surface. The proposed solutions are tested and validated by the microinjection experiments of zebrafish embryos using the in-house develop micro-robotic system. The technologies and strategies proposed in this paper significantly improve the automation level of the cell microinjections, and can be easily extended to any other micromanipulation of biological cells.

Keywords: Biological Cell Manipulation, Automation at Micro-Nano Scales, Surgical Robotics: Steerable Catheters/Needles
Abstract: Introducing alterations in the mtDNA sequence is challenging but needed for potential therapies and basic studies. Direct microinjection of mitochondria into small cells has been considered inefficient and impractical. To address this issue, we present a highly efficient and precise robotic approach for automatically transferring mitochondria from one single cell to another. A microfluidic cell positioning device is used to pattern two different types of cells in one dimensional array, and an image processing algorithm is applied to identify the location of the mitochondria and cell. A visual feedback control mechanism is developed to enhance the mitochondrial extraction efficiency. A robust adaptive sliding control algorithm is developed to precisely control an X–Y stage to accomplish the extraction of mitochondria from A type cell followed by injection of the mitochondria into B type cell automatically. The system can transfer mitochondria from one cell to another with an average duration of 15 s/mitochondria. Experiments of mitochondrial transfer from THP1 and NB4 cells to THP1 cells and fibroblasts are conducted to show the effectiveness of the developed approach

Keywords: Biomimetics, Force and Tactile Sensing, Biologically-Inspired Robots
Abstract: This work presents the design, modeling, and fabrication of a whisker-like sensor capable of measuring the whisker's angular displacement as well as the applied moments at the base of the whisker. The sensor takes advantage of readily accessible and low-cost 3D magnetic sensors to transduce whisker deflections, and a planar serpentine spring structure at the whisker base is used to provide a mechanical suspension for the whisker to rotate. The sensor prototype was characterized, calibrated, and compared with analytical models of the spring system and the magnetic field. The prototype showed a moment sensing range of 1.1 N-mm when deflected up to 19.7 degrees. The sensitivity of the sensor was 0.38 degree/LSB for the angular displacement sensing, and 0.021 N-mm/LSB for the moment sensing. A fully integrated system is demonstrated to display real-time information from the whisker on a graphical interface.

Keywords: Biomimetics, Neurorobotics, Biologically-Inspired Robots
Abstract: Identifying objects using sparse tactile sensor arrays requires movement across the surface and the integration of sensory information to support hypotheses. In this study we demonstrate a surface placement control strategy to orient and position a biomimetic tactile whisker array relative to the object surface. This reduces the variance in tactile view, or representation, of each region of the object surface thus relaxing the prior pose dependence for object recall. In addition we evaluate two regional search strategies in an attempt to further improve the robustness and speed of object classification.

Keywords: Biological Cell Manipulation, Micro/Nano Robots
Abstract: The capability to precisely rotate cells and other micrometer-sized biological samples is invaluable in biomedical, bioengineering, and biophysical applications. We propose a novel on-chip three-dimension (3D) cell rotation method based on whirling flows created by oscillating asymmetrical microstructures. In an acoustic field excited by the vibration of a piezoelectric transducer, two different modes of microvortices were generated around the specially designed microstructures, which were utilized to precisely achieve the in-plane and out-of-plane rotational manipulation. Experiments of the different sizes of microparticles and oocytes are conducted to demonstrate the claimed functions. We also investigated the effect of various parameters on the acoustically induced flow such as the frequency, the driving voltage and the distance from the microstructure tip to the oocyte center, indicating the rotational rate can be effectively tuned on demand for single-cell studies. Rotation by using this method demonstrated overall out-of-plane and in-plane rotational orientation control in a quite simple way. And comparing with the conventional works, our acoustofluidic cell rotation approach is simple-to-fabricate and easy-to-operate, permitting rotation irrespective of the magnetic, optical, or electrical properties of the specimen under investigation.

Keywords: Robust/Adaptive Control of Robotic Systems, Medical Robots and Systems, Dynamics
Abstract: In the present work we discuss a novel dynamic control approach for magnetically actuated robots, by proposing an adaptive control technique, robust towards parametric uncertainties and unknown bounded disturbances. The former generally arise due to partial knowledge of the robots’ dynamic parameters, such as inertial factors, the latter are the outcome of unpredictable interaction with unstructured environments. In order to show the application of the proposed approach, we consider controlling the Magnetic Flexible Endoscope (MFE) which is composed of a soft-tethered Internal Permanent Magnet (IPM), actuated with a single External Permanent Magnet (EPM). We provide with experimental analysis to show the possibility of levitating the MFE - one of the most difficult tasks with this platform - in case of partial knowledge of the IPM’s dynamics and no knowledge of the tether’s behaviour. Experiments in an acrylic tube show a reduction of contact of the 32% compared to non-levitating techniques and 1.75 times faster task completion with respect to levitating techniques. More realistic experiments, performed in a colon phantom, show that levitating the capsule achieves faster and smoother exploration and that the minimum time for completing the task is attained by the proposed approach.

Keywords: Localization, Calibration and Identification, Mapping
Abstract: In this paper, we present a sonar-based navigation system, designed to deploy a fleet of autonomous mobile platforms at a reasonable cost. In educational and hobbyist contexts, a large number of robots is required. By means of classical navigation approaches, every robot should be provided with accurate vision or range sensors. This limits the maximum number of robots in the fleet, due to the unaffordable cost of these sensors. In contrast to that, our system requires a single platform equipped with a higher quality sensor, used to perform calibration and mapping tasks. The rest of the fleet, able to localize and navigate, is equipped solely with low-cost sonars, providing a notable reduction in the overall cost. We achieve this task by presenting a novel calibration procedure to estimate the sonars extrinsic, and by adapting a classical Monte Carlo localization algorithm to the sonar model, focusing on efficiency. We release an open source implementation of the system to the community.

Keywords: Localization, Failure Detection and Recovery, Recognition
Abstract: Recognizing misalignment between sensor measurements and objects that exist on a map due to inaccuracies in localization estimation is challenging. This can be attributed to the fact that the sensor measurements are individually modelled for solving the localization problem, resulting in entire relations of the measurements being ignored. This paper proposes a misalignment recognition method using Markov random fields with fully connected latent variables for the detection of localization failures. The proposed method estimates the classes of each sensor measurement that are aligned, misaligned, and obtained from unknown obstacles. The full connection allows us to consider the entire relation of the measurements. A misalignment can be exactly recognized even when partial sensor measurements overlap with mapped objects. Based on the class estimation results, we are able to distinguish whether the localization has failed or not. The proposed method was compared with six alternative methods, including a convolutional neural network, using datasets composed of success and failure localization samples. Experimental results show that classification accuracy of the localization samples using the proposed method exceeds 95 % and outperforms the other examined methods.

Keywords: Localization, Humanoid and Bipedal Locomotion, Sensor Fusion
Abstract: Contemporary humanoids are equipped with visual and LiDAR sensors that are effectively utilized for Visual Odometry (VO) and LiDAR Odometry (LO). Unfortunately, such measurements commonly suffer from outliers in a dynamic environment, since frequently it is assumed that only the robot is in motion and the world is static. To this end, robust state estimation schemes are mandatory in order for humanoids to symbiotically co-exist with humans in their daily dynamic environments. In this article, the robust Gaussian Error-State Kalman Filter for humanoid robot locomotion is presented. The introduced method automatically detects and rejects outliers without relying on any prior knowledge on measurement distributions or finely tuned thresholds. Subsequently, the proposed method is quantitatively and qualitatively assessed in realistic conditions with the full-size humanoid robot WALK-MAN v2.0 and the mini-size humanoid robot NAO to demonstrate its accuracy and robustness when outlier VO/LO measurements are present. Finally, in order to reinforce further research endeavours, our implementation is released as an open-source ROS/C++ package.

Keywords: Localization, Mapping, Service Robots
Abstract: In outdoor scenarios changing conditions (e.g., seasonal, weather and lighting effects) have a substantial impact on the appearance of a scene, which often prevents successful visual localization. The application of an unsupervised Slow Feature Analysis (SFA) on the images captured by an autonomous robot enables self-localization from a single image. However, changes occurring during the training phase or over a more extended period can affect the learned representations. To address the problem, we propose to join long-term recordings from an outdoor environment based on their position correspondences. The established hierarchical model trained on raw images performs well, but as an extension, we extract Fourier components of the views and use that for learning of spatial representations, which reduces the computation time and makes it adequate to run on an ARM embedded system. We present the experimental results from a simulated environment and real-world outdoor recordings collected over a full year, which has effects like different day time, weather, seasons and dynamic objects. Results show an increasing invariance w.r.t. changing conditions over time, thus an outdoor robot can improve its localization performance during operation.

Keywords: Localization, Probability and Statistical Methods, Sensor Networks
Abstract: Given a nonlinear dynamical system, this paper considers the problem of selecting a subset of the total set of sen- sors that has provable guarantees on standard metrics related to the nonlinear observability Gramian. The key contribution is a simple randomized algorithm that samples the sensors uni- formly without replacement, and yields probabilistic guarantees on the minimum eigenvalue or the inverse of the condition number of the nonlinear observability Gramian relative to that of the complete set of sensors. Numerical studies reveal that the utility of the theoretical results lies in the regime of large total number of sensors wherein the combinatorial nature of the problem presents a significant computational challenge. The results are demonstrated numerically on a problem of moving target localization using an Extended Kalman Filter (EKF) in two scenarios: one using range sensors and another with time-difference-of-arrival (TDOA) measurements. A graceful degradation of performance with a decreased number of sensors is observed when compared to the use of all of the sensors for localization. It is also observed that for certain metrics, the proposed approach provides an improvement over a heuristic that selects the sensors in a greedy manner based on the contribution of an additional sensor toward the observability Gramian metric.

Keywords: Localization, Probability and Statistical Methods
Abstract: The Bayes filter is the basis for many state-of-the-art on-line robot localization algorithms. In the literature, its derivation typically requires the robot controls to be chosen independently of all other variables. However, this assumption is not valid for a robotic system which is to act purposefully. The contribution of this paper is twofold: We prove that the Bayes filter is also exact for an autonomous system which chooses the controls not randomly but depending on any subset of all observable variables. We further show how to augment the filter if a human agent chooses controls on the basis of parts of the state space that are not directly accessible to the robot. In this case, modeling the agent's purpose improves the pose estimate as the control provides additional information about the hidden state space. A careful derivation of the Bayes filter then leads to an additional pseudo measurement update step. Real-world experiments with our teleoperated mobile robot and evaluations on the KITTI dataset show that the localization accuracy significantly improves if we augment a particle filter with this pseudo measurement update. We further present an analytical example for an augmented Kalman filter, which leads to a more accurate estimate than the standard Kalman filter.

Keywords: AI-Based Methods, Recognition, Failure Detection and Recovery
Abstract: Human activity recognition is a crucial ingredient in safe and efficient human–robot collaboration. In this paper, we present a new model-based human activity recognition system called logical activity recognition system (LCARS). LCARS requires much less training data compared to learning-based works. Compared to other model-based works, LCARS requires minimal domain-specific modeling effort from users. The minimal modeling is for two reasons: i) we provide a systematic and intuitive way to encode domain knowledge for LCARS and ii) LCARS automatically constructs a probabilistic estimation model from the domain knowledge. Requiring minimal training data and modeling effort allows LCARS to be easily applicable to various scenarios. We verify this through simulations and experiments.

Keywords: Probability and Statistical Methods, Service Robots
Abstract: Some robots can interact with humans using natural language, and identify service requests through human-robot dialog. However, few robots are able to improve their language capabilities from this experience. In this paper, we develop a dialog agent for robots that is able to interpret user commands using a semantic parser, while asking clarification questions using a probabilistic dialog manager. This dialog agent is able to augment its knowledge base and improve its language capabilities by learning from dialog experiences, e.g., adding new entities and learning new ways of referring to existing entities. We have extensively evaluated our dialog system in simulation as well as with human participants through MTurk and real-robot platforms. We demonstrate that our dialog agent performs better in efficiency and accuracy in comparison to baseline learning agents. Demo video can be found at url{https://youtu.be/DFB3jbHBqYE}

Keywords: AI-Based Methods, Dynamics, Industrial Robots
Abstract: Inverse dynamics modeling is a critical problem for the computed-torque control of robotic manipulator. This paper presents a novel recurrent network based on the modified Simple Recurrent Unit (SRU) with hierarchical memory (SRU-HM), which is achieved by the nested SRU structure. In this way, it enables the capability to retain the long-term information in the distant past, compared with the conventional stacked structure. The hidden state of SRU is able to provide more complete information relevant to current prediction. Experimental results demonstrate that the proposed method can improve the accuracy of dynamics model greatly, and outperforms the state-of-the-art methods.

Keywords: AI-Based Methods, Learning and Adaptive Systems
Abstract: Bayesian Optimization (BO) is an effective method for optimizing expensive-to-evaluate black-box functions with a wide range of applications for example in robotics, system design and parameter optimization. However, scaling BO to problems with large input dimensions (> 10) remains an open challenge. In this paper, we propose to leverage results from optimal control to scale BO to higher dimensional control tasks and to reduce the need for manually selecting the optimization domain. The contributions of this paper are twofold: 1) We show how we can make use of a learned dynamics model in combination with a model-based controller to simplify the BO problem by focusing onto the most relevant regions of the optimization domain. 2) Based on (1) we present a method to find an embedding in parameter space that reduces the effective dimensionality of the optimization problem. To evaluate the effectiveness of the proposed approach, we present an experimental evaluation on real hardware, as well as simulated tasks including a 48-dimensional policy for a quadcopter.

Keywords: AI-Based Methods, Learning from Demonstration, Human Detection and Tracking
Abstract: In order for robots to share their workspace with people, they need to reason about human motion efficiently. In this work we leverage large datasets of paths in order to infer local models that are able to perform long-term predictions of human motion. Further, since our method is based on simple dynamics, it is conceptually simple to understand and allows one to interpret the predictions produced, as well as to extract a cost function that can be used for planning. The main difference between our method and similar systems, is that we employ a map of the space and translate the motion of groups of paths into vector fields on that map. We test our method on synthetic data and show its performance on the Edinburgh forum pedestrian long-term tracking dataset [1] where we were able to outperform a Gaussian Mixture Model tasked with extracting dynamics from the paths.

Keywords: AI-Based Methods, Manipulation Planning, Motion Control
Abstract: Accurate manipulation of a deformable body such as a piece of fabric is difficult because of its many degrees of freedom and unobservable properties affecting its dynamics. To alleviate these challenges, we propose the application of feedback-based control to robotic fabric strip folding. The feedback is computed from the low dimensional state extracted from a camera image. We trained the controller using reinforcement learning in simulation which was calibrated to cover the real fabric strip behaviors. The proposed feedback-based folding was experimentally compared to two state-of-the-art folding methods and our method outperformed both of them in terms of accuracy.

Keywords: Biologically-Inspired Robots, Biomimetics, Dynamics
Abstract: Recently, diverse research has actively been conducted to control the posture of jumping robots using an inertial tail mechanism. However, the inertial tail mechanism has a high probability of collision with obstacles. In this study, a momentum wheel mechanism is proposed to achieve the same attitude control performance while reducing the volume occupied by the inertial tail mechanism. To verify the performance of the momentum wheel mechanism, we proposed a jumping robot with a momentum wheel mechanism and performed a dynamic analysis, simulation, and experiments on a jumping robot with a momentum wheel mechanism. In addition, it has been demonstrated that the momentum wheel mechanism can contribute to control of the body angle of the jumping robot. As a result, the momentum wheel mechanism can enhance the stability of the jumping robot more than the tail mechanism, and the momentum wheel mechanism contributes to the attitude control of the body angle, which allows the jumping robot to perform continuous jumping.

Keywords: Biologically-Inspired Robots, Biomimetics, Soft Robot Materials and Design
Abstract: Soft robots are a burgeoning archetype in robotics due to their ability to perform intricate movements easily and seamlessly. They serve as an ideal concept for realistically emulating animal movements. However, the majority of soft robots today are unable to vary their motions due to the coupled interaction of the nature of their kinematics and structural composition. We therefore created Carpie, a robotic caterpillar adapted from a modular, pneumatic actuator. Carpie is designed to perform a variety of caterpillar movements by tuning its mechanical structure through physical reconfiguration. The robot has a completely soft body that enables a variety of movements and contortions into different shapes, making Carpie is perhaps the perfect example to showcase a soft robot’s aptness for animal mimicry. We analyzed the robot’s control aspect. Its capabilities were measured by performing gait velocity studies on three different configurations. Each configuration was executed by applying various modules on Carpie’s structure without fully refabricating the entire assembly. We also analyzed the accompanying change in the robot’s maximum height during gait. Finally, we demonstrate how physical reconfiguration was able to radically alter Carpie’s movement, allowing it to perform a turning motion.

Keywords: Biologically-Inspired Robots, Biomimetics
Abstract: This paper presents a distributed six-link two-dimensional electromechanical actuator that emulates a biological spine. The gearless actuator is made by stacking modules of E-shaped cores along with integrated springs. A coil excitation induces magnetic flux in the core, which produces electromechanical force in the air gap between two adjacent cores. The proposed actuator is driven by dc-dc converters with current control. Electromechanical force analysis for the proposed spine architecture with different air-gap distances and spring analysis that improve the actuator's performance are discussed. Experimental results show different biological motions with a prototype design. The prototype can produce a maximum torque of 1.4 N-m.

Keywords: Biologically-Inspired Robots, Humanoid and Bipedal Locomotion, Compliant Joint/Mechanism
Abstract: It is expected that human locomotion, in which the trunk twists and arms swing, provides stability as well as compensation for the angular momentum because the trunk and arms are placed over legs and then their position and the acceleration influence the bipedal locomotion. This paper presents a bipedal robot equipped with a trunk, embedding a viscoelastic joint and two arms away from the midline of the trunk. Physical experiments show that a passive oscillation of the viscoelastic trunk joint around the yaw axis vertical to the ground and a swing of heavy arms provided an oscillation of zero moment point (ZMP) in the lateral direction and a small oscillation in the anteroposterior direction. This is despite the fact that the robot does not have a roll joint and the arm swings back and forth. Numerical analysis supports the results of the physical experiments through a simple model and by expressing ZMP trajectory, showing that the arm swinging facilitates anti-phase locomotion in which right arm and left leg swings ahead and vice versa.

Keywords: Biologically-Inspired Robots, Humanoid Robots, Computer Vision for Other Robotic Applications
Abstract: Autonomous robots can rely on attention mechanisms to explore complex scenes and select salient stimuli relevant for behaviour. Stimulus selection should be fast to efficiently allocate available (and limited) computational resources to process in detail a subset of the otherwise overwhelmingly large sensory input. The amount of processing required is a product of the amount of data sampled by a robot's sensors; while a standard RGB camera produces a fixed amount of data for every pixel of the sensor, an textit{event-camera} produces data only for where there is a contrast change in the field of view, and does so with a lower latency. In this paper, we describe the implementation of a state-of-the-art bottom-up attention model, based on structuring the visual scene in terms of proto-objects. As an event-camera encodes different visual information compared to frame-based cameras, the original algorithm must be adapted and modified. We find that the event-camera's inherent detection of edges removes the need for some early stages of processing in the model. We describe the modifications, compare the event-driven algorithm to the original, and validate the potential for use on the iCub humanoid robot.

Keywords: Biologically-Inspired Robots, Learning and Adaptive Systems, Humanoid Robots
Abstract: The musculoskeletal humanoid has many benefits that human beings have, but the modeling of its complex flexible body is difficult. Although we have developed an online acquisition method of the nonlinear relationship between joints and muscles, we could not completely match the actual robot and its self-body image. When realizing a certain task, the direct relationship between the control input and task state needs to be learned. So, we construct a neural network representing the time-series relationship between the control input and task state, and realize the intended task state by applying the network to a real-time control. In this research, we conduct accelerator pedal control experiments as one application, and verify the effectiveness of this study.

Keywords: Aerial Systems: Applications, Aerial Systems: Mechanics and Control, Field Robots
Abstract: We propose an aerial robot system with a long-reach manipulator to allow the manipulated target to be at the desired distance away from the airframe. The system consists of a hexrotor with winch mechanism and wire-suspended hand. The winch mechanism is used to adjust the distance of the manipulator from the unmanned aerial vehicle (UAV). The wire-suspended hand is constructed with 4 ducted fans, IMU sensor, dual-gripper and vision system. The hand is autonomously able to stabilize its swing motion by controlling the ducted fans and position itself to the desired target using the vision system. Through experiment, we verified that for a 3m suspended hand, it is possible to control its position within 0.6m in X axis and 0.24m in Y axis of the hand.

Keywords: Aerial Systems: Mechanics and Control
Abstract: This work introduces the design, implementation, and control of an efficient and versatile tail-sitter VTOL UAV platform - Hong Hu. A multidisciplinary design and optimization framework is used to optimize the aircraft’s aerodynamic configuration and propulsion system. Then the carbon fiber structured aircraft is manufactured based on the optimal design. The resulting Hong Hu UAV platform could be used for various outdoor applications. In this abstract, we demonstrate its application in large-scale surveying and mapping

Keywords: Agricultural Automation, Multi-Robot Systems, Planning, Scheduling and Coordination
Abstract: Consider a team of mechanical-weeding robots managing herbicide-resistant weeds on any row-crop farm in the US. This team of robots needs to predict the weed growth across the whole farm in order to make intelligent decisions on robot coordination cite{mcallister2018weed}. However, none of the robots can observe the whole field, and even together, they can only observe a very limited part of the field at any given time. Furthermore, the robots do not have high bandwidth access to a High-Performance Computing (HPC) facility, nor would such a facility be able to provide highly accurate predictions of weed growth without real-time access to weed growth data being gathered through the weeding process, as weed distribution depends highly on the field and the day. The number of emerged seedlings at a given location in the field is governed by the latent seed bank density within the soil at that location, which vary randomly over different fields. For this reason, it is necessary to adapt any model trained on general weed growth data to the specific field.

We enable our team of robots to efficiently learn a predictive model of the specific field by adapting a general model which was trained using data from many fields. The predictive model enables this team of robots to pursue more intelligent coordination strategies, leveraging our earlier work cite{mcallister2018weed}, rather than resorting to grid search over the whole field or using pre-determined heuristics to plan its actions. Our work addresses the key challenge facing multi-agent field-robotic systems operating in spatiotemporally changing domains: managing the trade-off between learning a robust generalized model, and learning a model which is specialized in order to most accurately explain the current observations of the agents. Our work is based on the Evolving-Gaussian Process model, and is trained on ecologically realistic simulations of weed growth cite{mcallister2018weed}. The E-GP model has the valuable benefit of being able to predict spatiotemporal dynamics throughout a continuous domain using a finite linear dynamical system embedded in a feature space. We advance this modeling technique by showing the model may be decoupled into two components, the first based on aggregated past observations, and the second adapted in real-time to capture the unique properties of the current environment.

Keywords: AI-Based Methods, Deep Learning in Robotics and Automation
Abstract: With the significant achievements in the AI field, the investigation of learning-based methods in robotics area has received more and more attention in recent years. However, for the learning-based methods, the issue of data requirement must be addressed first since the size and diversity of training data are crucial for the performance. However, it is still challenging for the existing datasets of 2D environments to meet such demand since their size as well as the variability are limited. On the other hand, the processing time to build the map through Simultaneous Localization And Mapping (SLAM) is time-costing, which is a bottleneck in training the neuron networks which routinely involves millions of trial-and-error episodes. These issues motivate the authors to develop a large-scale dataset, HouseExpo, and a fast simulation platform, Pseudo-SLAM, to improve the training efficiency.

Keywords: AI-Based Methods, Deep Learning in Robotics and Automation, Probability and Statistical Methods
Abstract: Predicting the trajectory of traffic-agents is an essential module for autonomous driving system. It is a great challenge on urban streets due to various lighting conditions and traffic densities. In this paper, we propose multi-class LSTM encoder-decoder networks which involve multi-branches. Each branch has an encoder-decoder structure and predict one class traffic-agent. For each agent, we predict n trajectories by different noises and select better trajectory by manual rule.

Keywords: AI-Based Methods, Grasping, Big Data in Robotics and Automation
Abstract: A robot picking system incorporating a deepconvolutional-neural-network (DCNN) and depth-image-based visual servoing (VS) is presented for textureless-object picking. The DCNN predicts the grasp point belonging to the highest graspable surface among a pile of boxes. Then, based on the predicted grasp point, the visual servoing system tracks the graspable surface and controls the manipulator to pick the target box. The system has been validated by an experiment for consecutively picking 10 piled up boxes. As a result, the DCNN successfully predicted the grasp points, and the success rate of the picking task was 87.3%.

Keywords: AI-Based Methods, Learning from Demonstration, Social Human-Robot Interaction
Abstract: Hybrid Cloud-Edge robotic systems are scalable, adaptable, and intelligent. However, network latency between the Cloud and the Edge presents a problem for responsive human-robot interactions (HRI). Generative models shared between the Cloud and the Edge for motion segmentation and synthesis can help mitigate network latency, as we showed previously when a human tele-operated a dynamic robot to draw handwritten letters in two phases: (i) direct tele-operation with latency; (ii) automatic letter drawing so the robot completes the letter before the tele-operator. These generative models are GMM-HSMM models trained with supervision on labels of alphabetical letters. Similar motion segments across different letters can represent unlabeled structural concepts. Discovering and interpreting these concepts can help us train more expressive generative models for motion segmentation. It can also help us train a new model faster from previously unseen samples based on an ensemble of models. In this work, we use one-shot meta-learning to observe an unseen demonstration to: (i) discover and explain motion segments within the unseen demonstrations automatically; (ii) reconstruct a new generative model from ensemble of models for human-robot interactive controls.

Keywords: Assembly, Contact Modelling
Abstract: Since robotic assembly is involved in contact between assembly parts, the robot or parts may be damaged when the assembly fails. To prevent this, the contact force generated at contact should be monitored. In this study, the force and torque data generated during the battery assembly using a 6 DOF industrial robot is statistically modeled in the time domain using the Gaussian process.

Keywords: Autonomous Agents, AI-Based Methods
Abstract: The proposed work aims at the introduction and utilization of a complexity conditioned goals technique to enable a Reinforcement Learning Agent to train efficiently. RL agents (Sparse or Densely rewarded, Multi-Goal), in general, pursue randomly sampled goals irrespective of their training status or "ability". A premature agent (early phases of learning) seeking a complex goal extends training time and challenges as the episode rollout would most certainly fail and not contribute to learning. The proposed method bases the sampled goals on an exponentially growing complexity factor that sets "easier" goals for the agent improving the likelihood of a successful rollout. The experiments executed on a Bit Flipping Environment shows promising results and seamless coordination with the existing HER technique.

Keywords: Autonomous Vehicle Navigation, Computer Vision for Other Robotic Applications, Environment Monitoring and Management
Abstract: The objective of this work is to develop a semi-autonomous robot for the treatment, reuse and removal of poultry manure. With this robot, it’s expected to increase the productive capacity of each farm up to an additional of 20% during each productive cycle and reduce the exposure of the operators to possible sources of infections (due to the manure). The robot is designed for carving the manure bed in all the poultry warehouse implemented with a path following controller and computer vision algorithms. Likewise, the robot is equipped with a system that sprays medicine and sensors connected to the wireless system in order to get information about important environment parameters (temperature, luminosity, humidity, ammoniac and pressure).

Keywords: Autonomous Vehicle Navigation, Social Human-Robot Interaction, Collision Avoidance
Abstract: In densely populated environments, human-aware navigation is critical for autonomous robots where interaction is expected. Human-aware navigation is challenging given the constraints of human comfort and social rules. Traditional trajectory prediction-based methods are unreliable in dense crowds because the history of individual motion is often not precisely attainable. Other approaches such as reinforcement learning fail to address the naturalness aspect in socially compliant navigation. We present an autonomous navigation system capable of operating in dense crowds and complying with latent social rules. The underlying system incorporates a deep neural network to mimic human motion patterns. In experiments, our robot vehicle is able to drive naturally in a densely populated area (10+ people in a 10mx20m area).

Keywords: Big Data in Robotics and Automation, AI-Based Methods, Grasping
Abstract: We present Graph Element Networks, an architecture that uses Graph Neural Networks (GNNs) as a flexible model for physical systems. In contrast to previous work, nodes in the GNNs are not tied to objects or object subparts and are instead used to model computational meshes where the number and position of the nodes can be chosen by the user.

We show this has three particularly useful applications for robotics: first, we can choose the runtime complexity of the network at test time by using more or less nodes, trading off compute with model accuracy. Second, GENs can be used for end-effector design by putting a mesh on top of the gripper, predicting the probability of grasps being successful and then back-propagating this probability through the GEN to optimize the gripper mesh so that grasping is more likely to be successful. This could open the avenue of automatically designing new grippers in simulation, custom to the objects they are manipulating. Finally, we show that GENs can be used as a flexible spatial memory.

Keywords: Biological Cell Manipulation, Automation in Life Sciences: Biotechnology, Pharmaceutical and Health Care, Automation at Micro-Nano Scales
Abstract: In this brief work, we present an automatic single particle patterning system to create a pattern from individual cells. In contrast to the conventional microfluidic configuration, beads laying on the substrate can be displaced to different positions relative to the patterning electrodes in the microchip so that only the selected beads will be trapped and patterned by the microchip. This approach allows us to create individual cells pattern.

Keywords: Biologically-Inspired Robots, Legged Robots, Mechanism Design
Abstract: Animals often use their external appendages (such as tails, limbs) to achieve spectacular maneuverability, energy efficient locomotion, and robust stabilization to large perturbations, which may not be easily attained in the existing legged robots. Their appendages, particularly, the tails are very compact, light, highly dexterous with a large range of motion (ROM). Animals can also curl and straighten up their tails in less than one-tenth of a second to facilitate rapid adjustment for the moment of inertia (IoM) and the Center of Mass (CoM). Most of the existing robotic tail designs still lack dexterity, output force, dynamic response. And there is no viable and compact solution to adjust the tail IoM rapidly and effectively. This project aims to address the current limitations in existing tailed robotic systems, in particular, we develop a bipedal tailed hopping robot to investigate how a large ROM morphable inertial tail can enable agile locomotion.

Keywords: Brain Machine Interfaces, Human-Centered Automation, Human Factors and Human-in-the-Loop
Abstract: In this paper, we propose a continuous neural control method by integrating an electroencephalography (EEG)-based brain-computer interfaces (BCIs) with an adaptive assistant controller for steering a vehicle. The assistant controller includes the fuzzy logic module and adaptive law. Experimental results show that the proposed method has better performance and less workload to users than the existing methods.

Keywords: Brain Machine Interfaces, Human-Centered Robotics, Human Factors and Human-in-the-Loop
Abstract: Decoding human motor intention from electroencephalograms (EEG) signals is of great value for human-machine collaboration. Existing studies are concentrated on single-hand motion decoding from EEG signals given the opposite hand is maintained still. However, in many real human-machine interaction systems, it is unrealistic to require operators to keep the opposite hand still all the time. In this paper, we investigated how to decode right-hand movement direction from EEG signals under a left-hand movement distraction. Experimental results show the feasibility of single-hand movement direction decoding from EEG signals under the opposite-hand movement distraction.

Keywords: Climbing Robots, Cellular and Modular Robots, Field Robots
Abstract: In this paper, static analysis of the wall to wall conversion is performed and tested and verified. The condition that the force of the front part of the module should be 0 or more was confirmed and the optimum tail force was calculated to satisfy the condition. In addition, although it was not possible to switch the inner wall of two modules in the existing paper, what was possible in this paper was verified.

Keywords: Climbing Robots, Field Robots, Mechanism Design
Abstract: This late-breaking poster paper presents a novel design of a wheel mechanism for obstacle overcoming on a façade. We design a 1 DOF rotating leg with different length spokes. It assembled designated angle on the tapered disk. This leg mechanism have an spatial efficiency of the design layout for a robot, also reduce any chance of interference between legs with other parts of robot. The design, mechanism of leg wheel and movement of overcoming obstacle are going to be shared in the followings.

Keywords: Climbing Robots, Legged Robots, Mechanism Design
Abstract: Our long-term objective is to develop a small tree climbing robot for ecological surveys to protect endangered birds and mammals nesting in trees. When investigators climb trees, there is the danger of a crash. In addition, they can disturb their nesting because animals can abandon their nests with such disturbance stimuli: approaching a larger object, approaching more quickly, and louder noises. Therefore, we deem it necessary to develop a small robot that climbs trees quietly and slowly on behalf of the investigators and acts as a tree camera. We developed a small tree climbing robot driven only by shape memory alloy (SMA) actuators, which are extremely silent, as a prototype. It has six legs, and the fore and middle legs each have an antagonistic drive mechanism consisting of an SMA actuator and spring. Since the robot has six legs, the leg should be lightweight to make the robot small. Moreover, the legs must have spring elements for the antagonistic mechanism. In this paper, we present the design of a multifunctional light leg for a small tree climbing robot. We developed a stainless-steel wire leg. It was designed by modeling each component with rigid links. Its weight is 0.25 [g]. It has three coil parts. One is a spring element for roll rotation, the others are holes for passing the SMA actuator. The elasticity of the stainless-steel wire itself is used as a spring element for yaw rotation. The end of the leg is processed into a needle shape and plays the role of a nail. Thus, by processing one wire and giving various functions, the number of parts is reduced, and weight reduction is realized. In addition, the leg is reinforced with UV resin so that the through holes do not function as spring elements. To confirm effectiveness of the reinforcement, we reinforced with beams instead of UV resin and simulated the deformation of the leg during roll and yaw rotation. In both roll and yaw rotations, the reinforced one showed a behavior closer to the model. As a result, the robot climbs a cedar tree tilted up to 30 [deg]. The minimum climbing speed at that time was 0.18 [mm/s]. The reason why the robot failed to climb a surface inclined at more than 30 [deg] is that the tree contact position of the leg shifts in the positive z axis direction. This deviation occurs because the force direction of the SMA actuator changes slightly depending on how the leg is lifted. However, it is difficult to control the force direction. Therefore, a novel plate leg is presently designed to limit the direction of deformation by twisting a part of the thin plate and to eliminate the displacement.

Keywords: Cognitive Human-Robot Interaction, Sensor-based Control, Deep Learning in Robotics and Automation
Abstract: We propose a wearable sensor-based gait classification system. Our approach assumes that multiple IMU sensors attached to various body parts can capture the gait characteristics that are used to predict whether the subject has foot abnormalities or athletic performance. We first transform raw sensor signals into spectrogram images by using the Short-time Fourier transform (STFT) and feed this visual representation to four deep CNN networks, i.e., the newly proposed network, Resnet18, Resnet50, and Densenet121. The IMU data were acquired from 7 sensors attached to the pelvis, thighs, shanks, and feet while 29 semi-professional athletes (AT group), 19 normal participants (N group), and 21 participants with foot abnormalities (AB group) walked on a 20-m straight path. We investigated classification accuracy according to the number and location of attached sensors to optimize performance. In addition, we adopted IMU-based spectrogram approaches with deep CNN models for the gait classification with two different types of input layouts. Our experimental results show that in the case of using a single sensor, the best classification performance was achieved on the pelvic spectrograms. The accuracy increases significantly when we used the combinations of multiple sensors. The classification performance is remarkably increased in the stacked input layout than the flat one. Although the proposed model is simpler than the three existing well-known networks, it shows a comparable classification result. Our proposed network that only used a single IMU sensor data at pelvis can predict the subject groups with an accuracy of 97.58% even without requiring hand-craft extraction and selection of features. Moreover, when we combined the pelvis and two feet sensors data and produced the stacked input layout, the accuracy was increased up to 98.19%. Consequently, practical applications could use two IMU sensors attached in two shoes and one IMU sensor in a belt worn around the waist to help ambient intelligence easily recognize people’s gait group. Obviously, this use of a small number of sensors located on the human body could allow us to save power consumption and make applications more realistic. We made confusion matrices showing the number of correct and incorrect prediction of the proposed model in accordance with the types of input layout with different sensor combinations. No cases are reported, which incorrectly predict N or AT group to AB group, despite some confusions between the N group and the AT group. We plan to generalize our approach to predicting various health status information of people living in the ambient intelligent environment.

Keywords: Collision Avoidance, Visual Learning, Visual-Based Navigation
Abstract: We introduce AI technique in our new Robotic Vacuum Cleaner (RVC), named CodeZeroTM Hom-Bot. Hom-Bot is able to detect obstacles with a camera and learn them through state-of-the-art AI techniques. When Hom-Bot encounters pre-trained obstacles on the path, the robot can easily avoid them and keep its own cleaning task. In addition, Hom-Bot can recognize a known place in the cleaning environments with visual place recognition technique.

Keywords: Compliance and Impedance Control, Motion Control, Physical Human-Robot Interaction
Abstract: The proposed external force estimator calculates the net external force from the difference between the force/torque generated by the joint and the force/torque nominally needed for the robot motion using the trained recurrent neural network (RNN) inverse dynamics model. The experimental setup consists of a robot, PA10-7C (Mitsubishi Heavy Industries) and a force sensor (NITTA) installed at between the robot and the manipulated object. The object is an aluminum baton (hereinafter referred to as the baton), on which a weight movable on the linear guide is installed to apply the external force during the baton manipulation. In the experiment, a person touches the baton manipulated by the robot. Then, it is checked if the compliant motion to the external force calculated by the proposed method is properly generated. The experimental result showed that the robot gives way to the person and absorbs the collision force according to a desired mechanical impedance model. The advantages of the proposed external force estimator are summarized as follows: - Even while the robot dynamically manipulates the object, it is possible to estimate the net external force in distinction from the manipulation force, - The values of physical parameters of the robot are not explicitly needed to obtain inverse dynamics model, - If the friction force and/or the parameter error, which are difficult to be modeled, have reproducibility, a model as a part of the inverse dynamics model can be acquired.

Keywords: Compliant Joint/Mechanism, Legged Robots, Motion Control
Abstract: Considering the hardware complexity of torque-control motors, position-control motors are usually used in small-size quadruped robots and for prototype testing. However, due to the high gear ratio of position-control motors, elastic components are needed to achieve high dynamic motions including jumping or running. Instead of designing compliant legs, we propose using elastic surfaces (such as a trampoline) as energy storages to help a rigid-leg quadruped to jump. Thus, the jumping control algorithms could be developed first, then used to test further compliant legs design, and a jumping robot with elastic legs could be finally built. In this poster, we present the algorithms we used to achieve stable quadrupedal hopping on a trampoline. Experimental results are also breifly illustrated.

Keywords: Computer Vision for Automation, AI-Based Methods, Aerial Systems: Mechanics and Control
Abstract: This paper deals with classification and prediction of low-altitude rocket targets using deep neural networks. Conventionally, model-based methods such as the Kalman filter are widely used. However, they can lack robustness due to unexpected situations and noisy sensor measurements. To address this issue, this study proposes the use of various data-driven methods which are powerful on situations where no model is available. Specifically, three types of neural networks are used: DNN (deep neural network), CNN (convolutional neural network) and RNN (recurrent neural network). To verify the benefit and robustness of the proposed algorithms, comparisons with the model-based method are performed on several scenarios

Keywords: Computer Vision for Automation, Deep Learning in Robotics and Automation, AI-Based Methods
Abstract: This poster introduces an automated wrinkle detection method on semi-finished fiber products in the aerospace manufacturing industry. Machine learning, computer vision techniques, and evidential reasoning are combined to detect wrinkles during the draping process of fibre-reinforced materials with an industrial robot.

Keywords: Computer Vision for Automation, Deep Learning in Robotics and Automation, Computational Geometry
Abstract: In this paper, we investigate a problem of objects pose estimation for objects with symmetry. The proposed method determines ambiguity poses of objects with rotational symmetry. We use a deep neural network to estimate a set of angles which correspond to the projection of the object on the image plane.

Keywords: Computer Vision for Automation, Computer Vision for Other Robotic Applications, Object Detection, Segmentation and Categorization
Abstract: Recently, a lot of high-rise buildings have been built and the need for an unmanned exterior wall cleaning robot is increasing. Therefore, a system for detecting various façade contamination is required to robot just like as workers do. However, existing surface contamination detecting systems can only detect certain type of contaminant or not compact enough to be attached in mobile platforms. In this paper, we try to solve this problem with machine vision system through CNN and image processing method. This makes it possible to detect object, area and particle type contaminants using only one photograph

Keywords: Computer Vision for Automation, Object Detection, Segmentation and Categorization, Inventory management
Abstract: With the goal of specializing in the manufacturing of the world’s best steels, POSCO focuses on enhancing its competitive edge from raw material to final product. Especially, we have the world’s largest integrated steel mill with more than 2,700 hectares and it makes us interested in geospatial information technology including the inventory management system of ore and coal stockpiles. Huge raw material yard over 350 hectares has been managed based on the naked eye of many skilled operator in the past. On that account, the measurement accuracy was not ensured and there was a possibility of safety risk during field survey. Thus commercial unmanned aerial vehicle, in other words, drones are deployed for highly accurate surveys of stockpiles, Generally, drone can acquire a number of aerial images using auto-grid flight plan with a given mission. However, it is necessary to automate the image process after flight in order to increase the frequency of use, because the general aerial survey is professional and cumbersome task. One of the major challenges is automatic geolocation because manual correlation of ground control point (GCPs) in hundreds of photos with a high-resolution is very rigorous and tiresome. Thus we implement the auto-GCP algorithm to save time on manual marker placement and to match geolocation with more precision [2]. In addition, it is considered how many GCPs have to be required, as well as where and how they have to be measured considering the flying height, imager size, field of view (FOV), and overlap rate. Another challenge is that it is difficult to classify variety of stockpiles from a lot of facilities around the raw material yard in steelworks. The boom of stacker and reclaimer passes over the top of the stockpile and it makes occlusion shadows. Belt conveyors, concrete retaining walls, and embankment near stockpiles also complicate the automatic classification. Thus, the smoothing filter and triangulated digital terrain models (DTM) on specified criterion are implemented. In addition, because the terrains on which these stockpiles lie are not flat, the ground elevation is compensated [3]. The above whole process is summarized in Fig. 1. As a result, the proposed autonomous photogrammetry process can reflect high geometric fidelity of stockpile and calculate the volume of hundreds of stockpiles within an accuracy of 3% in four hours, respectively.

Keywords: Computer Vision for Other Robotic Applications, Energy and Environment-Aware Automation, Aerial Systems: Perception and Autonomy
Abstract: An autonomous system for the detection of solar panels using Unmanned Aerial Vehicles(UAVs) is proposed in this paper. There has been a huge growth in the solar energy field. In Dubai alone there been a massive group of solar farms providing a huge amount of electricity. If the inspection of the PV panels is done completely manually this would require a large amount of manpower and resources not to mention a lot of time. The aim is to have a system in which a drone flies over fixed waypoints that have been set previously based on GPS data. The drone will take images and capture videos which will be used to detect the panels.

Keywords: Contact Modelling, Simulation and Animation, Hydraulic/Pneumatic Actuators
Abstract: Co-simulation can be used to couple subsystems that present different time-scales, such as hydraulics. However, numerical stability of the co-simulation setup can be compromised by discontinuities and time delays of the coupling variables. Here, we use a reduced-order model of the mechanical subsystem based on the effective mass at the interface in order to obtain a prediction of the coupling variables, which allows for larger communication steps and increases the stability of co-simulation setups.

Keywords: Deep Learning in Robotics and Automation, Performance Evaluation and Benchmarking
Abstract: Meta-reinforcement learning algorithms can enable robots to acquire new skills much more quickly, by leveraging prior experience to learn how to learn. However, much of the current research on meta-reinforcement learning focuses on task distributions that are very narrow. For example, a commonly used meta-reinforcement learning benchmark uses different running velocities for a simulated robot as different tasks. When policies are meta-trained on such narrow task distributions, they cannot possibly generalize to more quickly acquire entirely new tasks. Therefore, if the aim of these methods is enable faster acquisition of entirely new behaviors, we must evaluate them on task distributions that are sufficiently broad to enable generalization to new behaviors. In this paper, we propose an open-source simulated benchmark for meta-reinforcement learning and multi-task learning consisting of 50 distinct robotic manipulation tasks, with the aim of making it possible to develop algorithms that generalize to accelerate the acquisition of entirely new, held-out tasks. We evaluate 6 state-of-the-art meta-reinforcement learning and multi-task learning algorithms on these tasks. Surprisingly, while each task and its variations (e.g., with different object positions) can be learned with reasonable success, these algorithms struggle to learn with multiple tasks at the same time, even with as few as nine distinct training tasks. Our analysis and open-source environments pave the way for future research in multi-task learning and meta-learning that can enable meaningful generalization, thereby unlocking the full potential of these methods.

Keywords: Dexterous Manipulation, Multifingered Hands
Abstract: This paper proposes subsynergy which provides a synergy-based control method for multi-fingered hands under selected joint spaces. Subsynergy is a synergy composed of subsets of fingers or joints needed for performing a specific task. By using subsynergy, we can perform several different dexterous tasks by controlling high DOF multi-fingered hand with lower dimensional inputs compared to than conventional synergies.

Keywords: Dual Arm Manipulation, Marine Robotics, Force Control
Abstract: This paper describes an efficient collaborative method of underwater robot-manipulator. We used the weighted pseudo-inverse of the manipulator's collaborative controller to derive the most efficient control method and experiment.

Keywords: Field Robots, Mechanism Design, Robotics in Hazardous Fields
Abstract: This late-breaking poster paper presents the experimental study on the parameters of high-pressure water cleaning on a facade. The cleaning performance parameters, such as water pressure, spray angle and spraying distance, were optimized by using Taguchi method. The performance was evaluated by using the image evaluation method. Also, we looked over the reaction force on nozzle tip and impact force on the specimens to know the effect on building surface.

Keywords: Field Robots, Sensor Fusion, Localization
Abstract: Despite the wide utility, RGB cameras and Light Detection and Ranging (LiDAR) have been reported to be vulnerable in low-visibility environments under fire or smoke. To Solve this issue, we introduce a sensor system that consists of a thermal infrared (IR) camera and an mmWave radar. In doing so, extrinsic calibration between two sensors is required, and 14-bit temperature and sparse range measurement from the radar are challenging to calibrate. We propose a solution to this multimodal calibration method by using an RGB-D sensor as an intermedium for the depth based optimization. As a validation for the relative pose estimation, we present the qualitative result by projecting the radar’s depth on the thermal camera frame.

Keywords: Force and Tactile Sensing, Soft Sensors and Actuators, Biomimetics
Abstract: The use of surgical robots in the field of minimally invasive neurosurgical procedures can offer several benefits and advantages. However, the lack of force sensing hinders and limits their use in such procedures. Equipping surgical robots with pressure sensors can enhance robot-environment interaction by enabling collision awareness and enhance human-robot interactions by providing surgeons the necessary force feedback for safe tissue manipulation. With the emergence of soft robotics in biomedical applications, the attached pressure sensors are required to be flexible and stretchable in order to comply with the mechanically dynamic robotic movements and deformations. Inspired by the multi-dimensional wrinkles of Shar-Pei dog’s skin, we have fabricated a flexible and stretchable piezoresistive pressure sensor consisting of MXene electrodes with biomimetic topographies.

Keywords: Force Control, Cellular and Modular Robots, Industrial Robots
Abstract: Industrial robots have multiple degrees-of-freedom (DOFs) and high dexterity but they currently have two mechanical challenges. First, they have excessive joint accelerations when encountering paths near the singular positions. Second, the large link inertia and gearbox friction make the contact force control of the robot end-effector difficult no matter where the torque sensors are placed. A macro-micro robot is proposed in this paper to address these two challenges. An existing 6-axis robot serves as the macro robot and a 2-axis wrist module is serially attached to the end of the macro robot to form a macro-micro robot. The micro robot is compact with high torque density and internal force sensing capability. The introduced redundant DOFs at the distal end of the macro robot allow more accurate force control and smoother path tracking. Simulation and experiment results are provided to show the advantages of the proposed design. We expect that this macro-micro robot can be used in applications where smooth motion or accurate contact force control is required.

Keywords: Force Control, Legged Robots, Motion and Path Planning
Abstract: The control of a legged robot's contact forces to lie within their friction cone is vital to avoid slippage during the robot's locomotion. However, a nominally actuated leg loses its ability to track the desired contact forces, when a fault develops, such as the locking of a joint. Current literature on fault tolerant control on multi-legged robots focuses on modifying the robot's gait but do not address the force tracking problem even with legs that have a force sensor attached to the foot. This research proposes two additional modules to the general architecture of a multi-legged robot to modulate the robot's pose such that all the feet are guaranteed ground contact and force tracking. Preliminary results using the Gazebo simulator shows that force tracking on all legs, including the faulty leg, is achievable using the proposed approach.

Keywords: Force Control, Motion Control, Contact Modelling
Abstract: This study proposes an admittance control method based on the stiffness ellipse. Some of assembly tasks in factories are quite difficult to automate, because the discontinuous transitions between contact and noncontact phases generate issues. In the contact phase, a control approach is required that handles assembly parts with compliance, while a control method that follows the trajectory in the noncontact phase. To achieve an assembly task, it is often necessary to switch the control when the contact occurs, while such a controller design often degrades the performance owing to the switchover. This study solves this problem by adjusting the stiffness ellipse by admittance control. By tilting the stiffness ellipse, the external force can be induced in a desired direction during contact, without switching the control or adjusting the position/force command. Using the proposed method, tasks including contact and noncontact phases can be achieved with continuous control. Experimental results on a drawing task and simulation results on drawing and peg-in-hole tasks show the advantage of the proposed method.

Keywords: Force Control, Soft Sensors and Actuators, Flexible Robots
Abstract: A series elastic actuator (SEA) combines an actuator in series with an elastic spring. By controlling the deformation of the elastic spring, an SEA provides more accurate force and impedance control than conventional rigid actuators. SEAs are ideal for robots and machines that need to interact safely with human or the environment. The majority of existing SEAs uses brushless or brushed DC motors as the actuators. The advantages of using step motors as the actuators of SEAs have not received enough attention. Step motors have much higher torque-to-rotor-inertia ratios than other DC motors. Hence they can provide better stability and high-speed accuracy of force control while maintaining lightweight. When rotor position feedback is used, step motors can achieve smooth and ultra-accurate dynamic force response. This paper develops the dynamic model of a linear series elastic step motor and presents its prototype. Forward and inverse force/impedance tracking control responses will be provided to show the advantages of the series elastic step motor. It is expected that the method presented here can offer a better actuator selection of SEAs when higher torque-to-rotor-inertia ratios are required.

Keywords: Grasping, Learning from Demonstration
Abstract: We propose a framework for generating coordinated reaching and grasping motions of a robotic hand-arm system by learning from human demonstration. Our framework consists of the three stages. The first stage is movement type determination. A mixture version of the conditional restricted Boltzmann machine (CRBM) lies at the core of this stage, which is called the convolutional implicit mixture of CRBMs (conv-imCRBM). To select a mixture mode for the next stage, a movement type is determined by the trained conv-imCRBM given the target object pose and the initial state of hand-arm system. The second stage is trajectory generation given the position and orientation of the target object. In this stage, a control variable l is added to the original CRBM model for encoding target object pose. K-step iterative Gibbs sampling is used in the inference process for the trajectory generation, and then control variable l is updated with gradient descent method to minimize the distance between the target object and robot hand. The final stage is to maximize grasp quality by fine-tuning the trajectory from the previous stage. Movement trajectory is modified to ensure the force closure condition by optimizing the minimum singular value of a grasp matrix as a grasp quality measure.

Keywords: Haptics and Haptic Interfaces, Deep Learning in Robotics and Automation, Force and Tactile Sensing
Abstract: Combining information from partial observations across multiple sensory modalities to execute goal directed actions is a key aspect of human intelligence. In order to build agents with similar capabilities, we consider the problem of identifying a target object from a drawer using tactile exploration only. We use a simulated anthropomorphic hand with continuous controls and high dimensional motors. The agent is able to interact with several objects in a box by touching them. The trial is a success if the agent is able to successfully identify the target object. Success at this task requires both the construction of a sequence of controls used to drive the hand's motors and learning to discriminate objects from the haptics signal received during an interaction. Formulated as a reinforcement learning problem, we compare the performance of various policies at this task, introducing intrinsic curiosity and classification-based extrinsic reward as a method for learning useful exploration policies in the domain of classification.

Keywords: Haptics and Haptic Interfaces, Recognition, Perception for Grasping and Manipulation
Abstract: Touch sensation of the finger is very important information for human behavior. The goal of this study is to investigate whether social perception and behavior are influenced by the touch sense of human finger using haptic interface device. In this paper as the pretest of a haptic interface device, experiments are tested whether a human finger attached to a real soft rubber ball or hard wire ball is affected on social perception and behavior of people. Experimental results show that information of a reaction force acted on a human finger has no influence on social perception, and show that the hardness of object has a few influence on social behavior.

Keywords: Haptics and Haptic Interfaces, Rehabilitation Robotics, Soft Robot Applications
Abstract: This paper introduces a tactile stimulation system for robot-assisted hand rehabilitation. The tactile stimulation was synchronized with the motion of the hand exoskeleton. Robot-assisted hand trainings with and without tactile stimulation were compared in an experiment involving healthy human subjects. The experimental results confirmed the increase of attention level of human subjects during the hand training process.

Keywords: Human Detection and Tracking, Object Detection, Segmentation and Categorization, Computer Vision for Transportation
Abstract: Pedestrians, the most vulnerable part in the transportation system, are more likely to get hurt because of careless drivers. Some Advanced Driver Assistance Systems (ADAS), like adaptive cruise control and pedestrian automatic emergency braking, are mature enough to be adopted in real-world vehicles. However, there is not an effective method to detect moving objects or pedestrians blocked by other vehicles. This paper aims to tackle this problem by sharing the view between vehicles through Vehicle-to-Vehicle (V2V) communication.

The overview of view sharing is illustrated in Figure 1. The camera in vehicle 1 detects the pedestrian whose bounding box location is then transformed to vehicle 2’s coordinate and displayed on its windshield. The flowchart of the view sharing system is shown in Figure 2. We detect pedestrians via the video stream from a camera placed inside vehicle 1, v1. The detected pedestrian can be located on the image plane as a 2D vector. On the other hand, to share the view, we need to estimate the real-world location (3D vector) of the pedestrian with respect to the vehicles. Therefore we need conduct coordinate transformation, which calls for camera calibration. Based on that, we can estimate the real-world location of the pedestrian with respect to vehicle v1. Then we transform the 3D-location from v1’s coordinate to v2's coordinate. Finally, a coordinate transformation is applied to estimate the corresponding 2D-pixel location of the pedestrian on the windshield of v2.

We conducted experiments on a custom-built assisted driving platform which includes two driving simulators running on two computers. Behind the driver seat of simulator 1, we mounted a camera which is connected to another computer to detect pedestrians. In this way, we can distribute the computation load among the three computers. The three computers communicate and share data with each other via UDP communication.

Figure 3 shows the bounding box of the detected pedestrian on the simulator 2. To evaluate the detection performance, we calculated the Intersection-over-Union (IoU) ratio between the detected bounding box and the ground truth one. The IoU ratio varies with different locations of vehicles and the average IoU is 70%. In addition, we also estimated the real-world error by comparing the ground truth location (gx, gy) with the estimated real-world location of the pedestrian with respect to v2. The average errors in both the x and y directions range from 0m to 1m.

Keywords: Human Factors and Human-in-the-Loop, AI-Based Methods, Wearable Robots
Abstract: We propose a structured motion classification strategy for high level wearable robot control and present preliminary results with lower limb motion classes.

Keywords: Human-Centered Automation, Motion and Path Planning, Telerobotics and Teleoperation
Abstract: The current state of the art of robot motion planning in a cluttered unstructured environment is subject to fail. However, introducing human intuition into the motion planning pipeline improves the performance of the planning algorithm. In this research, we propose a user interface to record a suggestion of a trajectory to ensure the convergence of the planning algorithm and reduce the computational time. Simulation results demonstrate the superior performance of the proposed motion planning method.

Keywords: Human-Centered Robotics, Human Factors and Human-in-the-Loop, Human-Centered Automation
Abstract: Motion intention estimation is critical to active human-robot collaboration. However, existing studies on electroencephalograms (EEG)-based motion intention estimation are all focused on motion intention decoding from EEG signals when subjects perform a motion task by using a single hand or foot, given no perception, cognitive, and motor distraction tasks. However, for healthy people, in many cases, they need to complete a motion task given a distraction task. In this paper, we propose a decoding model of the motion direction of the upper limb from EEG signals with a cognitive distraction task. Experimental results suggest that the decoding accuracy of the proposed model is 90%, close to that without a cognitive distraction task.

Keywords: Humanoid and Bipedal Locomotion, Compliant Joint/Mechanism, AI-Based Methods
Abstract: Most state-of-the-art bipedal robots are designed to be as anthropomorphic as possible, and therefore possess articulated legs with compliant elements in the joints to increase energy efficiency. Whilst this facilitates smoother, human-like locomotion, there are implementation issues that make walking with straight or near-straight legs difficult. Many robots have to move with a constant bend in the legs to avoid a singularity occurring at the knee joints or to keep the center of mass at a constant height for control purposes. The actuators must constantly work to maintain this stance, which can result in the complete negation of any energy-saving techniques employed. Furthermore, vertical compliance disappears when the leg is straight and the robot undergoes high-energy events such as impacts from running and jumping, as energy travels through the fully extended joints to the hips. We attempt to improve energy efficiency by attaching bungee cords as elastic elements in parallel to the legs of a novel, knee-less biped robot. Bayesian Optimization is utilized to find the optimal vertical motion behavior that achieves the largest reduction in energy consumption, taking into account the dynamics of the bungee cords as well as friction and other unmodelled effects. Our experiments find up to a 15% energy saving compared to the robot configuration without parallel elastic elements.

Keywords: Humanoid and Bipedal Locomotion, Deep Learning in Robotics and Automation
Abstract: Moving in dynamics and challenging environments is one of the key issues of robot locomotion. Most of the previous work on this area has focused on the task of solving specific problems, such as efficient learning of compact motor primitives, exploitation of bio-inspired motion models, or fall detection. We propose a general framework for learning general locomotion skills, and we propose a methodology to learn skills that are flexible and adapts to current sensory measurement and environment characteristics. To achieve high performances and generalization we build our framework on top of the previous works on Dynamic Motion Primitives and Synergies and improve their generalization capabilities with state of the art Deep Reinforcement Learning approaches.

Keywords: Humanoid and Bipedal Locomotion, Mechanism Design, Simulation and Animation
Abstract: This paper presents a Taguchi method optimizing hip joints in humanoid robots in terms of joint offsets. By using MuJoCo simulation, the effects of each offset condition were analyzed for the sum of the RMS power of three hip joints during target motion. The optimized model was compared to the initial model in terms of the RMS power of the sum of total hip joints to verify the improved energy efficiency.

Keywords: Humanoid Robots, Motion Control
Abstract: This article presents a Whole-Body Postural Control (WBPC) approach for a waiter humanoid robot. This approach is based on the concept of a multi-ZMP evaluation system to control body and transported object stability. Both controllers were developed independently, avoiding cross-linked perturbations. These controllers that deal with this complexity were proposed in previous research. The first one was an improvement for balance control of the body (locomotion) and, the second one was a method to apply classic body balance concepts for transporting objects on a tray (manipulation).

Keywords: Humanoid Robots, Optimization and Optimal Control, Climbing Robots
Abstract: This paper describes a method to generate a jump trajectory that will enable a humanoid robot to climb an obstacle. For a robot to climb an obstacle by jumping without colliding with the obstacle, the robot should jump higher than the height of the obstacle and move to the desired horizontal position. Therefore, to perform such a dynamic motion, the trajectory needs to consider the capacity of the actuator, impact force at landing, friction force, zero moment point, and so on. This process was defined as a nonlinear optimization problem, and we successfully generated the optimal jump trajectory. This trajectory was verified using ROK-3, an adult-sized humanoid robot.

Keywords: Hydraulic/Pneumatic Actuators, Flexible Robots, Soft Robot Materials and Design
Abstract: In this project, we present a 6 degrees of freedom hybrid robotics design with both hard and soft components. The robot is constructed with lightweight material and actuated with low pressure air, making it safe for human interaction and operation. The proposed design is modular and scalable.

Keywords: Industrial Robots, Human-Centered Robotics, Formal Methods in Robotics and Automation
Abstract: This paper presents a methodology that enables the exploitation of innovative technologies for collaborative robots through user involvement from the beginning of product development. The methodology will be applied in the EU-funded project CoLLaboratE that focuses on how industrial robots learn to collaborate with human workers in order to perform new manufacturing tasks. The presented methodology is preliminary and will be improved during the project runtime.

Keywords: Intelligent and Flexible Manufacturing, Tendon/Wire Mechanism, Parallel Robots
Abstract: XL-laser is a 6 DoFs spatial portable cable-driven parallel Robot (CDR) mounted with a 5.5W laser head to the end-effector. The leaser head powered by lithium battery emits high power visible light laser. It can cut and engrave on various materials. With the advantage of CDR, the XL-laser can also be scaled to cut large work-piece. It has been made with different sizes from centimeters to several meters. The 6-Dofs PoCaBot also allows cutting on an curved or inclined surface.

Keywords: Intelligent Transportation Systems, Cognitive Human-Robot Interaction, Visual-Based Navigation
Abstract: We have attempted to model the interactions between pedestrians and vehicles to create mechanisms that prompt safer interactions. In this paper, we discuss human-vehicle visual interactions that are effective at night and introduce our proposed framework and robotic platform.

Keywords: Learning from Demonstration
Abstract: Learning how to segment complex trajectories in meaningful simpler trajectories have become lately an important research topic for many fields from Computer Vision to Reinforcement Learning. In Robotics, trajectory segmentation could be applied for Imitation Learning. Robots would be able to split human demonstrations for a particular task in semantically meaningful segments and so, build a library of simple actions. While both supervised and unsupervised methods have been proposed, supervised methods are limited to a defined number of actions and so, unsupervised methods have shown better results adapting to new actions. We identified two main limitations of unsupervised methods for segmentation. On one side, the trajectories tend to be segmented based on some heuristics defined by the user. On the other side, these segments use to be clustered based on some similarity measurements defined by the user. In order to deal with these limitations, we have developed a semi-supervised learning procedure in which we will learn how to both segment and cluster the trajectories based on human demonstrations.

Keywords: Learning from Demonstration, Compliance and Impedance Control
Abstract: In many robot control problems, data are found in form of symmetric positive definite (SPD) matrices, e.g. stiffness and damping in impedance controllers, manipulability ellipsoids in tracking control, etc. Such data encapsulate specific geometry characteristics which imply special treatments. In this context, we introduce a new learning-from-demonstration platform that can learn SPD profiles (parametrized using Cholesky factorization or Riemannian metrics) from expert demonstrations and adapt to new situations. This approach uses kernelized movement primitives (KMP) whose predictions allow for the retrieval of proper SPD matrices. The proposed approach has been validated in simulations by learning and adapting stiffness ellipsoids.

Keywords: Legged Robots, Motion and Path Planning, Optimization and Optimal Control
Abstract: This paper addresses the problem of achieving energy-efficient locomotion in statically stable walking quadrupedal robot. Conventionally, both of the locomotion planner and locomotion controller of a quadrupedal robot do not consider energy efficiency, as a result, the energy consumption is usually quite huge. This paper focuses on maximizing locomotion efficiency in two ways: Firstly, a COG (center of gravity) trajectory optimizer is used to control the trunk kept a constant distance to its supporting surface. As a result, the COG fluctuations are significantly suppressed. Secondly, a foothold planner is used to select footholds which can generate postures for the stance legs with minimized sum of joint torque. Both of the COG trajectory optimizer and foothold planner are based on the real-time terrain map information. Our approach brings some constraints in locomotion, but results in lower energy consumption. To this end, we present a novel energy optimization approach that enables the robot to locomote efficiently. We experimentally validate our approach with the quadrupedal robot Pegasus using statically stable gait by autonomously traversing typical terrains such flat surface, slope and stairs. The experimental results indicate that our approach has a potential for achieving energy-efficient locomotion.

Keywords: Localization, Mapping, Autonomous Vehicle Navigation
Abstract: This paper presents a preliminary experimental result of testing pose-graph based indoor navigation for the purpose of application to unmanned underwater vehicles (UUV). To verify the usefulness and to estimate the performance of the pose-graph technique, we conduct an indoor test using a ground robot kobuki [1]. In this experiment, the robot travels indoor at an office and at a corridor. This allows creating a pose-graph of robot path continuously and building a map using a light detection and ranging (LIDAR) sensor at the front of the robot, simultaneously.

Keywords: Localization, Micro/Nano Robots, Automation at Micro-Nano Scales
Abstract: The rapidity developing field of nanomedicine can significantly impact human disease therapy. Magnetic drug targeting using magnetic microrobots offers a potential solution. To perform the technique of non-invasive drug delivery with high precision, we develop a magnetic sensor based probe for microrobot detection and localization

Keywords: Manipulation Planning, Motion and Path Planning, Kinematics
Abstract: This poster proposes an efficient and probabilistic complete planning algorithm to address the motion planning problem with soft constraints for spherical wrist manipulators. First, we present a novel configuration space termed Composite Configuration Space ("Composite Space" for short), which is composed of joint space and task space. Then, a collision-free path is generated in the Composite Space by the Rapidly-exploring Random Trees (RRT) algorithm. Finally, the planned path based on the Composite Space is mapped into the corresponding joint-space path by a local planner. The local planner utilizes the analytical inverse kinematics (IK) solver of spherical wrist joints to calculate the joint configurations, thus the proposed planner is characterized by high efficiency and no numerical iteration. This approach can effectively improve the smoothness of the end-effector and joint trajectories. The effectiveness of the proposed algorithm is demonstrated on the Willow Garage's PR2 simulation platform in a wide range of orientation-constrained scenarios.

Keywords: Mapping, Multi-Robot Systems, Optimization and Optimal Control
Abstract: This paper addresses the real-time optimization problem of grid map merging in embedded robotics systems. When sampling-based optimization such as particle swarm optimization (PSO) is applied to solve the problem, it should be accelerated to satisfy the real-time requirements of embedded robotic systems. This paper proposes a new variant of the PSO conducted on a field-programmable gate array (FPGA) and can reduce computation times by paralleling the computation blocks based on hardware resources on a FPGA. The proposed method was implemented with synthesizable hardware description languages and evaluated by post-layout simulations through FPGA development tools.

Keywords: Mapping, Visual-Based Navigation, Legged Robots
Abstract: In this work-in-progress report, we present a mapping method of stairs for a quadruped robot based on point-cloud measurements and stair's geometric property. Because of the sensor noise from the distance sensor, the mapping result from the conventional point-cloud processing mapping algorithm like occupancy mapping is highly affected by the noise. As a result, the mapping methods, which are only based on sensor measurement, can not be fully reliable. Unlike the other mobile robots that use the map information of their surroundings for path planning or localization, the quadrupedal robot uses the map information for actual contact. Therefore, accurate map data is required for the robot's locomotion. We overcome the sensor noise by representing the mapping by the plane base. Each step of the stairs is represented in its representative plane model, which is calculated by fusing measurement and estimation plane model. The suggested stair mapping algorithm can be used for locomotion and localization of quadrupedal robot on the stairs. The coverage of the algorithm is not limited only for a quadrupedal robot but any robot that tries to navigate on the stairs.

Keywords: Marine Robotics, Model Learning for Control, Kinematics
Abstract: A large portion of the earth surface is covered by ocean, most of which still remains unexplored due to the complex underwater environment. To explore the complex underwater world, autonomous underwater vehicle(AUV) with both high speed and high maneuverability is required. To fulfill the requirement, an AUV which is equipped with six fixed thrusters and two vectored thrusters is designed. By changing the configuration of the thrusters, it can achieve movement state transformation. The design, simulation and experimental results will be shown in this poster.

Keywords: Marine Robotics, Motion Control, Sensor-based Control
Abstract: This late-breaking poster paper presents hovering control of an underwater robot with tilting thruster. The tilting thruster mechanism can implement six-degree-of-freedom (DOF) motion with only four thrusters, but tilting motion makes the system nonlinear. In order to solve the nonlinear problem, nonlinear thrust vector is appropriately decomposed, then we added some constraints with extra degrees of freedom using null space approach. We designed PD controller obtaining thrust forces and continuous tilting angles and applied it to the robot system. We verify improving the hovering ability of the robot by simulation.

Keywords: Marine Robotics, Surveillance Systems, Field Robots
Abstract: This paper describes the framework for a vision-based detection system of unmanned surface vessels (USVs) with minimizing human intervention. In order to compensate each other's drawbacks due to characteristics of each camera, the system consists of an electro-optical (EO) camera, an infrared (IR) camera, and panorama cameras for omnidirectional situational awareness. For object identification and recognition, a systematic procedure using image processing and convolutional neural network (CNN) approaches is framed. As preliminary study, the performance and practical feasibility of the proposed approach demonstrates using field test dataset.

Keywords: Mechanism Design, Cooperating Robots
Abstract: The cooperative robot is one of the innovative robot which leads industrial automation by combining human skill with the strength, speed, repeatability and precision of robot in the human-robot coexistence environment. Many researches have been studied about the actuation modules for the cooperative robot in which a motor, a reducer, an encoder and sensors are integrated for the advantages of manufacturing, maintenance and flexibility. In the paper, a new concept of the actuation module with multi sensors is proposed for the safety and maintenance and is optimally designed through the integrated analysis of structural analysis and thermal analysis.

Keywords: Mechanism Design, Legged Robots, Gripper and Other End-Effectors
Abstract: This paper presents the combining function of the multi-legged modular robot, which is bio-inspired from ants. To this end, the pair of hook-link structure and holder are proposed as the coupling mechanisms with the ant’s claws structure. The steps of the robot combination using the proposed coupling mechanisms is exhibited. As a result, imitating the ant’s structure allows it to achieve the prosperity of merging the main concept of a modular robot, legged robot and, swarm robot into the developed robot for this project.

Keywords: Mechanism Design, Parallel Robots, Field Robots
Abstract: This paper introduces a 3-degree-of-freedom manifold composed of three linear actuators. The proposed mechanism has a workspace suitable for facade cleaning and can compensate horizontal position due to disturbance in gondola-based exterior wall cleaning. Position, velocity kinematic and Jacobian based singularity analysis is presented, and kinematic variables are defined to extend singularity free workspace. This study can be applied to robot arm for facade cleaning in the future

Keywords: Mechanism Design, Robot Safety, Physical Human-Robot Interaction
Abstract: In recent years, with increasing requirements of human–robot co-existence, safety has become one of the most important issues in the field of robotics. The counterbalance mechanism (CBM) can be a suitable solution to address this issue as it can ensure intrinsic safety and minimize the required torque of actuators. In this paper, we propose a variable counter -balance mechanism (VCBM) that improves upon the CBM; the proposed mechanism can adjust the counterbalancing torque according to the weight of the payload attached at the end effector of a robot arm such as a gripper. Therefore, the torque required to support not only the self-weight of the robot arm but also any additional external load can be effectively compensated for by the proposed VCBM.

Keywords: Mobile Manipulation
Abstract: Science fiction has long inspired pioneers of new areas of Engineering. When imagination meets the current needs of civil society, creative thinking then often gets real, and projects aiming at breakthroughs for advancing the scientific state of the art are put in place. Robotics is a scientific field that has always been driven by visionary applications of Engineering, often receiving impetus from the human will of having extended locomotion and interaction capacities. iRonCub breaks the ground for unifying Manipulation, Aerial, and Terrestrial Locomotion by implementing Aerial Humanoid Robotics. Aerial humanoid robots can then fly, walk, manipulate, and transport objects in the surrounding environment, thus being pivotal for disaster response and opening new branches of applications for humanoid robots. By doing so, aerial humanoid robots overcome the lack of terrestrial locomotion of aerial manipulators and extend the locomotion capabilities of humanoid robots to the flight case.

Keywords: Mobile Manipulation, Service Robots, Performance Evaluation and Benchmarking
Abstract: In this paper, we present a stabilization analysis of omni-mobile manipulator with 4K camera. For the image stabilization evaluation, we have adopted the inter-frame transformation fidelity (ITF) which have been used in the image stabilization literature. The experimental results show that the proposed approach has a high potential to be applicable to the image stabilization control in robot-based camera productions.

Keywords: Object Detection, Segmentation and Categorization, Visual Learning, Computer Vision for Other Robotic Applications
Abstract: We present a new dataset with the goal of advancing the state-of-the-art in object pose estimation especially for stored porcelain and glass crockery in kitchen scenes. Specifically, the ESKO6d (EASE Stored Kitchen Objects with 6d poses) dataset features texture-less, glossy or glassy ordinary used objects which were naturally stored in a cupboard, drawer or dishwasher. There is a large degree of occlusion being the specific challenge in these scenes. Each scene was recorded in video sequences by two cameras (RGB-D (Kinect) and binocular) within multiple setup stages. The dataset contains an RGB-D image or binocular RGB image plus stereo-matched depth image as well as 6d pose ground truth and instance segmentation. Our dataset contains twelve stored object scenes with a combined amount of 47 video sequences captured by each camera, resulting in over 17k annotated Kinect images and more than 42k annotated stereo images showing around 50 different objects. The ground truth annotation is precise to 3:5mm ADD (details see paper). The dataset can be accessed under http://www.informatik.uni-bremen.de/esko6d-dataset.

Besides the concrete dataset we propose a method of ground truth pose measurement based on an external 3d tracking system that allows on the one hand to precisely measure the object’s pose inside a tight packed storage and on the other hand to obtain the object pose in several images with just one manual measurement.

Keywords: Omnidirectional Vision, Localization, SLAM
Abstract: We propose a novel pose estimation method for geometric vision of omni-directional cameras. On the basis of the regularity of the pixel movement after camera pose changes, we formulate and prove the sinusoidal relationship between pixels movement and camera motion. We use the improved Fourier-Mellin invariant (iFMI) algorithm to find the motion of pixels, which was shown to be more accurate and robust than the feature-based methods. While iFMI works only on pin-hole model images and estimates 4 parameters (x, y, yaw, scaling), our method works on panoramic images and estimates the full 6 DoF 3D transform, up to an unknown scale factor. For that we fit the motion of the pixels in the panoramic images, as determined by iFMI, to two sinusoidal functions. The offsets, amplitudes and phase-shifts of the two functions then represent the 3D rotation and translation of the camera between the two images. We perform experiments for 3D rotation, which show that our algorithm outperforms the feature-based methods in accuracy and robustness. We leave the more complex 3D translation experiments for future work.

Keywords: Computer Vision for Other Robotic Applications, Cognitive Human-Robot Interaction
Abstract: In the context of mobile gaze tracking techniques, a 3D gaze point can be calculated as the middle point between two 3D visual axes. To infer gaze directions and eyeball positions, a nonlinear optimization problem is typically formulated to minimize the angular disparities between the training gaze directions and prediction ones. Nonetheless, the experimental results reported by some previous works show that this kind of approaches are very likely to yield large prediction errors hence considered less useful for human-machine interactions. In this study, we aim to address this widespread issue in three aspects. At first, instead of using a global regression model, a simple local polynomial model is proposed to back-project a pupil center onto its corresponding visual axis. Based on the Leave-One-Out cross-validation criterion, the partition structure is automatically learned in the process of resolving a homography-like relationship. Secondly, a good starting point for nonlinear-optimization is obtained by the image eyeball center, which can be estimated by systematic parallax errors. Meanwhile, it is necessary to add the suitable constraints for 3D eye positions. Otherwise, the optimization may end up with trivial solutions, i.e., faraway eye positions. Thirdly, we explore a strategy for designing the spatial distribution of calibration points in a principled manner. The experiment results demonstrate that an encouraging gaze estimation accuracy can be achieved by our proposed framework for both the normal vision and eyewear users.

Keywords: Computational Geometry, Computer Vision for Other Robotic Applications, RGB-D Perception
Abstract: In this paper, we address the problem of camera pose estimation using 2D and 3D line features, also known as PnL (Perspective-n-Line) with a known vertical direction.

The minimal number of line correspondences required to estimate the complete camera pose is 3 (P3L) in the general case, yielding to a minimum of 8 possible solutions. Prior knowledge of the vertical direction, such as provided by common sensors (e.g. Inertial Measurement Unit, or IMU), reduces the problem to a 4 Degree of Freedom (DoF) problem and outputs a single solution. We benefit this fact to decouple the remaining rotation estimation and the translation estimation and we present a two-fold contribution: (1) we present a linear formulation of the PnL problem in Plücker lines coordinates with a known vertical direction, including a Gauss-Newton-based orientation and location refinement to compensate IMU sensor noise. (2) we propose a new efficient RANdom SAmple Consensus (RANSAC) scheme for both feature pairing and outliers rejection based solely on rotation estimation from 2 line pairs. This greatly diminishes the computational cost compared to a RANSAC3 or RANSAC4 scheme.

We evaluate our algorithms on synthetic data and on our own real dataset. Experimental results show state of the art results in term of accuracy and runtime, when facing 2D noise, 3D noise and vertical direction sensor noise.