Keywords: Task Planning, Robot Reinforcement Learning, Integrated Task and Motion Planning
Abstract: The effectiveness of robot interaction depends on the robot's ability to perform task-relevant actions and on the degree to which it is able to predict the outcomes of these actions. In this paper we argue that the two learning problems - learning actions and learning forward models - must be tightly coupled for each of them to be successful. We present an approach that is able to learn a set of continuous action parameters and relational forward models from the robot's own experience. We formalize our approach as simultaneously clustering experiences in a continuous and a relational representation. Our experiments in a simulated manipulation experiment show that this form of coupled subsymbolic and symbolic learning is required for the robot to acquire task-relevant action capabilities.

Keywords: Learning Control, Motor Skill Learning, Learning from Demonstration
Abstract: We propose a novel method that arbitrates the control between the human and the robot actors in a teaching-by-demonstration setting to form synergy between the two and facilitate effective skill synthesis on the robot. We employed the human-in-the-loop teaching paradigm to teleoperate and demonstrate a complex task execution to the robot in real-time. As the human guides the robot to perform the task, the robot obtains the skill online during the demonstration. To encode the robotic skill we employed Locally Weighted Regression that fits local models to specific state region of the task based on the human demonstration. If the robot is in the state region where no local models exist, the control over the robotic mechanism is given to the human to perform the teaching. When local models are gradually obtained in that region, the control is given to the robot so that the human can examine its performance already during the demonstration stage, and take actions accordingly. This enables a co-adaptation between the agents and contributes to a faster and more efficient teaching. As a proof-of-concept, we realised the proposed robot teaching system on a haptic robot with the task of generation of a desired vertical force on a horizontal plane with unknown stiffness properties.

Keywords: Learning from Demonstration, Robot Reinforcement Learning, Imitation Learning
Abstract: In this paper, we propose a novel inverse reinforcement learning algorithm with leveraged Gaussian processes that can learn from both positive and negative demonstrations. While most existing inverse reinforcement learning (IRL) methods suffer from the lack of information near low reward regions, the proposed method alleviates this issue by incorporating (negative) demonstrations of what not to do. To mathematically formulate negative demonstrations, we introduce a novel generative model which can generate both positive and negative demonstrations using a parameter, called proficiency. Moreover, since we represent a reward function using a leveraged Gaussian process which can model a nonlinear function, the proposed method can effectively estimate the structure of a nonlinear reward function.

Keywords: Imitation Learning, Cooperating Robots, Motion and Trajectory Generation
Abstract: Dynamic Movement Primitives (DMPs) represent stable goal-directed or periodic movements, which are learned from observations or demonstrations. They rely on proper function approximators, which are sufficiently flexible to represent arbitrary movements but also ensure goal convergence in point-to-point motions. This work shows that Gaussian Processes (GPs) are suitable as a regressor for learning movements with DMPs ensuring stability. In addition, GPs provide a measure for the uncertainty about the current movement, which we exploit by proposing a new cooperation scheme for DMPs: For better reproduction of demonstrations, we follow the intuition, that individuals with more knowledge lead towards the goal, while others follow and focus on cooperation. Along with simulation results, we validate the presented methods in a robotic cooperative object manipulation task.

Keywords: Learning from Demonstration, Robot Learning, Imitation Learning
Abstract: Learning sequential force interaction tasks from kinesthetic demonstrations is a promising approach to transfer human manipulation abilities to a robot. In this paper we propose a novel concept to decompose such demonstrations into a set of Movement Primitives (MPs). The decomposition is based on a probability distribution we call Directional Normal Distribution (DND). To capture the sequential properties of the manipulation task, we model the demonstrations with a Hidden Markov Model (HMM). Here, we employ mixtures of DNDs as the HMM’s output emissions. The combination of HMMs and mixtures of DNDs allows to infer the MP’s composition, i.e., its reference frames, control variables and target coordinates from the demonstration data. In addition, it permits to determine an appropriate number of MPs that explains the demonstrations best. We evaluate the approach on kinesthetic demonstrations of a light bulb unscrewing task. Decomposing the task leads to intuitive and meaningful MPs that reflect the natural structure of the task.

Keywords: Robot Reinforcement Learning, Robot Learning, Force and Tactile Sensing
Abstract: For many tasks, tactile or visual feedback is helpful or even crucial. However, designing controllers that take such high-dimensional feedback into account is non-trivial. Therefore, robots should be able to learn tactile skills through trial and error by using reinforcement learning algorithms. The input domain for such tasks, however, might include strongly correlated or non-relevant dimensions, making it hard to specify a suitable metric on such domains. Auto-encoders specialize in finding compact representations, where defining such a metric is likely to be easier. Therefore, we propose a reinforcement learning algorithm that can learn non-linear policies in continuous state spaces, which leverages representations learned using auto-encoders. We first evaluate this method on a simulated toy-task with visual input. Then, we validate our approach on a real-robot tactile stabilization task.

Keywords: Learning from Demonstration, Motion and Trajectory Generation, Imitation Learning
Abstract: This work proposes a framework that enables robot arms to imitate human demonstrations to acquire a skill, and reproduce it in real-time. The number of different robots active in various environments is growing considerably, and it is preferable for users to be able to easily teach a skill to a robot with any body configuration. Our proposed method works as follows. First, a motion trajectory is obtained from human demonstrations via a Kinect sensor, and projected onto a corresponding human skeleton model. Second, the kinematics mapping between the robot and the human model is learned by employing Local Procrustes Analysis. Third, the mapping function transfers the demonstrated trajectory from the human model to the robot. Finally, the transferred trajectory is modeled to be reproduced in real-time, employing Dynamic Movement Primitives. Experiments in simulation on a 4 degree of freedom robot show that our method is able to correctly imitate a skill demonstrated by a human.

Keywords: Model Learning, Visual Learning, Learning Control
Abstract: The information used to determine the internal model of a robot system emerges from individual interactions with the environment. Knowledge about a specific internal model can be acquired by means of model learning techniques based on supervised machine learning tools, such as neural networks. One of the main challenges is to specify the starting point for the learning process. We propose a methodology for initial weight estimation based on the robot morphology. We use the generation of saccades in a robotic head as a case study to evaluate the performance of this approach. Our results suggest that this methodology, based on learning the relationship between the morphological parameters of the robot and those in its internal model, improves the quality of weight initialization, resulting in a considerable speed-up in the process of learning the internal model.

Keywords: Robot Reinforcement Learning, Learning Control, Neural and Fuzzy Control
Abstract: Recent years have seen a growing interest in the use of deep neural networks as function approximators in reinforcement learning. In this paper, an experience replay method is proposed that ensures that the distribution of the experiences used for training is between that of the policy and a uniform distribution. Through experiments on a magnetic manipulation task it is shown that the method reduces the need for sustained exhaustive exploration during learning. This makes it attractive in scenarios where sustained exploration is in-feasible or undesirable, such as for physical systems like robots and for life long learning. The method is also shown to improve the generalization performance of the trained policy, which can make it attractive for transfer learning. Finally, for small experience databases the method performs favorably when compared to the recently proposed alternative of using the temporal difference error to determine the experience sample distribution, which makes it an attractive option for robots with limited memory capacity.

Keywords: Imitation Learning, Learning from Demonstration
Abstract: Although many tasks intrinsically involve multiple modalities, often only data from a single modality are used to improve complex robots acquisition of new skills. We present a method to equip robots with multimodal learning skills to achieve multimodal imitation on-the-fly on multiple concurrent task spaces, including vision, touch and proprioception, only using self-learned multimodal sensorimotor relations, without the need of solving inverse kinematic problems or explicit analytical models formulation. We evaluate the proposed method on a humanoid iCub robot learning to interact with a piano keyboard and imitating a human demonstration. Since no assumptions are made on the kinematic structure of the robot, the method can be also applied to different robotic platforms.

Keywords: Robot Learning, Perception for Grasping and Manipulation, Cognitive Human-Robot Interaction
Abstract: Considering perception as an observation process only is the very reason for which robotic perception methods are to date unable to provide a general capacity of scene understanding. Related work in neuroscience has shown that there is a strong relationship between perception and action. We believe that considering perception in relation to action requires to interpret the scene in terms of the agent's own potential capabilities. In this paper, we propose a Bayesian approach for learning sensorimotor representations through the interaction between action and observation capabilities. We represent the notion of affordance as a probabilistic relation between three elements: objects, actions and effects. Experiments for affordances discovery were performed on a real robotic platform in an unsupervised way assuming a limited set of innate capabilities. Results show dependency relations that connect the three elements in a common frame: affordances. The increasing number of interactions and observations results in a Bayesian network that captures the relationships between them. The learned representation can be used for prediction tasks.

Keywords: Model Learning, Autonomous Agents, Motor Skill Learning
Abstract: This article studies algorithms used by a learner to explore high-dimensional structured sensorimotor spaces such as in tool use discovery. In particular, we consider goal babbling architectures that were designed to explore and learn solutions to fields of sensorimotor problems, i.e. to acquire inverse models mapping a space of parameterized sensorimotor problems/effects to a corresponding space of parameterized motor primitives. However, so far these architectures have not been used in high-dimensional spaces of effects. Here, we show the limits of existing goal babbling architectures for efficient exploration in such spaces, and introduce a novel exploration architecture called Model Babbling (MB). MB exploits efficiently a modular representation of the space of parameterized problems/effects. We also study an active version of Model Babbling (the MACOB architecture). These architectures are compared in a simulated experimental setup with an arm that can discover and learn how to move objects using two tools with different properties, embedding structured high-dimensional continuous motor and sensory spaces.

Keywords: Robot Learning, Field Robots, Learning from Demonstration
Abstract: This paper presents a learned, place-dependent terrain-assessment classifier that improves over time. Whereas typical methods aim to assess all of the terrain in a given environment, we exploit the fact that many robotic navigation tasks are well-suited to visual-teach-and-repeat navigation where robot motion is restricted to previously driven paths. In such scenarios, we argue that general terrain assessment is not required, and we can instead solve the much easier problem of detecting changes along the path; we sacrifice the ability to generalize off the path in favour of improved performance on the path. Terrain along a pretaught path is compared to terrain seen at the same location during previous, human-supervised traverses, and any significant differences cause that location to be labelled as unsafe. By storing all of our previous experiences, we are able to continuously improve our estimates as we revisit the same locations multiple times. We tested our method on two datasets collected at the University of Toronto and show that we improve over existing place-independent (both learned and not) methods, enabling nearly full autonomy in challenging, varied terrain using only a stereo camera.

Keywords: Robots and Embodied Art, Motion and Trajectory Generation, Entertainment Robotics
Abstract: We present an approach to generate rapid and fluid drawing movements on a compliant Baxter robot, by taking advantage of the kinematic redundancy and torque control capabilities of the robot. We concentrate on the task of reproducing graffiti-stylised letter-forms with a marker. For this purpose, we exploit a compact lognormal-stroke based representation of movement to generate natural drawing trajectories. An Expectation-Maximisation (EM) algorithm is used to iteratively improve tracking performance with low gain feedback control. The resulting system captures the aesthetic and dynamic features of the style under investigation and permits its reproduction with a compliant controller that is safe for users surrounding the robot.

Keywords: Robot Learning, Model Learning
Abstract: Robots often operate in built environments con- taining underlying structure that can be exploited to help predict future observations. In this work, we present a deep learning based approach to predict exit locations of buildings. This technique exploits the inherent structure of buildings to create a model. A convolutional neural network is trained using a database of building blueprints and used to guide a search within a building. This technique is compared to standard fron- tier exploration and a traditional image processing approach of extracting features through histogram of gradients (HOG) and training a support vector machine (SVM). After validation through simulation, we show that the proposed deep learning technique reduces the amount of building exploration required to find the goal by 36%.

Keywords: Robot Learning, Robot Reinforcement Learning, Learning Control
Abstract: No two robots are exactly the same—even for a given model of robot, different units will require slightly different controllers. Furthermore, because robots change and degrade over time, a controller will need to change over time to remain optimal. This paper leverages lifelong learning in order to learn controllers for different robots. In particular, we show that by learning a set of control policies over robots with different (unknown) motion models, we can quickly adapt to changes in the robot, or learn a controller for a new robot with a unique set of disturbances. Furthermore, the approach is completely model-free, allowing us to apply this method to robots that have not, or cannot, be fully modeled.

Keywords: Visual Navigation, Human-Robot Interaction
Abstract: The goal of this paper is to develop a robot with a grounded spatial vocabulary. Such a vocabulary would allow it to give and follow directions, and would give it valuable additional information in aiding localization and navigation. We approach the problem by defining and ontology of space (including corridor, doorway, and room) and by creating a Convolutional Neural Network (CNN) that allows the robot to classify LIDAR sensor data accordingly. In particular, we propose a CNN architecture that performs comparably or better than existing methods based on engineered features. Training CNNs can be fickle; we describe several specific aspects of our approach that are important for good performance in this task.

Keywords: Service Robots, Learning Control, Animation and Simulation
Abstract: For robotic agents to perform manipulation tasks in human environments at a human level or higher, they need to be able to relate the physical effects of their actions to how they are executing them; small variations in execution can have very different consequences. This paper proposes a framework for acquiring and applying action knowledge from naive user demonstrations in an interactive simulation environment under varying conditions. The framework combines a flexible constraint-based motion control approach with games-with-a-purpose-based learning using Random Forest Regression. The acquired action models are able to produce context-sensitive constraint-based motion descriptions to perform the learned action. A pouring experiment is conducted to test the feasibility of the suggested approach and shows the learned system can perform comparable to its human demonstrators.

Keywords: Model Learning, Robot Learning, Autonomous Agents
Abstract: The efficiency of autonomous robots depends on how well they understand their operating environment. While most of the traditional environment models focus on the spatial representation, long-term mobile robot operation in human populated environments requires that the robots have a basic model of human behaviour.

We present a framework that allows us to retrieve and represent aggregate human behaviour in large, populated environments on extended temporal scales. Our approach, based on time-varying Poisson process models and spectral analysis, efficiently retrieves long-term, re-occurring patterns of human activity from robot-gathered observations and uses these patterns to i) predict human activity level at particular times and places and ii) classify locations based on their periodic patterns of activity.

The application of our framework on real-world data, gathered by a mobile robot operating in an indoor environment for one month, indicates that its predictive capabilities outperform other temporal modelling methods while being computationally more efficient. The experiment also demonstrates that spectral signatures act as features that allow us to classify room types which semantically match with humans' expectations.

Keywords: Robot Reinforcement Learning
Abstract: One of the key challenges in applying reinforcement learning to complex robotic control tasks is the need to gather large amounts of experience in order to find an effective policy for the task at hand. Model-based reinforcement learning can achieve good sample efficiency, but requires the ability to learn a model of the dynamics that is good enough to learn an effective policy. In this work, we develop a model-based reinforcement learning algorithm that combines prior knowledge from previous tasks with online adaptation of the dynamics model. These two ingredients enable highly sample-efficient learning even in regimes where estimating the true dynamics is very difficult, since the online model adaptation allows the method to locally compensate for unmodeled variation in the dynamics. We encode the prior experience into a neural network dynamics model, adapt it online by progressively refitting a local linear model of the dynamics, and use model predictive control to plan under these dynamics. Our experimental results show that this approach can be used to solve a variety of complex robotic manipulation tasks in just a single attempt, using prior data from other manipulation behaviors.

Keywords: Recognition, Visual Learning
Abstract: Active Constraint Learning (ACL) is continuously gaining popularity in the area of constrained clustering due to its ability to achieve performance gains via incorporating minimal feedback from a human annotator for selected instances. For constrained clustering algorithms, such instances are integrated in the form of Must-Link (ML) and CannotLink (CL) constraints. Existing iterative uncertainty reduction schemes, introduce high computational burden particularly when they process larger datasets that are usually present in computer vision and visual learning applications. For scenarios that multiple agents (i.e., robots) require user feedback for performing recognition tasks, minimizing the interaction between the user and the agents, without compromising performance, is an essential task. In this study, a non-iterative ACL scheme with proven performance benefits is presented. We select to demonstrate the effectiveness of our methodology by building on the well known K-Means algorithm for clustering; one can easily extend it to alternative clustering schemes. The proposed methodology introduces the use of the Silhouette values, conventionally used for measuring clustering performance, in order to rank the degree of information content of the various samples. In addition, an efficient greedy selection scheme was devised for selecting the most informative samples for human annotation. To the best of our knowledge, this is the first active constrained clustering methodology with the ability to process computer vision datasets that this study targets. Performance results are shown on various computer vision benchmarks and support the merits of adopting the proposed scheme.

Keywords: Learning Control, Model Learning, Robot Learning
Abstract: Learning of inverse dynamics modeling errors is key for compliant or force control when analytical models are only rough approximations. Thus, designing real time capable function approximation algorithms has been a necessary focus towards the goal of online model learning. However, because these approaches learn a mapping from actual state and acceleration to torque, good tracking is required to observe data points on the desired path. Recently it has been shown how online gradient descent on a simple modeling error offset term to minimize tracking at acceleration level can address this issue. However, to adapt to larger errors a high learning rate of the online learner is required, resulting in reduced compliancy. Thus, here we propose to combine both approaches: The online adapted offset term ensures good tracking such that a nonlinear function approximator is able to learn an error model on the desired trajectory. This, in turn, reduces the load on the adaptive feedback, enabling it to use a lower learning rate. Combined this creates a controller with variable feedback and low gains, and a feedforward model that can account for larger modeling errors. We demonstrate the effectiveness of this framework, in simulation and on a real system.

Keywords: Recognition, Human Detection and Tracking, Rehabilitation Robotics
Abstract: Human locomotion and activity recognition systems form a critical part in a robot's ability to safely and effectively operate in a environment populated with human end users. Previous work in this area relies upon strong assumptions about the labels in the training data; e.g. that are noise-free and that they exist at all. Our approach does not predefine the relevant behaviours or their number, as both are learned directly from observations, similar to real-world human-robot interactions, where labels are neither available. Instead we introduce models that make no assumptions about the state space, by presenting a fully unsupervised nonparametric Bayesian recognition approach, in which we leverage recent advances in state space modelling with automatic inference using probabilistic programming. We demonstrate the utility of full model optimisation using Bayesian optimisation and validate our approach on several challenging problems, using different feature modalities.

Keywords: Imitation Learning, Learning from Demonstration, Learning Control
Abstract: Optimal control is a powerful approach to achieve optimal behavior. However, it typically requires a manual specification of a cost function which often contains several objectives, such as reaching goal positions at different time steps or energy efficiency. Manually trading-off these objectives is often difficult and requires a high engineering effort. In this paper, we present a new approach to specify optimal behavior. We directly specify the desired behavior by a distribution over future states or features of the states. For example, the experimenter could choose to reach certain mean positions with given accuracy/variance at specified time steps. Our approach also unifies optimal control and inverse optimal control in one framework. Given a desired state distribution, we estimate a cost function such that the optimal controller matches the desired distribution. If the desired distribution is estimated from expert demonstrations, our approach performs inverse optimal control. We evaluate our approach on several optimal and inverse optimal control tasks on non-linear systems using incremental linearizations similar to differential dynamic programming approaches.

Keywords: Field Robots, Navigation
Abstract: Real-time obstacle detection is a key component of autonomous vehicles. In this context, low obstacles are particularly challenging, as they are often discarded by traditional algorithms. Curb detection methods that can potentially be suitable for the problem usually target roads with clearly defined curbs and sidewalks. We propose a real-time algorithm for the detection of low obstacles (including, but not restricted to curbs), merging 2-D and 3-D information from stereo imaging. A set of candidate object lines is extracted based on their combined 2-D and 3-D features, tracked over time and clustered according to a novel similarity metric. Finally, a 3rd order polynomial spline is fitted to each cluster to represent the obstacle. The proposed system can deal with noisy and incomplete point clouds and keeps the model assumptions to a minimum. To evaluate the algorithm, a new stereo dataset is provided and made available online. We present experiments in different scenarios and lighting conditions, illustrating the applicability of the method.

Keywords: Robot Vision, Visual Learning
Abstract: This paper presents a method for 6D pose estimation from a single RGB image for complex texture-less objects. This class of objects are common in any environment but still challenging to deal with. This is due the fact that the distribution of surface brightness makes difficult to compute interest points or appearance-based descriptors. Here we propose a novel part-based method using an efficient template matching approach where each template independently encodes the similarity function using a Forest trained over the templates. Moreover, accuracy is even more incremented by using a cascade of the learned forest. These templates forests together with the simplicity of the computed image features allow a quick estimate of the pose achieving real-time performance. Performance are demonstrated both on synthetic and real images with known ground truth.

Keywords: Robot Vision, Manufacturing and Automation
Abstract: In this paper, an object detection and localization method for bin picking of plastic wrapped objects is described. Since such objects are deformable and have non-Lambertian surfaces, it is difficult to apply conventional feature point approaches or edge based template matching. To solve this problem, we propose a new method which is called “KCS (kernel convolution score)”. It measures the total score of convolution between local image and multiple kernels which are generated according to the characteristics of the target object. The local image having maximum score is considered as the most promising object for picking. In the last part of this paper, the limitation of existing edge and template matching in the given problem is discussed with the experimental results. Performance evaluation shows that the proposed method could localize the object which is appropriate for picking up with 91.67% success rate.

Keywords: SLAM, Robot Vision, Sensor Fusion
Abstract: We present a semi-direct visual odometry algorithm for a fisheye-stereo camera. In a tracking thread, we simultaneously track oriented patches and estimate the camera pose. In a mapping thread, we estimate the coordinates and surface normal for each new patch to be tracked. Estimation of the surface normals allows us to track patches over a wide variety of viewpoints. In our algorithm, we do not make use of descriptors and robust descriptor matching to find patch correspondences. Instead, we use photoconsistency-based techniques to find patch correspondences. For tracking, we use sparse model-based image alignment to find the relative motion estimate, and feature alignment to find 2D-3D patch correspondences. For mapping, we use plane-sweeping stereo to find matching patches between stereo images. We also implement a state estimator based on the Extended Kalman Filter (EKF) to fuse inertial measurements and relative pose estimates from our visual odometry implementation. We run experiments in two different outdoor environments to validate our algorithm, and discuss the experimental results. Our implementation runs at an average of 42 Hz on a commodity Intel CPU. To the best of our knowledge, there is no other existing semi-direct visual odometry algorithm for a fisheye-stereo camera.

Keywords: Visual Tracking, Visual Learning
Abstract: Object tracking over image sequences plays an remarkably crucial role in several computer vision applications, interalia, automated video surveillance, unmanned aerial vehicles and 3D reconstruction. In this paper, a novel, accurate, robust and recoverable real-time feature-based tracking framework is presented. The appearance modelling consists of a local and global layer. We propose a recommended keypoint-aware (RKA) tracker, which is fast and accurate, for the former, while the latter employs support vector machine (SVM) to determine the object and background, so that the RKA tracker can be recovered under possible target losing circumstances. Furthermore, the RKA tracker converts the tracking problem into the ranking of samples which provides a score of tracking confidence. Therefore, the priority switching between the local layer and global layer dependent upon the score becomes valid. Extensive experiments have been done by strictly following the visual tracking benchmark v1.0 protocol. The results demonstrate that the proposed novel method outperforms the start-of-the-art trackers in terms of robustness, speed and accuracy.

Keywords: Mapping, Field Robots
Abstract: Although Point Clouds Registration is a very well studied problem, with many different solutions, most of the approaches in the literature aims at aligning two dense point clouds. Instead, we tackle the problem of aligning a dense point cloud with a sparse one: a problem that has to be solved, for example, to merge maps produced by different sensors, such as a vision-based sensor and laser scanner or two different laser-based sensors. The most used approach to point clouds registration, Iterative Closest Point (ICP), is also applicable to this sub-problem. We propose an improvement over the standard ICP data association policy and we called it Probabilistic Data Association. It was derived applying statistical inference techniques on a fully probabilistic model. In our proposal, each point in the source point cloud is associated with a set of points in the target point cloud; each association is then weighted so that the weights form a probability distribution. The result is an algorithm similar to ICP but more robust w.r.t. noise and outliers. While we designed our approach to deal with the problem of dense-sparse registration, it can be successfully applied also to standard point clouds registration.

Keywords: Visual Servoing, Medical Robots and Systems
Abstract: This paper deals with the development of a vision- based controller for robot-assisted medical applications. It concerns the use of shearlet coefficients in case of ultrasounds (US) images as visual signal inputs and the design of the associated interaction matrix. The proposed controller was validated in both simulation and on an experimental test bench which consists of a robotic arm holding an US probe in contact with a realistic abdominal phantom. Also, the proposed control scheme was compared to the photometry-based visual servoing approach in order to evaluate its efficiency in different conditions of use (nominal and unfavorable conditions).

Keywords: Localization, SLAM, Navigation
Abstract: Matching 3D point clouds, a critical operation in map building and localization, is difficult with Velodyne-type sensors due to the sparse and non-uniform point clouds that they produce. Standard methods from dense 3D point clouds are generally not effective. In this paper, we describe a feature-based approach using Principal Components Analysis (PCA) of neighborhoods of points, which results in mathematically principled line and plane features. The key contribution in this work is to show how this type of feature extraction can be done efficiently and robustly even on non-uniformly sampled point clouds. The resulting detector runs in real-time and can be easily tuned to have a low false positive rate, simplifying data association. We evaluate the performance of our algorithm on an autonomous car at the MCity Test Facility using a Velodyne HDL-32E, and we compare our results against the state-of-the-art NARF keypoint detector.

Keywords: Robot Vision, Visual Learning, Robot Learning
Abstract: State-of-the-art techniques proposed for 6D object pose recovery depend on occlusion-free point clouds to accurately register objects in 3D space. To reduce this dependency, we introduce a novel architecture called Iterative Hough Forest with Histogram of Control Points that is capable of estimating occluded and cluttered objects’ 6D pose given a candidate 2D bounding box. Our Iterative Hough Forest is learnt using patches extracted only from the positive samples. These patches are represented with Histogram of Control Points (HoCP), a “scale-variant” implicit volumetric description, which we derive from recently introduced Implicit B-Splines (IBS). The rich discriminative information provided by this scale-variance is leveraged during inference, where the initial pose estimation of the object is iteratively refined based on more discriminative control points by using our Iterative Hough Forest. We conduct experiments on several test objects of a publicly available dataset to test our architecture and to compare with the state-of-the-art.

Keywords: Robot Vision, Intelligent Transportation Systems, Wheeled Robots
Abstract: Automated guided vehicles (AGVs) are useful for a variety of transportation tasks. They usually detect obstacles on the path they are following using 2D laser scanners. If an AGV should be deployed in a shared space with people, 3D information has to be considered as well to detect unforeseen obstacles. These can be small objects on the floor or overhanging parts of larger objects, which cannot be seen by the standard 2D safety scanners. We propose a generic object detection pipeline using 3D time-of-flight cameras, that can be used in real-time on AGVs of low height and demonstrate its robustness to different measurement artefacts.

Keywords: Recognition, Robot Vision
Abstract: We propose a novel approach for multi-view object detection in 3D scenes reconstructed from RGB-D sensor. We utilize shape based representation using local shape context descriptors along with the voting strategy which is supported by unsupervised object proposals generated from 3D point cloud data. Our algorithm starts with a single-view object detection where object proposals generated in 3D space are combined with object specific hypotheses generated by the voting strategy. To tackle the multi-view setting, the data association between multiple views enabled view registration and 3D object proposals. The evidence from multiple views is combined in simple bayesian setting. The approach is evaluated on the Washington RGB-D scenes datasets containing several classes of objects in a table top setting. We evaluated our approach against the other state-of-the-art methods and demonstrated superior performance on the same dataset.

Keywords: Human Detection and Tracking, Robot Vision, Gesture, Posture, Social Spaces and Facial Expressions
Abstract: We address the problem of extracting human body posture labels, upper body orientation and the spatial location of individual body parts from near-infrared (NIR) images depicting patterns of retro-reflective markers. The analyzed patterns originate from the observation of humans equipped with protective high-visibility garments that represent common safety equipment in the industrial sector. Exploiting the shape of the observed reflectors we adopt shape matching based on the chamfer distance and infer one of seven discrete body posture labels as well as the approximate upper body orientation with respect to the camera. We then proceed to analyze the NIR images on a pixel scale and estimate a figure-ground segmentation together with human body part labels using classification of densely extracted local image patches. Our results indicate a body posture classification accuracy of 80% and figure-ground segmentations with 87% accuracy.

Keywords: Visual Navigation, Localization, Distributed Robot Systems
Abstract: In this paper, we present an online landmark selection method for distributed long-term visual localization systems in bandwidth-constrained environments.Sharing a common map for online localization provides a fleet of autonomous vehicles the possibility to maintain and access a consistent map source, and therefore reduce redundancy while increasing efficiency. However, connectivity over a mobile network imposes strict bandwidth constraints and thus the need to minimize the amount of exchanged data. The wide range of varying appearance conditions encountered during long-term visual localization offer the potential to reduce data usage by extracting only those visual cues which are relevant at the given time. Motivated by this, we propose an unsupervised method of adaptively selecting landmarks according to how likely these landmarks will be observable under the prevailing appearance condition. The ranking function this selection is based upon exploits landmark co-observability statistics collected in past traversals through the mapped area. % to infer about landmark observability at the current time. Evaluation is performed over different outdoor environments, large time-scales and varying appearance conditions, including the extreme transition from day-time to night-time, demonstrating that with our appearance-dependent selection method, we can significantly reduce the amount of landmarks used for localization while maintaining or even improving the localization performance.

Keywords: Robot Vision, Visual Tracking, Neurorobotics
Abstract: The detection of consistent feature points in an image is fundamental for various kinds of computer vision techniques, such as stereo matching, object recognition, target tracking and optical flow computation. This paper presents an event-based approach to the detection of corner points, which benefits from the high temporal resolution, compressed visual information and low latency provided by an asynchronous neuromorphic event-based camera. The proposed method adapts the commonly used Harris corner detector to the event-based data, in which frames are replaced by a stream of asynchronous events produced in response to local light changes at (mu)s temporal resolution. Responding only to changes in its field of view, an event-based camera naturally enhances edges in the scene, simplifying the detection of corner features. We characterised and tested the method on both a controlled pattern and a real scenario, using the dynamic vision sensor (DVS) on the neuromorphic iCub robot. The method detects corners with a typical error distribution within 2 pixels. The error is constant for different motion velocities and directions, indicating a consistent detection across the scene and over time. We achieve a detection rate proportional to speed, higher than frame-based technique for a significant amount of motion in the scene, while also reducing the computational cost.

Keywords: Robot Vision, Visual Learning
Abstract: We consider the question of benchmarking the performance of methods used for estimating the depth of a scene from a single image. We describe various measures that have been used in the past, discuss their limitations and demonstrate that each is deficient in one or more ways. We propose a new measure of performance for depth estimation that overcomes these deficiencies, and has a number of desirable properties. We show that in various cases of interest the new measure enables visualisation of the performance of a method that is otherwise obfuscated by existing metrics. Our proposed method is capable of illuminating the relative performance of different algorithms on different kinds of data, such as the difference in efficacy of a method when estimating the depth of the ground plane versus estimating the depth of other generic scene structure. We showcase the method by comparing a number of existing single-view methods against each other and against more traditional depth estimation methods such as binocular stereo.

Keywords: Robot Vision, Visual Learning
Abstract: Object representation is one of the most challenging tasks in robotics because it must provide reliable information in real-time to enable the robot to physically interact with the objects in its environment. To ensure reliability, a global object descriptor must be computed based on a unique and repeatable object reference frame. Moreover, the descriptor should contain enough information enabling to recognize the same or similar objects seen from different perspectives. This paper presents a new object descriptor named Global Orthographic Object Descriptor (GOOD) designed to be robust, descriptive and efficient to compute and use. The performance of the proposed object descriptor is compared with the main state-of-the-art descriptors. Experimental results show that the overall classification performance obtained with GOOD is comparable to the best performances obtained with the state-of-the-art descriptors. Concerning memory and computation time, GOOD clearly outperforms the other descriptors. Therefore, GOOD is especially suited for real-time applications.

Keywords: SLAM, Localization, Robot Vision
Abstract: We present an object detection and tracking framework integrated into a simultaneous localization and mapping (SLAM) system using an RGB-D camera. We propose a compact representation of objects by grouping features hierarchically. Similar to a keyframe being a collection of features, an object is represented as a set of segments, where a segment is a subset of features in a frame. Just like keyframes, segments are registered with each other in a map, which we call an object map. We use the same SLAM procedure in both offline object scanning and online object detection modes. In the offline scanning mode, we scan an object using an RGB-D camera to generate an object map. In the online detection mode, a set of object maps for different objects is given, and the objects are detected via appearance-based matching between the segments in the current frame and in the object maps. In the case of a match, the object is localized with respect to the map being reconstructed by the SLAM system by a RANSAC registration. In the subsequent frames, the tracking is done by predicting the poses of the objects. We also incorporate constraints obtained from the objects into bundle adjustment to improve the object pose estimation accuracy as well as the SLAM reconstruction accuracy. We demonstrate our technique in an object picking scenario using a robot arm. Experimental results show that the system is able to detect and pick up objects successfully from different viewpoints and distances.

Keywords: Sensor Fusion, Intelligent Transportation Systems
Abstract: This paper focuses on a real system implementation, analysis, and evaluation of a cooperative sensor fusion algorithm based on a Gaussian Mixture Probability Hypothesis Density (GM-PHD) filter, using simulated and real vehicles endowed with automotive-grade sensors. We have extended our previously presented cooperative sensor fusion algorithm with a fusion weight optimization method and implemented it on a vehicle that we denote as the ego vehicle. The algorithm fuses information obtained from one or more vehicles located within a certain range (that we call cooperative), which are running a multi-object tracking PHD filter, and which are sharing their object estimates. The algorithm is evaluated on two Citroen C-ZERO prototype vehicles equipped with Mobileye cameras for object tracking and lidar sensors from which the ground truth positions of the tracked objects are extracted. Moreover, the algorithm is evaluated in simulation using simulated C-ZERO vehicles and simulated Mobileye cameras. The ground truth positions of tracked objects are in this case provided by the simulator. Multiple experimental runs are conducted in both simulated and real-world conditions in which a few legacy vehicles were tracked. Results show that the cooperative fusion algorithm allows for extending the sensing field of view, while keeping the tracking accuracy and errors similar to the case in which the vehicles act alone.

Keywords: Visual Servoing
Abstract: With respect to classical visual servoing (VS) technics based on geometrical features, the main drawback of direct visual servoing is its limited convergence area. In this paper we propose a new direct visual servoing control law that relies on a particle filter to achieve non-local and non-linear optimization in order to increase this convergence area. Thanks to multi-view geometry and image transfer techniques, a set of particles (which correspond to potential camera velocities) are drawn and evaluated in order to evaluate the best camera trajectory. This new control law is validated on a 6 DOF positioning task performed on a real gantry robot and statistical comparisons are also provided from simulation results.

Keywords: Human Detection and Tracking, Human-Robot Interaction, Sensor Fusion
Abstract: This paper describes a person identification method using a smartphone and laser range finders (LRFs) for a mobile service robot. The robot is equipped with LRFs and the target person holds a smartphone. The method first detects the foot strike timings of the target person using the smartphone and those of all people by using the LRFs. By finding the person whose foot strike timings captured by the LRFs are similar to those obtained by the smartphone, the robot can identify the target person. Person identification experiments and person following experiments are conducted in order to validate the method. Since the method only requires a person to simply hold a smartphone, it can be easily applied to daily situations.

Keywords: Robot Vision, Virtual Reality and Interfaces, Visual Navigation
Abstract: AprilTags and other passive fiducial markers require specialized algorithms to detect markers among other features in a natural scene. The vision processing steps generally dominate the computation time of a tag detection pipeline, so even small improvements in marker detection can translate to a faster tag detection system. We incorporated lessons learned from implementing and supporting the AprilTag system into this improved system.

This work describes AprilTag 2, a completely redesigned tag detector that improves robustness and efficiency compared to the original AprilTag system. The tag coding scheme is unchanged, retaining the same robustness to false positives inherent to the coding system. The new detector improves performance with higher detection rates, fewer false positives, and lower computational time. Improved performance on small images allows the use of decimated input images, resulting in dramatic gains in detection speed.

Keywords: Robot Vision, Recognition, Mapping
Abstract: Modern SLAM systems with a depth sensor are able to reliably reconstruct dense 3D geometric maps of indoor scenes. Representing these maps in terms of meaningful entities is a step towards building semantic maps for autonomous robots. One approach is to segment the 3D maps into semantic objects using Conditional Random Fields (CRF), which requires large 3D ground truth datasets to train the classification model. Additionally, the CRF inference is often computationally expensive. In this paper, we present an unsupervised geometric-based approach for the segmentation of 3D point clouds into objects and meaningful scene structures. We approximate an input point cloud by an adjacency graph over surface patches, whose edges are then classified as being either on or off. We devise an effective classifier which utilises both global planar surfaces and local surface convexities for edge classification. More importantly, we propose a novel global plane extraction algorithm for robustly discovering the underlying planes in the scene. Our algorithm is able to enforce the extracted planes to be mutually orthogonal or parallel which conforms usually with human-made indoor environments. We reconstruct 654 3D indoor scenes from NYUv2 sequences to validate the efficiency and effectiveness of our segmentation method.

Keywords: Range Sensing, Visual Navigation, Sensor-based Planning
Abstract: We introduce in this paper visibility maps for robots of any shape, representing the reachability limit of the robot’s motion and sensing in a 2D gridmap with obstacles. The brute-force approach to determine the optimal visibility map is computationally expensive, and prohibitive with dynamic obstacles. We contribute the Robot-Dependent Visibility Map (RDVM) as a close approximation to the optimal, and an effective algorithm to compute it. The RDVM is a function of the robot’s shape, initial position, and sensor model. We first overview the computation of RDVM for the circular robot case in terms of the partial morphological closing operation and the optimal choice for the critical points position. We then present how the RDVM for any-shape robots is computed. In order to handle any robot shape, we introduce in the first step multiple layers that discretize the robot orientation. In the second step, our algorithm determines the frontiers of actuation, similarly to the case of the the circular robot case. We then derive the concept of critical points to the any-shape robot, as the points that maximize expected visibility inside unreachable regions. We compare our method with the ground-truth in a simulated map compiled to capture a variety of challenges of obstacle distribution and type, and discuss the accuracy of our approximation to the optimal visibility map.

Keywords: Visual Navigation, Robot Vision, Visual Tracking
Abstract: Most approaches to visual odometry estimates the camera motion based on point features, consequently, their performance deteriorates in low-textured scenes where it is difficult to find a reliable set of them. This paper extends a popular semi-direct approach to monocular visual odometry known as SVO [1] to work with line segments, hence obtaining a more robust system capable of dealing with both textured and structured environments. The proposed odometry system allows for the fast tracking of line segments since it eliminates the necessity of continuously extracting and matching features between subsequent frames. The method, of course, has a higher computational burden than the original SVO, but it still runs with frequencies of 60Hz on a personal computer while performing robustly in a wider variety of scenarios.

Keywords: Robot Vision, SLAM, Mapping
Abstract: Cheap RGB-D sensors are ubiquitous in robotics. They typically contain a consumer-grade color camera that suffers from significant optical nonlinearities, often referred to as vignetting effects. For example, in Asus Xtion Live Pro cameras the pixels in the corners are two times darker than those in the center of the image. This deteriorates the visual appearance of 3D maps built with such cameras. We propose a simple calibration method that only requires a sheet of white paper as a calibration object and allows to reliably recover the vignetting response of a camera. We demonstrate calibration results for multiple popular RGB-D sensors and show that removal of vignetting effects using a nonparametric response model results in improved color coherence of the reconstructed maps. Furthermore, we show how to effectively compensate color variations caused by automatic white balance and exposure time control of the camera.

Keywords: Motion Planning for Manipulators, Task Planning
Abstract: Robot manipulation of liquid sees a lot of applications in daily lives. In this work, we present an optimization based planning framework to compute a smooth trajectory of a manipulator used to transfer a liquid from a source to target container. In order to handle the very high dimensional configuration space of a liquid body, we use a much reduced configuration space and simplified dynamics model based on simple heuristics and system identification to generate a path using trajectory optimization. Our optimization framework can readily incorporate various other constraints such as collision avoidance and different fluid properties. We demonstrate the performance of our planner on different benchmarks corresponding to various obstacle shape and fluid materials. Furthermore, we also evaluate its accuracy by comparing the resulting path with an expensive 3D liquid simulator governed by Navier-Stokes equation.

Keywords: Human-Robot Interaction, Human Performance Augmentation, Motion Planning for Manipulators
Abstract: This paper presents an anticipative robot kinematic limitation avoidance algorithm for collaborative robots. The main objective is to improve the performance and the intuitivity of the physical human-robot interaction. One obstacle to achieve this goal is the management of limitations such as joint position limitation, singularities and collisions with the environment. Indeed, in addition to performing a given principal task, human users must pay a close attention to the manipulator configuration in order to handle the kinematic limitations. The proposed anticipative algorithm aims at relieving the human user from having to deal with such limitations. The algorithm is first presented and detailed for each individual limitation of a planar RR serial robot. The framework developed to manage several limitations occurring simultaneously is then presented. Finally, experiments are performed in order to assess the performance of the algorithm.

Keywords: Motion Planning for Manipulators, Task Planning, Multi-Robot Systems
Abstract: This paper deals with the problem of combining motion and task assignment for a dual-arm robotic system. Each arm of the system performs independent tasks in a cluttered environment. Robot actions are determined to remove potential obstacles from the environment and obtain collision-free paths to grasp the target objects. The approach uses the information provided by the motion planner to build a graph structure in order to represent the obstacles to be removed. The graph is used, first, to decide which is the next motion path to be computed, and, second, to assign the tasks to each arm of the robotic system. The approach has been implemented for a dual-arm robotic system and real experiments are presented in the paper.

Keywords: Sensor-based Planning, Virtual Reality and Interfaces
Abstract: This work deals with interactive motion planning processes intended to assist a human operator when simulating industrial tasks such as assembly, maintenance or disassembly in Virtual Reality. Such applications need motion planning on surfaces. We propose an original interactive path planning algorithm with contact, I-RRT-C, based on a RRT-Connect approach. This algorithm is based on a real-time interactive approach allowing both an automatic motion planner and a human operator to jointly explore the workspace. A parameter balances the authority between the computer and the operator to reduce processing times. We improve the guidance by allowing to sample on the surfaces of obstacles. Our method allows to find a path in cluttered environments or to solve contact operations such as insertion or sliding tasks. Last, we present experimental results showing that our interactive path planner with contact brings a significant improvement over state of the art methods in both free and contact space.

Keywords: Visual Servoing, Unmanned Aerial Systems, Software and Architecture
Abstract: Self-localization using visual markers is widely used for precise control of Unmanned aerial vehicles (UAVs). This paper presents a novel method of 6-DoF pose estimation using a monocular camera and non co-planar markers. We conducted numerical simulations comparing pose estimation using co-planar markers and using non co-planar markers, and designed a non co-planar marker feasible for robotics applications. Our method uses a single CPU core and a field programmable gate array (FPGA) and can conduct fast and precise 6-DoF pose estimation at the same frequency of the camera input (100 fps).In addition, to show the performance of our method, we applied our 6-DoF pose estimation algorithm to a palm-sized quadcopter and flew it autonomously using only an on-board system with a monocular camera and three-axis gyro sensor.To the authors' knowledge, this is the first study on fully on-board autonomous flight control of an MAV in 6-DoF space without an accelerometer.

Keywords: Visual Servoing, Unmanned Aerial Systems, Motion Control
Abstract: This paper proposes a new adaptive repetitive visual-servo control system for a moving high-flying vehicle (HFV) with an uncalibrated camera to monitor, track, and precisely control the movements of a low-flying vehicle (LFV) or mobile ground robot. When deployed, a remote operator of the HFV defines the desired trajectory for the LFV in the HFV’s camera frame (image frame). Due to the circulatory motion of the HFV, the resulting motion trajectory of the LFV in the image frame is periodic in time, thus an adaptive repetitive control system is exploited to improve the tracking precision from one operating period to the next. Not only is the adaptive control law able to deal with uncertainties in the camera’s intrinsic and extrinsic parameters, but it can also tolerate uncertainties in the localization of the LFV. The design and stability analysis of the closed-loop control system is presented, where the Lyapunov approach is used to show stability. Simulations and experimental results are presented to demonstrate the effectiveness of the method for controlling the movement of a low-flying quadcopter vehicle. Results show good tracking performance for three simulated test cases, where the average maximum tracking error is reduced by approximately 75% compared to the performance of a standard adaptive visual-servo controller.

Keywords: Visual Servoing, Telerobotics, Manipulation Planning and Control
Abstract: Cleaning up the past half century of nuclear waste represents the largest environmental remediation project in the whole Europe. Nuclear waste must be sorted, segregated and stored according to its radiation level in order to optimize maintenance costs. The objective of this work is to develop a shared control framework for remote manipulation of objects using visual information. In the presented scenario, the human operator must control a system composed of two robotic arms, one equipped with a gripper and the other one with a camera. In order to facilitate the operator’s task, a subset of the gripper motion are assumed to be regulated by an autonomous algorithm exploiting the camera view of the scene. At the same time, the operator has control over the remaining null-space motions w.r.t. the primary (autonomous) task by acting on a force feedback device. A novel force feedback algorithm is also proposed with the aim of informing the user about possible constraints of the robotic system such as, for instance, joint limits. Human/hardware-in-the-loop experiments with simulated slave robots and a real master device are finally reported for demonstrating the feasibility and effectiveness of the approach.

Keywords: Micro/Nano Robots, Visual Servoing, Robot Vision
Abstract: Recently, high resolution visual methods based on direct-phase measurement of periodic patterns has been proposed with successful applications to microrobotics. This paper proposes a new implementation of direct-phase measurement methods to achieve 3-DoF (degrees of freedom) visual servoing. The proposed algorithm relies on a single frequency tracking rather than a complete 2D discrete Fourier transform that was required in previous works. The method does not require any calibration step and has many advantages such as high subpixelic resolution, high robustness and short computation time. Several experimental validations (in favorable and unfavorable conditions of use) were performed using a XYtheta microrobotic platform. The obtained results demonstrate the efficiency of the frequency-based controller, this in term of accuracy (micrometric error), convergence rate (30 iterations in nominal conditions) and robustness.

Keywords: SLAM, Robot Vision
Abstract: We present a robust real-time method which performs dense reconstruction of high quality height maps from monocular video. By representing the height map as a triangular mesh, and using efficient differentiable rendering approach, our method enables rigorous incremental probabilistic fusion of standard locally estimated depth and colour into an immediately usable dense model. We present results for the application of free space and obstacle mapping by a low-cost robot, showing that detailed maps suitable for autonomous navigation can be obtained using only a single forward-looking camera.

Keywords: Marine Robotics, SLAM, Mapping
Abstract: We propose a submap-based technique for mapping of underwater structures with complex geometries. Our approach relies on the use of probabilistic volumetric techniques to create submaps from multibeam sonar scans, as these offer increased outlier robustness. Special attention is paid to the problem of denoising/enhancing sonar data. Pairwise submap alignment constraints are used in a factor graph framework to correct for navigation drift and improve map accuracy. We provide experimental results obtained from the inspection of the running gear and bulbous bow of a 600-foot, Wright-class supply ship.

Keywords: Range Sensing, Robot Vision, Collision Detection and Avoidance
Abstract: Obstacle Detection is a central problem for any robotic system, and critical for autonomous systems that travel at high speeds in unpredictable environment. This is often achieved through scene depth estimation, by various means. When fast motion is considered, the detection range must be longer enough to allow for safe avoidance and path planning. Current solutions often make assumption on the motion of the vehicle that limit their applicability, or work at very limited ranges due to intrinsic constraints. We propose a novel appearance-based Object Detection system that is able to detect obstacles at very long range and at a very high speed (mathbf{sim300Hz}), without making assumptions on the type of motion. We achieve these results using a Deep Neural Network approach trained on real and synthetic images and trading some depth accuracy for fast, robust and consistent operation. We show how photo-realistic synthetic images are able to solve the problem of training set dimension and variety typical of machine learning approaches, and how our system is robust to massive blurring of test images.

Keywords: Navigation, Robot Vision, Range Sensing
Abstract: Optical depth cameras, including both time-of-flight and structured-light sensors, have led to dramatic improvement in robot sensing and perception. We propose the use of millimeter-wave (mmW) radar as an important complement to optical sensors. While the millimeter wavelengths of radar sensors do not support as high resolution as the nanometer wavelength of optical sensors, the ability of mmW signals to penetrate smoky and foggy environments as well as see through many opaque objects makes them a compelling sensor for navigation as well as manipulation in challenging environments. We present a series of 2D and 3D mmW images made with a hand-held antenna grasped by a PR2 robot. The radar image sensor uses a mechanical "painting" motion to acquire multiple views of the target object over the 17 to 26 GHz K-band. A GPU-based reconstruction algorithm synthesizes 2D and 3D images of the target object. We demonstrate a range resolution of 13.6 mm and a cross-range resolution of 7.1 mm for objects up to 0.5 m away from the robot. We further demonstrate imaging objects through fog, as well as through opaque paper.

Keywords: Surgical Robotics, Haptics and Haptic Interfaces, Medical Robots and Systems
Abstract: Haptically enabled hands-on or tele-operated surgical robotic systems provide a unique opportunity to integrate pre- and intra-operative information into physical actions through active constraints (also known as virtual fixtures). In many surgical procedures, including cardiac interventions, where physiological motion complicates tissue manipulation, dynamic active constraints can improve the performance of the intervention in terms of safety and accuracy. The non-energy-storing class of dynamic guidance constraints attempt to assist the clinician in following a reference path, while guaranteeing that the control system will not generate undesired motion due to stored potential energy. An important aspect that has not received much attention from the researchers is that while these methods help increase the performance, they should by no means distract the user systematically. In this paper, a viscosity-based dynamic guidance constraint is introduced that continuously redirects the tool’s motion towards the reference path. The proportionality and continuity of generated forces make the method less distracting and subjectively appealing. The performance is validated and compared with two existing non-energy-storing methods through extensive experimentation.

Keywords: Surgical Robotics, Medical Robots and Systems, Cooperative Manipulators
Abstract: Motor channelling is a concept to provide navigation and sensory feedback to operators in master-slave surgical setups. It is beneficial since the introduction of robotic surgery creates a physical separation between the surgeon and patient anatomy. Active Constraints/Virtual Fixtures are proposed which integrate Guidance and Forbidden Region Constraints into a unified control framework. The developed approach provides guidance and safe manipulation to improve precision and reduce the risk of inadvertent tissue damage. Online three-degree-of-freedom motion prediction and compensation of the target anatomy is performed to complement the master constraints. The presented Active Constraints concept is applied to two clinical scenarios; surface scanning for in situ medical imaging and vessel manipulation in cardiac surgery. The proposed motor channelling control strategy is implemented on the da Vinci Surgical System using the da Vinci Research Kit (dVRK) and its effectiveness is demonstrated through a detailed user study.

Keywords: Surgical Robotics, Flexible Arms, Medical Robots and Systems
Abstract: Concentric tube robots comprise telescopic precurved elastic tubes. The robot’s tip and shape are controlled via relative tube motions, i.e. tube rotations and translations. Non-linear interactions between the tubes, e.g. friction and torsion, as well as uncertainty in the physical properties of the tubes themselves, e.g. the Young’s modulus, curvature, or stiffness, hinder accurate kinematic modelling. In this paper, we present a machine-learning-based methodology for kinematic modelling of concentric tube robots and in situ model adaptation. Our approach is based on Locally Weighted Projection Regression (LWPR). The model comprises an ensemble of linear models, each of which locally approximates the original complex kinematic relation. LWPR can accommodate for model deviations by adjusting the respective local models at run-time, resulting in an adaptive kinematics framework. We evaluated our approach on data gathered from a three-tube robot, and report high accuracy across the robot’s configuration space.

Keywords: Surgical Robotics, Medical Robots and Systems, Parallel Robots
Abstract: We propose a new class of robotic device for minimally-invasive surgery that lies at the intersection of continuum, parallel, and reconfigurable robotics. This Continuum Reconfigurable Incisionless Surgical Parallel (CRISP) paradigm involves the use of multiple needle-diameter devices inserted through the skin and assembled into parallel structures inside the body. The parallel structure can be reconfigured inside the patient's body to satisfy changing task requirements such as reaching initially inaccessible locations or modifying mechanical stiffness for manipulation or palpation. Another potential advantage of the CRISP concept is that many small (needle-sized) entry points into the patient may be preferable in terms of both patient healing and cosmesis to the single (or multiple) larger ports needed to admit current surgical robots. This paper presents a mechanics-based model for CRISP forward and inverse kinematics, along with experimental validation.

Keywords: Kinematics, Parallel Robots
Abstract: Solving the direct kinematics (DK) of parallel robots, i.e. finding all the possible poses of the platform for given input values, is most of the time a difficult problem. This is evidently true for cable-driven parallel robots (CDPR) that are more complex than classical parallel robot because of the unilateral nature of the cable actions but also if cable deformations are taken into account. Furthermore there are many different deformable cable models and developing a DK algorithm for each of them will be a tremendous task. Consequently using a numerical continuation scheme that starts from the DK solutions for non-deformable cables and then moves toward the solution for deformable cables appears to be an interesting approach. We first investigate what assumption have to be satisfied by the cable model for using this approach and derive a possible maximum of solutions for the DK, whatever is the cable model. We then apply this approach for a specific complex cable model, the catenary case, to show that this approach is computer efficient but requires to address difficult theoretical issues in order to obtain a solving algorithm that is guaranteed to determine all DK solutions.

Keywords: Mechanism Design, Industrial Robots
Abstract: In most 6 DOF robot arms, considerable amounts of gravitational torques due to the robot’s own weight are applied to pitch joints of the robot, which causes most arms to use high capacity motors and speed reducers. A spring-based counterbalance mechanism can compensate for this gravitational torque, thus leading to a significant reduction in the effect of gravity. However, a simple installation of counterbalance mechanisms at each pitch joint does not work properly because the gravitational torque at each joint is dependent also on the other joints. To achieve multi-DOF counterbalancing, we propose a parallelogram linkage combined with dual counterbalance mechanisms, each being composed of a slider-crank mechanism and springs. Simulations and experimental results showed that the counterbalance robot arm based on the proposed counterbalance mechanisms effectively reduced the torques required to support the robot mass, thus allowing the prospective use of much smaller motors and speed reducers than traditional industrial robots.

Keywords: Parallel Robots, Sensor Fusion, Tendon/Wire Mechanisms
Abstract: Conventionally, a cable driven parallel mechanism (CDPM) pose is obtained through the forward kinematics from measurements of the cable lengths. However, this estimation method can be limiting for some applications requiring more precision. This paper proposes to use cable angle position sensors in addition to cable length measurements in order to improve the accuracy of such mechanisms. The robot pose is first obtained individually by the cable length measurements and the cable angle position measurements. A data fusion scheme combining these two types of measurements is then proposed in order to improve the CPDM accuracy. Finally, simulations and experiments are presented in order to assess the benefits of using cable angle position sensors on the CDPM.

Keywords: Mechanism Design, Manipulation and Compliant Assembly
Abstract: In this paper, we propose a new method for implementing a rotational nonlinear spring with user defined profile, based on the combination of a linear spring with a nonlinear transmission mechanism. The proposed structure consists of a non-circular cam, a roller which moves along outer circumference of the cam, and a stretched translational linear spring which is connected between center of the cam and center of the roller. We obtain a set of differential equations to design shape of the cam for any given torque-angle profile. Also, it will be shown that profiles with both positive and negative values can be implemented by the proposed method. At last, the cam of some popular nonlinear springs are designed, including constant, cubic, hyperbolic tangent and sinusoidal springs.

Keywords: Space Robotics and Automation, Visual Servoing, Motion Planning for Manipulators
Abstract: This work addresses path planning for reactionless visual servoing of a redundant dual-arm space robot through exploration in the image space. The planner explores the image moment based feature space, impends acceleration to the image features and extends the feature tree. A reactionless visual servoing control law is integrated to extend the tree in the configuration space simultaneously. The proposed algorithm is able to incorporate the necessary coupling between the motions of the the dual arms and the base of the robot to ensure zero base reactions. Additionally, it also gives the flexibility to apply multiple constraints in both the image space and the configuration space. The effectiveness of the proposed framework is exhibited by implementing the algorithm on a numerical model of a 14-DoF dual arm space robot.

Keywords: Space Robotics and Automation, Soft-bodied Robots, Motion and Path Planning
Abstract: This work presents a 10 kg tensegrity ball probe that can quickly and precisely deliver a 1 kg payload over a 1 km distance on the Moon by combining cable-driven rolling and thruster-based hopping. Previous research has shown that cable-driven rolling is effective for precise positioning, even in rough terrain. However, traveling large distances using thruster-based hopping, which is made feasible by the lightweight and compliant nature of the tensegrity structure, has not been explored. To evaluate the feasibility of a thruster-based tensegrity robot, a centrally-positioned cold gas thruster with nitrogen propellant was selected, and the system was simulated using the NASA Tensegrity Robotics Toolkit (NTRT) for four hopping profiles on hilly terrains. Optimizing energy efficiency and mechanical capabilities of the tensegrity robot, hopping profiles with a long flight distance per hop, followed by the higher accuracy rolling, are recommended. Simulations also show that thrust regulation can improve energy efficiency. Regulation of thrust magnitude can be achieved using a pressure regulator, but regulation of thrust orientation calls for additional control effort. In this paper, it is demonstrated that gimbal systems as well as shape-shifting control of the tensegrity structure have the potential to regulate thrust orientation. Finally, algorithms for localization and path planning that combine hopping and rolling for energy-efficient navigation are presented.

Keywords: Space Robotics and Automation, Localization, Mapping
Abstract: We present the localization approach for Astrobee, a new free-flying robot designed to navigate autonomously on the International Space Station (ISS). Astrobee will accommodate a variety of payloads and enable guest scientists to run experiments in zero-g, as well as assist astronauts and ground controllers. Astrobee will replace the SPHERES robots which currently operate on the ISS, whose use of fixed ultrasonic beacons for localization limits them to work in a 2 meter cube. Astrobee localizes with monocular vision and an IMU, without any environmental modifications. Visual features detected on a pre-built map, optical flow information, and IMU readings are all integrated into an extended Kalman filter (EKF) to estimate the robot pose. We introduce several modifications to the filter to make it more robust to noise, and extensively evaluate the localization algorithm.

Keywords: Space Robotics and Automation, Unmanned Aerial Systems, Service Robots
Abstract: This paper presents the design of a small aerial robot for inhabited microgravity environments, such as orbiting space stations (e.g., ISS). In particular, we target a fleet of robots, called Space CoBots, for collaborative tasks with humans, such as telepresence and cooperative mobile manipulation. The design is modular, comprising an hexrotor based propulsion system, and a stack of modules including batteries, cameras for navigation, a screen for telepresence, a robotic arm, space for extension modules, and a pair of docking ports. These ports can be used for docking and for mechanically attaching two Space CoBots together. The kinematics is holonomic, and thus the translational and the rotational components can be fully decoupled. We employ a multi-criteria optimization approach to determine the best geometric configuration for maximum thrust and torque across all directions. We also tackle the problem of motion control: we use separate converging controllers for position and attitude control. Finally, we present simulation results using a realistic physics simulator. These experiments include a sensitivity evaluation to sensor noise and to unmodeled dynamics, namely a load transportation.

Keywords: Human-Robot Interaction, Cognitive Human-Robot Interaction
Abstract: The objective of this study is to design a human-robots system that takes into account the non-deterministic nature of the human operator's behavior. Such a system is implemented in a proof of concept scenario relying on a (MO)MDP decision framework that takes advantage of an eye-tracker device to estimate the cognitive availability of the human operator, and, some human operator's inputs to deduce where he is focusing his attention. An experiment was conducted with ten participants interacting with a team of autonomous vehicles in a Search & Rescue scenario. Our results demonstrate the advantages of considering the cognitive availability of a human operator in a such complex context and also the interest of using such a decisional framework that can formally integrate the non-deterministic outcomes which model the human behavior.

Keywords: Human-Robot Interaction, Physical Human-Robot Interaction, Humanoid Interaction
Abstract: We propose an online iterative path optimisation method to enable a Baxter humanoid robot to assist human users to dress. The robot searches for the optimal personalised dressing path using vision and force sensor information: vision information is used to recognise the human pose and model the movement space of upper-body joints; force sensor information is used for the robot to detect external force resistance and to locally adjust its motion. We propose a new stochastic path optimisation method based on adaptive moment estimation. We first compare the proposed method with other path optimisation algorithms on synthetic data. Experimental results show that the performance of the method achieves the smallest error with fewer iterations and less computation time. We also evaluate real-world data by enabling the Baxter robot to assist real human users with their dressing.

Keywords: Human-Robot Interaction, Recognition
Abstract: Human activity recognition plays an important role in personal assistive robot, being able to recognize human activity and perform corresponding assistive action is a great challenges for personal assistive robot. Human body is an articulated system of rigid segments that can be divided into five parts. Many existing methods always identify actions based on the motion trajectories of whole body. In this paper, taking into account the fact that most actions can be performed by a few limbs and the other limbs should not impact on the action recognition, we proposed an activity recognition method based on limb weights. The weight of each limb is composed of consistence weight and uniqueness weight, which are learned according to the similarity degree among different sequences for each specific action. The covariance descriptor, which is the concatenation of eigenvalues extracted from covariance matrices, is adapted to represent the motion trajectory of each limb. In order to distinguish action instances from each other in the feature sequences, a simple annotation method is used. Experimental results on the public dataset and the Lab dataset show that the proposed method not only can outperform the state-of-the-art algorithms, but also is appropriate to recognize the actions whose non-core limbs’ trajectories are different from each other.

Keywords: Human-Robot Interaction, Unmanned Aerial Systems, Robot Companions and Social Human-Robot Interaction
Abstract: We present the first demonstration of end-to-end far-to-near situated interaction between an uninstrumented human user and an initially distant outdoor autonomous Unmanned Aerial Vehicle (UAV). The user uses an arm-waving gesture as a signal to attract the UAV's attention from a distance. Once this signal is detected, the UAV approaches the user using appearance-based tracking until it is close enough to detect the human's face. Once in this close-range interaction setting, the user is able to use hand gestures to communicate its commands to the UAV. Throughout the interaction, the UAV uses colored-light-based feedback to communicate its intent to the user. We developed this system to work reliably with a low-cost consumer UAV, with only computation off-board. We describe each component of this interaction system, giving details of the depth estimation strategy and the cascade predictive flight controller for approaching the user. We also present experimental results on the performance of the complete system and its individual components.

Keywords: Multi-Robot Systems, Path Planning for Multiple Mobile Robots or Agents
Abstract: In this paper we consider the problem of coordinating the motion of a group of Automated Guided Vehicles (AGVs) utilized in industrial environments for logistics operations. In particular, we consider a hierarchical coordination strategy, where the environment is partitioned into sectors: coordination on the top layer defines the sequence of sectors to be traveled, while coordination on the bottom layer deals with traffic management inside each sector. In this paper we introduce a novel partitioning algorithm, that defines the sectors in a dynamic manner, taking into account both the shape of the (generally non-convex) environment, and the current distribution of the AGVs. This is achieved exploiting a clustering algorithm, and subsequently defining the sectors based on the geodesic distance.

Keywords: Multi-Robot Systems, Autonomous Agents, Path Planning for Multiple Mobile Robots or Agents
Abstract: Autonomous multi-robot teams can be used in complex coordinated exploration tasks to improve exploration performance in terms of both speed and effectiveness. However, use of multi-robot systems presents additional challenges. Specifically, in domains where the robots' actions are tightly coupled, coordinating multiple robots to achieve cooperative behavior at the group level is difficult. In this paper, we demonstrate that reward shaping can greatly benefit learning in multi-robot explorations tasks. We propose a novel reward framework based on the idea of counterfactuals to tackle the coordination problem in tightly coupled domains. We show that the proposed algorithm provides superior performance by at least one order of magnitude compared to policies learned using either the global reward or the difference reward.

Keywords: Swarm Robotics, Environment Monitoring and Management, Search and Rescue Robots
Abstract: As robot swarms become more viable, efficient solutions to fundamental tasks such as swarm search and collection are required. We propose the distributed deterministic spiral algorithm (DDSA) which generalises a spiral search pattern to robot swarms. While being an effective search strategy in its own right, the DDSA is also a useful point of comparison for other swarm search strategies. Such a benchmark for robot swarm search is currently needed but missing. As a case study, we compare the DDSA to a biologically-inspired central-place foraging algorithm that uses stochastic search, memory, and communication.

Keywords: Path Planning for Multiple Mobile Robots or Agents, Motion and Path Planning, Agent-Based Systems
Abstract: We present a reactive multi-agent push system for a large set of objects. The behavior for the pushing agents consists of: 1) selecting and updating an object set to push, 2) reaching positions near the objects to start influencing, 3) pushing the objects along a path to the goal region, and 4) regrouping when needed to ensure the group is packed tightly enough. The emergent properties of the behavior allow us to test how effectively a group of agents can push a set of objects through the environment with different strategies.

Keywords: Biologically-Inspired Robots, Motion and Trajectory Generation, Legged Robots
Abstract: This paper presents Bezier curve based passive neural control applied in bio-inspired locomotion in order to decrease the computational cost implemented for 4 legged animal robot which has 3 joints in each leg. Neural oscillator model is applied for generating the walking pattern in bio-inspired locomotion. Bezier curve based optimization represents passive neural control supported by evolutionary algorithm for representing the relationship equation between neuron signal and reference joint signal. Passive neural control is implemented in order to reduce the neuron complexity in neuro-based locomotion by controlling 3 joints with one signal without decreasing the performance both in walking pattern and in its stability level, whereas one leg is represented by one motor neuron. Therefore, the 4 legged robot is controlled by 4 motor neurons which have feedback connection with ground and inertial sensor. In order to prove the effectiveness, we implemented the model in computer simulation and in a small 4 legged robot. This model can decrease the computational cost so it is possible to apply the model in either animal or humanoid robot with low frequency processor.

Keywords: Compliance and Impedance Control, Unmanned Aerial Systems, Flexible Arms
Abstract: This paper presents the design and experimental validation of a compliant and lightweight 3-DOF robotic arm – shoulder yaw, shoulder pitch and elbow pitch joints – equipped with a compliant finger module intended for aerial inspection and manipulation in contact with the environment. A simple transmission mechanism consisting in a pair of compression springs and a flange bearing is integrated in the shoulder pitch and elbow pitch joints between the servo shaft and the output frame. Joint deflection measurement with potentiometer allows joint torque but also contact force estimation and control. The low stiffness of the compliant finger has been exploited for soft collision detection and obstacle localization, in such a way that the contact forces do not significantly affect UAV stability. Fixed-base experiments have been performed with the arm, including the characterization of the compliant joints and the control of the contact force at wrist point.

Keywords: Wheeled Robots, Mechanism Design, Kinematics
Abstract: In this paper, new type of an active-caster with a single ball transmission (ACROBAT-S) is proposed. The proposed ball transmission is a novel mechanism which realizes combining of two motor powers together to rotate single ball in a 2Dway and simultaneously distributing the combined power to a wheel shaft and a steering shaft of an active-caster in an appropriate ratio. The transmission design enables to remove a sensor for detecting the wheel orientation for coordinated control. Also it would be possible to build an omnidirectional robot with three active casters to be controlled its 3D motion by three motors with no redundancy.

The new concept of ACROBAT-S gives many advantages compared with the original ACROBAT since the number of ball is reduced from two to one, therefore the number of friction drive between a ball and rollers or a ball to a ball is also reduced from five to three. These features contribute not only to simplify the mechanism design but also to enhance its performance since slippages and energy losses would be reduced. To verify the design concept, we derive and analyze a kinematic model of a single ball transmission mechanism. Furthermore the motions of ACROBAT-S are verified by Solid Works 3D simulator.

From the results, it is confirmed that the proposed single ball transmission is applicable for omnidirectional wheel mechanism to realize the omnidirectional motions.

Keywords: Perception for Grasping and Manipulation, Visual Learning, Robot Vision
Abstract: This paper presents a new method for parallel-jaw grasping of isolated objects from depth images, under large gripper pose uncertainty. Whilst most approaches aim to predict the single best grasp pose from an image, our method first predicts a score for every possible grasp pose, which we denote the grasp function. With this, it is possible to achieve grasping robust to the gripper's pose uncertainty, by smoothing the grasp function with the pose uncertainty function. Therefore, if the single best pose is adjacent to a region of poor grasp quality, that pose will no longer be chosen, and instead a pose will be chosen which is surrounded by a region of high grasp quality. To learn this function, we train a Convolutional Neural Network which takes as input a single depth image of an object, and outputs a score for each grasp pose across the image. Training data for this is generated by use of physics simulation and depth image simulation with 3D object meshes, to enable acquisition of sufficient data without requiring exhaustive real-world experiments. We evaluate with both synthetic and real experiments, and show that the learned grasp score is more robust to gripper pose uncertainty than when this uncertainty is not accounted for.

Keywords: Human Detection and Tracking, Kinematics, Human Centered Robotics
Abstract: In this paper, we present a novel kinematic framework using hybrid twists, that has the potential to improve the reliability of estimated human shoulder kinematics. This is important as the functional aspects of the human shoulder are evaluated using the information embedded in thoracohumeral kinematics. We successfully demonstrate in our results, that our approach is invariant of the body-fixed coordinate definition, is singularity free and has high repeatability; thus resulting in a flexible user-specific kinematic tracking not restricted to bony landmarks

Keywords: Software and Architecture, Architectures, Protocols And Middle-Ware, Networked Robots
Abstract: While the topic of security in industrial applications has gained some momentum in recent years, there are still severe security vulnerabilities which are actively exploited for attacks. The robot operating system (ROS) is expected to further grow in usage and to be used in many industrial applications. Analysis, however, shows that it lacks several security enhancements in order to make it suitable for industrial use. In its current state, false data and commands can be injected posing a possible safety risk for the resulting product and humans in the production. In addition, data may be eavesdropped and used by outsiders to gain insight into the production process. In this paper we propose a security architecture intended for use on top of ROS on the application level. We use a dedicated authorization server to ensure that only valid nodes are part of the application. Cryptographic methods ensure data confidentiality and integrity. We show in a demonstration with a collaborative robot how our architecture can be used to secure a ROS-based application.

Keywords: Architectures, Protocols And Middle-Ware, Swarm Robotics, Programming Environments
Abstract: This paper presents OpenSwarm, a lightweight easy-to-use open-source operating system. To our knowledge, it is the first operating system designed for and deployed on miniature robots. OpenSwarm operates directly on a robot's microcontroller. It has a memory footprint of 1 kB RAM and 12 kB ROM. OpenSwarm enables a robot to execute multiple processes simultaneously. It provides a hybrid kernel that natively supports preemptive and cooperative scheduling, making it suitable for both computationally intensive and swiftly responsive robotics tasks. OpenSwarm provides hardware abstractions to rapidly develop and test platform-independent code. We show how OpenSwarm can be used to solve a canonical problem in swarm robotics--clustering a collection of dispersed objects. We report experiments, conducted with five e-puck mobile robots, that show that an OpenSwarm implementation performs as good as a hardware-near implementation. The primary goal of OpenSwarm is to make robots with severely constrained hardware more accessible, which may help such systems to be deployed in real-world applications.

Keywords: Software and Architecture, Robot Safety
Abstract: Robots are being increasingly used in safety-critical contexts, such as transportation and health. The need for flexible behavior in these contexts, due to human interaction factors or unstructured operating environments, led to a transition from hardware- to software-based safety mechanisms in robotic systems, whose reliability and quality is imperative to guarantee. Source code static analysis is a key component in formal software verification. It consists on inspecting code, often using automated tools, to determine a set of relevant properties that are known to influence the occurrence of defects in the final product. This paper presents HAROS, a generic, plug-in-driven, framework to evaluate code quality, through static analysis, in the context of the Robot Operating System (ROS), one of the most widely used robotic middleware. This tool (equipped with plug-ins for computing metrics and conformance to coding standards) was applied to several publicly available ROS repositories, whose results are also reported in the paper, thus providing a first overview of the internal quality of the software being developed in this community.

Keywords: Localization, Robot Vision, Performance Evaluation and Benchmarking
Abstract: With the recent success of visual features from deep convolutional neural networks (DCNN) in visual robot self-localization, it has become important and practical to address more general self-localization scenarios. In this paper, we address the scenario of self-localization from images with small overlap. We explicitly introduce a localization difficulty index (LDI) as a decreasing function of view overlap between query and relevant database images and investigate performance versus difficulty for challenging cross-view self-localization tasks. We then reformulate the self-localization as a scalable bag-of-visual-features (BoVF) scene retrieval and present an efficient solution called PCA-NBNN, aiming to facilitate fast, yet discriminative correspondence between partially overlapping images. The proposed approach adopts recent findings in discriminativity preserving encoding of DCNN features using principal component analysis (PCA) and cross-domain scene matching using naive Bayes nearest neighbor distance metric (NBNN). We experimentally demonstrate that the proposed PCA-NBNN framework frequently achieves comparable results to previous DCNN features and that the BoVF model is significantly more efficient. We further address an important alternative scenario of ``self-localization from images with NO overlap" and report the result.

Keywords: SLAM, Robot Vision, Visual Tracking
Abstract: Real-time outdoor navigation in highly dynamic environments is an crucial problem. The recent literature on real-time static SLAM don't scale up to dynamic outdoor environments. Most of these methods assume moving objects as outliers or discard the information provided by them. We propose an algorithm to jointly infer the camera trajectory and the moving object trajectory simultaneously. In this paper, we perform a sparse scene flow based motion segmentation using a stereo camera. The segmented objects motion models are used for accurate localization of the camera trajectory as well as the moving objects. We exploit the relationship between moving objects for improving the accuracy of the poses. We formulate the poses as a factor graph incorporating all the constraints. We achieve exact incremental solution by solving a full nonlinear optimization problem in real time. The evaluation is performed on the challenging KITTI dataset with multiple moving cars.Our method outperforms the previous baselines in outdoor navigation.

Keywords: SLAM, Localization, Range Sensing
Abstract: Maps are an important component of most robotic navigation systems and building maps under uncertainty is often referred to as simultaneous localization and mapping or SLAM. Most SLAM approaches start from scratch and build a map only based on their own observations and odometry information. In this paper, we address the problem of how additional information can be exploited, for example from OpenStreetMap. We extend the standard graph-based SLAM formulation by relating the nodes of the pose-graph with an existing map. As this paper suggests, we can relate the newly built maps with information from publicly available maps with the laser range finder data from the robot and in this way improve the map quality. We implemented and evaluated our approach using real world data taken in urban environments. We illustrate that our extension to graph-based SLAM provides better aligned maps and adds only a marginal computational overhead.

Keywords: Localization, Mapping
Abstract: Localization in dynamic environments is still a challenging problem in robotics - especially if rapid and huge changes occur irregularly. Inspired by SLAM algorithms our Bayesian approach to this so-called dynamic localization problem divides it into a localization problem and a mapping problem, respectively. To tackle the localization problem we use a particle filter, coupled with a distance filter and a scan matching method, which achieves a more robust localization against dynamic obstacles. For the mapping problem we use an extended sensor model which results in an effective and precise map update effect. We compare our approach against other localization methods and evaluate the impact the map update effect has on the localization in dynamic environments.

Keywords: Mapping, Visual Navigation, Robot Vision
Abstract: Real-time 3D mapping with MAV (Micro Aerial Vehicle) in GPS-denied environment is a challenging problem. In this paper, we present an effective vision-based 3D mapping system with 2D laser-scanner. All algorithms necessary for this system are on-board. In this system, two cameras work together with the laser-scanner for motion estimation. The distance of the points detected by laser-scanner are transformed and treated as the depth of image features, which improves the robustness and accuracy of the pose estimation. The output of visual odometry is used as an initial pose in the Iterative Closest Point (ICP) algorithm and the motion trajectory is optimized by the registration result. We finally get the MAV’s state by fusing IMU with the pose estimation from mapping process. This method maximizes the utility of the point clouds information and overcomes the scale problem of lacking depth information in the monocular visual odometry. The results of the experiments prove that this method has good characteristics in real-time and accuracy.

Keywords: Recognition, Visual Navigation, SLAM
Abstract: In this paper we propose a novel technique for detecting loop closures on a trajectory by matching sequences of images instead of single instances. We build upon well established techniques for creating a bag of visual words with a tree structure and we introduce a significant novelty by extending these notions to describe the visual information of entire regions using Visual-Word-Vectors. The fact that the proposed approach does not rely on a single image to recognize a site allows for a more robust place recognition, and consequently loop closure detection, while reduces the computational complexity for long trajectory cases. We present evaluation results for multiple publicly available indoor and outdoor datasets using Precision-Recall curves, which reveal that our method outperforms other state of the art algorithms.

Keywords: Localization, Robot Audition, Cognitive Human-Robot Interaction
Abstract: Sound source localization (SSL) is an essential technique in many applications, such as robot audition, humanrobot interaction and speech capturing. However, SSL from a binaural input is still a challenging problem, particularly when multiple sources are active simultaneously. In this work, we propose a multi-sources localization framework based on the time-delay compensation (TDC) estimator and clustering analysis. The TDC estimator is a simultaneous operator to estimate binaural cues, which breaks the limitation of independent processors for binaural cues extraction. The multi-sources decision is realized by clustering analysis for the binaural cues of multiple signal frames. In experiments, we demonstrate that the localization performance is improved compared to the methods that assume the number of spatial stationary sources to be known. Results with both simulated and recorded impulse responses show that robust performance can be achieved with limited prior training, and our method is also adaptive to different sound activities.

Keywords: Visual Learning, Robot Vision, Localization
Abstract: We present two multi-modal panoramic 3D outdoor (MPO) datasets for semantic place categorization with six categories: forest, coast, residential area, urban area and indoor/outdoor parking lot. The first dataset consists of 650 static panoramic scans of dense (~9,000,000 points) 3D color and reflectance point clouds obtained using a FARO laser scanner with synchronized color images. The second dataset consists of 34,200 real-time panoramic scans of sparse (~70,000 points) 3D reflectance point clouds obtained using a Velodyne laser scanner while driving a car. The datasets were obtained in the city of Fukuoka, Japan and are publicly available in [1], [2]. In addition, we compare several approaches for semantic place categorization with best results of 96.42% (dense) and 89.67% (sparse).

Keywords: Localization, Sensor Fusion
Abstract: Localization of mobile robots can be challenging in highly dynamic environments (airport terminals, stores, hospitals) yet these areas offer great potential for robotics applications. High WiFi coverage provides a way to localize even low-cost robots that do not need to be equipped with expensive exteroceptive sensors or excessive computational power to run visual-based SLAM algorithms. Since current trend in robotics is in mass-deployment of low-cost, replaceable robots, we focus our efforts in this direction as well. Possible solution is combination of high-tech robots deployed as support assuring reliable localization for the low-cost robots that are actually doing the job. The proposed scenario is to use one SLAM robot to map WiFi signal strength in the working area and provide it to a low-cost robot to correct drift of its 6-DOF gyro-odometry localization system. For this purpose, we extend Gaussian-processes-based WiFi localization algorithms to full 3D and experimentally evaluate the proposed approach in 2 mapping and 7 localization sorties performed on different dates spanning four months.

Keywords: Mapping, SLAM, Service Robots
Abstract: We present a lifelong mapping and localisation system for long-term autonomous operation of mobile robots in changing environments. The core of the system is a spatio-temporal occupancy grid that explicitly represents the persistence and periodicity of the individual cells and can predict the probability of their occupancy in the future. During navigation, our robot builds temporally local maps and integrates then into the global spatio-temporal grid. Through re-observation of the same locations, the spatio-temporal grid learns the long-term environment dynamics and gains the ability to predict the future environment states. This predictive ability allows to generate time-specific 2d maps used by the robot's localisation and planning modules. By analysing data from a long-term deployment of the robot in a human-populated environment, we show that the proposed representation improves localisation accuracy and the efficiency of path planning. We also show how to integrate the method into the ROS navigation stack for use by other roboticists.

Keywords: Environment Monitoring and Management, SLAM, Unmanned Aerial Systems
Abstract: In this paper we present a real-time approach to stitch large-scale aerial images incrementally. A monocular SLAM system is used to estimate camera position and attitude, and meanwhile 3D point cloud map is generated. When GPS information is available, the estimated trajectory is transformed to WGS84 coordinates after time synchronized automatically. Therefore, the output orthoimage retains global coordinates without ground control points. The final image is fused and visualized instantaneously with a proposed adaptive weighted multiband algorithm. To evaluate the effectiveness of the proposed method, we create a publicly available aerial image dataset with sequences of different environments. The experimental results demonstrate that our system is able to achieve high efficiency and quality compared to state-of-the-art methods. In addition, we share the code on the website with detailed introduction and results.

Keywords: Mapping, Visual Navigation, Localization
Abstract: Precisely estimating the pose of an agent in a global reference frame is a crucial goal that unlocks a multitude of robotic applications, including autonomous navigation and collaboration. In order to achieve this, current state-of-the-art localization approaches collect data provided by one or more agents and create a single, consistent localization map, maintained over time. However, with the introduction of lengthier sorties and the growing size of the environments, data transfers between the backend server where the global map is stored and the agents are becoming prohibitively large. While some existing methods partially address this issue by building compact summary maps, the data transfer from the agents to the backend can still easily become unmanageable. In this paper, we propose a method that is designed to reduce the amount of data that needs to be transferred from the agent to the backend, functioning in large-scale, multi-session mapping scenarios. Our approach is based upon a landmark selection method that exploits information coming from multiple, possibly weak and correlated, landmark utility predictors; fused using learned feature coefficients. Such a selection yields a drastic reduction in data transfer while maintaining localization performance and the ability to efficiently summarize environments over time. We evaluate our approach on a data set that was autonomously collected in a dynamic indoor environment over a period of several months.

Keywords: Mapping, SLAM, Field Robots
Abstract: Having a team of robots to perform a task such as mapping is faster and more reliable than doing the same with a single robot, which can be crucial in scenarios such as search and rescue. We are developing a fully distributed framework for collaborative mapping with large robot swarms that is robust to abrupt departure of robots due to malfunctions or network problems. While several approaches to multi-robot mapping have been proposed, most of them either build a collection of local sub-maps, or rely on a central authority to merge maps built by individual robots. Our framework is unique in that it requires no central authority, yet allows robots to simultaneously contribute to a single global map, which is stored in a decentralized fashion. This greatly improves the scalability of our system with respect to number of robots. However, our approach requires systematic coordination among robots in order to make modifications to the map. Unannounced departure of the robots makes coordination challenging, and can potentially make the map inconsistent or result in loss of data. We borrow ideas from the domain of distributed computing to address those challenges. Further, we demonstrate the robustness of the proposed system by subjecting it to various conditions in which participating robots fail.

Keywords: Localization, Sensor Fusion, Service Robots
Abstract: We consider the problem of self-localisation for a mobile robot in an environment with a requested level of accuracy. The robot moves in a known environment following typical trajectories, which can be characterised in statistical terms. One of the main drivers of this paper is its application to assistive robots guiding senior or impaired users in shopping centres or in other public spaces. To localise itself the robot uses onboard sensors such as encoders and inertial platforms. The level of noise in these sensors and the lack of absolute measurements determines a steady growth of the uncertainty on its position. To alleviate the problem, we assume the presence of a number of visual markers deployed in the environment. Whenever the robot comes across one of these sensors, the uncertainty on its position is reset. In the paper, we show a methodology to minimise the number of these sensors and to select their position so that the uncertainty is never worse than a given target threshold with an assigned probability.

Keywords: SLAM, Motion and Path Planning
Abstract: In this work we address the problem of trajectory planning in Graph SLAM. We propose the use of Expected Value of the Final Uncertainty, which summarizes all the possible uncertainties that should be considered. In fully explored environments, this is used to determine the most reliable path to the final position. In partially explored environments, this criteria quantifies the reliability of the path planned in the free space. Tests demonstrate its ability to avoid unreliable paths in fully explored environments as compared to other uncertainty based criteria. In the exploration scenario, potential paths not present in the original graph are proposed using Voronoi Diagram of the space ahead observed by the sensors. A cost function is proposed considering the length of the path as well as the expected final uncertainty, thus including potentially shorter, but still reliable paths. Tests demonstrate the shortest path is preferred as long as it contains loop closures with low uncertainty.

Keywords: SLAM, Robot Vision, Mapping
Abstract: Mapping and self-localization in unknown environments are fundamental capabilities in many robotic applications. These tasks typically involve the identification of objects as unique features or landmarks, which requires the objects both to be detected and then assigned a unique identifier that can be maintained when viewed from different perspectives and in different images. The data association and simultaneous localization and mapping (SLAM) problems are, individually, well-studied in the literature. But these two problems are inherently tightly coupled, and that has not been well-addressed. Without accurate SLAM, possible data associations are combinatorial and become intractable easily. Without accurate data association, the error of SLAM algorithms diverge easily. This paper proposes a novel nonparametric pose graph that models data association and SLAM in a single framework. An algorithm is further introduced to alternate between inferring data association and performing SLAM. Experimental results show that our approach has the new capability of associating object detections and localizing objects at the same time, leading to significantly better performance on both the data association and SLAM problems than achieved by considering only one and ignoring imperfections in the other.

Keywords: Mapping, Localization
Abstract: Indoor robot localization systems using wireless signal measurements have gained popularity in recent years, as wireless Local Area Networks can be found practically everywhere. In this field, a popular approach is the use of fingerprinting techniques, such as Gaussian Processes. In our approach, we improve Gaussian Processes based mapping using path loss models as priors. Path loss models encode information regarding the signal propagation phenomena into the mapping. Our approach first fits training data to a simple path loss model, to then train a zero-mean Gaussian Process with the mismatches between the models and the data. Signal strength mean predictions are done using both the path loss model and the Gaussian Process output, while variances are calculated by bounding the Gaussian Process variance using the path loss models. Notably, the main improvement generated by our approach is not an enhanced mean value prediction, but rather a better model variance prediction. This translates into better likelihood estimations, leading to higher localization accuracy. Experiments using data acquired in an indoor environment and our approach as the perceptual likelihood of a dual Monte Carlo localization algorithm are used to demonstrate this improvement. Furthermore, this idea can be extrapolated to other fingerprinting techniques and to applications other than wireless-based localization.

Keywords: Recognition, SLAM, Visual Navigation
Abstract: In this paper we present an approach for efficiently retrieving the most similar image, based on point-to-point correspondences, within a sequence that has been acquired through continuous camera movement. Our approach is entailed to the use of standardized binary feature descriptors and exploits the temporal form of the input data to dynamically adapt the search structure. While being straightforward to implement, our method exhibits very fast response times and its Precision/Recall rates compete with state of the art approaches. Our claims are supported by multiple large scale experiments on publicly available datasets.

Keywords: Virtual Reality and Interfaces, Human-Robot Interaction, Mapping
Abstract: Map building is a fundamental task in many robotic applications. In this paper, we propose a novel approach for 3D mapping of indoor environments which allows a robot avatar to collaborate with a human seamlessly through a virtual reality (VR) device. The 3D map is created using the 3D data from an RGB-D camera mounted on the robot and simultaneously transmitted to a remote server, and then rendered to the VR device. On the other hand, the intentions of the user are inferred using the motion of the head movement based on hidden Markov models (HMMs), and then interpreted into commands to control the robot. We implement the proposed approach based on a modified Pioneer robot platform. The experimental results show the feasibility of the proposed system.

Keywords: SLAM, Localization, Mapping
Abstract: Graph-based SLAM has proved to be one of the most effective solutions to the Simultaneous Localization and Mapping problem. This approach relies on nonlinear iterative optimization methods that in practice perform both accurately and efficiently. However, due to the non-convexity of the problem, the obtained solutions come with no guarantee of global optimality and may get stuck in local minima. The application of SLAM to many real-world applications cannot be conceived without additional control tools that detect possible suboptimalities as soon as possible in order to take corrective action and avoid catastrophic failure of the entire system. This paper builds upon the state-of-the-art framework [1] in verification for this problem and introduces a novel superior formulation that leads to a much higher efficiency. While retaining the same high effectiveness, the verification times of our proposal reduce up to >50x, paving the way for faster verification in critical real applications or in embedded low-power systems. We support our claims with extensive experiments with real and simulated data.

Keywords: Visual Navigation, SLAM, Localization
Abstract: Multi-camera clusters used for visual SLAM assume a fixed calibration between the cameras, which places many limitations on its performance, and directly excludes all configurations where a camera in the cluster is mounted to a moving component. In this work, we present a calibration method for dynamic multi-camera clusters, where one or more of the cluster cameras is mounted to an actuated mechanism, such as a gimbal or robotic manipulator. Our calibration approach parametrizes the actuated mechanism using the Denavit-Hartenberg convention, then determines the calibration parameters which allow for the estimation of the time varying extrinsic transformations between camera frames. We validate our calibration approach using a dynamic camera cluster consisting of a static camera and a camera mounted to a pan-tilt unit, and demonstrate that the dynamic camera cluster can provide accurate tracking when used to perform SLAM.

Keywords: Mapping, SLAM, Model Learning
Abstract: Autonomous service mobile robots need to consistently, accurately, and robustly localize in human environments despite changes to such environments over time. Episodic non-Markov Localization~(EnML) addresses the challenge of localization in such changing environments by classifying observations as arising from Long-Term, Short-Term, or Dynamic Features. However, EnML relies on an estimate of the Long-Term Vector Map (LTVM) that does not change over time. In this paper, we introduce a recursive algorithm to build and update the LTVM over time by reasoning about visibility constraints of objects observed over multiple robot deployments. Using a signed distance function (SDF) to filter out observations of short-term and dynamic features, the remaining long-term observations are used to build a vector map by robust local linear regression, with uncertainty in the resulting LTVM computed by Monte Carlo resampling. Using scatter matrix decomposition, the map features remain faithful to the entire observation history without storing each observation explicitly, leading to significant memory savings. We present experimental results demonstrating the accuracy, robustness, and compact nature of the extracted LTVMs from several long-term robot datasets.

Keywords: Localization, Human-Robot Interaction, Human Detection and Tracking
Abstract: Localization of a human or a robot in indoor environment prior to initial human-robot interaction is the motivation of this paper. The utilization of Wi-Fi received signal strength (RSS) by adopting fingerprinting technique is used to compute the location estimation. However, Wi-Fi RSS instability incurred by mutable channel characteristics hampers a wide-spread adoption of RSS based location fingerprinting to real world applications. To overcome RSS instability problem, we propose a new method based on the concept of “invariant RSS statistics”, where it is defined as the RSS samples collected at each calibration location under minimal random spatiotemporal disturbances. The invariant RSS statistics thus collected forms the reference pattern classes in the space formed by Wi-Fi sources. Fingerprinting is done by identifying the reference pattern class that maximally supports the RSS readings collected at an unknown location for available Wi-Fi sources. The support of RSS readings is defined here as the sum of the likelihood probabilities of individual RSS readings. Experimental results show that the proposed method provides superior performance to conventional ones with the success rate higher by 17%, the printing resolution finer by 30% and, naturally, no performance degradation in time without recalibration. Lastly, we present a design guideline that relates statistically the different levels of the random spatiotemporal disturbances inducing RSS instability to the minimum number of Wi-Fi sources required for achieving a certain class separation degree under the given number of calibration locations to be identified.

Keywords: Localization, Visual Navigation, Robot Vision
Abstract: The extreme variability in the appearance of a place across the four seasons of the year is one of the most challenging problems in life-long visual topological localization for mobile robotic systems and intelligent vehicles. Traditional solutions to this problem are based on the description of images using hand-crafted features, which have been shown to offer moderate invariance against seasonal changes. In this paper, we present a new proposal focused on automatically learned descriptors, which are processed by means of a technique recently popularized in the computer vision community: Convolutional Neural Networks (CNNs). The novelty of our approach relies on fusing the image information from multiple convolutional layers at several levels and granularities. In addition, we compress the redundant data of CNN features into a tractable number of bits for efficient and robust place recognition. The final descriptor is reduced by applying simple compression and binarization techniques for fast matching using the Hamming distance. An exhaustive experimental evaluation confirms the improved performance of our proposal (CNN-VTL) with respect to state-of-the-art methods over varied long-term datasets recorded across seasons.

Keywords: SLAM, Motion and Path Planning, Marine Robotics
Abstract: This paper reports on an active SLAM framework that leverages the Bayes tree data structure for efficient planning. Evaluating information-theoretic objective functions in the context of active SLAM is a very expensive process that requires significant computational overhead. The contributions of this work involve exploiting the structure of the planning problem integrated with SLAM via the Bayes tree graphical model. Specifically, we propose a constrained variable ordering and subtree caching scheme that reduce computational complexity by eliminating redundant computations between candidate actions that are similar. We also propose an active SLAM framework that utilizes these concepts, and demonstrate the benefits of the approach with an underwater robot performing visual SLAM in a hybrid simulation environment.

Keywords: Sensor-based Planning, Marine Robotics, Collision Detection and Avoidance
Abstract: In this paper we propose a new vision-based obstacle avoidance strategy using the Underwater Dark Channel Prior (UDCP) that can be applied to any Unmanned Underwater Vehicle (UUV) equipped with a simple monocular camera and minimal on-board processing capabilities. For each incoming image, our method first computes a relative depth map to estimate the obstacles nearby. Then, the map is segmented and the most promising Region of Interest (RoI) is identified. Finally, an escape direction is computed within the RoI and a control action is performed accordingly to avoid the obstacles. We tested our approach on a video sequence in a natural environment and compared it against a state-of-the-art method showing better performance, specially in light changing conditions. We also provide online results on a low-cost Remotely Operated Vehicle (ROV) in a controlled environment.

Keywords: Task Planning, Planning, Scheduling and Coordination
Abstract: In this paper, a new task-oriented active-sensing method is presented. Most active sensing methods choose sensing actions that minimize the uncertainty of the state according to some information-theoretic measure. While this is reasonable for most applications, minimizing state uncertainty may not be most relevant when the state information is used to perform a task. This is because the uncertainty in some subspace of the state space could have more impact on the performance of the task than the others at a given time. The active-sensing method presented in this paper takes the task into account when selecting sensing actions by minimizing the uncertainty in future task action.

Keywords: Haptics and Haptic Interfaces, Motor Skill Learning, Wearable Robots
Abstract: In this paper we present a wearable Haptic Feedback Device to convey intuitive motion direction to the user through haptic feedback based on vibrotactile illusions. Vibrotactile illusions occur on the skin when two or more vibrotactile actuators in proximity are actuated in coordinated sequence, causing the user to feel combined sensations, instead of separate ones. By combining these illusions we can produce various sensation patterns that are discernible by the user, thus allowing to convey different information with each pattern.

A method to provide information about direction through vibrotactile illusions is introduced on this paper. This method uses a grid of vibrotactile actuators around the arm actuated in coordination. The sensation felt on the skin is consistent with the desired direction of motion, so the desired motion can be intuitively understood. We show that the users can recognize the conveyed direction, and implemented a proof of concept of the proposed method to guide users' elbow flexion/extension motion.

Keywords: Reactive and Sensor-Based Planning, Nonholonomic Motion Planning
Abstract: This paper addresses the problem of real-time, non-holonomic motion planning in environments with moving obstacles of unforeseen, arbitrary motion. An approach is introduced to smoothly switch trajectories by generating feasible non-holonomic trajectory segments on the fly as the robot moves in such an environment, extending the real-time adaptive motion planning (RAMP) approach that is used for holonomic motion. It allows efficient on-line simultaneous planning and execution of non-holonomic trajectories and enables a robot to adapt to changes in the environment while taking into account robot motion uncertainty. The effectiveness and efficiency of the method has been verified through real experiments with a mobile robot and several dynamic obstacles of unforeseen motion to the robot.

Keywords: Medical Systems, Healthcare, and Assisted Living, Service Robots, Human-Robot Interaction
Abstract: An important aspect to be taken care of while designing assistive robots for mobility is that they need to operate among humans. Thus understanding human spatial social conventions and incorporating them in the assistive solutions, is important. In this paper, we introduce a semi-autonomous framework for assistive wheelchair navigation in human environments, which is driven by the intention of the wheelchair user. Safe and textit{socially compliant} motion provided by a user intention driven local motion planner is fused with user teleoperation in order to create such a system. Taking into account the fact that the user is the primary controller, our proposed system aims to provide progressive assistance whenever the user is in danger of collision or at risk of disturbance to other humans. We also thus propose generalized formulations for estimating user intentions and for sharing control within the context of wheelchair mobility assistance, that is adaptable in order to be deployed in real world systems. We then evaluate the proposed framework in simulation in order to obtain a quantitative analysis. We also provide experimental evidence using an off-the-shelf robotized wheelchair equipped with a single 2D laser scanner.

Keywords: Motion and Path Planning, Nonholonomic Motion Planning, Sensor-based Planning
Abstract: In this paper, we present a motion planning algorithm for autonomous navigation in highly constrained urban environments. Since common approaches to on-road trajectory planning turned out to be unsuitable for this task, we instead extended an A*-based planner originally designed for navigation in unstructured environments. Two novel node expansion methods were added to obtain smooth and accurate trajectories that consider the structure of the environment. The first one attempts to find a trajectory connecting the current node directly to the goal by solving a boundary value problem using numerical optimization. The second method leverages a simulated pure-pursuit controller to generate edges (i.e. short motion primitives) that guide the vehicle toward or along the global reference path. As a result, the planner is able to produce smooth paths while retaining the explorative power of A* that is needed to deal with challenging situations in urban driving (e.g., reversing in order to pass a vehicle that stopped unexpectedly). Its practical usefulness was demonstrated during extensive tests on an electric vehicle navigating a mock urban environment as well as on our own autonomous vehicle MuCAR-3.

Keywords: Soft-bodied Robots, Compliance and Impedance Control, Biologically-Inspired Robots
Abstract: From the viewpoint of evolution, vertebrates first accomplished locomotion via motion of the spine. Legs evolved later, to enhance mobility, but the spine remains central. Contrary to this, most robots have rigid torsos and rely primarily on movement of the legs for mobility. The force distributing properties of tensegrity structures presents a potential means of developing compliant spines for legged robots, with the goal of driving motion from the robots core. We present an initial exploration of the morphological design of a tensegrity quadruped robot, the first to the authors’ knowledge, which we call MountainGoat, and its impact on controllable locomotion. All parts of the robot, including legs and spine, are compliant. Locomotion is aided by the use of central pattern generators, feedback control via a neural network, and machine learning techniques involving the Monte Carlo method as well as genetic evolution for parameter optimization. Control is demonstrated with three variations of MountainGoat, focusing on actuation of the spine as central to the locomotion process.

Keywords: Motion and Path Planning, Humanoid Robots, Motion Planning for Manipulators
Abstract: The task of whole-body motion planning for humanoid robots is challenging due to its high-DOF nature, stability constraints, and the need for obstacle avoidance and movements that appear human-like. Over the years, various approaches have been adopted to solve this problem such as bounding-box models and jacobian-based techniques. More commonly though, sampling-based algorithms are employed for this task since they perform admirably well in high dimensional spaces. As an alternative, search-based planners offer improvements in terms of optimality and consistency of the solution. However, they are normally considered impractical for motion planning because their performance in high dimensional spaces is generally considered to be poor. In this paper, we present a heuristic search-based motion planning framework for humanoid robots that circumvents the drawbacks traditionally associated with search-based planners while catering to the specific requirements of humanoid motion planning. This is achieved primarily through a combination of informative yet computationally inexpensive heuristics, carefully crafted motion primitives as atomic actions, and a whole body inverse kinematics (IK) solver for achieving desired end effector orientations. The experimental results show the ability of our framework to perform complex motion planning tasks quickly and efficiently.

Keywords: Motion and Trajectory Generation, Motor Skill Learning, Recognition
Abstract: An effective robot trajectory representation should encode all relevant aspects of the desired motion. For kinematic representations, this means that both the spatial course of the trajectory and its speed profile must be specified. The concept of dynamic movement primitives (DMP) provides a kinematic representation that fully specifies these two aspects of motion. They are, however, not separated from each other within the DMP representation. This can be problematic when movements with significant speed variations are compared within movement recognition and skill learning algorithms. In such comparisons it is often important to distinguish between the spatial and temporal aspects of motion. In this paper we propose a new representation based on dynamic movement primitives, where spatial and temporal aspects are well separated. We demonstrate the effectiveness of the proposed representation for statistical learning of robot skills and movement recognition and compare the performance with standard DMPs, where temporal and spatial aspects of motion are intertwined.

Keywords: Nonholonomic Motion Planning, Motion and Path Planning, Intelligent Transportation Systems
Abstract: Autonomous vehicles are actively being developed from ADAS(Advanced Driver Assistance Systems) toward fully autonomous vehicles. Motion planning is one of the most important key technologies for fully autonomous vehicles, especially when they are operated in constrained narrow space such as parking lot. In this environment, the motion planning is challenging because it requires many changes in forward and reverse directions and adjustments of position and orientation. In this paper, an efficient motion planning algorithm is proposed based on Rapidly-exploring Random Trees (RRT) by specifying desired orientation during the tree expansion. A tangential vector space for desired orientation is used to model nonholonomic constraints of a vehicle and geometric constraints of obstacles. The proposed algorithm has been tested on various situations and its results demonstrated much faster performance compared to a nonholonomic RRT algorithm.

Keywords: Motion and Path Planning, Integrated Planning and Control, Service Robots
Abstract: This paper presents a novel Local-Path Planning approach for an independent four-wheel steering (I4WS) mobile base. The approach smoothly drives and rotates the platform while still following a given path and avoiding obstacles. I4WS vehicles are omnidirectional vehicles, similar to shopping carts, if one neglects the time needed to steer the wheels. However, due to the danger of mechanical parts breaking while actuating the wheels, they are commonly controlled in a stop-and-go fashion, where the vehicle stops to turn its wheels perpendicular to a desired instantaneous center of curvature (ICC) before starting to move again. The approach overcomes this limitation and ensures an ICC-based kinematic constraint during continuous motion using a flat-input controller running at 100 Hz. Therefore, the trajectory of the ICC is both predictable and suitable for model predictive control (MPC). The MPC implemented generates optimized collision-free trajectories of up to several meters ahead at 10 Hz, given a set of points along a path and laser contour readings. Furthermore, the planner realized here is able to deal with additional constraints such as the vehicle's view direction to focus on attention points while driving. Experimental results highlight the capabilities of the approach on a simulated robot, using GazeboSim, and demonstrate its applicability for the field of service robotics.

Keywords: Integrated Task and Motion Planning
Abstract: Solutions to robotic manipulation problems can be substantially improved through integrated task and motion planning. Existing approaches typically focus on satisfaction, finding a feasible solution, instead of optimization. We formulate large-scale robotic manipulation problems as multi-level optimization, incorporating task, action, and motion planning. We develop an integrated planning approach for solving this optimization problem and generating a combined motion plan for a robot to optimize a task-level objective. This approach utilizes a combinatorial search algorithm for task planning and incrementally exploits information from lower-level optimization to improve the high-level task plan. Empirical results show that this integrated approach not only significantly outperforms a traditional top-down approach in solution quality, but also avoids infeasible lower-level motion plans.

Keywords: Biologically-Inspired Robots, Legged Robots, Mechanism Design
Abstract: New work in robotics targets the development of controllable agile motions such as leaping. In this work, we examine animal and robotic systems on the metric of jumping agility and find that animals can outperform the most agile robots by a factor of two. These specially adapted animals use a jumping strategy we term power modulation to generate more peak power for jumping than otherwise possible. A novel eight-bar revolute mechanism designed with a new linkage synthesis approach encodes the properties for power modulation as well as constraints which assure rotation-free jumping motion. We fabricate an 85 gram prototype and demonstrate that it can perform a range of jumps while constrained by a linear slide. The prototype can deliver 3.63 times more peak jumping power than the maximum its motor can produce. A simulation matched to the physical parameters of the prototype predicts that the robot can attain an agility exceeding that of the most agile animals if the actuator power is increased to 15W.

Keywords: Reactive and Sensor-Based Planning, Motion and Trajectory Generation, Recognition
Abstract: We present an online trajectory optimization approach that optimizes a trajectory such that object recognition performance is improved. Inspired by prior work, we formulate the optimization as a derivative-free stochastic optimization, allowing us to express the cost function in an arbitrary way. The cost function is defined such that information acquisition of target objects is improved, while simultaneously moving towards the goal point. We show the evaluation of our approach on a quadrotor platform in simulation as well as on a real robot. The results show that by using an online optimization approach recognition accuracy is greatly improved, but more importantly the optimized trajectory reduces the uncertainty of the posterior class distribution greatly. Hence, verifying that the optimized trajectory collects more valuable information.

Keywords: Motion and Path Planning, Motion and Trajectory Generation, Medical Systems, Healthcare, and Assisted Living
Abstract: This work focuses on how to compute trajectories for an autonomous wheelchair based on indoor GIS maps, in particular on IndoorGML maps, which set the standard in this context. Good wheelchair trajectories are safe and comfortable for the user and the people sharing the space with him, turn gently, are high legible, and smooth (at least G² continuos). We derive a navigation graph from a given IndoorGML map. We define and solve an optimization problem to find the desired path: given a succession of cells to traverse, the path corresponds to the best composite B'ezier trajectory for the wheelchair. We discuss a related multi-objective path planning problem. Experimental results and an implementation on real robots show the planner performance.

Keywords: Nonholonomic Motion Planning, Optimal Control, Motion and Trajectory Generation
Abstract: Inertial reorientation of articulated bodies has been an active area of research in the robotics community, as this behavior can help guide dynamic robots to a safe landing with minimal damage. The main objective of this work is emulating the aggressive and large angle correction maneuvers, like somersaults, that are performed by human divers. To this end, a planar three link robot, called DiverBot, is proposed. By considering a gravity-free scenario, a local connection is obtained between joint angles and the body orientation, resulting in a reduction in the system dynamics. An optimal control policy applied on this reduced configuration space yielded diving maneuvers that appear human-like while being dynamically feasible. Numerical results show that the DiverBot can execute one somersault without drift and multiple somersaults with drift.

Keywords: Motion and Trajectory Generation, Motion and Path Planning
Abstract: We present an efficient anytime motion planner for mobile robots that considers both other dynamic obstacles and uncertainty caused by various sensors and low-level controllers. Our planning algorithm, which is an anytime extension of the Rapidly-exploring Random Belief Tree (RRBT), maintains the best possible path throughout the robot execution, and the generated path gets closer to the optimal one as more computation resources are allocated. We propose a branch-andbound method to cull out unpromising areas by considering path lengths and uncertainty. We also propose an uncertaintyaware velocity obstacle as a simple local analysis to avoid dynamic obstacles efficiently by finding a collision-free velocity. We have tested our method with three benchmarks that have non-linear measurement regions or potential collisions with dynamic obstacles. By using the proposed methods, we achieve up to five times faster performance given a fixed path cost.

Keywords: Motion and Path Planning
Abstract: Sampling-based techniques are often employed to solve various complex motion planning problems — the problem of computing a valid path under various robot and/or obstacle constraints. As these methods are random in nature, the probability of their success is directly related to the expansiveness, or openness, of the underlying planning space. However, little is known theoretically in qualifying the conditions under which user (human)-guided approaches improve the efficiency of sampling-based planners.

In this paper, we classify and create simplistic models of common user-guided approaches, and we extend the concept of expansiveness to analyze these models to understand both when and how much user-guidance aids sampling-based planners.

Keywords: Motion and Path Planning
Abstract: Motion planning in stochastic dynamic uncertain environments is critical in several applications such as human interacting robots, autonomous vehicles and assistive robots. In order to address these complex applications, several methods have been developed. The most successful methods often predict future obstacle locations in order identify collision-free paths. Since prediction can be computationally expensive, offline computations are commonly used ,and simplifications such as the inability to consider the dynamics of interacting obstacles or possible stochastic dynamics are often applied. Online methods can be preferable to simulate potential obstacle interactions, but recent methods have been restricted to Gaussian interaction processes and uncertainty.

In this paper we present an online motion planning method, Runtime Stochastic Ensemble Simulation (Runtime SES) planning, an inexpensive method for predicting obstacle motion with generic stochastic dynamics while maintaining a high planning success rate despite the potential presence of obstacle position error. Runtime SES planning evaluates the likelihood of collision for any state-time coordinate around the robot by performing Monte Carlo simulations online. This prediction is used to construct a customized Rapidly Exploring Random Tree (RRT) in order to quickly identify paths that avoid obstacles while moving toward a goal. We demonstrate Runtime SES planning in problems that benefit from online predictions, environments with strongly-interacting obstacles with stochastic dynamics and positional error. Through experiments that explore the impact of various parametrizations, robot dynamics and obstacle interaction models, we show that real-time capable planning with a high success rate is achievable in several complex environments.

Keywords: Motion and Trajectory Generation, Planning, Scheduling and Coordination, Unmanned Aerial Systems
Abstract: This paper presents an online optimization-based approach to compute trajectories to enable substitution of robots in formation-based deployments with durations that exceed the energy capacity of individual systems. The proposed algorithm computes trajectories in a multi-robot context to ensure a collision-free exchange, even where congestion is a concern. The quality of the resulting trajectories is determined by the amount of time spent deviating from the original plan while maintaining collision-free, speed-limited polynomial splines. The algorithm is shown through simulation and experiments to be viable with average deviation time gaps of less than 20 seconds and average computation times of under 3 minutes for the presented scenarios with varying numbers of robots and deployment specifications.

Keywords: Biologically-Inspired Robots, Legged Robots, Motion and Trajectory Generation
Abstract: While numerous gaits in the horizontal regime (e.g. walking or running) have been defined for legged systems on level ground, no dynamically grounded definitions have been developed for dynamic vertical running. Gaits have clear implications to robotic control strategy, efficiency, and stability. However, while several climbing robotic systems have been described as achieving `running', the question of whether distinct dynamic gaits exist and what classifies these gaits has not been rigorously explored. In this paper, by applying definitions developed in the horizontal regime, we show evidence of three distinct gaits as well as discuss the implications of these gaits on the development of dynamic climbing systems.

Keywords: Motion and Path Planning, Task Planning, Cloud Robotics
Abstract: Motions of a robot interacting with its environment can be described by a set of constraints. This paper introduces an approach, called motion template, which can quickly program and compose the constraints for the motion planner to generate the trajectory. Two types of motion templates, grasp and turn, are specifically described to explain the details of the technique. The reusability and shareability properties of the motion template are demonstrated using a variety of the motion planning applications across different robot platforms. A motion template framework is used to implement the motion template with the trajectory optimization.

Keywords: Motion and Trajectory Generation, Flexible Arms, Virtual Reality and Interfaces
Abstract: The paper presents an analysis of human-like reaching movements in manipulation of parallel flexible objects. To predict the trajectory of human hand, a minimum hand jerk model, based on the minimization of integral of squared hand jerk over the movement duration is established. It is shown that within this model the optimal hand trajectory is composed of a fifth order polynomial and two trigonometric terms depending on the natural frequencies of the system and the movement time. A virtual reality-based experimental setup with a haptic simulator is designed, and the prediction by the minimum hand jerk model is verified against experimental data. The theoretical prediction matches the collected data with a reasonable accuracy. The initial experimental results confirm the applicability of the minimum hand jerk model for modeling of human-like reaching movements in dynamic environments.

Keywords: Path Planning for Multiple Mobile Robots or Agents, Motion and Path Planning, Multi-Robot Systems
Abstract: We study formation change for robot groups in known environments. We are given a team of robots partitioned into groups, where robots in the same group are interchangeable with each other. A formation specifies the locations occupied by each group. The objective is to find collision-free paths that move all robots from a given start formation to a given goal formation. Our algorithm TAPF* has the following features: (a) it incorporates kinematic constraints of robots in form of velocity limits; (b) it maintains a user-specified safety distance between robots; (c) it attempts to minimize the makespan; and (d) it runs efficiently for hundreds of robots and dozens of groups even in dense 3D environments with narrow corridors and other occlusions. We demonstrate the efficiency and effectiveness of TAPF* in simulation and on robots.

Keywords: Unmanned Aerial Systems, Mechanism Design, Dynamics
Abstract: The paper focuses on the mechatronic design of a robotic manipulator that is meant to be mounted on an Unmanned Aerial Vehicle (UAV) and to be used in industrial applications, for both aerial inspection by contact and aerial manipulation. The combination of an UAV and the robotic manipulator realizes the aerial manipulator. The robotic manipulator is designed to be versatile so that the aerial manipulator can perform both trajectory tracking in free flight and physical interaction. Moreover, the robotic manipulator can be mounted on commercially available UAVs a modular way without interfering with the existing onboard control architecture. Experimental test are validating the overall mechatronic design.

Keywords: Manipulation Planning and Control, Unmanned Aerial Systems, Mobile Manipulation
Abstract: In this paper, we model an aerial vehicle, specifically a quadrotor, and a load attached to each other by a rigid link. We assume a torque input at the joint between the aerial vehicle and the rigid link is available. After modeling, we decouple the system dynamics in two separate subsystems, one concerning the position of the center of mass, which we control independently from the chosen torque input; and a second subsystem, concerning the attitude of the rigid link, which we control by appropriately designing a torque control law. Differential flatness is used to show that controlling these two separate systems is equivalent to controlling the complete system. We design control laws for the quadrotor thrust, the quadrotor angular velocity and the torque input, and provide convergence proofs that guarantee that the quadrotor follows asymptotically a desired position trajectory while the manipulator follows a desired orientation. Simulation and experimental works are presented which validate the proposed algorithms.

Keywords: Robotics in Agriculture and Forestry, Visual Tracking, Unmanned Aerial Systems
Abstract: We present an aerial robotic platform for remote tree cavity inspection, based on a hexacopter Micro-Aerial vehicle (MAV) equipped with a dexterous manipulator. The goal is to make the inspection process safer and more efficient and facilitate data collection about tree cavities, which are important for the conservation of biodiversity in forest ecosystems. This work focuses on two key enabling technologies, namely a vision- based cavity detection system and strategies for high level control of the MAV and manipulator. The results of both simulation and real-world experiments are discussed at the end of the paper and demonstrate the effectiveness of our approach.

Keywords: Unmanned Aerial Systems, Robot Vision, Visual Navigation
Abstract: With society and industry pushing for robot-assisted systems to automate cumbersome tasks, such as inspection and maintenance, a vast amount of research effort has been dedicated to relevant technologies. Right at the forefront are small Unmanned Aerial Vehicles (UAVs) equipped with onboard cameras, recently demonstrating that vision-based autonomous flights without reliance on GPS are possible, sparking great interest in a plethora of areas. Current solutions, however, still lack in portability and generality struggling to perform outside the controlled laboratory environment, with onboard robotic perception constituting the biggest impediment.

Driven by the need for real-time denser scene estimation, in this work we present a dramatically low-computation approach enabling estimation of the immediate surroundings of a UAV using the inertial and visual cues from a single onboard camera. Instead of following the recent trend towards dense scene reconstruction, we trade detail of reconstruction for efficiency of estimation, albeit without compromising accuracy. We present results against scene ground truth obtained by a millimetre-precise laser scanner which we make publicly available together with our code. The ETHZ CAB Building dataset contains the ground-truth and visual-inertial data captured from both handheld and flying setups.

Keywords: Unmanned Aerial Systems, Dexterous Manipulation, Mechanism Design
Abstract: In this paper, the design, modeling and experimental verification of a large workspace parallel aerial manipulator is presented. The proposed manipulator has 3 Degrees of Freedom (DoFs) and enables physical interaction on the sides, as well as below the aerial robot. The design parameters of the manipulator are chosen such that it achieves large and singularity-free workspace in combination with high dexterity. A textit{Global Conditioning Index (GCI)} is defined and used as a performance index for the manipulator over its complete workspace. Given the manipulator design parameters, a holistic model of the redundant aerial manipulation system is derived capturing the coupled dynamics of the aerial vehicle and the manipulator. Free flight experimental studies are utilized to validate the system, demonstrate its redundant DoFs, and evaluate its performance.

Keywords: Visual Learning, Robot Vision, Autonomous Agents
Abstract: In the context of visual object search and localization, saliency maps provide an efficient way to find object candidates in images. Unlike most approaches, we propose a way to learn saliency maps directly on a robot, by exploring the environment, discovering salient objects using geometric cues, and learning their visual aspects. More importantly, we provide an autonomous exploration strategy able to drive the robot for the task of learning saliency. For that, we describe the Reinforcement Learning-Intelligent Adaptive Curiosity algorithm (RL-IAC), a mechanism based on IAC (Intelligent Adaptive Curiosity) able to guide the robot through areas of the space where learning progress is high, while minimizing the time spent to move in its environment without learning. We demonstrate first that our saliency approach is an efficient tool to generate relevant object boxes proposal in the input image and significantly outperforms the state-of-the-art EdgeBoxes algorithm. Second, we show that RL-IAC can drastically decrease the required time for learning saliency compared to random exploration.

Keywords: Visual Learning, Robot Vision, Recognition
Abstract: This paper addresses the problem of road scene segmentation in conventional RGB images by exploiting recent advances in semantic segmentation via convolutional neural networks (CNNs). Segmentation networks are very large and do not currently run at interactive frame rates. To make this technique applicable to robotics we propose several architecture refinements that provide the best trade-off between segmentation quality and runtime. This is achieved by a new mapping between classes and filters at the expansion side of the network. The network is trained end-to-end and yields precise road/lane predictions at the original input resolution in roughly 50ms. Compared to the state of the art, the network achieves top accuracies on the KITTI dataset for road and lane segmentation while providing a 20x speed-up. We demonstrate that the improved efficiency is not due to the road segmentation task. Also on segmentation datasets with larger scene complexity, the accuracy does not suffer from the large speed-up.

Keywords: Visual Learning, Robot Vision, Visual Tracking
Abstract: Binary descriptors allow fast detection and matching algorithms in computer vision problems. Though binary descriptors can be computed at almost two orders of magnitude faster than traditional gradient-based descriptors, they suffer from poor matching accuracy in challenging conditions. In this paper, we propose three improvements for binary descriptors in their computation and matching that enhance their performance in comparison to traditional binary and non-binary descriptors without compromising their speed. This is achieved by learning some weights and threshold parameters that allow customized matching under some variations such as lighting and viewpoint. Our suggested improvements can be easily applied to any binary descriptor. We demonstrate our approach on the ORB (Oriented FAST and Rotated BRIEF) descriptor and compare its performance with the traditional ORB and SIFT descriptors on a wide variety of datasets. In all instances, our enhancements outperform standard ORB and is comparable to SIFT.

Keywords: Visual Learning, Recognition, Mapping
Abstract: Recent research in appearance-based mapping has introduced a diverse range of techniques to deal with environments under varying conditions. Common to almost all the existing methods is that they expect a supervised or offline training. Furthermore, some approaches match sequences, assuming constant velocity over the routes. This work addresses the challenges of appearance-based mapping in an online setup without making assumptions or acquiring prior knowledge of the environment. For this purpose, we exploit the topology preserving capability of self-organizing neural networks and learn the perceptual representation of environments using GIST features. Due to the fact that real-world environments are complex and large-scale, we let the network grow while accounting for the amount of error in the network due to perceptual differences among the places. Given the current state of the network and a query image at any time instant, it is possible to identify whether a place comes from the visited location by computing maximum a posteriori estimate over the network. The extensive experiments on three standard datasets, St. Lucia (4 videos), KITTI (2 sequences), and the Oxford City Center dataset, demonstrate the strength of our approach for real-time place recognition while concurrently learning the spatial representation of scenes.

Keywords: Visual Learning, Robot Vision, Recognition
Abstract: The development of reliable and robust visual recognition systems is a main challenge towards the deployment of autonomous robotic agents in unconstrained environments. Learning to recognize objects requires image representations that are discriminative to relevant information while being invariant to nuisances, such as scaling, rotations, light and background changes, and so forth. Deep Convolutional Neural Networks can learn such representations from large web-collected image datasets and a natural question is how these systems can be best adapted to the robotics context where little supervision is often available.

In this work, we investigate different training strategies for deep architectures on a new dataset collected in a real-world robotic setting. In particular we show how deep networks can be tuned to improve invariance and discriminability properties and perform object identification tasks with minimal supervision.

Keywords: Force and Tactile Sensing, Recognition, Sensor Fusion
Abstract: Tactile sensing has been used in robotics for object identification, grasping, and material recognition. Most material recognition approaches use vibration signals from a tactile exploration, typically above one second long, to identify the material. This work proposes a tactile multi-modal (vibration and thermal) material identification approach based on recursive Bayesian estimation. Through the frequency response of the vibration induced by the material and thermal features, like an estimate of the thermal power loss of the finger, we show that it is possible to identify materials in less than half a second. Moreover, a comparison between vibration only and multi-modal identification shows that both recognition time and classification errors are reduced by adding thermal information.

Keywords: Tactile Sensing, Force and Tactile Sensing, Sensor Networks
Abstract: In this paper we describe how we realized event-based signaling for large scale artificial robotic skin. We developed a new algorithm for the event generation on multi-modal skin cells. The skin cells have two modes, the conventional data sampling mode and the event mode. A comprehensive performance evaluation and comparison of these two modes is presented. We perform different experiments on our robot TOMM which has two UR5 robot arms, each covered with 260 multi-modal skin cells. Each skin cell samples 9 signals of 4 different modalities. Finally we derive models for extrapolating CPU usage and network traffic for larger numbers of skin cells and higher sample rates. The results show that the event-based system has superior performance and its performance edge increases with larger numbers of skin cells and higher sample rates. Experimental validation on our real robot system shows that in reactive control the event-based system reduces in comparison to the conventional system the packet rate by 48.2% and the CPU usage by 17.79%. We extrapolate the worst case for 5000 cells and show that the event-based system can at least reduce the packet rate by 21.2% and the CPU usage by 17.46%.

Keywords: Tactile Sensing, Force and Tactile Sensing, Robot Learning
Abstract: Accurate object shape knowledge provides important information for performing stable grasping and dexterous manipulation. When modeling an object using tactile sensors, touching the object surface at a fixed grid of points can be sample inefficient. In this paper, we present an active touch strategy to efficiently reduce the surface geometry uncertainty by leveraging a probabilistic representation of object surface. In particular, we model the object surface using a Gaussian process and use the associated uncertainty information to efficiently determine the next point to explore. We validate the resulting method for tactile object surface modeling using a real robot to reconstruct multiple, complex object surfaces.

Keywords: Tactile Sensing, Recognition, Perception for Grasping and Manipulation
Abstract: We present a new descriptor for tactile 3D object classification. It is invariant to object movement and simple to construct, using only the relative geometry of points on the object surface. We demonstrate successful classification of 185 objects in 10 categories, at sparse to dense surface sampling rate in point cloud simulation, with an accuracy of 77.5% at the sparsest and 90.1% at the densest. In a physics-based simulation, we show that contact clouds resembling the object shape can be obtained by a series of gripper closures using a robotic hand equipped with sparse tactile arrays. Despite sparser sampling of the object's surface, classification still performs well, at 74.7%. On a real robot, we show the ability of the descriptor to discriminate among different object instances, using data collected by a tactile hand.

Keywords: Tactile Sensing, Haptics and Haptic Interfaces, Force and Tactile Sensing
Abstract: All the people have their own individual force ranges to feel the sense of touch and its minimum value is called light touch threshold (in short LTT). For instance, if a touching force is very weak, people do not know whether the object is contacted to their skin, even though it is pushing the skin already. This paper presents a novel apparatus to measure the LTT felt by each subject, which is referred to as Active von Frey (AvF) in this paper. As far as the authors know, the LTT measurement device is for the first time proposed in this paper. It is possible for the AvF to provide the touching force ranges from 1[mgf] to 400[mgf]. In order to provide an accurate touching force for the subject, D’Arsonval movement is chosen as an actuator to rotate a touching AvF pin. Both an electric current applied to the D’Arsonval movement and a rotational angle of AvF pin are utilized to calculate the touching force by the electro-mechanical statics. In addition, the image processing technique is used to measure the rotational angle of AvF pin. Finally we show that the LTTs are very different from individual to individual through the experimental results of 10 participants. We hope that the personalized parameters is used to design or control haptic or tele-operation devices for various applications.

Keywords: Biologically-Inspired Robots, Biomimetics, Mechanism Design
Abstract: Many kinds of mobile robots have been developed through biomimetic research. In this research, we mimicked a basilisk lizard’s ability to run on water for the maneuverability of a hexapedal robot, especially steering locomotion on the water. The robot has a circular plate as a tail, which the robot rotates to steer on water. We dynamically modeled the platform and conducted simulations and experiments on steering locomotion with a bang-bang controller. The robot can steer on water by rotating the tail, and the controlled steering locomotion is stable. The dynamic modelling approximates the robot’s steering locomotion and the trends of the simulation and experiment are similar, although there are errors between the desired and actual angles. The robot’s maneuverability on water can be improved through further research.

Keywords: Biomimetics, Biologically-Inspired Robots, Localization
Abstract: Artificial sensory system is promising to navigate in cluttered and turbid underwater environment by achieving similar functions of biological electrosense of weakly electric fish. In this paper we designed an electrosensory membrane that can operate in a full 3-dimensional mode. Algorithms on the object localization were also designed and tested based on numerical methods of electric field forward simulation. We combined the statistic learning method by training a multi-layer neural network and a probabilistic approach by applying a constrained unscented Kalman filter (CUKF). This exploits the merits of fast estimation and precise signal marching process. Experimental results showed that the detection and localization with the reported sensor were quick and accurate, with errors of around 10 mm using one-step neural network mapping and about 5 mm in close-range using CUKF. This work demonstrated the effectiveness of proposed electrosensory membrane and algorithms.

Keywords: Biologically-Inspired Robots, Marine Robotics, Soft-bodied Robots
Abstract: Dielectric elastomer actuators (DEAs), a soft actuator technology, hold great promise for biomimetic underwater robots. The high-voltages required to drive DEAs can however make them challenging to use in water. This paper demonstrates a method to create DEA-based biomimetic swimming robots that operate reliably even in conductive liquids. We ensure the insulation of the high-voltage DEA electrodes without degrading actuation performance by laminating silicone layers. A fish and a jellyfish were fabricated and tested in water. The fish robot has a length of 120 mm and a mass of 3.8 g. The jellyfish robot has a 61 mm diameter for a mass of 2.6 g. The measured swimming speeds for a periodic 3 kV drive voltage were ~8 mm/s for the fish robot, and ~1.5 mm/s for the jellyfish robot.

Keywords: Biologically-Inspired Robots, Micro/Nano Robots, Sensor Fusion
Abstract: Despite an intensive research on flapping flight and flapping wing MAVs in recent years, there are still no accurate models of flapping flight dynamics. This is partly due to lack of free flight data, in particular during manoeuvres. In this work, we present, for the first time, a comparison of free flight forces estimated using solely an on-board IMU with wind tunnel measurements. The IMU based estimation brings higher sampling rates and even lower variation among individual wingbeats, compared to what has been achieved with an external motion tracking system in the past. A good match was found in comparison to wind tunnel measurements; the slight differences observed are attributed to clamping effects. Further insight was gained from the on-board rpm sensor, which showed motor speed variation of +/- 15% due to load variation over a wingbeat cycle. The IMU based force estimation represents an attractive solution for future studies of flapping wing MAVs as, unlike wind tunnel measurements, it allows force estimation at high temporal resolutions also during manoeuvres.

Keywords: Biologically-Inspired Robots, Mechanism Design, Field Robots
Abstract: A water beetle is skillful at drag-powered swimming by using its oar-like legs. Inspired by this mechanism, a miniature robot, whose mobility is obtained by a pair of legs attached with swimming hairs, was studied in this work. The robot in this paper has optimally designed linkage structures to maximize the sweep angle, which is actuated by a single DC motor with series of gears and a spring. A simplified swimming hair model was proposed to calculate deflection by drag force applied, and COMSOL simulations were used to verify our model. Also, each of swimming hair was optimized individually by considering the attached locations on the legs using the two fitness functions, and three different configurations were selected. In the experiments, the performance of the proposed robots was verified using a high-speed camera, and motion capture cameras. The robot with the proposed configuration showed the fastest movement comparing with other robots.

Keywords: Unmanned Aerial Systems, Field Robots
Abstract: We present a long-range visual signal detection system that is suitable for an unmanned aerial vehicle to find an optical signal released at a desired landing site for the purposes of cargo delivery or rescue situations where radio signals or other communication systems are not available or the wind conditions at the landing site need to be signaled. The challenge here is to have a signal and detection system that works from long range (> 1000m) amongst ground clutter during various seasonal conditions on passive imagery. We use a smoke-grenade as a ground signal, which has the advantageous properties of being easy to carry by ground crews because of its light weight and small size, but when released has a long visual signaling range. We employ a camera system on the UAV with a visual texture feature extraction approach in a machine learning framework to classify image patches as 'signal' or 'background'. We study conventional approaches and develop a visual feature descriptor that can better differentiate the appearance of the visual signal under varying conditions and, when used to train a random-forest classifier, outperforms commonly used feature descriptors. The system was rigorously and quantitatively evaluated on data collected from a camera mounted on a helicopter and flown towards a plume of signal smoke over a variety of seasons, ground conditions, weather conditions, and environments. Our system was capable of detecting the smoke cloud with both precision and recall rates greater than 0.95 from ranges between 1000m and 1500m. Further, we develop a method to estimate wind orientation and approximate wind strength by assessing the shape of the smoke signal. We present a preliminary evaluation of the wind estimation in conditions with different wind intensities and orientations relative to the approach direction.

Keywords: Unmanned Aerial Systems, Sensor Fusion, Visual Navigation
Abstract: We address the important problem of achieving robust and easy-to-deploy visual state estimation for micro aerial vehicles (MAVs) operating in complex environments. We use a sensor suite consisting of multiple cameras and an IMU to maximize perceptual awareness of the surroundings and provide sufficient redundancy against sensor failures. Our approach starts with an online initialization procedure that simultaneously estimates the transformation between each camera and the IMU, as well as the initial velocity and attitude of the platform, without any prior knowledge about the mechanical configuration of the sensor suite. Based on the initial calibrations, a tightly-coupled, optimization-based, generalized multi-camera-inertial fusion method runs onboard the MAV with online camera-IMU calibration refinement and identification of sensor failures. Our approach dynamically configures the system into monocular, stereo, or other multi-camera visual-inertial settings, with their respective perceptual advantages, based on the availability of visual measurements. We show that even under random camera failures, our method can be used for feedback control of the MAVs. We highlight our approach in challenging indoor-outdoor navigation tasks with large variations in vehicle height and speed, scene depth, and illumination.

Keywords: Collision Detection and Avoidance, Unmanned Aerial Systems
Abstract: Despite the recent flight control regulations, Unmanned Aerial Vehicles (UAVs) are still gaining popularity in civilian and military applications, as much as for personal use. Such emerging interest is pushing the development of effective collision avoidance systems. Such systems play a critical role UAVs’ operations especially in a crowded airspace setting. Because of cost and weight limitations associated with UAVs’ payload, camera based technologies are the de-facto choice for collision avoidance navigation systems. This requires multi-target detection and tracking algorithms from a video, which can be run on board efficiently. While there has been a great deal of research on object detection and tracking from a stationary camera, few have attempted to detect and track small UAVs from a moving camera.

In this paper, we present a new approach to detect and track UAVs from a single camera mounted on a different UAV. Initially, we estimate background motions via a perspective transformation model and then identify distinctive points in the background subtracted image. We find spatio-temporal traits of each moving object through optical flow matching and then classify those candidate targets based on their motion patterns compared with the background. The performance is boosted through Kalman filter tracking. This results in temporal consistency among the candidate detections. The algorithm was validated on video datasets taken from a UAV. Results show that our algorithm can effectively detect and track small UAVs with limited computing resources.

Keywords: Unmanned Aerial Systems, Service Robots, Visual Navigation
Abstract: In the last decade, multi-rotor Micro Aerial Vehicles (MAVs) have attracted great attention from robotics researchers. Offering affordable agility and maneuverability, multi-rotor aircrafts have become the most commonly used platforms for robotics applications. Amongst the most promising applications are inspection of power-lines, cell-towers, large and constrained infrastructures and precision agriculture. While GPS offers an easy solution for outdoor autonomy, using on-board sensors is the only solution for autonomy in constrained indoor environments. In this paper, we present our results on autonomous inspection of completely dark, featureless, symmetric dam penstocks using cameras and range sensors. We use a hex-rotor platform equipped with an IMU, four cameras and two lidars. One of the cameras tracks features on the walls using the on-board illumination to estimate the position along the tunnel axis unobservable to range sensors while all of the cameras are used for panoramic image construction. The two lidars estimate the remaining degrees of freedom (DOF). Outputs of the two estimators are fused using an Unscented Kalman Filter (UKF). A moderately trained operator defines waypoints using the Remote Control (RC). We demonstrate our results from Carters Dam, GA and Glen Canyon Dam, AZ which include panoramic images for crack and rusty spot detection and 6-DOF estimation results with ground truth comparisons. To our knowledge ours is the only study that can autonomously inspect environments with no geometric cues and poor to no external illumination using MAVs.

Keywords: Unmanned Aerial Systems, Search and Rescue Robots, Image Processing
Abstract: This paper presents a real-time image restoration method for aerial inspection images that are degraded by the appearance of a passive rotating shell of an unmanned aerial vehicle (UAV).

Only images are required as the input. The method mainly consists of feature-based object detection for joints of the UAV shell, outlier rejection based on a sample consensus (SAC) approach and the geometrical model of the shell, reconsuruction of the entire region that the shell appears in, and inpainting of the shell.

Not only a pure algorithm for general UAVs with passive rotating shells but also an implementation specialized for a UAV that we have been developing are presented. The successfully implemented algorithm was evaluated for actual inspection flights under different conditions. The average rate of successful image restorations was 81.1% at 4.3 fps.

Keywords: Task Planning, Domestic Robots and Home Automation, Integrated Planning and Control
Abstract: We consider an online planning algorithm for partially observable Markov decision processes (POMDPs), called Anytime Error Minimization Search 2 (AEMS2). Despite the considerable success it has enjoyed in robotics and other problems, no quantitative analysis exists of the relationship between its near-optimality and the computation invested. Exploiting ideas from fully-observable MDP planning, we provide here such an analysis, in which the relationship is modulated via a measure of problem complexity called near-optimality exponent. We illustrate the exponent for some interesting POMDP structures, and examine the role of the informative heuristics used by AEMS2 in the guarantees. In the second part of the paper, we introduce a domestic assistance problem in which a robot monitors partially observable switches and turns them off if needed. AEMS2 successfully solves this task in real experiments, and also works better than several state of the art planners in simulation comparisons.

Keywords: Planning, Scheduling and Coordination, Multi-Robot Systems, Intelligent Transportation Systems
Abstract: In this paper, we address the problem of time-optimal coordination of mobile robots under kinodynamic constraints along specified paths. We propose a novel approach based on time discretization that leads to a mixed-integer linear programming (MILP) formulation. This problem can be solved using general-purpose MILP solvers in a reasonable time, resulting in a resolution-optimal solution. Moreover, unlike previous work found in the literature, our formulation allows an exact linear modeling (up to the discretization resolution) of second-order dynamic constraints. Extensive simulations are performed to demonstrate the effectiveness of our approach.

Keywords: Planning, Scheduling and Coordination, Unmanned Aerial Systems, Autonomous Agents
Abstract: In this paper we consider an online planning problem for unmanned aerial vehicle (UAV) operations. Specifically, a UAV has the task of reaching a goal from a set of possible goals while minimizing the amount of energy required. Due to unforeseen disturbances, it is possible that initially attractive goals might end up being very expensive during the execution. Thus, two main problems are investigated here: i) how to predict and plan the motion of the UAV at run time to minimize its energy consumption and ii) when to schedule next replanning time to avoid unnecessary periodic re-evaluation executions. Our approach considers a nonlinear model of the system for which a model predictive controller is used to determine the desired control inputs for each possible goal. These control inputs are then used to estimate the energy required to reach the different goals. Finally, a self-triggered scheduling policy determines how long to wait before replanning the goal to aim for. The proposed framework is validated through simulations and experiments in which a quadrotor must choose and reach some goal while being subject to external disturbances.

Keywords: Planning, Scheduling and Coordination, Task Planning, Unmanned Aerial Systems
Abstract: This article introduces a framework to compute decision making policies (mission planning) for a UAV-based delivery system serving impatient customers. Customers arrive on a finite number of locations L separated by arbitrary but fixed distances and eventually leave if not served. Policies seek to minimize the average net cost (maximize the average net revenue), i.e. loss from customers’ abandonment deprived of the revenue from successful services. We introduce a novel model for the stochastic and dynamic pickup and delivery problem based on semi-Markov decision processes, and show the dependence of the optimal average net cost on the minimum distance between locations δ. Furthermore, we propose a feature-based state aggregation method to overcome the curse of dimensionality due to the exact modeling. The selection of relevant features is based on their correlation with the optimal performance. We show that the distance to the nearest pickup location dominates on other features for almost all δ. Based on this observation, we introduce the policy nearest neighbor or none, a policy with computational complexity O(L³) in many cases of interest. We show that this policy performs considerably better that the nearest neighbor (greedy) policy, and reaches the optimum for some δ.

Keywords: Integrated Task and Motion Planning, Mobile Manipulation, Motion and Trajectory Generation
Abstract: We consider the problem of refining an abstract task plan into a motion trajectory. Task and motion planning is a hard problem that is essential to long-horizon mobile manipulation problems. Many approaches divide the problem into two steps: a search for a task plan and task plan refinement to find a feasible trajectory. We apply sequential quadratic programming to jointly optimize over the parameters in a task plan (e.g., trajectories, grasps, put down locations). We provide two modifications that make our formulation more suitable to task and motion planning. We show how to use movement primitives to reuse previous solutions (and so save optimization effort) without trapping the algorithm in a poor basin of attraction. We also derive an early convergence criterion that lets us quickly detect unsatisfiable constraints so we can re-initialize their variables. We present experiments in a navigation amongst movable objects domain show substantial improvement in cost over a backtracking refinement algorithm.

Keywords: Human-Robot Interaction, Humanoid Interaction, Human Detection and Tracking
Abstract: Robots are at the position to become our everyday companions in the near future. Still, many hurdles need to be cleared to achieve this goal. One of them is the fact that robots are still not able to perceive some important communication cues naturally used by humans, e.g. gaze. In the recent past, eye gaze in robot perception was substituted by its proxy, head orientation. Such an approach is still adopted in many applications today. In this paper we introduce performance improvements to an eye tracking system we previously developed and use it to explore if this approximation is appropriate. More precisely, we compare the impact of the use of eye- or head-based gaze estimation in a human robot interaction experiment with the iCub robot and naïve subjects. We find that the possibility to exploit the richer information carried by eye gaze has a significant impact on the interaction. As a result, our eye tracking system allows for a more efficient human-robot collaboration than a comparable head tracking approach, according to both quantitative measures and subjective evaluation by the human participants.

Keywords: Gesture, Posture, Social Spaces and Facial Expressions, Human-Robot Interaction, Medical Systems, Healthcare, and Assisted Living
Abstract: Social activity based on body motion is a key feature for non-verbal and physical behavior defined as function for communicative signal and social interaction between individuals. Social activity recognition is important to study human-human communication and also human-robot interaction. Based on that, this research has threefold goals: (1) recognition of social behavior (e.g. human-human interaction) using a probabilistic approach that merges spatio-temporal features from individual bodies and social features from the relationship between two individuals; (2) learn priors based on physical proximity between individuals during an interaction using proxemics theory to feed a probabilistic ensemble of activity classifiers; and (3) provide a public dataset with RGB-D data of social daily activities including risk situations useful to test approaches for assisted living, since this type of dataset is still missing. Results show that using the proposed approach designed to merge features with different semantics and proximity priors improves the classification performance in terms of precision, recall and accuracy when compared with other approaches that employ alternative strategies.

Keywords: Collision Detection and Avoidance, Humanoid Robots
Abstract: In order for robots to interact safely and intelligently with their environment they must be able to reliably estimate and localize external contacts. This paper introduces CPF, the Contact Particle Filter, which is a general algorithm for detecting and localizing external contacts on rigid body robots without the need for external sensing. CPF finds external contact points that best explain the observed external joint torque, and returns sensible estimates even when the external torque measurement is corrupted with noise. We demonstrate the capability of the CPF to track multiple external contacts on a simulated Atlas robot, and compare our work to existing approaches.

Keywords: Human-Robot Interaction, Cooperating Robots, Telerobotics
Abstract: Non-humanoid robots are becoming increasingly utilized for collaborative tasks that rely on each collaborator’s ability to effectively convey their mental state while accurately estimating and interpreting their partner’s knowledge, intent, and actions. During these tasks, it may be beneficial or even necessary for the human collaborator to assist the robot. Consequently, we explore the use of nonverbal signals to request help during a collaborative task. We focus on light and sound as they are commonly used communication channels across many domains. This paper analyzes the effectiveness of three nonverbal help signals that vary in urgency. Our results show that these signals significantly influence the human collaborator’s and their perception of the collaboration.

Keywords: Human-Robot Interaction, Multi-Robot Systems, Robot Vision
Abstract: This paper presents a computer vision based system for interaction between multiple humans and multiple robots. Each human can ``select'' (obtain the undivided attention of) a robot by simply looking directly at it. This extends previous work whereby a single human can select one or more robots from a population. Each robot optimally assigns human identities to tracked faces in its camera view using a local Hungarian algorithm. The gaze-direction and location of the faces are estimated via vision, and a score for each robot-face pair is assigned. Then the system finds the global optimal allocation of robot-to-human selections using a centralized Hungarian algorithm. A useful feature of this method is that robots can be selected by people they cannot see. This is the first demonstration of optimal many-to-many robot-selection HRI.

Keywords: Multi-Robot Systems, Sensor Fusion, Visual Servoing
Abstract: In this paper, we propose a novel decentralized active perception strategy that maximizes the convergence rate in estimating the (unmeasurable) formation scale in the context of bearing-based formation localization for robots evolving in R³ x S¹. The proposed algorithm does not assume presence of a global reference frame and only requires bearing-rigidity of the formation (for the localization problem to admit a unique solution), and presence of (at least) one pair of robots in mutual visibility. Two different scenarios are considered in which the active scale estimation problem is treated either as a primary task or as a secondary objective with respect to the constraint of attaining a desired bearing formation. The theoretical results are validated by realistic simulations.

Keywords: Multi-Robot Systems, Unmanned Aerial Systems, Haptics and Haptic Interfaces
Abstract: In this paper, we propose a bilateral tele-operation scheme for cooperative aerial manipulation in which a human operator drives a team of Vertical Take-Off and Landing (VTOL) aerial vehicles, that grasped an object beforehand, and receives a force feedback depending on the states of the system. For application scenarios in which dexterous manipulation by each robot is not necessary, we propose using a rigid tool attached to the vehicle through a passive spherical joint, equipped with a simple adhesive mechanism at the tool-tip that can stick to the grasped object. Having more than two robots, we use the extra degrees of freedom to find the optimal force allocation in term of minimum power and forces smoothness. The human operator commands a desired trajectory for the robot team through a haptic interface to a pose controller, and the output of the pose controller along with system constraints, e.g., VTOL limited forces and contact maintenance, defines the feasible set of forces. Then, an on-line optimization allocates forces by minimizing a cost function of forces and their variation. Finally, propeller thrusts are computed by a dedicated attitude and thrust controller in a decentralized fashion. Human/Hardware in the loop simulation study shows efficiency of the proposed scheme, and the importance of haptic feedback to achieve a better performance.

Keywords: Multi-Robot Systems, Visual Servoing, Unmanned Aerial Systems
Abstract: This paper considers the problem of controlling a formation of quadrotor UAVs equipped with onboard cameras and able to measure relative bearings in their local body frames w.r.t. neighboring UAVs. The control goal is twofold: (i) steering the agent group towards a formation defined in terms of desired bearings, and (ii) actuating the group motions in the `null-space' of the current bearing formation. The proposed control strategy strongly relies on an extension of the rigidity theory to the case of directed bearing frameworks in R³ x S¹. This extension allows to devise a decentralized bearing controller which, unlike most of the present literature, does not need presence of a common reference frame or of reciprocal bearing measurements for the agents. Simulation and experimental results are then presented for illustrating and validating the approach.

Keywords: Multi-Robot Systems, Visual Learning, Recognition
Abstract: This paper considers the merging of appearance-based spatial knowledge among robots having compatible visual sensing. Each robot is assumed to retain its knowledge in its individual long-term spatial memory where i) the place knowledge and their spatial relations are retained in an organized manner in place and map memories respectively; and ii) a ‘place’ refers to a spatial region as designated by a collection of associated appearances. In the proposed approach, each robot communicates with another robot, receives its memory and then merges the received knowledge with its own. The novelty of the merging process is that it is done in two stages: merging of place knowledge followed by the merging of map knowledge. As each robot’s place memory is processed as a whole or in portions, the merging process scales easily with respect to the amount and overlap of the appearance data. Furthermore, the merging can be done in decentralized manner. Our experimental results with a team of three robots demonstrate that the resulting merged knowledge enables the robots to reason about learned places.

Keywords: Multi-Robot Systems, Path Planning for Multiple Mobile Robots or Agents, Planning, Scheduling and Coordination
Abstract: One of the standing challenges in multi-robot systems is the ability to reliably coordinate motions of multiple robots in environments where the robots are subject to disturbances. We consider disturbances that force the robot to temporarily stop and delay its advancement along its planned trajectory which can be used to model, e.g., passing-by humans for whom the robots have to yield. Although reactive collision-avoidance methods are often used in this context, they may lead to deadlocks between robots. We design a multi-robot control strategy for executing coordinated trajectories computed by a multi-robot trajectory planner and give a proof that the strategy is safe and deadlock-free even when robots are subject to delaying disturbances. Our simulations show that the proposed strategy scales significantly better with the intensity of disturbances than the naive liveness-preserving approach. The empirical results further confirm that the proposed approach is more reliable and also more efficient than state-of-the-art reactive techniques.

Keywords: Micro/Nano Robots
Abstract: Magnetotactic bacteria have the potential to controllably reach deep-seated regions of the body via vessels and achieve targeted drug delivery. In this application, motion of the magnetotactic bacteria is influenced by the strength of the external magnetic field. Here, we investigate the swimming characteristics of magnetotactic bacteria (Magnetospirillum gryphiswaldense strain MSR-1) under the influence of uniform and adaptive magnetic fields inside microfluidic chip with depth of 5 µm. This depth enables tracking of single bacterium and comparison of uniform and adaptive magnetic field on the positioning accuracy. We find that under the influence of magnetic field reversal with approximately twice the field strength, the diameter of the U-turn trajectories taken by the magnetotactic bacteria is decreased by 63%. In addition, the adaptive magnetic field decreases the size of region-of convergence of the controlled bacteria within the vicinity of the reference position by 65.5%, compared to control using uniform magnetic field. The comparisons between motion control using uniform and adaptive magnetic fields are done on the same culture of magnetotactic bacteria and using the same cell in each motion control trial.

Keywords: Mechanism Design, Manufacturing and Automation, Micro/Nano Robots
Abstract: In this paper, we propose manipulation ability extension of the optical tweezers by developing microhands, which are to use as end-effectors of the laser beam. First, three different 3D micro-scale handles are designed, then manufactured by the two-photon polymerization method with nano-scale resolution of 100 nm. Second, printed microhands are manipulated by multi-spot laser beam which traps and manipulates numerous objects simultaneously. Third, where direct trapping of the target object is not possible due to target objects’ features such as size, shape, material, index of refraction, etc., indirect manipulation of the target microobjects is achieved by using the microhands as an extension of optical tweezers. Finally, three different microhand designs are compared in terms of speed and success rate. Furthermore, suitability of different shapes of microhandles against common usage of spherical shape is discussed.

Keywords: Micro/Nano Robots
Abstract: Mobile microrobots are typically fabricated in a multi-step microfabrication process and then transported into an enclosed workspace for operation. This paper presents a new, 3D printing inspired method for in-situ fabrication of mobile magnetic microrobots with complex topology from a polymer filament on demand directly inside an enclosed operational environment. Through the use of a tip magnet on the filament, the target shape is formed by magnetic guidance from external electromagnetic coils which wirelessly project fields into the workspace as the filament is fed through a hot needle which is inserted into the workspace. A bending model and a shape planner are developed for predicting and controlling the fabrication process. Magnetically-active millimeter-scale robotic devices of different shapes and sizes are fabricated using polylactic acid (PLA) filament with diameter as small as 50 micrometers. As a demonstration of the in-situ formation of a functional microrobotic device, a force-sensing microrobot with integrated sensing spring is fabricated inside an enclosed space, and then is used to measure the manipulation force during a pushing experiment by optical deformation measurement. We thus show the utility of the fabrication method for creating complex microrobot shapes remotely in enclosed environments for advanced microrobotic applications, with the potential for scaled down applications in healthcare and microfluidics.

Keywords: Micro/Nano Robots, Motion Control, Visual Servoing
Abstract: In this paper, a closed-loop strategy for controlling the transient regime of Marangoni flows and thus to selectively manipulate and separate micro-particles is reported. Marangoni flows were generated by an infrared laser as a heat source. Simulations of the transient regime of Marangoni flows were performed with COMSOL Multiphysics to analyze the temperature field and velocity profile. The convection cell dynamic growth was controlled with the laser beam exposure time. Then, selected particles, glass beads ranging from 150 up to 212 microns, were dragged by small convection flows without reaching undesired particles. The closed-loop control improved the manipulation speed and precision in comparison to manual manipulation.

Keywords: Micro/Nano Robots, Biological Applications of Micro Robots, Smart Actuators
Abstract: A surprising phenomenon is observed in a microfluidic channel where suspending microbeads are spontaneously aligned into lines. Microbeads are randomly distributed in the channel at a relatively high velocity, but start to align into lines at a relatively low velocity. The alignment has been repeated with and without obstacles in the channel. The phenomenon is interpreted as an unintended acoustic wave being around the experimental environment, and the wave resulted in a standing wave which moves the microbeads towards the nodes of the standing wave. By using a frequency analyzer, it is found that a pulse-width-modulation controller in the system generated high-frequency signals, which is the most possible wave source for the alignment. The experimental results are presented, and characteristics of acoustic wave are analyzed. The phenomenon could contribute to microfluidic applications for achieving acoustic alignment without complex fabrication of interdigital transducers.

Keywords: Humanoid and Bipedal Locomotion, Motion and Trajectory Generation, Humanoid Robots
Abstract: For a biped robot to keep walking, basic walking patterns are needed. One method to generate these patterns is to use an extrapolated center of mass (XcoM) concept and constant zero-moment point (ZMP) patterns. However, center of mass (CoM) patterns generated from such a method have discontinuous ZMP and CoM accelerations. For solving this problem, we propose a method in which continuous ZMP patterns can be simply generated from constant ones. In addition, experimental results of forward walking are presented. The results demonstrate that this pattern generation method can give a starting point for walking robots.

Keywords: Humanoid and Bipedal Locomotion, Legged Robots, Dynamics
Abstract: This paper presents a simple 2D passive dynamic walking model with straight legs based on a novel hip joint, called T-joint. The model directly solves the common foot-scuffing problem in straight-legged walkers without introducing any new degree of freedom or additional motion phase, which is unavoidable in kneed robots. The mathematic model of the new walker is a 4D hybrid dynamical system, which can be reduced to a 3D Poincare map. A stable period-1 walking gait is found by searching the map with a proposed algorithm. Numerical studies show that the model performs well for disturbances on initial conditions and also on a large slope. The stable walking gait is verified by a virtual prototype in ADAMS. This study has been successfully used to make an efficient walking robot called Xingzhe, which has set a new world record for the longest distance after 134km non-stoping walking.

Keywords: Humanoid and Bipedal Locomotion, Humanoid Robots
Abstract: In this paper, we address the problem of planning optimal zero moment point (ZMP) trajectories for the double support phase in bipedal gaits that alternate between single and double support. This is achieved by allowing pre- and post-actuation during the single support phases. Thus, we solve two coupled problems: exact tracking of a given desired ZMP trajectory in the pre- and post-phases (single support), and determination of the desired ZMP during the transition phase (double support). Both are solved while minimizing the overall control energy. We also provide a formal method to assess how the choice of desired ZMP trajectory during the single support phases impacts the overall energy expended during the footstep cycle. Although the obtained solution may not be physically feasible in general, it represents a benchmark to which alternative feasible solutions may be compared. Our approach generalizes previous results that consider only constant output in the pre- and post-phases, e.g., allowing pre- and post-phase output from a family of polynomial splines. We evaluate the approach via simulations.

Keywords: Humanoid and Bipedal Locomotion, Legged Robots, Dynamics
Abstract: In this paper, we propose a novel planar biped model which achieves asymptotically stable walking, based on ankle actuation and the virtual pendulum concept. Simple models for walking have provided useful insights in understanding fundamental principles of human locomotion as well as developing controllers for biped robots. Existing simplest walking models include a point mass with either rigid legs or compliant legs. Both models are able to describe different gaits such as walking and running, but are not able to address posture stabilization, which is another important issue in bipedalism. Recently, the virtual pendulum (VP) concept was proposed as an intuitive posture stabilization strategy and successfully demonstrated on the compliant leg scheme. However, the model does not address inherent peripheral mechanics of walking, i.e., ankle actuation and collisional energy loss from foot-ground interaction. The model proposed in this paper consists of a rigid trunk, rigid legs and ankle actuation. The energy dissipation due to collision and compensation by ankle actuation play an essential role for asymptotically stable walking, while the trunk posture is stabilized based on the VP concept. Nonlinear dynamic simulation verifies that the model constructs a stable limit cycle with a small angular oscillation of trunk.

Keywords: Haptics and Haptic Interfaces, Virtual Reality and Interfaces, Surgical Robotics
Abstract: Epidural anaesthesia is a medical gesture commonly performed by an anaesthesiologist. However, it remains one of the most difficult gestures to master for medical students. Given the lack of sufficiently realistic training devices available for future physicians, we propose a new haptic simulator which reproduces the haptic sensations felt by anaesthesiologists when performing this kind of operation. The originality of this simulator is the coupling of a Geomagic Touch device with a pneumatic cylinder to reproduce the "Loss of Resistance" phenomenon which helps the physician to control the needle depth. In this paper, we introduce the parametric 3D model of the region of interest and the control laws used jointly. Even though this device could not reproduce the right level of forces required in this type of anaesthesia, an anaesthesiologist involved in the project gave positive feedback about its haptic tissue rendering.

Keywords: Micro/Nano Robots, Force and Tactile Sensing
Abstract: Robotic handling inside the SEM is well known, but mostly a teleoperated task, but automation of in-situ handling would be highly advantageous and could enable serious production and assembly in the SEM. However, reliable techniques who facilitate an automation are still lacking. Especially the positioning along the optical axis of the SEM remains as a major challenge. In this paper, an universal approach addressing this particular issue is proposed, allowing to assist and improve handling and assembly task. The approach exploits typical charging effects in the SEM and derives information about the positions of objects and end-effectors as well as about assembly statuses. Assembly accuracies better than 100 nm are achieved and controlled approaching speeds of 10 μm/s.

Keywords: Micro/Nano Robots, Force Control, Medical Robots and Systems
Abstract: Control of the exerted force on objects as well as the position of the actuator during a cell characterization operation has an essential role in achieving an efficient performance. In such operations, any undesired applied force could degrade the efficiency or make damages to the object. A challenging problem associated with application to cell characterization using a piezoelectric actuator is to design a suitable force-position switching control law and also to deal with nonlinear hysteretic behavior of such actuators. The main concern in this operation is to provide a smooth switching transition between position and force control modes. A modified PrandtlIshlinskii (PI) operator and its inverse are utilized for both identification and real time compensation of the hysteresis nonlinear behavior. By proposing an appropriate force-position switching control approach, the smooth transition between these two modes is guaranteed. The stability of each controller as well as the switching system is demonstrated analytically. Experimental results depict that the proposed approach achieves precise force-position control within each modes and also the switching remains smooth.

Keywords: Micro/Nano Robots, Manufacturing and Automation
Abstract: This paper proposed a phase-closed-loop nanomachining control method to realize the directly control of machining depth based on ultrasonic vibration-assisted AFM. By using applied force to control the machining depth, conventional AFM machining approaches unable to machining a nanostructure with specified machined depth. With the proposed method, the vibration phase of micro-cantilever has a specific relationship with machining depth. Therefore, the nano-grooves with desired depth can be machined by using phase value as feedback of PID control. In this paper, the theoretical analysis and simulation are carried out, and the experiments of phase-closed-loop control method are conducted. The experimental results verify the primary feasibility of the proposed method. The present method also demonstrates the potential on the fabrication of three-dimension nanostructures and nanoelectronic device.

Keywords: Robot Learning, Robot Vision, Neurorobotics
Abstract: Autonomous robots and humans need to create a coherent 3D representation of their peripersonal space in order to interact with nearby objects. Recent studies in visual neuroscience suggest that the small coordinated head/eye movements that humans continually perform during fixation provides useful depth information. In this work, we mimic such a behavior on a humanoid robot and propose a computational model that extracts depth information without requiring the kinematic model of the robot. First, we show that, during fixational head/eye movements, proprioceptive cues and optic flow lie on a low dimensional subspace that is a function of the depth of the target. Then, we use the generative adaptive subspace self-organizing map (GASSOM) to learn these depth-dependent subspaces. The depth of the target is eventually decoded using a winner-take-all strategy. The proposed model is validated on a simulated model of the iCub robot.

Keywords: Micro/Nano Robots, Biological Applications of Micro Robots, Medical Robots and Systems
Abstract: Cellular mechanical properties as the main physical performance characteristics have been actively studied in the past years for the study of cytobiology and the development of medicine. In this study, by combining Hertz model, a novel strategy is proposed to simultaneously measure the cellular mechanical properties including cellular mass, elasticity and viscidity, based on the principle of forced vibration stimulated by simple harmonic force, with piezoelectric transducer (PZT) as vibrator and Atomic Force Microscope (AFM) as detector. The corresponding theoretical model was derived and the simulation was realized based on the proposed model. The experiments of indentations and vibrations with myoblasts and myotubes were implemented to calculate the three mechanical parameters of cells according to the proposed strategy. The results validated the proposed approach. This work would be useful for the development of cytology, medicine, previously diagnose, specific therapy and so on.

Keywords: Micro/Nano Robots, Robot Vision, Visual Tracking
Abstract: Visual tracking and estimation of the 3D pose of a micro/nano-object is a key issue in the development of automated manipulation tasks using a visual feedback. The 3D pose of the micro object is estimated based on a template matching algorithm. Nevertheless, a key challenge for visual tracking in a scanning electron microscope (SEM) is the difficulty to observe the motion along the depth direction. In this paper, we propose a template-based hybrid visual tracking scheme that uses luminance information to estimate the object displacement on x-y plane and uses defocus information to estimate object depth. This approach is experimentally validated on 4 DoF-motion of a sample in a SEM.

Keywords: Soft-bodied Robots, Flexible Arms
Abstract: A closed elastica, which is an elastic rod whose both ends are fixed to prescribed two places, can be found in various situations in robotics.　In this paper, we investigate validity of the discretized spatial model of a closed elastica. The discretized closed elastica model can be regarded as a series of rigid bodies connected with passive elastic joints physically, and represented by a set of nonlinear equations derived by discretizing differential equations with boundary conditions as well as Euler equations expressing local minimum elastic energy. The validity of the considered model was studied by comparing the measured shape and the calculated shape from the model. Experimental results using a laser position sensor for shape measurement show that good correspondence between measured and calculated shapes, which supports the validity of the considered model. The constrained force and torque at the tip of an elastica, which can be obtained simultaneously associated with shape calculation, are also verified from measurement by a 6-axes force/torque sensor.

Keywords: Soft-bodied Robots, Haptics and Haptic Interfaces
Abstract: Haptic rendering of soft bodies is essential in medical simulations of procedures such as surgery or palpation. The most commonly used approach is to recreate the sense of touch using a specific design and control of a robotic arm. In this paper, we propose a new approach, based on soft-robotics technology. We create a tangible deformable device that allows users to "touch" soft tissues and perceive mechanical material properties, in a realistic manner. The device is able to dynamically provide user touch with different stiffness perceptions, thanks to actuators placed at the boundaries. We introduce a control algorithm, based on inverse Finite Element Analysis, which controls the actuators in order to recreate a desired stiffness that corresponds to the contact with soft tissues in the virtual environment. The approach uses antagonistic actuation principle to create a wide range of stiffness. We validate our algorithm and demonstrate the method using prototypes based on simple mechanisms.

Keywords: Recognition, Visual Tracking, Surgical Robotics
Abstract: There are many applications where domain-specific sensing, such as accelerometers, kinematics, or force sensing, provide unique and important information for control or for analysis of motion. However, it is not always the case that these sensors can be deployed or accessed beyond laboratory environments. For example, it is possible to instrument humans or robots to measure motion in the laboratory in ways that it is not possible to replicate in the wild. An alternative, which we explore in this paper, is to address situations where accurate sensing is available while training an algorithm, but for which only video is available for deployment. We present two examples of this sensory substitution methodology. The first variation trains a convolutional neural network to regress a real-valued signal -- robot end-effector pose -- from video. The second example regresses binary signals detecting when specific objects are in motion. We evaluate these on the JIGSAWS dataset for robotic surgery training assessment and the 50 Salads dataset for modeling complex structured cooking tasks. We evaluate the trained models for video-based action recognition and show that the trained models provide information that is comparable to the sensory signals they replace.

Keywords: Robot Vision, Perception for Grasping and Manipulation
Abstract: One of the central tasks for a household robot is searching for specific objects. It does not only require localizing the target object but also identifying promising search locations in the scene if the target is not immediately visible. As computation time and hardware resources are usually limited in robotics, it is desirable to avoid expensive visual processing steps that are exhaustively applied over the entire image. The human visual system can quickly select those image locations that have to be processed in detail for a given task. This allows us to cope with huge amounts of information and to efficiently deploy the limited capacities of our visual system. In this paper, we therefore propose to use human fixation data to train a top-down saliency model that predicts relevant image locations when searching for specific objects. We show that the learned model can successfully prune bounding box proposals without rejecting the ground truth object locations. In this aspect, the proposed model outperforms a model that is trained only on the ground truth segmentations of the target object instead of fixation data.

Keywords: Haptics and Haptic Interfaces, Medical Robots and Systems, Compliance and Impedance Control
Abstract: This paper introduces a new controller for dual-user training systems, designed by way of an energy based approach. Dual-user training systems are useful for supervised hands-on training when a trainer shows the right gestures to a trainee and where the forces to apply on the tools are difficult to dose. An energy shared control (ESC) based architecture is proposed, based on an intrinsically passive authority sharing mechanism which is enhanced to provide a full force feedback to both users. As this enhancement may violate the natural passivity of the system, a passivity controller is introduced. A task based comparative study with two other dual-user schemes (Complementary Linear Combination (CLC) and Masters Correspondence with Environment Transfer (MECT) from [1] is conducted, which reveals analogous performances. Real-time experiments demonstrate good tracking performances.

Keywords: Soft-bodied Robots, Hydraulic/Pneumatic Actuators, Smart Actuators
Abstract: Bellows-like actuators are popular in soft robotic systems. Sensing the movement of these actuators with traditional sensors is challenging. This work proposes and tests a sensing system based on the changing mutual inductance of wire coils on bellows. A method for modeling the changes in mutual inductance between coils of tightly-packed wires is introduced. Changes in mutual inductance are measured through the self-inductance of a circuit made up of the coils in series. The self-inductance of the circuit measures the bellows bend-angle. The experiments show an approximately quadratic relationship between the bend angle and the measured inductance. From a bend angle of 121 degrees to -67 degrees the inductance of the circuit increases by 19%. The bias-inducing effects of shear strain, torsional strain, non-uniform bending, and nearby metal are also explored.

Keywords: Multi-Robot Systems, Distributed Robot Systems, Communication-aware Sensor and Motion Planning
Abstract: Multi-robot information gathering teams typically require communication for data fusion and cooperative decision making. However, when communication takes place over wireless networks, stringent bandwidth limits apply. These limits raise the need for efficient utilisation of available communication resources in a manner that balances information gathering utility with communication costs or limits. In our previous work, we introduced the dynamic information flow (DIF) problem as a general formulation of this trade-off. We introduced two variants of the problem addressing the issue of communication efficiency for data fusion only. In this paper, we extend one of the variants to address communication efficiency for both data fusion and cooperative decision making in a synergistic manner. We present a solution to this new variant that integrates a multi-cast routing algorithm with information structure optimisation. This solution allows teams that involve high-data-rate sensors and tight coordination to respect bandwidth limits. Through several simulations we verify that our solution significantly reduces bandwidth usage of such teams while maintaining information gathering performance.

Keywords: Micro/Nano Robots, Biological Applications of Micro Robots
Abstract: We propose a fluidic monostable multivibrator (fMMV) circuit, which is a new flow-switching mechanism in microfluidic chips. Flow control using parallel fMMV circuits does not require an external controller; this is a distinct advantage over various conventional micro-flow control methods. The fMMV circuit mainly consists of an inverter and an RC circuit. Each of the parts uses a microchannel as a resistor, a membrane chamber as a capacitor and a membrane valve as a transistor. The inverter acts as a trigger which opens the membrane valve and the RC circuit delays the closing of the valve, so that flow switching can occur over a time interval. A serial fMMV circuit can switch flows in multiple channels and regulate flowrates with pre-defined durations. By modifying the resistances in fMMV circuits, the flow rates and duration of each phase could be adjusted. For a proof of concept demonstration, we performed experiments using 3 parallel fMMV and 4 parallel fMMV circuit systems and these experiments show the applicability of fMMV circuit to various biochemical processes.

Keywords: Robot Vision, Recognition, Visual Tracking
Abstract: In this paper, we propose a keyframe-based online object learning and detection method. To manage appearance changes of target objects, the proposed method incrementally updates an object database using detection results. One of the major problems in updating the appearance model is that the object model can gradually be degraded by accumulated errors and biased to specific views. To solve this problem, our object model is updated according to the selected keyframes, which not only help memorize important views of target objects, but also prevent the holistic appearance model from overfitting. The database is represented as a graph of the registered images, and the importance of the database images is measured by analyzing the constructed graph. Then, the redundant or less important images are discarded from the database. As a result, the database is efficiently maintained while new views of the objects are gradually added. The experimental results demonstrate that the proposed algorithm efficiently maintains the object database and improves the detection performance compared to previous incremental object learning and detection algorithms.

Keywords: Micro/Nano Robots
Abstract: Targeted drug delivery by using magnetic nanoparticles (MNPs) is an efficient technique to deliver drug molecules towards specific tissues in a human body. The MNPs’ 1D guidance system is a combined electromagnetic actuation (EMA) and monitoring system, which can provide an accurate control scheme with nanoparticle’s localization for more precise targeting of the drug delivery. The localization of the MNPs is done by magnetic particle imaging (MPI) with low amplitude excitation field. In this paper, we have developed a novel coil topology for 1D MNPs feedback control, by alternate supply of different currents to coils set in time sequence, the coil set alternates functions between MPI and EMA. Motion of MNPs is controlled by a gradient of the magnetic field in EMA period, the distribution of MNPs is reconstructed in MPI period and provides feedback to the EMA. The guidance system will provide navigation and tracking interleaved for targeted drug delivery of MNPs in compact and efficient ways. The 1D MNPs guidance system has 2Hz of EMA and MPI hybrid frequency, allowing a position control of MNPs with 90nm diameter.