Keywords: Virtual Reality and Interfaces, Human-Centered Robotics, Learning from Demonstration

Keywords: Medical Robots and Systems, Mechanism Design

Keywords: Motion and Path Planning, Optimization and Optimal Control

Keywords: Representation Learning, Force and Tactile Sensing, In-Hand Manipulation

Keywords: Aerial Systems: Perception and Autonomy, Visual-Based Navigation, Representation Learning

Keywords: Visual Learning, Robot Safety, Deep Learning for Visual Perception

Keywords: Product Design, Development and Prototyping, Mechanism Design, Software, Middleware and Programming Environments

Keywords: Deep Learning for Visual Perception, Representation Learning, Deep Learning in Grasping and Manipulation

Keywords: Representation Learning, Probability and Statistical Methods, Cognitive Control Architectures

Keywords: Biologically-Inspired Robots, Multi-legged Robots, Sensorimotor Learning

Keywords: Human-Centered Robotics, Virtual Reality and Interfaces

Keywords: Cognitive Control Architectures, AI-Based Methods, Learning from Demonstration

Keywords: Behavior-Based Systems, Task Planning

Keywords: Service Robots, Behavior-Based Systems, Software, Middleware and Programming Environments

Keywords: Visual Tracking, Mapping, SLAM

Keywords: Legged Robots, Multi-legged Robots, Optimization and Optimal Control

Keywords: Assembly, Compliant Assembly

Keywords: Cellular and Modular Robots, Mechanism Design

Keywords: Product Design, Development and Prototyping, Mechanism Design, Software, Middleware and Programming Environments

Keywords: Soft Robot Materials and Design, Soft Robot Applications

Keywords: Motion and Path Planning, Optimization and Optimal Control, Modeling, Control, and Learning for Soft Robots

Keywords: Entertainment Robotics, Dexterous Manipulation, Model Learning for Control

Keywords: Marine Robotics, Field Robots, Robotics in Hazardous Fields

Keywords: Simulation and Animation

Keywords: Robot Safety, Collision Avoidance, Perception-Action Coupling

Keywords: Motion and Path Planning, Space Robotics and Automation, Field Robots

Keywords: Field Robots, Autonomous Vehicle Navigation, Robotics in Hazardous Fields

Keywords: Autonomous Vehicle Navigation, Motion Control, Kinematics

Keywords: Robotics in Agriculture and Forestry, Agricultural Automation

Keywords: Agricultural Automation, Novel Deep Learning Methods, Computer Vision for Automation

Keywords: Robotics in Agriculture and Forestry, Localization, Probability and Statistical Methods

Keywords: Soft Sensors and Actuators, Biomimetics, Agricultural Automation

Keywords: Big Data in Robotics and Automation, Localization, Multi-Modal Perception

Keywords: Energy and Environment-Aware Automation, Field Robots, Marine Robotics

Keywords: Aerial Systems: Applications

Keywords: Legged Robots, Field Robots, Multi-legged Robots

Keywords: Aerial Systems: Applications, Mechanism Design, Cellular and Modular Robots
Abstract: Energy sources such as batteries do not decrease in mass after consumption, unlike combustion-based fuels. We present the concept of staging energy sources, i.e. consuming energy in stages and ejecting used stages, to progressively reduce the mass of aerial vehicles in-flight which reduces power consumption, and consequently increases flight time. A flight time vs. energy storage mass analysis is presented to show the endurance benefit of staging to multirotors. We consider two specific problems in discrete staging -- optimal order of staging given a certain number of energy sources, and optimal partitioning of a given energy storage mass budget into a given number of stages. We then derive results for a continuously staged case of an internal combustion engine driving propellers. Notably, we show that a multirotor powered by internal combustion has an upper limit on achievable flight time independent of the available fuel mass. Lastly, we validate the analysis with flight experiments on a custom two-stage battery-powered quadcopter. This quadcopter can eject a battery stage after consumption in-flight using a custom-designed mechanism, and continue hovering using the next stage. The experimental flight times match well with those predicted from the analysis for our vehicle. We achieve a 19% increase in flight time using the batteries in two stages as compared to a single stage.

Keywords: Aerial Systems: Applications, Surveillance Systems, Planning, Scheduling and Coordination
Abstract: We study a range-constrained variant of the multi-UAV target search problem where commercially available UAVs are used for target search in tandem with ground-based mobile recharging vehicles (MRVs) that can travel, via the road network, to meet up with and recharge a UAV. We propose a pipeline for representing the problem on real-world road networks, starting with a map of the road network and yielding a final routing graph that permits UAVs to recharge via rendezvous with MRVs. The problem is then solved using mixed-integer linear programming (MILP) and constraint programming (CP). We conduct a comprehensive simulation of our methods using real-world road network data from Scotland. The assessment investigates accumulated search reward compared to ideal and worst-case scenarios and briefly explores the impact of UAV speeds. Our empirical results indicate that CP is able to provide better solutions than MILP, overall, and that the use of a fleet of MRVs can improve the accumulated reward of the UAV fleet, supporting their inclusion for surveillance tasks.

Keywords: Aerial Systems: Applications, Novel Deep Learning Methods, Hybrid Logical/Dynamical Planning and Verification
Abstract: Autonomous systems operating in uncertain environments under the effects of disturbances and noises can reach unsafe states even while using fine-tuned controllers and precise sensors and actuators. To provide safety guarantees on such systems during motion planning operations, reachability analysis (RA) has been demonstrated to be a powerful tool. RA however suffers from computational complexity especially when dealing with complex systems characterized by high order dynamics, making it hard to be deployed for runtime monitoring. To deal with this issue, in this work, a neural network (NN)-based framework is proposed to perform fast online monitoring for safety and an approach for verification of NNs is presented. Training is performed offline using precise RA tools while the trained NN is used online as a fast safety checker for motion planning. In this way, at runtime, a planned trajectory can be quickly predicted to be safe or unsafe: when unsafe, a replanning procedure is triggered until a safe trajectory is obtained. The results of the trained network are tested for verification using our recent tool Verisig in which the NN is transformed into a hybrid system in order to provide guarantees before deployment. In case of unverified NN, the outputs of the verification are used to retrain the network until verification is achieved. Two illustrative case studies on a quadrotor aerial vehicle - a pick-up drop-off operation and a navigation in a cluttered environment - are presented to validate the proposed framework both in simulations and experiments.

Keywords: Aerial Systems: Applications, Aerial Systems: Mechanics and Control, Aerial Systems: Perception and Autonomy
Abstract: Anomaly detection is a key goal of autonomous surveillance systems that should be able to alert unusual observations. In this paper, we propose a holistic anomaly detection system using deep neural networks for surveillance of critical infrastructures (e.g., airports, harbors, warehouses) using an unmanned aerial vehicle (UAV). First, we present a heuristic method for the explicit representation of spatial layouts of objects in bird-view images. Then, we propose a deep neural network architecture for unsupervised anomaly detection (UAV-AdNet), which is trained on environment representations and GPS labels of bird-view images jointly. Unlike studies in the literature, we combine GPS and image data to predict abnormal observations. We evaluate our model against several baselines on our aerial surveillance dataset and show that it performs better in scene reconstruction and several anomaly detection tasks. The codes, trained models, dataset, and video will be available at https://bozcani.github.io/uavadnet.

Keywords: Aerial Systems: Applications, Intelligent Transportation Systems, Field Robots
Abstract: Delivery drones used by logistics companies today are equipped with unshielded propellers, which represent a major hurdle for in-hand parcel delivery. The exposed propeller blades are hazardous to unsuspecting bystanders, pets, and untrained users. One solution to provide safety is to enclose a drone with an all-encompassing protective cage. However, the structures of existing cage designs have low density in order to minimize obstruction of propeller airflow, so as to not decrease efficiency. The relatively large openings in the cage do not protect hands and fingers from fast rotating propellers. Here we describe a novel approach to safety and aerodynamic efficiency by means of a high-density cage and morphing arms loosely inspired by the box turtle. The drone cage is made of a dense and lightweight grid. When flying in proximity of humans, the arms and propellers are retracted and fully sealed within the cage, thus making the drone safe and also reducing the total footprint. When flying at cruising altitude far from people and objects, the arms and propellers extend out of the protective grid, thus increasing aerodynamic efficiency by more than 20%.

Keywords: Aerial Systems: Applications
Abstract: ROSflight is a lean, open-source autopilot system developed with the primary goal of supporting the needs of researchers working with micro aerial vehicle systems. The project consists of firmware designed to run on low-cost, readily available flight controller boards, as well as ROS packages for interfacing between the flight controller and application code and for simulation. The core objectives of the project are as follows: maintain a small, easy-to-understand code base; provide high-bandwidth, low-latency communication between the flight controller and application code; provide a straightforward interface to research application code; allow for robust safety pilot integration; and enable true software-in-the-loop simulation capability.

Keywords: Aerial Systems: Applications
Abstract: Nonlinear Model Predictive Control (NMPC) is a powerful and widely used technique for nonlinear dynamic process control under constraints. In NMPC, the state and control weights of the corresponding state and control costs are commonly selected based on human-expert knowledge, which usually reflects the acceptable stability in practice. Although broadly used, this approach might not be optimal for the execution of a trajectory with the lowest positional error and sufficiently "smooth" changes in the predicted controls. Furthermore, NMPC with an online weight update strategy for fast, agile, and precise unmanned aerial vehicle navigation, has not been studied extensively. To this end, we propose a novel control problem formulation that allows online updates of the state and control weights. As a solution, we present an algorithm that consists of two alternating stages: (i) state and command variable prediction and (ii) weights update. We present a numerical evaluation with a comparison and analysis of different trade-offs for the problem of quadrotor navigation. Our computer simulation results show improvements of up to 70% in the accuracy of the executed trajectory compared to the standard solution of NMPC with fixed weights.

Keywords: Software, Middleware and Programming Environments, Simulation and Animation, Aerial Systems: Applications
Abstract: Unmanned Aerial Vehicles (UAVs) are becoming increasingly popular in the film and entertainment industries, in part because of their maneuverability and perspectives they enable. While there exists methods for controlling the position and orientation of the drones for visibility, other artistic elements of the filming process, such as focal blur, remain unexplored in the robotics community. The lack of cinematographic robotics solutions is partly due to the cost associated with the cameras and devices used in the filming industry, but also because stateof-the-art photo-realistic robotics simulators only utilize a full in-focus pinhole camera model which does not incorporate these desired artistic attributes. To overcome this, the main contribution of this work is to endow the well-known drone simulator, AirSim, with a cinematic camera as well as extend its API to control all of its parameters in real time, including various filming lenses and common cinematographic properties. In this paper, we detail the implementation of our AirSim modification, CinemAirSim, present examples that illustrate the potential of the new tool, and highlight the new research opportunities that the use of cinematic cameras can bring to research in robotics and control.

Keywords: Aerial Systems: Applications
Abstract: With a growing number of applications in the world for UAVs, there is a clear limitation regarding the need for extended battery life. With the current flight times, many users would benefit greatly with an innovative option of field charging these devices. The objective of this project is to investigate feasibility of inductively harvesting energy from a power line cable for applications such as charging a UAV drone. Research investigates a dual hook perching device that securely attaches to a power cable and aligns an inductive core with the cable for harvesting energy from its electro-magnetic field. Modeling and analysis of the core highlights critical design parameters, leading to evaluation of circular, semi-cylindrical, and u-shaped prototypes designed to interface with a 1” power cable. Underactuated two jaw manipulators at each end of the coil are proposed for grasping the cable and aligning it with the charging coil, ultimately providing a firm grasp and perch. An open source hexacopter drone was used in this study to integrate with the charging novelty. The results provided can be used as a starting point to study the reliability of this method of charging and to further investigate perching abilities of UAVs.

Keywords: Aerial Systems: Applications, Energy and Environment-Aware Automation, Motion and Path Planning
Abstract: Communication is a key aspect in modern life. Unfortunately, when natural disasters occur, the communication system and infrastructure of a city can be partially lost, and in the worst case, completely destroyed. In this case, communication is a crucial part for the search-and-rescue missions. This paper focuses on developing an aerial communication relay platform as an effective solution for communication loss in a natural disaster. The model used considers the aircraft altitude and attitude, which affects the energy acquisition and consumption, and the signal fading effects. The flight path planning is performed adopting a nonlinear optimization technique, Hermite-Simpson collocation method. For a realistic communication model regarding urban signal loss and path propagation, the building deployment of a 2km radius circular area of two cities in South Korea (Seoul and Jeonju) was obtained. Simulation experiments for the different urban environments are performed to test the communication reliability focusing on the relation between the Unmanned Aerial Vehicle (UAV) and the Ground Users (GU). As a result of the simulation, an optimal flight path in a high-rise urban and urban microcell environment is obtained. The flight path indicates the feasibility of endurance flights for low-altitude communication aid aircrafts including signal fading model alongside solar power energy acquisition into the case study.

Keywords: Aerial Systems: Mechanics and Control, Aerial Systems: Applications, Cellular and Modular Robots
Abstract: We introduce SplitFlyer--a novel quadcopter with an ability to disassemble into two self-contained bicopters through human assistance. As a subunit, the bicopter is a severely underactuated aerial vehicle equipped with only two propellers. Still, each bicopter is capable of independent flight. To achieve this, we provide an analysis of the system dynamics by relaxing the control over the yaw rotation, allowing the bicopter to maintain its large spinning rate in flight. Taking into account the gyroscopic motion, the dynamics are described and a cascaded control strategy is developed. We constructed a transformable prototype to demonstrate consecutive flights in both configurations. The results verify the proposed control strategy and show the potential of the platform for future research in modular aerial swarm robotics.

Keywords: Aerial Systems: Mechanics and Control, Aerial Systems: Applications, Cooperating Robots
Abstract: This paper addresses the problem of cooperative transport of a point mass hoisted by two aerial robots. Treating the robots as a leader and a follower, the follower stabilizes the system with respect to the leader using only feedback from its Inertial Measurement Units (IMU). This is accomplished by neglecting the acceleration of the leader, analyzing the system through the generalized coordinates or the cables' angles, and employing an observation model based on the IMU measurements. A lightweight estimator based on an Extended Kalman Filter (EKF) and a controller are derived to stabilize the robot-payload-robot system. The proposed methods are verified with extensive flight experiments, first with a single robot and then with two robots. The results show that the follower is capable of realizing the desired quasi-static trajectory using only its IMU measurements. The outcomes demonstrate promising progress towards the goal of autonomous cooperative transport of a suspended payload via small flying robots with minimal sensing and computational requirements.

Keywords: Aerial Systems: Mechanics and Control, Marine Robotics
Abstract: To extend the mission duration of smaller unmanned aerial vehicles, this paper presents a solar recharge approach that uses lakes as landing, charging, and standby areas. The Sherbrooke University Water-Air VEhicle (SUWAVE) is a small aircraft capable of vertical takeoff and landing on water. A second-generation prototype has been developed with new capabilities: solar recharging, autonomous flight, and a larger takeoff envelope using an actuated takeoff strategy. A 3D dynamic model of the new takeoff maneuver is conceived to understand the major forces present during this critical phase. Numerical simulations are validated with experimental results from real takeoffs made in the laboratory and on lakes. The final prototype is shown to have accomplished repeated cycles of autonomous takeoff, followed by assisted flight and landing, without any human physical intervention between cycles.

Keywords: Aerial Systems: Applications, Cooperating Robots, Telerobotics and Teleoperation
Abstract: The landing of a fixed-wing UAV on top of a mobile landing platform requires a cooperative control strategy, which is based on relative motion estimates. These estimates typically suffer from communication or processing time delays, which can render an otherwise stable control system unstable. Such effects must therefore be considered during the design process of the cooperative landing controller. In this letter the application of a model-free passivity-based stabilizing controller is proposed, which is based on the monitoring of energy flows in the system, and actively dissipating any given active energy by means of adaptive damping elements. In doing so, overall system passivity and consequently stability is enforced in a straightforward and easy to implement way. The proposed control system is validated in numerical simulations for round trip delays of up to 4 seconds.

Keywords: Aerial Systems: Applications, Aerial Systems: Mechanics and Control
Abstract: Currently, drone research and development has received significant attention worldwide. Particularly, delivery services employ drones as it is a viable method to improve delivery efficiency by using a several unmanned drones. Research has been conducted to realize complete automation of drone control for such services. However, regarding the takeoff and landing port of the drones, conventional methods have focused on the landing operation of a single drone, and the continuous landing of multiple drones has not been realized. To address this issue, we propose a completely novel port system, “EAGLES Port,” that allows several drones to continuously land and takeoff in a short time. Experiments verified that the landing time efficiency of the proposed port is ideally 7.5 times higher than that of conventional vertical landing systems. Moreover, the system can tolerate 270 mm of horizontal positional error, ±30 degrees of angular error in the drone’s approach (±40 degrees with the proposed gate mechanism), and up to 1.9 m/s of drone’s approach speed. This technology significantly contributes to the scalability of drone usage. Therefore, it is critical for the development of a future drone port for the landing of automated drone swarms.

Keywords: Aerial Systems: Mechanics and Control, Grippers and Other End-Effectors, Mobile Manipulation
Abstract: To perch on or grasp the objective surface using the suction cup-based manipulator, the precise contact control is commonly required. Improper contact angle or insufficient contact force may cause failure. To enhance the tolerance to flight control insufficiency, a suction cup that comprises an inner soft cup and an outer firm cup to facilitate its engagement without reducing the adhesion stiffness is investigated. The soft cup is adaptable to the angular error induced by the multicopter and the resulting adhesion force can draw the firm cup and correct the angular error between the firm cup and the surface. These effects increase the engagement rate and reduce the dependence on precise control. The outer firm cup is devoted to providing a large adhesion force and a stiff base for subsequent tasks. To reduce the air evacuation time in the firm cup, a novel self-sealing structure is designed. Based on the combined cup, we build a multifunctional aerial manipulation system which can execute perching or lateral aerial grasping tasks. With the proposed prototype, the comparative flight experiments involving perching on a wall under disturbance and grasping an object are conducted. The results demonstrate that our proposed suction cup outperforms the conventional cup.

Keywords: Aerial Systems: Applications, Environment Monitoring and Management, Field Robots
Abstract: A high-quality estimate of wind fields can potentially improve the safety and performance of Unmanned Aerial Vehicles (UAVs) operating in dense urban areas. Computational Fluid Dynamics (CFD) simulations can help provide a wind field estimate, but their accuracy depends on the knowledge of the distribution of the inlet boundary conditions. This paper provides a real-time methodology using a Particle Filter (PF) that utilizes wind measurements from a UAV to solve the inverse problem of predicting the inlet conditions as the UAV traverses the flow field. A Gaussian Process Regression (GPR) approach is used as a surrogate function to maintain the real-time nature of the proposed methodology. Real-world experiments with a UAV at an urban test-site prove the efficacy of the proposed method. The flight test shows that the 95% confidence interval for the difference between the mean estimated inlet conditions and mean ground truth measurements closely bound zero, with the difference in mean angles being between -3.67 degrees and 1.2 degrees and the difference in mean magnitudes being between -0.206 m/s and 0.020 m/s.

Keywords: Multi-Robot Systems, Aerial Systems: Applications, Search and Rescue Robots
Abstract: Mobile robots of various types have been proposed for infrastructure inspection and disaster investigation. For such mobile robot applications, accessing the areas is of primary importance for missions. Therefore, various locomotive mechanisms have been studied. We introduce a novel mobile robot system, named DIR-3, combining a crawler robot and a microdrone. By rotating its arm back and forth, DIR-3, a very simple, lightweight crawler robot with a single 360-degree rotatable U-shaped arm, can climb up/down an 18 cm high step, 1.5 times its height. Furthermore, to inspect high places, which is considered difficult for conventional mobile robots, a drone mooring system for mobile robots is presented. The tethered microdrone of DIR-3 can be controlled freely as a flying camera by switching operating modes on the graphic user interface. The drone mooring system has a unique tension-controlled winding mechanism that enables stable landing on DIR-3 from any location in the air, in addition to measurement and estimation of relative positions of the drone. We evaluated the landing capability, position estimation accuracy, and following control of the drone using the winding mechanism. Results show the feasibility of the proposed system for inspection of cracks in a 5 m high concrete wall.

Keywords: Aerial Systems: Applications
Abstract: In this paper, we propose a new concept of robot which is hybrid, including aerial and crawling subsystems and an arm, and also modular with interchangeable crawling subsystems for different pipe configurations, since it has been designed to cover most industrial oil & gas end-users’ requirements. The robot has the same ability than aerial robots to reach otherwise inaccessible locations, but makes the inspection more efficient, increasing operation time since crawling requires less energy than flying, and achieving better accuracy in the inspection. It also integrates safety-related characteristics for operating in the potentially explosive atmosphere of a refinery, being able to immediately interrupt the inspection if a hazardous situation is detected and carry the sensible parts such as batteries and electronic devices away as soon as possible. The paper presents the design of this platform in detail and shows the feasibility of the whole system performing indoor experiments.

Keywords: Aerial Systems: Applications, Field Robots, Environment Monitoring and Management
Abstract: We present a pipeline for geomorphological analysis that uses structure from motion (SfM) and deep learning on close-range aerial imagery to estimate spatial distributions of rock traits (size, roundness, and orientation) along a tectonic fault scarp. The properties of the rocks on the fault scarp derive from the combination of initial volcanic fracturing and subsequent tectonic and geomorphic fracturing, and our pipeline allows scientists to leverage UAS-based imagery to gain a better understanding of such surface processes. We start by using SfM on aerial imagery to produce georeferenced orthomosaics and digital elevation models (DEM). A human expert then annotates rocks on a set of image tiles sampled from the orthomosaics, and these annotations are used to train a deep neural network to detect and segment individual rocks in the entire site. The extracted semantic information (rock masks) on large volumes of unlabeled, high-resolution SfM products allows subsequent structural analysis and shape descriptors to estimate rock size, roundness, and orientation. We present results of two experiments conducted along a fault scarp in the Volcanic Tablelands near Bishop, California. We conducted the first, proof-of-concept experiment with a DJI Phantom 4 Pro equipped with an RGB camera and inspected if elevation information assisted instance segmentation from RGB channels. Rock-trait histograms along and across the fault scarp were obtained with the neural network inference. In the second experiment, we deployed a hexrotor and a multispectral camera to produce a DEM and five spectral orthomosaics in red, green, blue, red edge, and near infrared. We focused on examining the effectiveness of different combinations of input channels in instance segmentation.

Keywords: Aerial Systems: Applications, Surveillance Systems
Abstract: The limited in-flight battery lifetime of centimeter-scale flying robots is a major barrier to their deployment, especially in applications which take advantage of their ability to reach high vantage points. Perching, where flyers remain fixed in space without use of flight actuators by attachment to a surface, is a potential mechanism to overcome this barrier. Electroadhesion, a phenomenon where an electrostatic force normal to a surface is generated by induced charge, has been shown to be an increasingly viable perching mechanism as robot size decreases due to the increased surface-area-to-volume ratio. Typically electroadhesion requires high (> 1 kV) voltages to generate useful forces, leading to relatively large power supplies that cannot be carried on-board a micro air vehicle. In this paper, we motivate the need for application-specific power electronics solutions for electroadhesive perching, develop a useful figure of merit (the "specific voltage") for comparing and guiding efforts, and walk through the design methodology of a system implementation. We conclude by showing that this high voltage power supply enables, for the first time in the literature, tetherless electroadhesive perching of a commercial micro quadrotor.

Keywords: Aerial Systems: Applications, Robotics in Hazardous Fields, Sensor Networks
Abstract: Unmanned aerial vehicles (UAVs) have been shown to be useful for the installation of wireless sensor networks (WSNs). More notably, the accurate placement of sensor nodes using UAVs, opens opportunities for many industrial and scientific uses, in particular, in hazardous environments or inaccessible locations.

This publication proposes and demonstrates a new aerial sensor placement method based on impulsive launching. Since direct physical interaction is not required, sensor deployment can be achieved in cluttered environments where the target location cannot be safely approached by the UAV, such as under the forest canopy.

The proposed method is based on mechanical energy storage and an ultralight shape memory alloy (SMA) trigger. The developed aerial system weighs a total of 650 grams and can execute up to 17 deployments on a single battery charge. The system deploys sensors of 30 grams up to 4 meters from a target with an accuracy of pm 10 cm. The aerial deployment method is validated through more than 80 successful deployments in indoor and outdoor environments.

The proposed approach can be integrated in field operations and complement other robotic or manual sensor placement procedures. This would bring benefits for demanding industrial applications, scientific field work, smart cities and hazardous environments [Video attachment: https://youtu.be/duPRXCyo6cY].

Keywords: Multi-Robot Systems, Aerial Systems: Perception and Autonomy, Optimization and Optimal Control
Abstract: In the paper, a pair of auto-tuning methods for fixed-parameter controllers is presented, in application to multirotor unmanned aerial vehicles (UAVs) control. In both cases, the automatized process of searching the best altitude controller parameters is carried out with the use of a modified golden-search method, for a selected cost function, during the flight of a pair of UAVs. All the calculations are performed in real-time in the iterative manner using only basic sensory information available concerning current altitude information for a pair of UAVs. The auto-tuning process of the controller is characterized by neglectfully low computational demand, and the parameters are obtained rapidly with no dynamic model of a UAV needed. In both methods, by using a pair of UAVs in tuning process, the level of control performance can be increased, what has been proved by means of multiple outdoor experiments. The first method increases precision of the obtained controller parameters by averaging sensory information over a pair of UAVs, whereas in the second, by exchanging measurement information between the units, the search space is explored faster. The latter is of special importance when seeking the best controller parameters, what is especially expected when a limited experiment duration of multirotor UAVs is taken into account.

Keywords: Optimization and Optimal Control, Motion and Path Planning, Autonomous Vehicle Navigation
Abstract: Trajectory optimization under affine motion models and convex cost functions are often solved through the convex-concave procedure (CCP), wherein the non-convex collision avoidance constraints are replaced with its affine approximation. Although mathematically rigorous, CCP has some critical limitations. First, it requires a collision-free initial guess of solution trajectory which is difficult to obtain, especially in dynamic environments. Second, at each iteration, CCP involves solving a convex constrained optimization problem which becomes prohibitive for real-time computation even with a moderate number of obstacles, if long planning horizons are used.

In this paper, we propose a novel trajectory optimization algorithm which like CCP involves solving convex optimization problems but can work with an arbitrary initial guess. Moreover, our proposed optimizer can be computationally upto a few orders of magnitude faster than CCP while achieving similar or better optimal cost. The reduced computation time, in turn, stems from some interesting mathematical structures in our optimizer which allows for distributed computation and obtaining solutions in symbolic form. We validate the proposed optimizer on several benchmarks with static and dynamic obstacles.

Keywords: Underactuated Robots
Abstract: Landing is an essential part of multicopter task operations. A multicopter has relatively stringent requirements for landing, particularly for achieving flatness. Currently, landing on rough terrain with normal skids is difficult. Therefore, research is being conducted to obtain skids capable of landing on rough terrain. In this paper, a passive skid for multicopter landing on rough terrain is proposed. The proposed device is based on an existing previous study of the multicopter carried with a electric robo-arm only for object manipulation. This innovative idea stems from the aim of giving the multicopter carried with a electric robo-arm the ability to land on various occasions and then the passive skid is designed. By using a slope to simulate a rough terrain, the range of available landing in which a multicopter can maintain its pose and the frictional torque of the passive joint are analyzed. Further, experiments are conducted to demonstrate that landing can be achieved using the skid proposed in our study.

Keywords: Optimization and Optimal Control, Aerial Systems: Mechanics and Control
Abstract: We present a novel paradigm and algorithm for optimal design of underactuated robot platforms in highly-constrained nonconvex parameter spaces. We apply this algorithm to two variants of the mature RoboBee platform, numerically demonstrating predicted performance improvements of over 10% in some cases by algorithmically reasoning about variable effective-mechanical-advantage (EMA) transmissions, higher aspect ratio (AR) wing designs, and force-power tradeoffs. The algorithm can currently be applied to any underactuated mechanical system with one actuated degree of freedom (DOF), and can be easily extended to arbitrary configuration spaces and dynamics.

Keywords: Mechanism Design, Micro/Nano Robots, Aerial Systems: Perception and Autonomy
Abstract: This work presents the design, fabrication, and characterization of an airflow sensor inspired by the whiskers of animals. The body of the whisker was replaced with a fin structure in order to increase the air resistance. The fin was suspended by a micro-fabricated spring system at the bottom. A permanent magnet was attached beneath the spring, and the motion of fin was captured by a readily accessible and low-cost 3D magnetic sensor located below the magnet. The sensor system was modeled in terms of the dimension parameters of fin and the spring stiffness, which were optimized to improve the performance of the sensor. The system response was then characterized using a commercial wind tunnel and the results were used for sensor calibration. The sensor was integrated into a micro aerial vehicle (MAV) and demonstrated the capability of capturing the velocity of the MAV by sensing the relative airflow during flight.

Keywords: Mechanism Design, Aerial Systems: Mechanics and Control, Human-Centered Robotics
Abstract: We present PufferBot, an aerial robot with an expandable structure that may expand to protect a drone's propellers when the robot is close to obstacles or collocated humans. PufferBot is made of a custom 3D printed expandable scissor structure, which utilizes a one degree of freedom actuator with rack and pinion mechanism. We propose four designs for the expandable structure, each with unique characterizations which may be useful in different situations. Finally, we present three motivating scenarios in which PufferBot might be useful beyond existing static propeller guard structures.

Keywords: Aerial Systems: Mechanics and Control, Dynamics
Abstract: Rotary-wing flying machines draw attention within the UAV community for their in-place hovering capability, and recently, holonomic motion over fixed-wings. In this paper, we investigate about the power-optimality in a mono-spinner, i.e., a class of rotary-wing UAVs with one rotor only, whose main body has a streamlined shape for producing additional lift when counter-spinning the rotor. We provide a detailed dynamic model of our mono-spinner. Two configurations are studied: (1) a symmetric configuration, in which the rotor is aligned with the fuselage’s COM, and (2) an asymmetric configuration, in which the rotor is located with an offset from the fuselage’s COM. While the former can generate an in-place hovering flight condition, the latter can achieve trajectory tracking in 3D space by resolving the yaw and precession rates. Furthermore, it is shown that by introducing a tilting angle between the rotor and the fuselage, within the asymmetric design, one can further minimize the power consumption without compromising the overall stability. It is shown that an energy optimal solution can be achieved through the proper aerodynamic design of the mono-spinner for the first time.

Keywords: Aerial Systems: Mechanics and Control, Optimization and Optimal Control, Motion Control
Abstract: Robots are usually programmed for particular tasks with a considerable amount of hand-crafted tuning work. Whenever a new robot with different dynamics is presented, the well-designed control algorithms for the robot usually have to be re-tuned to guarantee good performance. It remains challenging to directly program a robot to automatically learn from the experiences gathered by other dynamically different robots. With such a motivation, this paper proposes a learning algorithm that enables a quadrotor unmanned aerial vehicle (UAV) to automatically improve its tracking performance by learning from the tracking errors made by other UAVs with different dynamics. This learning algorithm utilizes the relationship between the dynamics of different UAVs, named the target and training UAVs, respectively. The learning signal is generated by the learning algorithm and then added to the feedforward loop of the target UAV, which does not affect the closed-loop stability. The learning convergence can be guaranteed by the design of a learning filter. With the proposed learning algorithm, the target UAV can improve its tracking performance by learning from the training UAV without baseline controller modifications. Both numerical studies and experimental tests are conducted to validate the effectiveness of the proposed learning algorithm.

Keywords: Aerial Systems: Mechanics and Control, Robust/Adaptive Control of Robotic Systems, Motion Control
Abstract: The conceptual design and flight controller of a novel kind of quadcopter are presented. This design is capable of morphing the shape of the UAV during flight to achieve position and attitude control. We consider a dynamic center of gravity (CoG) which causes continuous variation in a moment of inertia (MoI) parameters of the UAV. These dynamic structural parameters play a vital role in the stability and control of the system. The length of quadcopter arms is a variable parameter, and it is actuated using attitude feedback-based control law. The MoI parameters are computed in real-time and incorporated in the equations of motion of the system. The UAV utilizes the angular motion of propellers and variable quadcopter arm lengths for position and navigation control. The movement space of the CoG is a design parameter and it is bounded by actuator limitations and stability requirements of the system. A detailed information on equations of motion, flight controller design and possible applications of this system are provided. Further, the proposed shape-changing UAV system is evaluated by comparative numerical simulations for way point navigation mission and complex trajectory tracking.

Keywords: Aerial Systems: Applications, Aerial Systems: Mechanics and Control
Abstract: This paper presents the design and control of a novel quadrotor with a variable geometry to physically interact with cluttered environments and fly through relatively narrow gaps and passageways. This compliant quadrotor with passive morphing capabilities is designed using torsional springs at every arm hinge to allow for rotation with external forces. We derive the dynamic model of this variable geometry quadrotor (SQUEEZE), and develop a low-level adaptive controller for trajectory tracking. The corresponding Lyapunov stability proof of attitude tracking is also presented. Further, an admittance controller is designed to account for change in yaw due to physical interactions with the environment. Finally, the proposed design is validated in real-time flight tests in two setups: a relatively small gap and a passageway. The experimental results demonstrate unique capability of SQUEEZE in navigating through constrained narrow spaces.

Keywords: Aerial Systems: Mechanics and Control, Underactuated Robots, Mechanism Design
Abstract: Exploiting contacts with environment structures provides extra force support to a UAV, often reducing the power consumption and hence extending the mission time. This paper investigates one such way to exploit flat surfaces in the environment by a novel aerial-ground hybrid locomotion. Our design is a single passive wheel integrated at the UAV bottom, serving a minimal design to date. We present the principle and implementation of such a simple design as well as its control. Flight experiments are conducted to verify the feasibility and the power saving caused by the ground locomotion. Results show that our minimal design allows successful aerial-ground hybrid locomotion even with a less-controllable bi-copter UAV. The ground locomotion saves up to 77% battery without much tuning effort.

Keywords: Aerial Systems: Mechanics and Control, Mechanism Design, Aerial Systems: Applications
Abstract: Multi-directional aerial platforms can fly in almost any orientation and direction, often maneuvering in ways their underactuated counterparts cannot match. A subset of multi-directional platforms is fully-actuated multirotors, where all six degrees of freedom are independently controlled without redundancies. Fully-actuated multirotors possess much greater freedom of movement than conventional multirotor drones, allowing them to perform complex sensing and manipulation tasks. While there has been comprehensive research on multi-directional multirotor control systems, the spectrum of hardware designs remains fragmented. This paper sets out the hardware design architecture of a fully-actuated quadrotor and its associated control framework. Following the novel platform design, a prototype was built to validate the control scheme and characterize the flight performance. The resulting quadrotor was shown in operation to be capable of holding a stationary hover at 30 degrees incline, and track position commands by thrust vectoring [Video attachment: url{https://youtu.be/8HOQl_77CVg}].

Keywords: Aerial Systems: Mechanics and Control, Biologically-Inspired Robots
Abstract: This work presents a model-free nonlinear controller for an ornithopter prototype with bioinspired wings and tail. The size and power requirements have been thought to allocate a customized autopilot onboard. To assess the functionality and performance of the full mechatronic design, a controller has been designed and implemented to execute a prescribed perching 2D trajectory. Although functional, its 'handmade' nature forces many imperfections that cause uncertainty that hinder its control. Therefore, the controller is based on adaptive backstepping and does not require any knowledge of the aerodynamics. The controller is able to follow a given reference in flight path angle by actuating only on the tail deflection. A novel space-dependent nonlinear guidance law is also provided to prescribe the perching trajectory. Mechatronics, guidance and control system performance is validated by conducting indoor flight tests.

Keywords: Aerial Systems: Mechanics and Control, Contact Modeling, Flexible Robots
Abstract: Conventional multirotors are unable to land on inclined surfaces without specialized suspensions and adhesion devices. With the development of a bidirectional rotor, landing maneuvers could benefit from rapid thrust reversal, which would increase the landing envelope without involving the addition of heavy and complex landing gears or reduction of payload capacity. This article presents a model designed to accurately simulate quadrotor landings, the behavior of their stiff landing gear, and the limitations of bidirectional rotors. The model was validated using experimental results on both low-friction and high-friction surfaces, and was then used to test multiple landing algorithms over a wide range of touchdown velocities and slope inclinations to explore the benefits of reverse thrust. It is shown that thrust reversal can nearly double the maximum inclination on which a quadrotor can land and can also allow high vertical velocity landings.

Keywords: Aerial Systems: Perception and Autonomy, Autonomous Vehicle Navigation, Neurorobotics
Abstract: Flying insects are capable of vision-based navigation in cluttered environments, reliably avoiding obstacles through fast and agile maneuvers, while being very efficient in the processing of visual stimuli. Meanwhile, autonomous micro air vehicles still lag far behind their biological counterparts, displaying inferior performance at a much higher energy consumption. In light of this, we want to mimic flying insects in terms of their processing capabilities, and consequently show the efficiency of this approach in the real world. This letter does so through evolving spiking neural networks for controlling landings of micro air vehicles using optical flow divergence from a downward-looking camera. We demonstrate that the resulting neuromorphic controllers transfer robustly from a highly abstracted simulation to the real world, performing fast and safe landings while keeping network spike rate minimal. Furthermore, we provide insight into the resources required for successfully solving the problem of divergence-based landing, showing that high-resolution control can be learned with only a single spiking neuron. To the best of our knowledge, this work is the first to integrate spiking neural networks in the control loop of a real-world flying robot. Videos of the experiments can be found at https://bit.ly/neuro-controller.

Keywords: Aerial Systems: Mechanics and Control, Search and Rescue Robots, Robotics in Hazardous Fields
Abstract: Aerial vehicles with collision resilience can operate with more confidence in environments with obstacles that are hard to detect and avoid. This paper presents the methodology used to design a collision resilient aerial vehicle with icosahedron tensegrity structure. A simplified stress analysis of the tensegrity frame under impact forces is performed to guide the selection of its components. In addition, an autonomous controller is presented to reorient the vehicle from an arbitrary orientation on the ground to help it take off. Experiments show that the vehicle can successfully reorient itself after landing upside-down and can survive collisions with speed up to 6.5m/s.

Keywords: Aerial Systems: Mechanics and Control, Robot Safety, Aerial Systems: Applications
Abstract: In this paper, we introduce a new quadrotor fail-safe flight solution that can perform the same four controllable degrees-of-freedom flight as a standard multirotor even when a single thruster fails. The new solution employs a novel multirotor platform known as the T3-Multirotor and utilizes a distinctive strategy of actively controlling the center of gravity position to restore the controllable degrees of freedom. A dedicated control structure is introduced, along with a detailed analysis of the dynamic characteristics of the platform that change during emergency flights. Experimental results are provided to validate the feasibility of the proposed solution.

Keywords: Aerial Systems: Mechanics and Control, Redundant Robots, Cellular and Modular Robots
Abstract: Robotic manipulators using thrusters for weight compensation are an active research topic due to their potential to exceed the limits of maximum length. However, existing manipulators that use thrusters have limitations of maximum length because the hardware design is not sufficiently refined. This paper focuses on overcoming these limitations and realizing an articulated manipulator more than twice the length of conventional ones. To cancel the moment for each link, we performed static analysis considering the torsional deformation around the link axis to derive the thruster position. Weight compensation and joint angle control of the manipulator can be realized with simple proportional integral derivative control for each link by numerical simulation. Consequently, we demonstrated the feasibility of the proposed manipulator by lifting a 0.6 kg payload at the arm end with a prototype of length 6.6 m. Theoretically, each thrust force control input was almost constant, regardless of link attitude. This suggests modular properties that contribute to the practicality of the proposed manipulator for various tasks.

Keywords: Aerial Systems: Applications, Reactive and Sensor-Based Planning
Abstract: This work presents the deployment of UAVs for the exploration of clouds, from the system architecture and simulation tests to a real-flight campaign and trajectory analyzes. Thanks to their small size and low altitude, light UAVs have proven to be adapted for in-situ cloud data collection. The short life time of the clouds and limited endurance of the planes require to focus on the area of maximum interest to gather relevant data. Based on previous work on cloud adaptive sampling, the article focuses on the overall system architecture, the improvements made to the system based on preliminary tests and simulations, and finally the results of a field campaign. The Barbados experimental flight campaign confirmed the capacity of the system to map clouds and to collect relevant data in dynamic environment, and highlighted areas for improvement.

Keywords: Aerial Systems: Perception and Autonomy, Reactive and Sensor-Based Planning, Motion and Path Planning
Abstract: In micro aerial vehicle (MAV) operations, the success of mission is highly dependent on navigation performance, which has raised recent interests on navigation-aware path planning. One of the challenges lies in that optimal motions for successful navigation and a designated mission are often different in unknown, unstructured environments, and only sub-optimality may be obtained in each aspect. We aim to organize a two-MAV team that can effectively execute mission and simultaneously guarantee navigation quality, which consists of a main-agent responsible for mission and a sub-agent for navigation of the team. Especially, this paper focuses on path planning of the sub-agent to provide navigational assistance to the main-agent using a monocular camera. We adopt a graph-based receding horizon planner to find a dynamically feasible path in order for the sub-agent to help the main-agent's navigation. In this process, we present a metric for evaluating the localization performance utilizing the distribution of the features projected to the image plane. We also design a map management strategy and pose-estimation support mechanism in a monocular camera setup, and validate their effectiveness in two scenarios.

Keywords: Aerial Systems: Perception and Autonomy, Motion and Path Planning, Autonomous Agents
Abstract: Coverage path planning (CPP) is the task of designing a trajectory that enables a mobile agent to travel over every point of an area of interest. We propose a new method to control an unmanned aerial vehicle (UAV) carrying a camera on a CPP mission with random start positions and multiple options for landing positions in an environment containing no-fly zones. While numerous approaches have been proposed to solve similar CPP problems, we leverage end-to-end reinforcement learning (RL) to learn a control policy that generalizes over varying power constraints for the UAV. Despite recent improvements in battery technology, the maximum flying range of small UAVs is still a severe constraint, which is exacerbated by variations in the UAV’s power consumption that are hard to predict. By using map-like input channels to feed spatial information through convolutional network layers to the agent, we are able to train a double deep Q-network (DDQN) to make control decisions for the UAV, balancing limited power budget and coverage goal. The proposed method can be applied to a wide variety of environments and harmonizes complex goal structures with system constraints.

Keywords: Aerial Systems: Perception and Autonomy, Motion and Path Planning, Reactive and Sensor-Based Planning
Abstract: This work investigates an efficient trajectory generation for chasing a dynamic target, which incorporates the detectability objective. The proposed method actively guides the motion of a cinematographer drone so that the color of a target is well-distinguished against the colors of the background in the view of the drone. For the objective, we define a measure of color detectability given a chasing path. After computing a discrete path optimized for the metric, we generate a dynamically feasible trajectory. The whole pipeline can be updated on-the-fly to respond to the motion of the target. For the efficient discrete path generation, we construct a directed acyclic graph (DAG) for which a topological sorting can be determined analytically without the depth-first search. The smooth path is obtained in quadratic programming (QP) framework. We validate the enhanced performance of state-of-the-art object detection and tracking algorithms when the camera drone executes the trajectory obtained from the proposed method.

Keywords: Aerial Systems: Perception and Autonomy, Surveillance Systems, Motion and Path Planning
Abstract: This paper tackles the problem of positioning a swarm of UAVs inside a completely unknown terrain, having as objective to maximize the overall situational awareness. The situational awareness is expressed by the number and quality of unique objects of interest, inside the UAVs' fields of view. YOLOv3 and a system to identify duplicate objects of interest were employed to assign a single score to each UAVs' configuration. Then, a novel navigation algorithm, capable of optimizing the previously defined score, without taking into consideration the dynamics of either UAVs or environment, is proposed. A cornerstone of the proposed approach is that it shares the same convergence characteristics as the block coordinate descent (BCD) family of approaches. The effectiveness and performance of the proposed navigation scheme were evaluated utilizing a series of experiments inside the AirSim simulator. The experimental evaluation indicates that the proposed navigation algorithm was able to consistently navigate the swarm of UAVs to ``strategic'' monitoring positions and also adapt to the different number of swarm sizes, utilizing the dynamics of UAVs to the full extent. The source code and a video demonstration are available at https://github.com/dimikout3/ConvCAO_AirSim.

Keywords: Aerial Systems: Applications, Motion and Path Planning, Autonomous Vehicle Navigation
Abstract: With much research has been conducted into trajectory planning for quadrotors, planning with spatial and temporal optimal trajectories in real-time is still challenging. In this paper, we propose a framework for large-scale waypoint-based polynomial trajectory generation, with highlights on its superior computational efficiency and simultaneous spatial-temporal optimality. Exploiting the implicitly decoupled structure of the problem, we conduct alternating minimization between boundary conditions and time durations of trajectory pieces. Algebraic convenience of both sub-problems is leveraged to escape poor local minima and achieve the lowest time consumption. Theoretical analysis for the global/local convergence rate of our method is provided. Moreover, based on polynomial theory, an extremely fast feasibility checker is designed for various kinds of constraints. By incorporating it into our alternating structure, a constrained minimization algorithm is constructed to optimize trajectories on the premise of feasibility. Benchmark evaluation shows that our algorithm outperforms state-of-the-art waypoint-based methods regarding efficiency, optimality, and scalability. The algorithm can be incorporated in a high-level waypoint planner, which can rapidly search over a three-dimensional space for aggressive autonomous flights. The capability of our algorithm is experimentally demonstrated by quadrotor fast flights in a limited space with dense obstacles.

Keywords: Motion and Path Planning, Multi-Robot Systems, Autonomous Vehicle Navigation
Abstract: For heterogeneous unmanned systems composed of unmanned aerial vehicles (UAVs) and unmanned ground vehicles (UGVs), using UAVs serve as eyes to assist UGVs in motion planning is a promising research direction due to the UAVs' vast view scope. However, its limitations on flight altitude prevent the UAVs from observing the global map. Thus motion planning in the local map becomes a Partially Observable Markov Decision Process (POMDP) problem. This paper proposes a motion planning algorithm for heterogeneous unmanned systems under partial observation from UAV without reconstruction of global maps. Our algorithm consists of two parts designed for perception and decision-making, respectively. For the perception part, we propose the Grid Map Generation Network (GMGN), which is used to perceive scenes from UAV's perspective and classify the pathways and obstacles. For the decision-making part, we propose the Motion Command Generation Network (MCGN). Due to the addition of the memory mechanism, MCGN has planning and reasoning abilities under partial observation from UAVs. We evaluate our proposed algorithm by comparing it with baseline algorithms. The results show that our method effectively plans the motion of heterogeneous unmanned systems and achieves a relatively high success rate.

Keywords: Motion and Path Planning, Task Planning
Abstract: We present a multi-UAV Coverage Path Planning (CPP) framework for the inspection of large-scale, complex 3D structures. In the proposed sampling-based coverage path planning method, we formulate the multi-UAV inspection applications as a multi-agent coverage path planning problem. By combining two NP-hard problems: Set Covering Problem (SCP) and Vehicle Routing Problem (VRP), a Set-Covering Vehicle Routing Problem (SC-VRP) is formulated and subsequently solved by a modified Biased Random Key Genetic Algorithm (BRKGA) with novel, efficient encoding strategies and local improvement heuristics. We test our proposed method for several complex 3D structures with the 3D model extracted from OpenStreetMap. The proposed method outperforms previous methods, by reducing the length of the planned inspection path by up to 48%

Keywords: Motion and Path Planning, Optimization and Optimal Control, Nonholonomic Motion Planning
Abstract: We propose an algorithm for generating minimum-snap trajectories for quadrotors with linear computational complexity with respect to the number of segments in the spline trajectory. Our algorithm is numerically stable for large numbers of segments and is able to generate trajectories of more than 500,000 segments. The computational speed and numerical stability of our algorithm makes it suitable for real-time generation of very large scale trajectories. We demonstrate the performance of our algorithm and compare it to existing methods, in which it is both faster and able to calculate larger trajectories than state-of-the-art. We also show the feasibility of the trajectories experimentally with a long quadrotor flight.

Keywords: Reactive and Sensor-Based Planning, Collision Avoidance, Aerial Systems: Perception and Autonomy
Abstract: We present RAPPIDS: a novel collision checking and planning algorithm for multicopters that is capable of quickly finding local collision-free trajectories given a single depth image from an onboard camera. The primary contribution of this work is a new pyramid-based spatial partitioning method that enables rapid collision detection between candidate trajectories and the environment. By leveraging the efficiency of our collision checking method, we shown how a local planning algorithm can be run at high rates on computationally constrained hardware, evaluating thousands of candidate trajectories in milliseconds. The performance of the algorithm is compared to existing collision checking methods in simulation, showing our method to be capable of evaluating orders of magnitude more trajectories per second. Experimental results are presented showing a quadcopter quickly navigating a previously unseen cluttered environment by running the algorithm on an ODROID-XU4 at 30 Hz.

Keywords: Path Planning for Multiple Mobile Robots or Agents, Networked Robots, Motion and Path Planning
Abstract: Persistent surveillance with aerial vehicles (drones) subject to connectivity and power constraints is a relatively uncharted domain of research. To reduce the complexity of multi-drone motion planning, most state-of-the-art solutions ignore network connectivity and assume unlimited battery power. Motivated by this and advances in optimization and constraint satisfaction techniques, we introduce a new persistent surveillance motion planning problem for multiple drones that incorporates connectivity and power consumption constraints. We use a recently developed constrained optimization tool (Satisfiability Modulo Convex Optimization (SMC)) that has the expressivity needed for this problem. We show how to express the new persistent surveillance problem in the SMC framework. Our analysis of the formulation based on a set of simulation experiments illustrates that we can generate the desired motion planning solution within a couple of minutes for small teams of drones (up to 5) confined to a 7 x 7 x 1 grid-space.

Keywords: Multi-Robot Systems, Planning, Scheduling and Coordination, Aerial Systems: Applications
Abstract: This paper proposes a planning algorithm for autonomous media production with multiple Unmanned Aerial Vehicles (UAVs) in outdoor events. Given filming tasks specified by a media Director, we formulate an optimization problem to maximize the filming time considering battery constraints. As we conjecture that the problem is NP-hard, we consider a discretization version, and propose a graph-based algorithm that can find an optimal solution of the discrete problem for a single UAV in polynomial time. Then, a greedy strategy is applied to solve the problem sequentially for multiple UAVs. We demonstrate that our algorithm is efficient for small teams (3-5 UAVs) and that its performance is close to the optimum. We showcase our system in field experiments carrying out actual media production in an outdoor scenario with multiple UAVs.

Keywords: Path Planning for Multiple Mobile Robots or Agents, Aerial Systems: Perception and Autonomy, Multi-Robot Systems
Abstract: In this work, we present a system architecture to enable autonomous navigation of multiple agents across user-selected global interest points in a partially unknown environment. The system is composed of a server and a team of agents, here small aircrafts. Leveraging this architecture, computationally demanding tasks, such as global dense mapping and global path planning can be outsourced to a potentially powerful central server, limiting the onboard computation for each agent to local pose estimation using Visual-Inertial Odometry (VIO) and local path planning for obstacle avoidance. By assigning priorities to the agents, we propose a hierarchical multi-robot global planning pipeline, which avoids collisions amongst the agents and computes their paths towards the respective goals. The resulting global paths are communicated to the agents and serve as reference input to the local planner running onboard each agent. In contrast to previous works, here we relax the common assumption of a previously mapped environment and perfect knowledge about the state, and we show the effectiveness of the proposed approach in photo-realistic simulations with up to four agents operating in an industrial environment.

Keywords: Swarms, Distributed Robot Systems, Multi-Robot Systems
Abstract: Reliance on external localization infrastructure and centralized coordination are main limiting factors for formation flying of vehicles in large numbers and in unprepared environments. While solutions using onboard localization address the dependency on external infrastructure, the associated coordination strategies typically lack collision avoidance and scalability. To address these shortcomings, we present a unified pipeline with onboard localization and a distributed, collision-free motion planning strategy that scales to a large number of vehicles. Since distributed collision avoidance strategies are known to result in gridlock, we also present a decentralized task assignment solution to deconflict vehicles. We experimentally validate our pipeline in simulation and hardware. The results show that our approach for solving the optimization problem associated with motion planning gives solutions within seconds in cases where general purpose solvers fail due to high complexity. In addition, our lightweight assignment strategy leads to successful and quicker formation convergence in 96-100% of all trials, whereas indefinite gridlocks occur without it for 33-50% of trials. By enabling large-scale, deconflicted coordination, this pipeline should help pave the way for anytime, anywhere deployment of aerial swarms.

Keywords: Multi-Robot Systems, Cooperating Robots, Aerial Systems: Applications
Abstract: Cooperative robotics is a trending topic nowadays as it makes possible a number of tasks that cannot be performed by individual robots, such as heavy payload transportation and agile manipulation. In this work, we address the problem of cooperative transportation by heterogeneous, manipulator- endowed robots. Specifically, we consider a generic number of robotic agents simultaneously grasping an object, which is to be transported to a prescribed set point while avoiding obstacles. The procedure is based on a decentralized leader-follower Model Predictive Control scheme, where a designated leader agent is responsible for generating a trajectory compatible with its dynamics, and the followers must compute a trajectory for their own manipulators that aims at minimizing the internal forces and torques that might be applied to the object by the different grippers. The Model Predictive Control approach appears to be well suited to solve such a problem, because it provides both a control law and a technique to generate trajectories, which can be shared among the agents. The proposed algorithm is implemented using a system comprised of a ground and an aerial robot, both in the robotic Gazebo simulator as well as in experiments with real robots, where the methodological approach is assessed and the controller design is shown to be effective for the cooperative transportation task.

Keywords: Collision Avoidance, Aerial Systems: Applications, Object Detection, Segmentation and Categorization
Abstract: This article proposes a novel Nonlinear Model Predictive Control (NMPC) framework for Micro Aerial Vehicle (MAV) autonomous navigation in indoor enclosed environments. The introduced framework allows us to consider the nonlinear dynamics of MAVs, nonlinear geometric constraints, while guarantees real-time performance. Our first contribution is to reveal underlying planes within a 3D point cloud, obtained from a 3D lidar scanner, by designing an efficient subspace clustering method. The second contribution is to incorporate the extracted information into the nonlinear constraints of NMPC for avoiding collisions. Our third contribution focuses on making the controller robust by considering the uncertainty of localization in NMPC using Shannon's entropy to define the weights involved in the optimization process. This strategy enables us to track position or velocity references or none in the event of losing track of position or velocity estimations. As a result, the collision avoidance constraints are defined in the local coordinates of the MAV and it remains active and guarantees collision avoidance, despite localization uncertainties, e.g., position estimation drifts. The efficacy of the suggested framework has been evaluated using various simulations in the Gazebo environment.

Keywords: Energy and Environment-Aware Automation, Robust/Adaptive Control of Robotic Systems, Aerial Systems: Perception and Autonomy
Abstract: Limited flight range is a common problem for multicopters. To alleviate this problem, we propose a method for finding the optimal speed and heading of a multicopter when flying a given path to achieve the longest flight range. Based on a novel multivariable extremum seeking controller with adaptive step size, the method (a) does not require any power consumption model of the vehicle, (b) can adapt to unknown disturbances, (c) can be executed online, and (d) converges faster than the standard extremum seeking controller with constant step size. We conducted indoor experiments to validate the effectiveness of this method under different payloads and initial conditions, and showed that it is able to converge more than 30% faster than the standard extremum seeking controller. This method is especially useful for applications such as package delivery, where the size and weight of the payload differ for different deliveries and the power consumption of the vehicle is hard to model.

Keywords: Aerial Systems: Perception and Autonomy, Autonomous Vehicle Navigation, Aerial Systems: Applications
Abstract: There has been a considerable amount of recent work on high-speed micro-aerial vehicle flight in unknown and unstructured environments. Generally these approaches either use active sensing or fly slowly enough to ensure a safe braking distance with the relatively short sensing range of passive sensors. The former generally requires carrying large and heavy LIDARs and the latter only allows flight far away from the dynamic limits of the vehicle. One of the significant challenges for high-speed flight is the computational demand of trajectory planning at sufficiently high rates and length scales required in outdoor environments. We tackle both problems in this work by leveraging semantic information derived from an RGB camera on-board the vehicle. We first describe how to use semantic information to increase the effective range of perception on certain environment classes. Second, we present a sparse representation of the environment that is sufficiently lightweight for long distance path planning. We show how our approach outperforms more traditional metric planners which seek the shortest path, demonstrate the semantic planner's capabilities in a set of simulated and excessive real-world autonomous quadrotor flights in an urban environment.

Keywords: Aerial Systems: Perception and Autonomy, Computer Vision for Automation, Computer Vision for Other Robotic Applications
Abstract: The outstanding computational efficiency of discriminative correlation filter (DCF) fades away with various complicated improvements. Previous appearances are also gradually forgotten due to the exponential decay of historical views in traditional appearance updating scheme of DCF framework, reducing the model’s robustness. In this work, a novel tracker based on DCF framework is proposed to augment memory of previously appeared views while running at real-time speed. Several historical views and the current view are simultaneously introduced in training to allow the tracker to adapt to new appearances as well as memorize previous ones. A novel rapid compressed context learning is proposed to increase the discriminative ability of the filter efficiently. Substantial experiments on UAVDT and UAV123 datasets have validated that the proposed tracker performs competitively against other 26 top DCF and deep-based trackers with over 40fps on CPU.

Keywords: Aerial Systems: Perception and Autonomy, Reactive and Sensor-Based Planning, Path Planning for Multiple Mobile Robots or Agents
Abstract: In this paper, we propose a novel cluster-based Next-Best-View path planning algorithm to simultaneously explore and inspect large-scale unknown environments with multiple Unmanned Aerial Vehicles (UAVs). In the majority of existing informative path-planning methods, a volumetric criterion is used for the exploration of unknown areas, and the presence of surfaces is only taken into account indirectly. Unfortunately, this approach may lead to inaccurate 3D models, with no guarantee of global surface coverage. To perform accurate 3D reconstructions and minimize runtime, we extend our previous online planner based on TSDF (Truncated Signed Distance Function) mapping, to a fleet of UAVs. Sensor configurations to be visited are directly extracted from the map and assigned greedily to the aerial vehicles, in order to maximize the global utility at the fleet level. The performances of the proposed TSGA (TSP-Greedy Allocation) planner and of a nearest neighbor planner have been compared via realistic numerical experiments in two challenging environments (a power plant and the Statue of Liberty) with up to five quadrotor UAVs equipped with stereo cameras.

Keywords: Aerial Systems: Perception and Autonomy, Aerial Systems: Applications, Surveillance Systems
Abstract: Object tracking has been broadly applied in unmanned aerial vehicle (UAV) tasks in recent years. However, existing algorithms still face difficulties such as partial occlusion, clutter background, and other challenging visual factors. Inspired by the cutting-edge attention mechanisms, a novel object tracking framework is proposed to leverage multi-level visual attention. Three primary attention, i.e., contextual attention, dimensional attention, and spatiotemporal attention, are integrated into the training and detection stages of correlation filter-based tracking pipeline. Therefore, the proposed tracker is equipped with robust discriminative power against challenging factors while maintaining high operational efficiency in UAV scenarios. Quantitative and qualitative experiments on two well-known benchmarks with 173 challenging UAV video sequences demonstrate the effectiveness of the proposed framework. The proposed tracking algorithm favorably outperforms 12 state-of-the-art methods, yielding 4.8% relative gain in UAVDT and 8.2% relative gain in UAV123@10fps against the baseline tracker while operating at the speed of ~28 frames per second.

Keywords: Aerial Systems: Perception and Autonomy
Abstract: Inspection for structural properties (surface stiffness and coefficient of restitution) is crucial for understanding and performing aerial manipulations in unknown environments, with little to no prior knowledge on their state. Inspection-on-the-fly is the uncanny ability of humans to infer states during manipulation, reducing the necessity to perform inspection and manipulation separately. This paper presents an infrastructure for inspection-on-the-fly method for aerial manipulators using hybrid physical interaction control. With the proposed method, structural properties (surface stiffness and coefficient of restitution) can be estimated during physical interactions. A three-stage hybrid physical interaction control paradigm is presented to robustly approach, acquire and impart a desired force signature onto a surface. This is achieved by combining a hybrid force/motion controller with a model-based feed-forward impact control as intermediate phase. The proposed controller ensures a steady transition from unconstrained motion control to constrained force control, while reducing the lag associated with the force control phase. And an underlying Operational Space dynamic configuration manager permits complex, redundant vehicle/arm combinations. Experiments were carried out in a mock-up of a Dept. of Energy exhaust shaft, to show the effectiveness of the inspection-on-the-fly method to determine the structural properties of the target surface and the performance of the hybrid physical interaction controller in reducing the lag associated with force control phase.

Keywords: Reinforecment Learning, Aerial Systems: Perception and Autonomy, Multi-Robot Systems
Abstract: In this letter, we introduce a deep reinforcement learning (DRL) based multi-robot formation controller for the task of autonomous aerial human motion capture (MoCap). We focus on vision-based MoCap, where the objective is to estimate the trajectory of body pose and shape of a single moving person using multiple micro aerial vehicles. State-of-the-art solutions to this problem are based on classical control methods, which depend on hand-crafted system and observation models. Such models are difficult to derive and generalize across different systems. Moreover, the non-linearities and non-convexities of these models lead to sub-optimal controls. In our work, we formulate this problem as a sequential decision making task to achieve the vision-based motion capture objectives, and solve it using a deep neural network-based RL method. We leverage proximal policy optimization (PPO) to train a stochastic decentralized control policy for formation control. The neural network is trained in a parallelized setup in synthetic environments. We performed extensive simulation experiments to validate our approach. Finally, real-robot experiments demonstrate that our policies generalize to real world conditions.

Keywords: Aerial Systems: Applications, Computer Vision for Automation, Aerial Systems: Perception and Autonomy
Abstract: Visual tracking has yielded promising applications with unmanned aerial vehicle (UAV). In literature, the advanced discriminative correlation filter (DCF) type trackers generally distinguish the foreground from the background with a learned regressor which regresses the implicit circulated samples into a fixed target label. However, the predefined and unchanged regression target results in low robustness and adaptivity to uncertain aerial tracking scenarios. In this work, we exploit the local maximum points of the response map generated in the detection phase to automatically locate current distractors. By repressing the response of distractors in the regressor learning, we can dynamically and adaptively alter our regression target to leverage the tracking robustness as well as adaptivity. Substantial experiments conducted on three challenging UAV benchmarks demonstrate both excellent performance and extraordinary speed (~50fps on a cheap CPU) of our tracker.

Keywords: Aerial Systems: Perception and Autonomy, Aerial Systems: Applications, Compliance and Impedance Control
Abstract: The integration of computer vision techniques for the accomplishment of autonomous interaction tasks represents a challenging research direction in the context of aerial robotics. In this paper, we consider the problem of contact-based inspection of a textured target of unknown geometry and pose. Exploiting state of the art techniques in computer graphics, tuned and improved for the task at hand, we designed a framework for the projection of a desired trajectory for the robot end-effector on a generically-shaped surface to be inspected. Combining these results with previous work on energy-based interaction control, we are laying the basis of what we call vision-based impedance control paradigm. To demonstrate the feasibility and the effectiveness of our methodology, we present the results of both realistic ROS/Gazebo simulations and preliminary experiments with a fully-actuated hexarotor interacting with heterogeneous curved surfaces whose geometric description is not available a priori, provided that enough visual features on the target are naturally or artificially available to allow the integration of localization and mapping algorithms.

Keywords: Aerial Systems: Perception and Autonomy, Deep Learning for Visual Perception, Computer Vision for Automation
Abstract: In this work, we present a pragmatic approach to enable unmanned aerial vehicle (UAVs) to autonomously perform highly complicated tasks of object pick and place. This paper is largely inspired by challenge-2 of MBZIRC 2020 and is primarily focused on the task of assembling large 3D structures in outdoors and GPS-denied environments. Primary contributions of this system are: (i) a novel computationally efficient deep learning based unified multi-task visual perception system for target localization, part segmentation, and tracking, (ii) a novel deep learning based grasp state estimation, (iii) a retracting electromagnetic gripper design, (iv) a remote computing approach which exploits state-of-the-art MIMO based high speed (5000Mb/s) wireless links to allow the UAVs to execute compute intensive tasks on remote high end compute servers, and (v) system integration in which several system components are weaved together in order to develop an optimized software stack. We use DJI Matrice-600 Pro, a hex-rotor UAV and interface it with the custom designed gripper. Our framework is deployed on the specified UAV in order to report the performance analysis of the individual modules. Apart from the manipulation system, we also highlight several hidden challenges associated with the UAVs in this context.

Keywords: Aerial Systems: Applications, Computer Vision for Automation, Visual Tracking
Abstract: We present a method to reconstruct the 3D trajectory of an airborne robotic system only from videos recorded with cameras that are unsynchronized, may feature rolling shutter distortion, and whose viewpoints are unknown. Our approach enables robust and accurate outside-in tracking of dynamically flying targets, with cheap and easy-to-deploy equipment. We show that, in spite of the weakly constrained setting, recent developments in computer vision make it possible to reconstruct trajectories in 3D from unsynchronized, uncalibrated networks of consumer cameras, and validate the proposed method in a realistic field experiment. We make our code available along with the data, including cm-accurate ground-truth from differential GNSS navigation.

Keywords: Aerial Systems: Perception and Autonomy, Sensor Fusion, Visual Tracking
Abstract: This paper presents a method for target localization and tracking in clutter using Bayesian fusion of vision and Radio Frequency (RF) sensors used aboard a small Unmanned Aircraft System (sUAS). Sensor fusion is used to ensure tracking robustness and reliability in case of camera occlusion or RF signal interference. Camera data is processed using an off-the-shelf algorithm that detects possible objects of interest in a given image frame, and the true RF emitting target needs to be identified from among these if it is present. These data sources, as well as the unknown motion of the target, lead to a heavily non-linear non-Gaussian target state uncertainties, which are not amenable to typical data association methods for tracking. A probabilistic model is thus first rigorously developed to relate conditional dependencies between target movements, RF data, and visual object detections. A modified particle filter is then developed to simultaneously reason over target states and RF emitter association hypothesis labels for visual object detections. Truth model simulations are presented to compare and validate the effectiveness of the RF + visual data fusion filter.

Keywords: Aerial Systems: Perception and Autonomy, Visual-Based Navigation, Representation Learning
Abstract: Machines are a long way from robustly solving open-world perception-control tasks, such as first-person view (FPV) aerial navigation. While recent advances in end-to-end Machine Learning, especially Imitation Learning and Reinforcement appear promising, they are constrained by the need of large amounts of difficult-to-collect labeled real-world data. Simulated data, on the other hand, is easy to generate, but generally does not render safe behaviors in diverse real-life scenarios. In this work we propose a novel method for learning robust visuomotor policies for real-world deployment which can be trained purely with simulated data. We develop rich state representations that combine supervised and unsupervised environment data. Our approach takes a cross-modal perspective, where separate modalities correspond to the raw camera data and the system states relevant to the task, such as the relative pose of gates to the drone in the case of drone racing. We feed both data modalities into a novel factored architecture, which learns a joint low-dimensional embedding via Variational Auto Encoders. This compact representation is then fed into a control policy, which we trained using imitation learning with expert trajectories in a simulator. We analyze the rich latent spaces learned with our proposed representations, and show that the use of our cross-modal architecture significantly improves control policy performance as compared to end-to-end learning or purely unsupervised feature extractors. We also present real-world results for drone navigation through gates in different track configurations and environmental conditions. Our proposed method, which runs fully onboard, can successfully generalize the learned representations and policies across simulation and reality, significantly outperforming baseline approaches. Supplementary video available at: https://youtu.be/AxE7qGKJWaw and open-sourced code available at: https://github.com/microsoft/AirSim-Drone-Racing-VAE-Imitation

Keywords: Sensor Fusion, Aerial Systems: Perception and Autonomy, Aerial Systems: Applications
Abstract: Disturbance estimation for Micro Aerial Vehicles (MAVs) is crucial for robustness and safety. In this paper, we use novel, bio-inspired airflow sensors to measure the airflow acting on a MAV, and we fuse this information in an Unscented Kalman Filter (UKF) to simultaneously estimate the three-dimensional wind vector, the drag force, and other interaction forces (e.g. due to collisions, interaction with a human) acting on the robot. To this end, we present and compare a fully model-based and a deep learning-based strategy. The model-based approach considers the MAV and airflow sensor dynamics and its interaction with the wind, while the deep learning-based strategy uses a Long Short-Term Memory (LSTM) to obtain an estimate of the relative airflow, which is then fused in the proposed filter. We validate our methods in hardware experiments, showing that we can accurately estimate relative airflow of up to 4 m/s, and we can differentiate drag and interaction force.

Keywords: Marine Robotics, Range Sensing, Sensor Fusion
Abstract: We propose a novel approach to handling the ambiguity in elevation angle associated with the observations of a forward looking multi-beam imaging sonar, and the challenges it poses for performing an accurate 3D reconstruction. We utilize a pair of sonars with orthogonal axes of uncertainty to independently observe the same points in the environment from two different perspectives, and associate these observations. Using these concurrent observations, we can create a dense, fully defined point cloud at every time-step to aid in reconstructing the 3D geometry of underwater scenes. We will evaluate our method in the context of the current state of the art, for which strong assumptions on object geometry limit applicability to generalized 3D scenes. We will discuss results from laboratory tests that quantitatively benchmark our algorithm's reconstruction capabilities, and results from a real-world, tidal river basin which qualitatively demonstrate our ability to reconstruct a cluttered field of underwater objects.

Keywords: Sensor Fusion, Autonomous Vehicle Navigation, Computer Vision for Transportation
Abstract: Automated driving requires highly precise and robust vehicle state estimation for its environmental perception, motion planning and control functions. Using GPS and environmental sensors can compensate for the deficits of the estimation based on traditional vehicle dynamics sensors. However, each type of sensor has specific strengths and limitations in accuracy and robustness due to their different properties regarding the quality of detection and robustness in diverse environmental conditions. For these reasons, we present a scalable concept for vehicle state estimation using an error-state extended Kalman filter (ESEKF) to fuse classical vehicle sensors with environmental sensors. The state variables, i.e., position, velocity and orientation, are predicted by a 6-degree-of-freedom (DoF) vehicle kinematic model that uses a low-cost inertial measurement unit (IMU) on a customer vehicle. The Error of the 6-DoF rigid body motion model is estimated using observations of global position using the global navigation satellite system (GNSS) and of the environment using radar, a camera and low-cost lidar. Our concept is scalable such that it is compatible with different sensor setups on different vehicle configurations. The experimental results compare various sensor combinations with measurement data in scenarios such as dynamic driving maneuvers on a test field. The results show that our approach ensures accuracy and robustness with redundant sensor data under regular and dynamic driving conditions.

Keywords: Automation Technologies for Smart Cities, Service Robots, Field Robots
Abstract: All-day and all-weather navigation is a critical capability for autonomous driving, which requires proper reaction to varied environmental conditions and complex agent behaviors. Recently, with the rise of deep learning, end-to-end control for autonomous vehicles has been well studied. However, most works are solely based on visual information, which can be degraded by challenging illumination conditions such as dim light or total darkness. In addition, they usually generate and apply deterministic control commands without considering the uncertainties in the future. In this paper, based on imitation learning, we propose a probabilistic driving model with multi-perception capability utilizing the information from the camera, lidar and radar. We further evaluate its driving performance online on our new driving benchmark, which includes various environmental conditions (e.g., urban and rural areas, traffic densities, weather and times of the day) and dynamic obstacles (e.g., vehicles, pedestrians, motorcyclists and bicyclists). The results suggest that our proposed model outperforms baselines and achieves excellent generalization performance in unseen environments with heavy traffic and extreme weather.

Keywords: Sensor Fusion, Computer Vision for Automation, Autonomous Vehicle Navigation
Abstract: We present a probabilistic framework for detailed 3-D shape estimation and tracking using only vision measurements. Vision detections are processed via a bird’s eye view representation, creating accurate detections at far ranges. A probabilistic model of the vision based point cloud measurements is learned and used in the framework. A 3-D shape model is developed by fusing a set of point cloud detections via a recursive Best Linear Unbiased Estimator (BLUE). The point cloud fusion accounts for noisy and inaccurate measurements, as well as minimizing growth of points in the 3-D shape. The use of a tracking algorithm and sensor pose enables 3-D shape estimation of dynamic objects from a moving car. Results are analyzed on experimental data, demonstrating the ability of our approach to produce more accurate and cleaner shape estimates.

Keywords: Aerial Systems: Perception and Autonomy, Reactive and Sensor-Based Planning, Computational Geometry
Abstract: This paper presents Polylidar, an efficient algorithm to extract non-convex polygons from 2D point sets, including interior holes. Plane segmented point clouds can be input into Polylidar to extract their polygonal counterpart, thereby reducing map size and improving visualization. The algorithm begins by triangulating the point set and filtering triangles by user configurable parameters such as triangle edge length. Next, connected triangles are extracted into triangular mesh regions representing the shape of the point set. Finally each region is converted to a polygon through a novel boundary following method which accounts for holes. Real-world and synthetic benchmarks are presented to comparatively evaluate Polylidar speed and accuracy. Results show comparable accuracy and more than four times speedup compared to other concave polygon extraction methods

Keywords: Marine Robotics, Optimization and Optimal Control
Abstract: Achieving and maintaining line-of-sight (LOS) is challenging for underwater optical communication systems, especially when the underlying platforms are mobile. In this work, we propose and demonstrate an active alignment control-based LED-communication system that uses the DC value of the communication signal as feedback for LOS maintenance. Utilizing the uni-modal nature of the dependence of the light signal strength on local angles, we propose a novel triangular exploration algorithm, that does not require the knowledge of the underlying light intensity model, to maximize the signal strength that leads to achieving and maintaining LOS. The method maintains an equilateral triangle shape in the angle space for any three consecutive exploration points, while ensuring the consistency of exploration direction with the local gradient of signal strength. The effectiveness of the approach is first evaluated in simulation by comparison with extremum-seeking control, where the proposed approach shows a significant advantage in the convergence speed. The efficacy is further demonstrated experimentally, where an underwater robot is controlled by a joystick via LED communication.

Keywords: Biologically-Inspired Robots, Biomimetics, Marine Robotics
Abstract: Underwater communication is extremely challenging for small underwater robots that have stringent power and size constraints. In our previous work, we have demonstrated that electrocommunication is an alternative method for small underwater robot communication. This paper presents a new electrocommunication system which utilizes Binary Frequency Shift Keying (2FSK) modulation and deep-learning-based demodulation for underwater robots. We first derive an underwater electrocommunication model which covers both the near-field area and a large transition area outside of the near-field area. The 2FSK modulation is adopted to improve the anti-interference ability of the signal. A deep learning algorithm is used to demodulate the signal by the receiver. Simulations and experiments show that at the same testing condition, the new communication system has a lower bit error rate and higher data rate than the previous electrocommunication system. The communication system achieves stable communication within the distance of 10 m at a data transfer rate of 5 Kbps with a power consumption of less than 0.1 W. The large improvement of the communication distance in this study further advances the application of electrocommunication.

Keywords: Marine Robotics, Underactuated Robots, Biologically-Inspired Robots
Abstract: Underwater roll rotation is a basic but essential maneuver that allows many biological swimmers to achieve high maneuverability and complex locomotion patterns. In particular, sea mammals (e.g., sea otter) with flexible vertebra structures have a unique mechanism to efficiently achieve roll rotation, not propelled mainly by inter-digital webbing or fin, but by bending and twisting their body. In this work, we attempt to implement and effectively control the roll rotation by mimicking this kind of efficient biomorphic roll mechanism on our two degrees of freedom (DOF) soft modular swimming robot. The robot also allows the achievement of other common maneuvers, such as pitch/yaw rotation and linear swimming patterns. The proposed 2DOF soft swimming robot platform includes an underactuated, cable-driven design that mimics the flexible cascaded skeletal structure of soft spine tissue and hard spine bone seen in many fish species. The cable-driven actuation mechanism is oriented laterally for forwarding motion and steering in a 3D plane.

The robot can perform a steady and controllable roll rotation with a maximum angular speed of 41.6 deg/s. A hypothesis explaining this novel roll rotation mechanism is set forth, and the phenomenon is systematically studied at different frequencies and phase lag gait conditions. Preliminary results show a linear relationship between roll angular velocity and frequency within a specific range. Additionally, the roll rotation can be controlled independently in some special conditions. These abilities form the foundation for future research on 3D underwater locomotion with adaptive, controllable maneuvering capabilities.

Keywords: Underactuated Robots, Marine Robotics
Abstract: We describe motion primitives and closed-loop control for a unique low-cost single-motor oscillating aquatic system: the Modboat. The Modboat is driven by the conservation of angular momentum, which is used to actuate two passive flippers in a sequential paddling motion for propulsion and steering. We propose a discrete description of the motion of the system, which oscillates around desired trajectories, and propose two motion primitives - one frequency based and one pause-based - with associated closed-loop controllers. Testing is performed to evaluate each motion primitive, the merits of each are presented, and the pause-based primitive is shown to be significantly superior. Finally, waypoint following is implemented using both primitives and shown to be significantly more successful using the pause-based motion primitive.

Keywords: Marine Robotics, Motion and Path Planning, Computational Geometry
Abstract: In this paper, we present a novel algorithm for constructing a maximally informative path for a robot in an information gathering task. We use a Self-Organizing Map (SOM) framework to discover important topological features in the information function. Using these features, we identify a set of distinct classes of trajectories, each of which has improved convexity compared with the original function. We then leverage a Stochastic Gradient Ascent (SGA) optimization algorithm within each of these classes to optimize promising representative paths. The increased convexity leads to an improved chance of SGA finding the globally optimal path across all homotopy classes. We demonstrate our approach in three different simulated experiments. First, we show that our SOM is able to correctly learn the topological features of a gyre environment with a well-defined topology. Then, in the second set of experiments, we compare the effectiveness of our algorithm in an information gathering task across the gyre world, a set of randomly generated worlds, and a set of worlds drawn from real-world ocean model data. In these experiments our algorithm performs competitively or better than a state-of-the-art Branch and Bound while requiring significantly less computation time. Lastly, the final set of experiments show that our method scales better than the comparison methods across different planning mission sizes in real-world environments.

Keywords: Marine Robotics, Robotics in Hazardous Fields, Autonomous Agents
Abstract: The SPIR, Submersible Pylon Inspection Robot, is developed to provide an innovative and practical solution to keep workers safe during maintenance of underwater structures in shallow waters, which involves working in dangerous water currents, and high-pressure water-jet cleaning. More advanced than work-class Remotely Operated Vehicles technology, the SPIR is automated and required minimum involvement of humans into the working process, thus effectively lowered the learning curve required to conduct work. To make SPIR operate effectively in poor visibility and highly disturbed environments, the multiple new technologies are developed and implemented into the system, including SBL-SONAR-based navigation, 6-DOF stabilisation, and vision-based 3D mapping. Extensive testing and field trials in various bridges are conducted to verify the robotic system. The results demonstrate the suitability of the SPIR in substituting humans for underwater hazardous tasks such as autonomous cleaning and inspection of bridge and wharf piles.

Keywords: Marine Robotics, Task Planning, Cooperating Robots
Abstract: A heterogeneous swarm of marine robots was developed with the goal of autonomous long-term monitoring of environmental phenomena in the highly relevant ecosystem of Venice, Italy. As logistics are a continuing challenge in the field of marine robotics, especially when dealing with a large number of agents to be collected and redeployed per experimental run, an approach is needed that provides the benefits of simulation while also reflecting the complexity of the real world. This paper focuses on the development of a vehicle-in-the-loop test environment in which a surface station simulates and transmits the data of any number of simulated agents, while a real marine platform operates based on the received information. Several experimental runs of a specific use-case test scenario using the developed framework and carried out in the field are described and their results are examined.

Keywords: Marine Robotics, Autonomous Vehicle Navigation, Automation Technologies for Smart Cities
Abstract: This paper presents a novel autonomous surface vessel (ASV), called Roboat II for urban transportation. Roboat II is capable of accurate simultaneous localization and mapping (SLAM), receding horizon tracking control and estimation, and path planning. Roboat II is designed to maximize the internal space for transport, and can carry payloads several times of its own weight. Moreover, it is capable of holonomic motions to facilitate transporting, docking, and inter-connectivity between boats. The proposed SLAM system receives sensor data from a 3D LiDAR, an IMU, and a GPS, and utilizes a factor graph to tackle the multi-sensor fusion problem. To cope with the complex dynamics in the water, Roboat II employs an online nonlinear model predictive controller (NMPC), where we experimentally estimated the dynamical model of the vessel in order to achieve superior performance for tracking control. The states of Roboat II are simultaneously estimated using a nonlinear moving horizon estimation (NMHE) algorithm. Experiments demonstrate that Roboat II is able to successfully perform online mapping and localization, plan its path and robustly track the planned trajectory in the confined river, implying that this autonomous vessel holds the promise on potential applications in transporting humans and goods in many of the waterways nowadays.

Keywords: Marine Robotics, Field Robots, Intelligent Transportation Systems
Abstract: Automatic latching for charging in a disturbed environment for Unmanned Surface Vehicle (USVs) is always a challenging problem. In this paper, we propose a two-stage automatic latching system for USVs charging in berth. In Stage I, a vision-guided algorithm is developed to calculate an optimal latching position for charging. In Stage II, a novel latching mechanism is designed to compensate the movement misalignments from the water disturbance. A set of experiments have been conducted in real-world environments. The results show the latching success rate has been improved from 40% to 73.3% in the best cases with our proposed system. Furthermore, the vision-guided algorithm provides a methodology to optimize the design radius of the latching mechanism with respect to different disturbance levels accordingly. Outdoor experiments have validated the efficiency of our proposed automatic latching system. The proposed system improves the autonomy intelligence of the USVs and provides great benefits for practical applications.

Keywords: Marine Robotics, Field Robots, Mining Robotics
Abstract: This paper presents the results of the experimental tests performed to validate the functionality of a variable pitch system (VPS), designed for pitch attitude control of the novel underwater robotic vehicle explorer UX-1. The VPS is composed of a mass suspended from a central rod mounted across the hull. This mass is rotated around the transverse axis of the vehicle in order to perform a change in the inclination angle for navigation in vertical mine shafts. In this work, the equations of motion are first derived with a quaternion attitude representation, and are then extended to include the dynamics of the VPS. The performance of the VPS is demonstrated in real underwater experimental tests that validate the pitch angle control independently, and coupled with the heave motion control system.

Keywords: Marine Robotics, Field Robots
Abstract: In this paper we present the LoCO AUV, a Low-Cost, Open Autonomous Underwater Vehicle. LoCO is a general-purpose, single-person-deployable, vision-guided AUV, rated to a depth of 100 meters. We discuss the open and expandable design of this underwater robot, as well as the design of a simulator in Gazebo. Additionally, we explore the platform’s preliminary local motion control and state estimation abilities, which enable it to perform maneuvers autonomously. In order to demonstrate its usefulness for a variety of tasks, we implement a variety of our previously presented human-robot interaction capabilities on LoCO, including gestural control, diver following, and robot communication via motion. Finally, we discuss the practical concerns of deployment and our experiences in using this robot in pools, lakes, and the ocean. All design details, instructions on assembly, and code will be released under a permissive, open-source license.

Keywords: Marine Robotics, Field Robots, Object Detection, Segmentation and Categorization
Abstract: In this paper, we present the first large-scale dataset for semantic Segmentation of Underwater IMagery (SUIM). It contains over 1500 images with pixel annotations for eight object categories: fish (vertebrates), reefs (invertebrates), aquatic plants, wrecks/ruins, human divers, robots, and sea-floor. The images have been rigorously collected during oceanic explorations and human-robot collaborative experiments, and annotated by human participants. We also present a comprehensive benchmark evaluation of several state-of-the-art semantic segmentation approaches based on standard performance metrics. Additionally, we present SUIM-Net, a fully-convolutional deep residual model that balances the trade-off between performance and computational efficiency. It offers competitive performance while ensuring fast end-to-end inference, which is essential for its use in the autonomy pipeline by visually-guided underwater robots. In particular, we demonstrate its usability benefits for visual servoing, saliency prediction, and detailed scene understanding. With a variety of use cases, the proposed model and benchmark dataset open up promising opportunities for future research in underwater robot vision.

Keywords: Field Robots, Deep Learning for Visual Perception, Localization
Abstract: In this paper, we propose a real-time deep-learning approach for determining the 6D relative pose of Autonomous Underwater Vehicles (AUV) from a single image. A team of autonomous robots localizing themselves in a communication-constrained underwater environment is essential for many applications such as underwater exploration, mapping, multi-robot convoying, and other multi-robot tasks. Due to the profound difficulty of collecting ground truth images with accurate 6D poses underwater, this work utilizes rendered images from the Unreal Game Engine simulation for training. An image-to-image translation network is employed to bridge the gap between the rendered and the real images producing synthetic images for training. The proposed method predicts the 6D pose of an AUV from a single image as 2D image keypoints representing 8 corners of the 3D model of the AUV, and then the 6D pose in the camera coordinates is determined using RANSAC-based PnP. Experimental results in real-world underwater environments (swimming pool and ocean) with different cameras demonstrate the robustness and accuracy of the proposed technique in terms of translation error and orientation error over the state-of-the-art methods. The code is publicly available.

Keywords: Marine Robotics, SLAM, Sensor Fusion
Abstract: This paper proposes a methodology for real-time depth estimation of underwater monocular camera images, fusing measurements from a single-beam echosounder. Our system exploits the echosounder's detection cone to match its measurements with the detected feature points from a monocular SLAM system. Such measurements are integrated in a monocular SLAM system to adjust the visible map points and the scale. We also provide a novel calibration process to determine the extrinsic between camera and echosounder to have reliable matching. Our proposed approach is implemented within ORB-SLAM2 and evaluated in a swimming pool and in the ocean to validate image depth estimation improvement. In addition, we demonstrate its applicability for improved underwater color correction. Overall, the proposed sensor fusion system enables inexpensive underwater robots with a monocular camera and echosounder to correct the depth estimation and scale in visual SLAM, leading to interesting future applications, such as underwater exploration and mapping.

Keywords: Marine Robotics, Deep Learning for Visual Perception, Aerial Systems: Perception and Autonomy
Abstract: Underwater localization is a challenging task due to the lack of a Global Positioning System (GPS). However, the capability to match georeferenced aerial images and acoustic data can help with this task. Autonomous hybrid aerial and underwater vehicles also demand a new localization method capable of combining the perception from both environments. This study proposes a cross-domain and cross-view image matching, using a color aerial image and an underwater acoustic image to identify if these images are captured in the same place. The method is designed to match images acquired in partially structured environments with shared features, such as harbors and marinas. Our pipeline combines traditional image processing methods and deep neural network techniques. Real-world datasets from multiple regions are used to validate our work, obtaining a matching precision of up to 80%

Keywords: Marine Robotics, Computer Vision for Other Robotic Applications
Abstract: ACMarker is an acoustic camera-based fiducial marker system designed for underwater environments. Optical camera-based fiducial marker systems have been widely used in computer vision and robotics applications such as augmented reality (AR), camera calibration, and robot navigation. However, in underwater environments, the performance of optical cameras is limited owing to water turbidity and illumination conditions. Acoustic cameras, which are forward-looking sonars, have been gradually applied in underwater situations. They can acquire high-resolution images even in turbid water with poor illumination. We propose methods to recognize a simply designed marker and to estimate the relative pose between the acoustic camera and the marker. The proposed system can be applied to various underwater tasks such as object tracking and localization of unmanned underwater vehicles. Simulation and real experiments were conducted to test the recognition of such markers and pose estimation based on the markers.

Keywords: Marine Robotics, Collision Avoidance, Autonomous Vehicle Navigation
Abstract: This paper presents a novel risk vector-based near miss prediction and obstacle avoidance that can be used for computing an efficient, dynamic, and robust action in real-time. Simulation experiments with parameters inferred from experiments in the ocean with our custom-made robotic boat show flexibility and adaptability to many obstacles present in the environment.

Keywords: Marine Robotics, Calibration and Identification, Dynamics
Abstract: This work presents a practical method of obtaining a dynamic system model for small omnidirectional aquatic vehicles. The models produced can be used to improve vehicle localisation, aid in the design or tuning of control systems and facilitate the development of simulated environments. The use of a dynamic model for onboard real-time velocity prediction is of particular importance for aquatic vehicles because, unlike ground vehicles, fast and direct measurement of velocity using encoders is not possible. Previous work on model identification of aquatic vehicles has focused on large vessels that are typically underactuated and have low controllability in the sway direction. In this paper it is demonstrated that the procedure for identifying the model coefficients can be performed quickly, without specialist equipment and using only onboard sensors. This is of key importance because the dynamic model coefficients will change with the payload. Two different thrust allocation schemes are tested, one of which is a known method and another is proposed here. Validation tests are performed and the models generated are shown to be suitable for their intended applications. Significant reduction in model error is demonstrated using the novel thrust allocation method that is designed to avoid deadbands in the thruster responses.

Keywords: Marine Robotics, Field Robots, Robotics in Hazardous Fields
Abstract: Despite the recent progress, guidance, navigation, and control (GNC) are largely unsolved for agile micro autonomous underwater vehicles (micro AUVs). Hereby, robust and accurate self-localization systems which fit micro AUVs play a key role and their lack is, thus, a severe bottleneck in micro underwater robotics research. In this work we present, first, a small-size low-cost high performance vision-based self-localization module which solves this bottleneck even for the requirements highly agile robot platforms. Second, we present its integration into a powerful GNC-framework which allows the deployment of micro AUVs in fully autonomous mission. Finally, we critically evaluate the performance of the localization system and the GNC-framework in two experimental scenarios.

Keywords: Marine Robotics, Motion and Path Planning, Robotics in Hazardous Fields
Abstract: In this paper we address the mine countermeasures (MCM) search problem for an autonomous underwater vehicle (AUV) surveying the seabed using a side-looking sonar. We propose a coverage path planning method that adapts the AUV track spacing with the objective of collecting better data. We achieve this by shifting the coverage overlap at the tail of the sensor range where the lowest data quality is expected. To assess the algorithm, we collected data from three at-sea experiments. The adaptive survey allowed the AUV to recover from a situation where the sensor range was overestimated and resulted in reducing area coverage gaps. In another experiment, the adaptive survey showed a 4.2% improvement in data quality for nearly 30% of the 'worst' data.

Keywords: Marine Robotics, Multi-Robot Systems, Autonomous Agents
Abstract: In this paper, we present a dynamic median consensus protocol for multi-agent systems using acoustic communication. The motivating target scenario is a multi-agent system consisting of underwater robots acting as intelligent sensors, applied to continuous monitoring of the state of a marine environment. The proposed protocol allows each agent to track the median value of individual measurements of all agents through local communication with neighbouring agents. Median is chosen as a measure robust to outliers, as opposed to average value, which is usually used. In contrast to the existing consensus protocols, the proposed protocol is dynamic, uses a switching communication topology and converges to median of measured signals. Stability and correctness of the protocol are theoretically proven. The protocol is tested in simulation, and accuracy and influence of protocol parameters on the system output are analyzed. The protocol is implemented and validated by a set of experiments on an underwater group of robots comprising of aMussel units. This experimental setup is one of the first deployments of any type of consensus protocol for an underwater setting. Both simulation and experimental results confirm the correctness of the presented approach.

Keywords: Space Robotics and Automation, Flexible Robots, Calibration and Identification
Abstract: Space manipulator systems on orbit are subject to link flexibilities since they are designed to be lightweight and long reaching. Often, their joints are driven by harmonic gear-motor units, which introduce joint flexibility. Both of these types of flexibility may cause structural vibrations. To improve endpoint tracking, advanced control strategies that benefit from the knowledge of system parameters, including those describing link and joint flexibilities, are required. In this paper, first, the equations of motion of space manipulator systems whose manipulators are subject to both link and joint flexibilities are derived. Then, a parameter estimation method is developed, based on the energy balance during the motion of a flexible space manipulator. The method estimates all system parameters including those that describe both link and joint flexibilities and can reconstruct the system full dynamics required for the application of advanced control strategies. The method, developed for spatial systems, is illustrated by a planar example.

Keywords: Space Robotics and Automation, Assembly, Dual Arm Manipulation
Abstract: In-space assembly (ISA) is the next step to building larger and more permanent structures in orbit. The use of a robotic in-space assembler will save on costly and potentially risky EVAs. Determining the best robot for ISA is difficult as it will depend on the structure being assembled. A comparison between two categories of robots are presented: a stationary robot and robot which crawls along the truss. The estimated mass, energy, and time are presented for each system as it, in simulation, builds a desired truss system. There are trade-offs to every robot design and understanding those trade-offs is essential to building a system that is not only efficient but also cost-effective.

Keywords: Telerobotics and Teleoperation, Virtual Reality and Interfaces, Space Robotics and Automation
Abstract: Ground-based teleoperation of robot manipulators for on-orbit servicing of spacecraft represents an example of high-payoff, high-risk operations that are challenging to perform due to high latency communications, with telemetry time delays of several seconds. In these scenarios, confidence of operating without failure is paramount. We report the development of an Interactive Planning and Supervised Execution (IPSE) system that takes advantage of accurate 3D reconstruction of the remote environment to enable operators to plan motions in the virtual world, evaluate and adjust the plan, and then supervise execution with the ability to pause and return to the planning environment at any time. We report the results of an experimental evaluation of a representative on-orbit telerobotic servicing task from NASA's upcoming OSAM-1 mission to refuel a satellite in low earth orbit; specifically, to change the robot tool to acquire the fuel supply line and then to insert it into the satellite fill/drain valve. Results of a pilot study show that the operators preferred, and were more successful with, the IPSE system when compared to a conventional teleoperation implementation.

Keywords: Space Robotics and Automation, Calibration and Identification
Abstract: A novel identification method is developed which identifies the accumulated angular momentum (AAM) of spinning reaction wheels (RWs) of an uncooperative satellite captured by a robotic servicer. In contrast to other methods that treat captured satellite’s RWs as non-spinning, the developed method provides simultaneously accurate estimates of the AAM of the captured satellite’s RWs and of the inertial parameters of the entire system consisting of the robotic servicer and of the captured satellite. These estimates render the system free-floating dynamics fully identified and available to model-based control. Three-dimensional simulations demonstrate the method’s validity. To show its usefulness, the performance of a model-based controller is evaluated with and without knowledge of the captured satellite’s RWs AAM.

Keywords: Space Robotics and Automation, Contact Modeling, Simulation and Animation
Abstract: This paper presents the modeling and analysis of a novel moving mechanism ``tumbling'' for asteroid exploration. The system actuation is provided by an internal motor and torque wheel; elastic spring-mounted spikes are attached to the perimeter of a circular-shaped robot, protruding normal to the surface and distributed uniformly. Compared with the conventional motion mechanisms, this simple layout enhances the capability of the robot to traverse a diverse microgravity environment. Technical challenges involved in conventional moving mechanisms, such as uncertainty of moving direction and inability to traverse uneven asteroid surfaces, can now be solved. A tumbling locomotion approach demonstrates two beneficial characteristics in this environment. First, tumbling locomotion maintains contact between the rover spikes and the ground. This enables the robot to continually apply control adjustments to realize precise and controlled motion. Second, owing to the nature of the mechanical interaction of the spikes and potential uneven surface protrusions, the robot can traverse uneven surfaces. In this paper, we present the dynamics modeling of the robot and analyze the motion of the robot experimentally and via numerical simulations. The results of this study help establish a moving strategy to approach the desired locations on asteroid surfaces.

Keywords: Space Robotics and Automation, Simulation and Animation, Compliance and Impedance Control
Abstract: In this paper, we propose three novel Hardware-in-the-loop simulation (HLS) methods for a fully-actuated orbital robot in the presence of external interactions using On-Ground Facility Manipulators (OGFM). In particular, a fixed-base and a vehicle-driven manipulator are considered in the analyses. The key idea is to describe the orbital robot's dynamics using the Lagrange-Poincare (LP) equations, which reveal a block-diagonalized inertia. The resulting advantage is that noisy joint acceleration/torque measurements are avoided in the computation of the spacecraft motion due to manipulator interaction even while considering external forces. The proposed methods are a consequence of two facilitating theorems, which are proved herein. These theorems result in two actuation maps between the simulated orbital robot and the physical OGFM. The chief advantage of the proposed methods is physical consistency without level-set assumptions on the momentum map. We validate this through experiments on both types of OGFM in the presence of external forces. Finally, the effectiveness of our approach is validated through a HLS of a fully-actuated orbital robot while interacting with the environment.

Keywords: Space Robotics and Automation, SLAM, Visual Tracking
Abstract: With the booming development in aerospace technology,the video satellite has gradually emerged as a new Earth observation method, which observes the live phenomena on the ground by video shooting and opens a “dynamic” era of remote sensing. Thus, some new techniques are needed, especially the near-real-time tracking and positioning algorithm for ground moving targets. However, many researches only extract pixel-level trajectories in the post-processed video product, resulting in fairly limited applications. We regard the video satellite as a robot flying in space and adopt the SLAM framework for the positioning of ground moving targets. We design our framework based on the representative ORB-SLAM and make improvements mainly in feature extraction, satellite pose estimation, moving target tracking, and positioning. We install GPS-RTK (Real-time Kinematic) devices on a fishing boat to measure its ground truth and use the Zhuhai-1 video satellite to observe it simultaneously. We conduct experiments on this video and demonstrate that our framework can provide the geolocation of the moving target in satellite videos.

Keywords: Space Robotics and Automation, Mapping, SLAM
Abstract: The ability to recognize previously mapped locations is an essential feature for autonomous systems.Unstructured planetary-like environments pose a major challenge to these systems due to the similarity of the terrain. As a result, the ambiguity of the visual appearance makes state-of-the-art visual place recognition approaches less effective than in urban or man-made environments. This paper presents a method to solve the loop closure problem using only spatial information. The key idea is to use a novel continuous and probabilistic representations of terrain elevation maps. Given 3D point clouds of the environment, the proposed approach exploits Gaussian Process (GP) regression with linear operators to generate continuous gradient maps of the terrain elevation information. Traditional image registration techniques are then used to search for potential matches. Loop closures are verified by leveraging both the spatial characteristic of the elevation maps (SE(2) registration) and the probabilistic nature of the GP representation. A submap-based localization and mapping framework is used to demonstrate the validity of the proposed approach. The performance of this pipeline is evaluated and benchmarked using real data from a rover that is equipped with a stereo camera and navigates in challenging, unstructured planetary-like environments in Morocco and on Mt. Etna.

Keywords: Space Robotics and Automation, Force Control, Visual Servoing
Abstract: We propose a system for visually monitoring and servoing the cutting of a multi-layer insulation (MLI) blanket that covers the envelope of satellites and spacecraft. The main contributions of this paper are: 1) to propose a model for relating visual features describing the engagement depth of the blade to the force exerted on the MLI blanket by the cutting tool, 2) a blade design and algorithm to reliably detect the engagement depth of the blade inside the MLI, and 3) a servoing mechanism to achieve the desired applied force by monitoring the engagement depth. We present results that validate these contributions by comparing forces estimated from visual feedback to measured forces at the blade. We also demonstrate the robustness of the blade design and vision processing under challenging conditions.

Keywords: Space Robotics and Automation, Robotics in Hazardous Fields, Autonomous Vehicle Navigation
Abstract: This work aims at developing an efficient path planning algorithm for the driving objective of a Martian day(sol) that can take into account terrain information for application to the proposed Mars Sample Return (MSR) mission. To prepare the planning process for one sol (i.e., with a limited time allocated to driving), a map of expected rover velocity over a chosen area is constructed, obtained by combining terrain classes, rock abundance and slope at that location.The planning phase starts offline by computing several paths that can be traversed in one sol (i.e., a few hours), which will later provide suitable options to the rover if replanning is necessary due to unexpected mobility difficulties. Online, the rover gains information about its environment as it drives (via slip monitoring and/or instrument deployment) and updates the map if major discrepancies are found. If an update is made, the remaining driving time along the different options is recalculated and the most efficient path is chosen. The online process is repeated until the rover has reached its daily goal.When simulated on different maps of expected rover speed at Gusev Crater, Mars, the algorithm correctly captured changes of terrain initially not mapped, and rerouted the rover to a more efficient path only when necessary, in which case it effectively complied with the time constraint to reach the goal.

Keywords: Space Robotics and Automation, Multi-Modal Perception
Abstract: Terrain classification is critically important for Mars rovers, which rely on it for planning and autonomous navigation. On-board terrain classification using visual information has limitations, and is sensitive to illumination conditions. Classification can be improved if one fuses visual imagery with additional infrared (IR) imagery of the scene, yet unfortunately there are no IR image sensors on the current Mars rovers. A virtual IR sensor, estimating IR from RGB imagery using deep learning, was proposed in the context of a MU-Net architecture. However, virtual IR estimation was limited by the fact that slope angle variations induce temperature differences within the same terrain. This paper removes this limitation, giving good IR estimates and as a consequence improving terrain classification by including the additional angle from the surface normal to the Sun and the measurement of solar radiation. The estimates are also useful when estimating thermal inertia, which can enhance slip prediction and small rock density estimation. Our approach is demonstrated in two applications. We collected a new data set to verify the effectiveness of the proposed approach and show its benefit by applying to the two applications.

Keywords: Space Robotics and Automation, Field Robots
Abstract: This paper presents a novel approach to sampling subsurface asteroidal regolith under severe time constraints. Sampling operations that must be completed within a few hours require techniques that can manage subsurface obstructions that may be encountered. The large uncertainties due to our lack of knowledge of regolith properties also make sampling difficult. To aid in managing these challenges, machine learning-based detection methods using tactile feedback can detect the presence of rocks deeper than the length of the probe, ensuring reliable sampling in unobstructed areas. In addition, given the variability of soil hardness and the short time available, a corer shooting mechanism has been developed that uses a special shape-memory alloy to collect regolith in about a minute. Experiments on subsurface obstacle detection and shooting-corer ejection tests were conducted to demonstrate the functionality of this approach.

Keywords: Space Robotics and Automation, Product Design, Development and Prototyping, Wheeled Robots
Abstract: Modular robotic systems with self-repair or self-replication capabilities have been presented as a robust, low cost solution to extraterrestrial or Arctic exploration. This paper explores using ice as the sole structure element to build robots. The ice allows for increased flexibility in the system design, enabling the robotic structure to be designed and built post deployment, after tasks and terrain obstacles have been better identified and analyzed. However, ice presents many difficulties in manufacturing. The authors explore a structure driven approach to examine compatible manufacturing processes with an emphasis on conserving process energies. The energy analysis shows the optimal manufacturing technique depends on the volume of the final part relative to the volume of material that must be removed. Based on experiments three general design principles are presented. A mobile robotic platform made from ice is presented as a proof of concept and first demonstration.

Keywords: Space Robotics and Automation, Field Robots, Whole-Body Motion Planning and Control
Abstract: The ocean world Europa is a prime target for exploration given its potential habitability. We propose a mobile platform that is capable of autonomously traversing tens of meters to visit multiple sites of interest on a Europan analogue surface. Due to the topology of Europan terrain being largely unknown, it is desired that this mobility system traverse a large variety of terrain types. The mobility system should also be capable of crossing unstructured terrain in an autonomous manner given the communications limitations between Earth and Europa.

A wheel-on-limb robotic rover is presented that may actively conform to terrain features up to 1.5 wheel diameters tall while driving. The robot uses a sampling-based motion planner to generate paths that leverage its unique locomotive capabilities. The planner assesses terrain hazards and wheel workspace limits as obstacles. It may also select a mobility mode based on predicted energy usage and the need for limb articulation on the terrain being traversed. This autonomous mobility was evaluated on chaotic salt-evaporite terrain found in Death Valley, CA, an analogue to the Europan surface. Over the course of 38 trials, the rover autonomously traversed 435 m of extreme terrain while maintaining a rate of 0.64 traverse ending failures for every 10 m driven.

Keywords: Space Robotics and Automation, Cooperating Robots, Robotics in Construction
Abstract: To allow for the construction of large space structures to support future space endeavors, autonomous robotic solutions would serve to reduce cost and risk of human extravehicular activity (EVA). Practicality of autonomous assembly requires both theoretical and algorithmic advances, and hardware experimentation across a spectrum of technological readiness levels. Analysis of hardware experiments provides novel insights not readily apparent in simulations alone, which serves to inform future developments. This paper describes analysis and insights gained from an autonomous assembly experiment consisting of a dexterous manipulator, a gross positioning serial arm, and a 1 degree of freedom (DOF) turntable to facilitate the assembly and deployment of a solar array mockup. This experiment combined state estimation in an uncertain environment with contact-heavy robot operations such as grasping, self-reconfiguring, joining, and deploying. Insights gained are presented here due to their applicability to other field-based manipulation tasks by teams of robots.

Keywords: Space Robotics and Automation, Multi-Robot Systems, Autonomous Agents
Abstract: Teams of mobile robots will play a crucial role in future missions to explore the surfaces of extraterrestrial bodies. Setting up infrastructure and taking scientific samples are expensive tasks when operating in distant, challenging, and unknown environments. In contrast to current single-robot space missions, future heterogeneous robotic teams will increase efficiency via enhanced autonomy and parallelization, improve robustness via functional redundancy, as well as benefit from complementary capabilities of the individual robots. In this article, we present our heterogeneous robotic team, consisting of flying and driving robots that we plan to deploy on scientific sampling demonstration missions at a Moon-analogue site on Mt. Etna, Sicily, Italy in 2021 as part of the ARCHES project. We describe the robots' individual capabilities and their roles in two mission scenarios. We then present components and experiments on important tasks therein: automated task planning, high-level mission control, spectral rock analysis, radio-based localization, collaborative multi-robot 6D SLAM in Moon-analogue and Mars-like scenarios, and demonstrations of autonomous sample return.

Keywords: Space Robotics and Automation, Path Planning for Multiple Mobile Robots or Agents, Motion and Path Planning
Abstract: This paper proposes a routing framework for heterogeneous multi-robot teams in exploration tasks. The proposed framework deals with a combinatorial optimization problem and provides a new solving algorithm, for Generalized Team Orienteering Problem (GTOP). In this paper, a route optimization problem is formulated for a heterogeneous multi-robot system. A novel problem solver is also proposed based on self-organizing map. The proposed framework has a strong advantage in its scalability because the processing time is independent from the number of robots and the heterogeneity of the team. The validity of the proposed framework is evaluated in the exploration and mapping tasks by heterogeneous robot team with overlapping abilities. The simulation results show the effectiveness of the proposed framework and how it outperforms the conventional greedy exploration scheme.

Keywords: Deep Learning for Visual Perception, Computer Vision for Automation, Autonomous Vehicle Navigation
Abstract: Autonomous racing provides the opportunity to test safety-critical perception pipelines at their limit. This paper describes the practical challenges and solutions to applying state-of-the-art computer vision algorithms to build a low-latency, high-accuracy perception system for DUT18 Driverless (DUT18D), a 4WD electric race car with podium finishes at all Formula Driverless competitions for which it raced. The key components of DUT18D include YOLOv3-based object detection, pose estimation, and time synchronization on its dual stereovision/monovision camera setup. We highlight modifications required to adapt perception CNNs to racing domains, improvements to loss functions used for pose estimation, and methodologies for sub-microsecond camera synchronization among other improvements. We perform a thorough experimental evaluation of the system, demonstrating its accuracy and low-latency in real-world racing scenarios.

Keywords: Autonomous Vehicle Navigation, Big Data in Robotics and Automation, Model Learning for Control
Abstract: Recent works have proved that combining spatial and temporal visual cues can significantly improve the performance of various vision-based robotic systems. However, for the ultrasonic sensors used in most robotic tasks (e.g. collision avoidance, localization and navigation), there is a lack of benchmark ultrasonic datasets that consist of spatial and temporal data to verify the usability of spatial and temporal ultrasonic cues. In this paper, we are the first to propose a Spatio-Temporal Ultrasonic Dataset (STUD), which aims to develop the ability of ultrasonic sensors by mining spatial and temporal information from multiple ultrasonic measurements. In particular, we first propose a novel Spatio-Temporal (ST) ultrasonic data gathering scheme, in which an innovatory data instance is designed. Besides, part of the data in the STUD is collected in a robot simulator, in which a well-designed corridor map is utilized to increase data diversity. Then a selection algorithm is proposed to find a proper length of data sequences to obtain the best description of the navigation environments. Finally, we present an end-to-end learning benchmark model that learns driving policies by extracting spatial and temporal ultrasonic cues from the STUD. With the help of our STUD and this benchmark model, more powerful deep neural networks can be trained for addressing the tasks of indoor navigation or motion planning of mobile robots, which is unachievable by simply using the existing ultrasonic datasets. Comparison experiments verified the effectiveness of spatial and temporal ultrasonic cues for the driving policy learning.

Keywords: AI-Based Methods, Autonomous Vehicle Navigation, Social Human-Robot Interaction
Abstract: Drivers and other road users often encounter situations (e.g., arriving at an intersection simultaneously) where priority is ambiguous or unclear but must be resolved via communication to reach agreement. This poses a challenge for autonomous vehicles, for which no direct means for expressing intent and acknowledgment has yet been established. This paper contributes a minimal model to manage ambiguity and produce actions that are expressive and encode aspects of intent. Specifically, intent is treated as a latent variable, communicated implicitly through a partially observable Markov decision process (POMDP). We validate the model in a simple setting: a simulation of a prototypical crossing with a vehicle and one pedestrian at an unsignalized intersection. We further report use of our self-driving Ford Lincoln MKZ platform, through which we conducted experimental trials of the method involving real-time interaction. The experiment shows the method achieves safe and efficient navigation.