Keywords: Engineering for Robotic Systems, Hardware-Software Integration in Robotics, Embedded Systems for Robotic and Automation
Abstract: HILARE stands for: Heuristiques Intégrées aux Logiciels et aux Automatismes dans un Robot Evolutif. The HILARE project started by the end of 1977 at LAAS (Laboratoire d'Automatique et d'Analyse des Systèmes at this time) under the leadership of Georges Giralt. The project's goal was to perform general research in robotics and robot perception and planning. The video features HILARE robot and delivers explanations.

This work is part of SH-ROB project ”Historical sociology of a composite discipline: Robotics within LAAS (1967-2005)” for collecting and studying archives from the beginning of robotics at LAAS. Contact us if your are interested by this project or have information to share. This project has received financial support from the CNRS through the MITI interdisciplinary programs.

Keywords: Reactive and Sensor-Based Planning, Perception for Grasping and Manipulation, Deep Learning in Grasping and Manipulation
Abstract: In this paper, we present a novel approach for planning an object's Next Best Views (NBV) so that a depth camera can collect the object's surface point cloud and reconstruct its 3D model with a small number of consequent views. Our focus is especially on thin and curved metal plates, and we use a robot manipulator and an externally installed stationary depth sensor as the experimental system. The targeted objects have shiny and flat surfaces, which leads to noisy point cloud data and low guidance in the surface normal for completion. To overcome these challenges, we propose using a Point cloud Completion Network (PCN) to find heuristics for NBV (when the depth sensor is mounted on a robot's end flange) or Next Best robot Configuration (NBC, when the depth sensor is externally fixed) optimization. Unlike previous methods, our approach predicts NBV by considering a holistic view of the object predicted by neural networks, which is not limited by the local information captured by the sensors and is, therefore, robust to deficiencies in known point cloud data and normal. We conducted simulation and real-world experiments to evaluate the proposed method's performance. Results show that the proposed method efficiently solves the NBV problems and can satisfactorily model thin and curved metal plates.

Keywords: Model Learning for Control, Robust/Adaptive Control of Robotic Systems, Learning and Adaptive Systems, Industrial Robots
Abstract: Model-based reinforcement learning and control have demonstrated great potential in various sequential decision mak- ing problem domains, including in robotics settings. However, real-world robotics systems often present challenges that limit the applicability of those methods. In particular, we note two prob- lems that jointly happen in many industrial systems: first, irregu- lar/asynchronous observations and actions and, second, dramatic changes in environment dynamics from an episode to another (e.g., varying payload inertial properties). We propose a general frame- work that overcomes those difficulties by meta-learning adaptive dynamics models for continuous-time prediction and control. The proposed approach is task-agnostic and can be adapted to new tasks in a straight-forward manner. We present evaluations in two different robot simulations and on a real industrial robot.

Keywords: Reactive and Sensor-Based Planning, Perception for Grasping and Manipulation, Task and Motion Planning
Abstract: Efficiency and reliability are critical in robotic bin-picking as they directly impact the productivity of automated industrial processes. However, traditional approaches, demanding static objects and fixed collisions, lead to deployment limitations, operational inefficiencies,

and process unreliability. This paper introduces a Dynamic Bin-Picking Framework (DBPF) that challenges traditional static assumptions. The DBPF endows the robot with the reactivity to pick multiple moving arbitrary objects while avoiding dynamic obstacles, such as the moving bin. Combined with scene-level pose generation, the proposed pose selection metric leverages the Tendency-Aware Manipulability Network optimizing suction pose determination. Heuristic task-specific designs like velocity-matching, dynamic obstacle avoidance, and the resight policy, enhance the picking success rate and reliability. Empirical experiments demonstrate the importance of these components. Our method achieves an average 84% success rate, surpassing the 60% of the most comparable baseline, crucially, with zero collisions. Further evaluations under diverse dynamic scenarios showcase DBPF's robust performance in dynamic bin-picking. Results suggest that our framework offers a promising solution for efficient and reliable robotic bin-picking under dynamics.

Keywords: Machine Learning for Robot Control, Dexterous Manipulation, Collision Avoidance
Abstract: Deformable linear object (DLO) manipulation in constrained environments with obstacles has received limited investigations due to DLO's complex intrinsic deformation. In this study, we focus on addressing constrained DLO manipulation problems, especially in the context of avoiding cluttered environment obstacles. Unlike sampling-based planners, which struggle with the high-dimensional state space or require modifications to ensure DLO's kinematic feasibility, we propose a novel obstacle avoidance approach by combining a learning-based predictive control method and an efficient control-theoretic technique. Specifically, we utilize a learning-based model predictive control (MPC) with an attention-based global deformation model to generate low-dimensional reference actions that inherently align with DLO's physics. The attention-based model outperforms multilayer perceptron and bi-directional long short-term memory models by capturing contextual relationships among feature points on DLOs. To mitigate the inevitable modeling errors, a safety-critical filter is designed based on the control barrier function (CBF) principle. An online local linear model is employed in the filter to steer clear of obstacles in close proximity. The proposed approach was validated with extensive simulations and physical experiments on constrained DLO manipulation tasks.

Keywords: Telerobotics and Teleoperation, Space Robotics and Automation, Assembly
Abstract: Virtual Fixtures (VFs) provide haptic feedback for teleoperation, typically requiring distinct input modalities for different phases of a task. This often results in vision- and position-based fixtures. Vision-based fixtures, particularly, require the handling of visual uncertainty, as well as target appearance/disappearance for increased flexibility. This creates the need for principled ways to add/remove fixtures, in addition to uncertainty-aware assistance regulation. Moreover, the arbitration of different modalities plays a crucial role in providing an optimal feedback to the user throughout the task. In this paper, we propose a Mixture of Experts (MoE) model that synthesizes visual servoing fixtures, elegantly handling full pose detection uncertainties and teleoperation goals in a unified framework. An arbitration function combining multiple vision-based fixtures arises naturally from the MoE formulation, leveraging uncertainties to modulate fixture stiffness and thus the degree of assistance. The resulting visual servoing fixtures are then fused with position-based fixtures using a Product of Experts (PoE) approach, achieving guidance throughout the complete workspace. Our results indicate that this approach not only permits human operators to accurately insert printed circuit boards (PCBs) but also offers added flexibility and retains the performance level of a baseline with carefully handtuned VFs, without requiring the manual creation of VFs for individual connectors. An exemplary video showcasing our method is available at: https://youtu.be/6BDB3g0QyFg

Keywords: Data Sets for Robotic Vision, Object Detection, Segmentation and Categorization, Performance Evaluation and Benchmarking, Service Robots
Abstract: We introduce the UT Campus Object Dataset (CODa), a mobile robot egocentric perception dataset collected on the University of Texas Austin Campus. Our dataset contains 8.5 hours of multimodal sensor data: synchronized data from a 128-channel 3D LiDAR, two stereo RGB cameras; RGB-D cameras, and a 9-DOF IMU. CODa contains 58 minutes of ground-truth annotations with 1.3 million 3D bounding boxes, 5000 frames of 3D semantic annotations for terrain, and pseudo-ground truth localization. Using CODa, we empirically show that: 1) 3D object detection performance in urban settings is significantly higher when trained using CODa compared to existing datasets, 2) sensor-specific fine-tuning improves object detection accuracy, and 3) pre-training on CODa improves cross-dataset object detection performance in urban settings compared to pre-training on AV datasets. We publicly release CODa on the Texas Data Repository, benchmarks for 3D object detection and semantic segmentation, pre-trained 3D perception models, and a dataset development package. We expect CODa to be a valuable dataset for research in egocentric 3D perception and planning for autonomous navigation in urban environments.

Keywords: Computational Geometry, Computer Vision for Automation, Perception for Grasping and Manipulation
Abstract: The curve skeleton is known to geometric modelling and computer graphics communities as one of the shape descriptors which intuitively indicates the topological properties of the objects. In recent years, studies have also suggested the potential of applying curve skeletons to assist robotic reasoning and planning. However, the raw scanned point cloud model is typically incomplete and noisy. Besides, dealing with a large point cloud is also computationally inefficient. Focusing on the curve skeletonisation of incomplete and poorly distributed point clouds of objects, an efficient geometric Laplacian-based skeletonisation framework (GLSkeleton) is proposed in this work. We also present the computational efficiency of the introduced local reduction strategy (LPR) approach without sacrificing the main topological structure. Comprehensive experiments have been conducted to benchmark performance using an open-source dataset, and they have demonstrated a significant improvement in both contraction and overall skeletonisation computational speed.

Keywords: Sensor Fusion, Intelligent Transportation Systems, RGB-D Perception
Abstract: Precise and robust extrinsic parameter calibration is fundamental for LiDAR-camera multi-modal sensing applications. However, most existing methods assume that sensors have the same orientation, limiting their effectiveness in feature extraction and feature alignment from different angle of view in multi-angle sensing scenarios. Moreover, the calibration accuracy of existing methods is insufficient for high-performance applications. To address these limitations, we propose a novel automatic extrinsic parameter calibration method utilizing a spherical target. We propose the Curvature Consistency Spherical Detection (CCSD) algorithm for LiDAR point cloud sphere recognition. The CCSD leverages the sphere's structural attributes, enabling robust detection against noise and partial occlusion. To improve camera sphere detection, we present an enhanced ellipse detection technique and compensate the eccentricity error arising from spherical projection based on the principle of perspective transformation. Extensive simulations and real-world experiments demonstrate the proposed method's superiority in accuracy and practicality over state-of-the-art (SOTA) methods.

Keywords: Recognition, SLAM, Autonomous Vehicle Navigation
Abstract: The paper introduces a novel Visual Place Recognition (VPR) method called Lightweight Pooling-centric Saliency-aware VPR (LPS-VPR), a high-performance VPR method capable of exploiting saliency information without computational burden. The key contribution of the method is a pooling-based saliency encoder (PSE) that efficiently extracts and integrates local context saliency cues into the image embedding, avoiding the computational complexity of convolution and transformer attention module. The method employs a multi-level feature pyramid fusion with a cascaded backtracking structure, merging multi-scale embedding information from different levels of the backbone network with negligible computational demand. The LPS-VPR method demonstrates superior performance on mainstream VPR benchmarks, showcasing state-of-the-art runtime efficiency and detailed ablation studies highlight the effectiveness of each LPS-VPR component. Extended visual demonstrations further underscore the effectiveness and superiority of LPS-VPR, making it a promising solution for real-world large-scale VPR applications.

Keywords: Aerial Systems: Perception and Autonomy, Design and Human Factors, Emotional Robotics
Abstract: Robots are becoming increasingly important for supporting people's well-being, such as their use for emotional support, which leverages their physical embodiment to enhance companionship. Recent research works have opened avenues for aerial robots (i.e., drones) to serve as emotional support providers. This paper investigates such social drones' visual qualities for emotional support by exploring five components: color, flight mechanism, body shape, material, and interaction modality. By considering key visual qualities, this work contributes to the understanding of how drones should look and interact with users as emotional support companions, opening new avenues for future research in human-drone interaction research and in the provision of emotional support mechanisms by technologies.

Keywords: Data Sets for Robot Learning, Software Tools for Benchmarking and Reproducibility, Bimanual Manipulation
Abstract: Realistic physics engines play a crucial role for learning to manipulate deformable objects such as garments in simulation. By doing so, researchers can circumvent challenges such as sensing the deformation of the object in the real-world. In spite of the extensive use of simulations for this task, few works have evaluated the reality gap between deformable object simulators and real-world data. We present a benchmark dataset to evaluate the sim-to-real gap in cloth manipulation. The dataset is collected by performing a dynamic as well as a quasi-static cloth manipulation task involving contact with a rigid table. We use the dataset to evaluate the reality gap, computational time, and simulation stability of four popular deformable object simulators: MuJoCo, Bullet, Flex, and SOFA. Additionally, we discuss the benefits and drawbacks of each simulator. The benchmark dataset is open-source. Supplementary material, videos, and code, can be found at https://sites.google.com/view/cloth-sim2real-benchmark.

Keywords: Sensor Fusion, Kinematics, SLAM
Abstract: Continuous and reliable ego-velocity information is significant for high-performance motion control and planning in a variety of robotic tasks, such as autonomous navigation and exploration. While linear velocities as first-order kinematics can be simultaneously estimated with other states or explicitly obtained by differentiation from positions in ego-motion estimators such as odometers, the high coupling leads to instability and even failures when estimators degenerate. To this end, we present River: an accurate and continuous linear velocity estimator that efficiently fuses high-frequency inertial and radar target measurements based on continuous-time optimization. Specifically, a dynamic initialization procedure is first performed to rigorously recover the initials of states, followed by batch estimations, where the velocity and rotation B-splines would be optimized incrementally to provide continuous body-frame velocity estimates. Results from both simulated and real-world experiments demonstrate that River is capable of high accuracy, repeatability, and consistency for ego-velocity estimation. We open-source our implementations at (https://github.com/Unsigned-Long/River) to benefit the research community.

Keywords: SLAM, Sensor Fusion, Visual-Inertial SLAM
Abstract: Visual simultaneous localization and mapping (VSLAM) has broad applications, with state-of-the-art methods leveraging deep neural networks for better robustness and applicability. However, there is a lack of research in fusing these learning-based methods with multi-sensor information, which could be indispensable to push related applications to large-scale and complex scenarios. In this paper, we tightly integrate the trainable deep dense bundle adjustment (DBA) with multi-sensor information through a factor graph. In the framework, recurrent optical flow and DBA are performed among sequential images. The Hessian information derived from DBA is fed into a generic factor graph for multi-sensor fusion, which employs a sliding window and supports probabilistic marginalization. A pipeline for visual-inertial integration is firstly developed, which provides the minimum ability of metric-scale localization and mapping. Furthermore, other sensors (e.g., global navigation satellite system) are integrated for driftless and geo-referencing functionality. Extensive tests are conducted on both public datasets and selfcollected datasets. The results validate the superior localization performance of our approach, which enables real-time dense mapping in large-scale environments. The code has been made open-source (https://github.com/GREAT-WHU/DBA-Fusion).

Keywords: Multi-Robot Systems, Formal Methods in Robotics and Automation, Collision Avoidance
Abstract: Model-based development allows faster prototyping, earlier experimentation and validation of design intents. For a multi-agent system with complex asynchronous interactions and concurrency, model checking is a stronger and automated mechanism for verifying desired system properties. Timed Rebeca is an actor-based modelling language supporting both concurrent and time semantics, accompanied with a model-checking compiler. These capabilities allow using Timed Rebeca to correctly model ROS2 node topography, recurring physical signals, motion primitives and other timed and time-convertible behaviours. In this work we modelled a multiple autonomous mobile robots system in Timed Rebeca then developed corresponding ROS2 simulation code, ensured semantic synchronization between the model and the code, and set up experiments to use the model for revealing problems that do not always show up in simulation and verifying different properties (goal reachability, collision freedom, deadlock freedom, arrival times). The biggest challenges lie in abstracting complex information in robotics, bridging the gap between a discrete model and a continuous system and minimizing the state space, while maintaining the model's accuracy. We devised different discretization strategies for different kinds of information, identifying the 'enough' thresholds of abstraction, and applying efficient optimization techniques to boost computations to reduce model checking time. The benefits of formal verification are clear, however applying them in practice is difficult; our approach is a well explained, easy to understand, working solution in a realistic context.

Keywords: Whole-Body Motion Planning and Control, Motion and Path Planning, Humanoid and Bipedal Locomotion
Abstract: Performing acrobatic maneuvers like dynamic jumping in bipedal robots presents significant challenges in terms of actuation, motion planning, and control. Traditional approaches to these tasks often simplify dynamics to enhance computational efficiency, potentially overlooking critical factors such as the control of centroidal angular momentum (CAM) and the variability of centroidal composite rigid body inertia (CCRBI). This paper introduces a novel integrated dynamic planning and control framework, termed centroidal dynamics model-based model predictive control (CDM-MPC), designed for robust jumping control that fully considers centroidal momentum and non-constant CCRBI. The framework comprises an optimization-based kinodynamic motion planner and an MPC controller for real-time trajectory tracking and replanning. Additionally, a centroidal momentum-based inverse kinematics (IK) solver and a landing heuristic controller are developed to ensure stability during high-impact landings. The efficacy of the CDM-MPC framework is validated through extensive testing on the full-sized humanoid robot KUAVO in both simulations and experiments.

Keywords: Virtual Reality and Interfaces, Transfer Learning, Robot Companions
Abstract: Understanding and respecting personal space preferences is important for many service robots, especially for some user populations, such as older adults. This work introduces and evaluates a novel personalized context-aware method for modeling users’ proxemic preferences during human-robot interactions. The method involves an engaging, interactive augmented reality interface for collecting user preference data in the form of user-preferred robot distances. The data were sampled using an Active Transfer Learning (ATL) approach that fine-tuned a specialized deep learning model. The method was implemented on a complete robot system and evaluated in two user studies: 1) a convenience population study (N = 24) to validate the efficacy of the ATL approach and 2) a user study involving older adults (N = 15) to assess the system’s usability for the target user population. Results from the convenience population user study demonstrate a significant enhancement in model performance following fine-tuning with samples obtained through ATL: on average, the error in testing decreased by 26.97% after fine-tuning. Insights gathered from semi-structured interviews with older adults in the second us

Keywords: Dexterous Manipulation, Model Learning for Control, Deep Learning in Robotics and Automation, Trajectory Prediction
Abstract: Accurate measuring and modeling of dynamic robot manipulation (e.g. tossing and catching) is particularly challenging, due to the high-speed robot motions and highly dynamic robot-object interactions happening in very short distances and times. In this work, we investigate whether using solely the on-board proprioceptive sensory modalities can effectively capture and characterize dynamic manipulation processes. We present an object-agnostic strategy to learn the robot toss dynamics of arbitrary unknown objects from the spatio-temporal variations of robot toss movements and wrist-F/T observations. Then we propose TossNet, an end-to-end formulation that jointly measures the robot toss dynamics and predicts the resulting flying trajectories of the tossed objects. Experimental results demonstrate that our methods can accurately model the robot toss dynamics of both seen and unseen objects, and predict their flying trajectories with superior prediction accuracy in nearly real time. Case studies show that TossNet allows various robot platforms to easily perform challenging tossing-centric robot applications, such as blind juggling and high-precise robot pitching.

Keywords: Data Sets for Robotic Vision, Object Detection, Segmentation and Categorization, Intelligent Transportation Systems
Abstract: LiDAR is an essential sensor for autonomous driving by collecting precise geometric information regarding a scene. Exploiting this information for perception is interesting as the amount of available data increases.

As the performance of various LiDAR perception tasks has improved, generalizations to new environments and sensors has emerged to test these optimized models in real-world conditions.

Unfortunately, the various annotation strategies of data providers complicate the computation of cross-domain performances.

This paper provides a novel dataset, ParisLuco3D, specifically designed for cross-domain evaluation to make it easier to evaluate the performance utilizing various source datasets. Alongside the dataset, online benchmarks for LiDAR semantic segmentation, LiDAR object detection, and LiDAR tracking are provided to ensure a fair comparison across methods.

The ParisLuco3D dataset, evaluation scripts, and links to benchmarks can be found at the website : https://npm3d.fr/parisluco3d

Keywords: Sensor Networks, Probability and Statistical Methods, Optimization and Optimal Control
Abstract: This paper examines the spatial coverage optimization problem for multiple sensors in a known convex environment, where the coverage service of each sensor is heterogeneous and anisotropic. We introduce the Stein Coverage algorithm, a distribution-matching coverage approach

that aims to place sensors at positions and orientations such that their collective coverage distribution is as close as possible to the event distribution. To select the most important representative points from the coverage event distribution, Stein Coverage utilizes the Stein

Variational Gradient Descent (SVGD), a deterministic sampling method from the variational inference literature. An innovation in our work is

the introduction of a repulsive force between the samples in the SVGD algorithm to spread the samples and avoid footprint overlap for the deployed sensors. After pinpointing the points of interest for deployment, Stein Coverage solves the multisensor assignment problem using a bipartite optimal matching process. Simulations demonstrate the

advantages of the Stein Coverage method compared to conventional Voronoi partitioning multisensor deployment methods.

Keywords: Computational Geometry, Computer Vision for Automation, Perception for Grasping and Manipulation
Abstract: In recent years, there has been notable interest in skeletonization methods for 3D object models, driven by their broad applicability in fields such as computer graphics and robotics. However, existing studies have lacked a clear quantitative standard for evaluating skeletonization quality. This paper extends prior research on point cloud skeletonization to examine the intrinsic properties of the process across diverse object shapes, aiming to provide intuitive insights into the quality of resulting skeletons. Additionally, we propose a novel concept of stable convergence of contraction, leveraging distributions of geometric curvature and vectorial normal changes.

Keywords: Aerial Systems: Applications, Aerial Systems: Mechanics and Control, Aerial Systems: Perception and Autonomy
Abstract: My team at the University of Southern Denmark has been pioneering the field of self-recharging drones since 2017. We have developed autonomous drone systems designed for extended operations near powerlines. These drones are equipped with a robust perception and navigation system, enabling them to identify powerlines and approach them for landing. A unique feature of our drones is their self-recharging capability. They accomplish this by landing on powerlines and utilizing a passively actuated gripping mechanism to secure themselves to the powerline cable. A control circuit then manipulates the magnetic field within a split-core current transformer, ensuring sufficient holding force and facilitating battery recharging. We have successfully demonstrated these drones’ ability to perform several hours of fully autonomous operations, which include multiple cycles of flight, landing, recharging, and takeoff. This achievement underscores the potential for virtually unlimited operational endurance. We were the first in the world to recharge a drone on powerlines. Furthermore, we have designed and implemented an electromagnetic shielding system with onboard electronics to mitigate interference from high-voltage lines and protect the drone’s electronics. We have also integrated AI techniques onboard the drone for fault detection and navigational algorithms, marking a significant advancement in the field of powerline inspections. This work represents a major step forward in the autonomous inspection technologies.

Keywords: Swarm Robotics
Abstract: Robot swarms provide a promising means for inspection of infrastructure. In a previous work, we introduced an experimental setup around a swarm of miniaturized vibration- sensing robots that complete a binary decision task over a 2D surface. There, we used two previously developed and proposed a new implementation of a Bayesian decision-making algorithm and showed that our proposed algorithm compels the swarm to make decisions at an accelerated rate. In this work, we study how the swarm size impacts the decision time in our algorithm.

Keywords: Logistics, Networked Robots, Intelligent Transportation Systems
Abstract: Future 6G communication systems must be tested under near real-life conditions to validate their performance beyond theoretical and simulated scenarios. The 3D-automated intralogistics environment of the Fraunhofer IML and FLW chair of the Technical University (TU) Dortmund provide a 6G validation environment for performing reproducible and cost-effective tests with unique features such as ground and aerial vehicles with different scaling options and high dynamics. In cooperation with the TU Communication Networks Institute, we have developed a system to show either real world scales (1:1) or higher scales by adjusting the mobile communication parameters to emulate bigger ranges and harsher environments. To emulate shadowing effects and reduced signal strengths, we introduce reflective obstacles and attenuation pads. A 3D-aerial cable robot bridges the gaps in the radio communication and a Digital Network Twin simulates the test bed, containing the robot platforms, obstacles, and the radio communication, in real-time. The digital twin core is used as the main controller of the laser projection system. The virtual environment emulates the warehouse shelves with specific path loss modeling. Here, the communication link represents the Reference Signal Received Power between devices, where a red line represents obstructed path, and a green line illustrates a good signal.

Keywords: Force and Tactile Sensing, Simulation and Animation, Soft Robot Materials and Design
Abstract: Camera-based tactile sensors have shown great promise in dexterous manipulation and perception of object properties. However, the design process for vision-based tactile sensors (VBTS) is largely driven by domain experts through a trial-and-error process using real-world prototypes. In this work, we formulate the design process as a systematic and objective-driven design problem using physically accurate optical simulation. We introduce an interactive and easy-touse design toolbox in Blender, OptiSense Studio. The toolbox consists of (1) a set of five modularized widgets to express optical elements with user-definable parameters; (2) a simulation panel for the visualization of tactile images; and (3) an optimization panel for automatic selection of sensor designs. To evaluate our design framework and toolbox, we quickly prototyped and improved a novel VBTS sensor, GelBelt. We fully design and optimize GelBelt in simulation and show the benefits with a real-world prototype.

Keywords: Modeling, Control, and Learning for Soft Robots, Soft Robot Materials and Design
Abstract: Tendon-driven Continuum Robots (TDCRs) are promising candidates for applications in confined spaces due to their unique shape, compliance, and miniaturization capability. Non-parallel tendon routing for TDCRs have shown definite advantages including segments with higher degrees of free- dom, larger workspace and higher dexterity. However, most works have focused on parallel tendons to achieve constant-curvature shapes, which yields analytically simple kinematics but overly restricts the design possibilities. We believe this under-utilization of general tendon rout- ing can be attributed to the lack of a general kinematic model that estimates shape from only tendon geometry and displacements. Cosserat rod-based models are capable of modeling general tendon routing, but they require accurate tendon tension measurements and extensive system identification, hindering their usability for design purposes. Recent attempts in developing a kinematic model are limited to simple scenarios like actuation with a single tendon or tendons on perpendicular planes. Moreover, model formulations are often disconnected from hardware, making designs challenging to build under manufacturing constraints. Our first contribution is a novel design for TDCRs based on a synovial universal joint module, which provides a mechanically discretized and feasible design space. Based on the design, our second contribution is the formulation and evaluation of an optimization-based kinematic model, capable of handling actuation of multiple general routed tendons. Lastly, we present an example application of a TDCR designed for gaited locomotion, demonstrating our method’s potential for an unified model-based design pipeline.

Keywords: Modeling, Control, and Learning for Soft Robots, Visual Servoing, Surgical Robotics: Laparoscopy
Abstract: This study presents a soft robotic instrument for in-situ cancer detection during surgery, focusing on the precise servoing of a projected laser dot in space. Using optical servoing and closed-loop control, the system accurately positions the laser dot for tissue analysis, showcasing the robot's adaptability, even with external disturbances.

Keywords: SLAM, Sensor Fusion, Modeling, Control, and Learning for Soft Robots
Abstract: For robots with low rigidity, determining the robot's state based solely on kinematics is challenging. This is particularly crucial for robots whose entire body is in contact with the environment, as accurate state estimation is essential for environmental interaction. We propose a method for simultaneous articulated robot posture estimation and environmental mapping by integrating data from proximity sensors distributed on the whole body. Our method extends the discrete-time model, typically used for state estimation, to the spatial direction of the articulated structure. Simulations demonstrate that this approach significantly reduces estimation errors.

Keywords: Modeling, Control, and Learning for Soft Robots, Soft Robot Applications
Abstract: Rehabilitation robots and robotic prosthetics are cutting-edge technologies that help individuals recover from disabilities, regain mobility, and provide personalised therapies. Rigid and soft robotic prosthetics designs are continually advancing in technology, allowing for more personalized and efficient solutions for individuals with limb impairments. These advancements are providing hope for those in need of prosthetic solutions to lead more active and independent lives. In this work, we introduce a position controller that has been designed for a soft wrist section of a prosthetic hand called 'Prisma Hand II'. We present both a simulation study and an experimental validation to demonstrate the effectiveness of the controller. A mathematical model of the wrist section derived based on the bending beam theory is used to compute positions of the wrist section and tensions of tendons present in the wrist section. The payload carrying capacity of the wrist section during the motions of hand is analysed through the application of the proposed controller. Experimental validations are conducted on a constructed wrist section attached to the hand for evaluating the effectiveness of the proposed controller in real-time scenarios.

Keywords: Modeling, Control, and Learning for Soft Robots, Compliance and Impedance Control, Soft Robot Applications
Abstract: Our research focuses on compliant control of soft robots using a simulation-based model to select the end effector's dynamics. We use a linearized model to introduce a novel compliance-tuning control structure. It comprises a reference model representing the desired behavior, a force estimator, and a state feedback control law. Through real-world tests, we demonstrate the validity of our approach in terms of position control accuracy and adaptive force response. Our approach achieves less than 0.5mm error in position control and up to 55% force reduction in targeted directions, highlighting its potential for safer and more effective human-robot interactions.

Keywords: Visual-Inertial SLAM
Abstract: Accurate disturbance estimation is crucial for reliable robotic physical interaction. To estimate environmental interference in a low-cost and sensorless way (without force sensor), a variety of tightly-coupled visual inertial external force estimators are proposed in the literature. However, existing solutions may suffer from relatively low-frequency preintegration. In this paper, a novel estimator is designed to overcome this issue via high-frequency instantaneous accelerometer update.

Keywords: SLAM, Wheeled Robots
Abstract: Pose graph optimization (PGO) is a fundamental technique for robot localization. It is typically encoded with a sparse graph. The recent work on the cycle-based PGO reveals the merits of solving PGOs in the graph cycle space, which brings the computation of the minimum cycle basis (MCB) into the robotics community. However, due to batch-MCB's inability to handle the graph topology changes, it is hard for its use in real-time applications. In practice, PGOs are constructed incrementally, which requires us to solve MCB problems in an incremental setting. In this letter, we propose an exact method to solve MCB problem in an incrementally constructed graph. Methodology-wise, we first compute a tight superset called isometric set which contains an MCB, and then apply independence tests to evaporate redundant cycles to form an MCB. Our main contribution is the construction of an effective algorithm to update the superset, namely the isometric set, in an incremental setting. Our update rules preserve the optimality, thus yielding an exact incremental MCB algorithm, which is termed as iMCB. We integrate our iMCB algorithm into the cycle-based PGO, forming the iMCB-PGO approach. We validate the superior performance of our iMCB-PGO on a range of simulated and real-world datasets.

Keywords: Autonomous Vehicle Navigation, Cognitive Modeling, Biologically-Inspired Robots
Abstract: Autonomous Navigation and spatial awareness is one of the fundamental requirements for a cognitive robot assisting in a range of unstructured environments (farms, hospitals). In this paper, we propose a bio-inspired process of swift and cumulative learning of cognitive maps of new environments. Inversely, goal-directed behaviours are generated through structural planning by creating a high-level path and sub-goals to reach the final desired goal, exploiting the learnt internal model. The framework is tested in the scenario of outdoor navigation in a vertical farm using the Clearpath Husky UGV. In future work, we would like to combine spatial awareness with human awareness and validate the framework in more complex settings- farms or hospitals.

Keywords: Tendon/Wire Mechanism, Soft Robot Materials and Design, Force and Tactile Sensing
Abstract: Inspired by caterpillar movements, this paper addresses the challenge of environment exploration by developing a spring-based tendon-driven continuum robot. Using a constant curvature kinematic model, the robot achieves positional control during both bending and compressing motions, with an average error of 7.88 mm. Additionally, equipped with an artificial bristle, the robot demonstrates remarkable capabilities in confined-space inspection tasks. With its compact design and compliant features, this robot system offers an economical upgrade option for commercial robots and promises effective exploration in challenging environments.

Keywords: Mechanism Design, Cellular and Modular Robots, Nonholonomic Mechanisms and Systems
Abstract: This paper presents a new type of mobile rolling robot module as a platform for non-prehensile manipulation and the control challenges involved in accurate control of the robotic system. The designed robot modules include a novel internally actuated magnetic-pendulum coupling mechanism, which presents an interesting control problem involving the frictional/sliding and magnetic effects between each of the modules.

Keywords: Space Robotics and Automation, Dexterous Manipulation, Robotics in Hazardous Fields
Abstract: Given the accumulation of space debris in key orbits around the Earth, robots capable of in-orbit repair, refueling and assembly are crucial for sustainable space exploration. DLR’s contributions to space manipulation began in 1993 with the ROTEX experiment. A small six-axis robotic arm was launched aboard the D2 Space Shuttle mission, where it performed grasping of a free-floating object using various control modes, including teleoperation, shared autonomy and full autonomy modes. This was followed by ROKVISS, a two-joint arm mounted outside the ISS for more than five years. This experiment provided valuable scientific data on the behavior of torque-controlled joints in the harsh space environment. DLR’s latest space robot, the CAESAR robotic arm developed in 2018, is a lightweight, compliant and fully redundant seven-joint manipulator designed for on-orbit operations. Ground testing of the space-destined CAESAR is performed using a cable-driven Motion Suspension System, which minimizes the torques acting on the joints. This allows complex tasks such as docking, latching and grasping to be validated in 3D space. This video details the advances in orbital manipulation made by DLR’s Robotic and Mechatronics Center over the past 30 years, paving the way for the development of robotic technology for space sustainability.

Keywords: Path Planning for Multiple Mobile Robots or Agents, Motion and Path Planning, Multi-Robot Systems
Abstract: This work explores constraint-based search algorithms for multi-agent pathfinding (MAPF) in distributed settings with multi-hop communication. Traditional approaches typically assume centralized network-wide planning without defining individual agents' roles. Our research innovates by distributing the planning process among agents, using constraint-based methods for efficient and safe pathfinding. We present Hierarchical Composition for Multi-Hop Distributed Communication (HCMDC), a framework that strategically determines communication paths within distributed networks along which the MAPF problem is built and solved. This initial investigation lays the groundwork for more nuanced and effective strategies in distributed MAPF, highlighting the potential for improved coordination and communication among agents in complex networks by using constraint-based search.

Keywords: Mechanism Design, Actuation and Joint Mechanisms, Tendon/Wire Mechanism
Abstract: This paper introduces a novel underactuated geometric compliant (UGC) robot and explores the behavior of modules with variable radial stiffness to enhance UGC robot versatility. We design and fabricate semi-rigid geometric joints tailored to specific objectives, validating their stiffness and durability through physical testing. A Gaussian process regression model incorporates joint characteristics, including thickness, facilitating the development of easily 3D-printable prototypes. We present various configurations for constructing the overall UGC module, demonstrating a prototype that dynamically reduces its radius while maintaining structural integrity. This study also discusses the potential, challenges, and limitations of UGC modules, providing insights for future UGC robotics research.

Keywords: Autonomous Vehicle Navigation, Machine Learning for Robot Control, Audio-Visual SLAM
Abstract: This paper outlines the research directions related to the Project "LiSA - Listen, See and Act: fusing audio-video cues to perceive visible and invisible events and develop perception-to-action solutions for autonomous vehicles". In this project, we aim to develop novel approaches to autonomous vehicle navigation that integrate auditory and visual cues using deep learning techniques. Traditional navigation systems, primarily rely on RGB cameras, LiDAR and RADAR sensors. Yet those systems face limitations when dealing with poor light, noisy environments, or weather conditions. Our approach, on the other hand, addresses those challenges by incorporating sound, to leverage the multimodal model's ability to grasp missing information that can be heard but not seen (eg. incoming ambulances) and vice-versa, based on human environment perception. This perception is then meant to be implemented into an end-to-end navigation model.

Keywords: Mechanism Design, Multifingered Hands, Manipulation Planning
Abstract: Traditional robotic hands are typically viewed as end-effectors mounted on robotic arms, dedicated to grasping. Such configuration, however, may limit the functionality of the robotic systems in complex scenarios. This paper presents a novel design of a emph{bimodal} robotic hand: when mounted on a robotic arm, it can grasp objects using both its palm and back; when detached from the arm, it can crawl on the ground to reach objects beyond the original reachable space. The design features fingers that act as legs and an automatic mechanism for seamless role transitions. An optimization framework is proposed to maximize the two capabilities. Experimental results demonstrate the system's effectiveness in both manipulation and mobility tasks.

Keywords: Autonomous Vehicle Navigation, Path Planning for Multiple Mobile Robots or Agents, Model Learning for Control
Abstract: Highway driving represents a critical domain in the advancement of autonomous vehicle technologies due to its complex dynamics and the high consequences of operational failures. Traditionally, model-based methods have approached this task by integrating various stages of decision-making processes, from perception to control. However, this segmentation can introduce compounded errors as inaccuracies in one stage may degrade the performance of subsequent stages, highlighting the need for robust alternatives. Conversely, end-to-end deep reinforcement learning (DRL) methods offer promising avenues by directly mapping sensory inputs to driving actions, thereby bypassing the stage-wise integration challenges. Nevertheless, conventional DRL approaches such as Deep Q-Networks (DQN) often suffer from limited capability in effectively focusing on relevant surrounding Highway Driving Vehicles (HDVs) and exhibit slow convergence rates in learning optimal policies.

To address these deficiencies, our work introduces a novel balanced-attention mechanism that integrates an imaging attention mechanism with a balanced reward function. The imaging attention mechanism prioritizes critical stimuli among adjacent HDVs by selecting emphasis areas with high risks, enhancing situational awareness and decision accuracy. Concurrently, the balanced reward function discounts learning rewards in areas with low risks, thereby emphasizing learning during high-risk situations. This combined approach not only improves the focus on relevant stimuli but also accelerates the convergence velocity of the learning process.

Extensive simulations demonstrate that our adapted DQN, equipped with the balanced-attention mechanism, significantly outperforms benchmark DRL models in terms of quicker convergence speed, fewer collisions, and higher average driving velocity across various driving situations.

Keywords: SLAM, Mapping, Localization
Abstract: Initial applications of 3D Gaussian Splatting (3DGS) in Visual Simultaneous Localization and Mapping (VSLAM) demonstrate the generation of high-quality volumetric reconstructions from monocular video streams. However, despite these promising advancements, current 3DGS integrations have reduced tracking performance and lower operating speeds compared to traditional VSLAM. To address these issues, we propose integrating 3DGS with Direct Sparse Odometry, a monocular photometric SLAM system. We have done preliminary experiments showing that using Direct Sparse Odometry point cloud outputs, as opposed to standard structure-from-motion methods, significantly shortens the training time needed to achieve high-quality renders. Reducing 3DGS training time enables the development of 3DGS-integrated SLAM systems that operate in real-time on mobile hardware. These promising initial findings suggest further exploration is warranted in combining traditional VSLAM systems with 3DGS.

Keywords: SLAM, Visual-Inertial SLAM, Localization
Abstract: The optical flow odometry based on panoramic images offers significant advantages. However, the conversion of the panoramic image from the spherical structure to the 2D plane during feature tracking results in distortion inevitably, which leads to the failure of optical flow tracking, particularly in scenarios involving large-scale displacement. To address this issue, we proposed the Spherical Congruence Projection, which preserves the pixel structure of the spherical image after the plane mapping. The experimental results in the public dataset demonstrate the effectiveness of the proposed method.

Keywords: Integrated Planning and Learning, Perception-Action Coupling
Abstract: In this work, we argue that new methods are needed to generate robot motion for navigation or manipulation while effectively achieving perception goals. We support our argument by conducting experiments with a simulated robot that must accomplish a primary task, such as manipulation or navigation, while concurrently monitoring an object in the environment. Our preliminary study demonstrates that a decoupled approach fails to achieve high success in either action-focused motion generation or perception goals, motivating further developments of approaches that holistically consider both goals.

Keywords: Virtual Reality and Interfaces, Touch in HRI, Physical Human-Robot Interaction
Abstract: In this video, we want to demonstrate a novel method to physically interact with virtual objects in Mixed-Reality (MR) through robotics. The user, wearing an MR head mounted display (HMD), is standing in front of a holographic object, or scene. A robot arm is used to emulate the surface of the object at any given interaction point, once the user tries to touch it. The objects material properties can also be simulated by adjusting the robot dynamics. This allows for deformation of the virtual objects with force feedback. These control schemes have to potential to be applied to the teleoperation of robots without the need of wearable robotics, while still being able to receive feedback and sense the remote environment. It also has potential in simulating environments, as the dynamics of the virtual objects can be chosen freely. In one of the demonstrations in the video, we show the user playing with a ball in zero gravity, hence the title “ballin' in space”. Furthermore the system could act as shape-shifting, all-purpose interface that can be physically interacted with, as well as serve as a demonstration tool in education.

Keywords: Logistics, Computer Vision for Automation, Task and Motion Planning
Abstract: In this work, we perform an autonomous robot system for the container unloading task. The demand for automation in logistics is increasing due to high ergonomic hazards and labor shortages. Therefore, we try to solve the problem by adopting robotic systems to the logistics, especially for the container unloading task. Two key technologies are pivotal in the development of the autonomous unloading system: detection of the packages, and planning to take the packages out of the container safely and efficiently. Our integrated robotic system in this work incorporates vision-based package detection and unloading planning. Firstly, the system captures the image inside the logistics container using the in-hand RGB-D vision sensor. From the color image, we extract the instances with a deep neural network and parameterize the packages into 3-dimensional oriented bounding boxes. The vision detection method is based on the SipMask instance segmentation network and the Cuboid-RANSAC algorithm. Subsequently, the planning module establishes the unloading sequence preventing package damage. Finally, the robotic manipulator picks and pulls out the package using the vacuum gripper and repeats the capture, detection, and planning procedures. The developed autonomous system presents the generalized robotic unloading system which contains vision-based detection and planning methods. Further details of the proposed system are available in our previous publications.

Keywords: Machine Learning for Robot Control, Planning under Uncertainty, Reactive and Sensor-Based Planning
Abstract: Air hockey is a highly reactive game in which the player needs to quickly reason over stochastic puck and contact dynamics. Due to high-speed motion requirements, the presence of constraints, and the possibility of adapting the playing style against an opponent, the air hockey setting has seen extensive use in robotics as a testbed for learning and control algorithms. This video demonstrates a novel learning framework for efficient contact planning in real-time, subject to uncertain contact dynamics. Based on a learned stochastic model of puck dynamics, we formulate contact planning for shooting actions as a stochastic optimal control problem with a chance constraint on hitting the goal. To achieve online re-planning capabilities, we propose training an energy-based model to generate optimal shooting plans in real-time. Example shots are shown in the real world with comparisons to control-based and reinforcement learning baselines. Our approach was further tested in a competitive setting as part of the NeurIPS 2023 Robot Air Hockey Challenge. The proposed framework outperformed all other approaches in real-robot matches, establishing a new state-of-the-art in robot air hockey. Qualitative comparisons between shots produced by the baselines and shots produced by our framework are shown in the video.

Keywords: Virtual Reality and Interfaces, Sensor-based Control, Robotics in Hazardous Fields
Abstract: This extended abstract describes the latest ad- vancements in the multi-robot measurement and inspection of the Super Proton Synchrotron (SPS) accelerator at CERN. In this setup, two mobile robots with manipulation capabilities simultaneously generate a radiation map along a 7-kilometer path. These robots are readily deployed for long-term operation within the SPS accelerator, with a novel Human-Robot Interface enhancing data accuracy and mission robustness. Due to constrained 4G communication, the need for improved operator comfort, and the assurance and safety of scientific equipment, a supervised autonomous control system is required. This system allows the operator to dynamically adjust high-level mission parameters, such as the navigation distance to the accelerator support structure and tunnel walls, velocity, mission execution, and stoppage. Recent results with this new user interface have shown increased mission safety and greater operator comfort, enabling operators to focus on critical information provided via the ”Mixed Reality” module.

Keywords: Robotics and Automation in Agriculture and Forestry, Visual Tracking, Field Robots
Abstract: Tracking plant features is crucial for various agricultural tasks like phenotyping, pruning, or harvesting, but the unstructured, cluttered, and deformable nature of plant environments makes it a challenging task. In this context, the recent advancements in foundational models show promise in addressing this challenge. In our work, we propose PlantTrack where we utilize DINOv2 which provides high-dimensional features, and train a keypoint heatmap predictor network to identify the locations of semantic features such as fruits and leaves which are then used as prompts for point tracking across video frames using TAPIR. We show that with as few as 20 synthetic images for training the keypoint predictor, we achieve zero-shot Sim2Real transfer, enabling effective tracking of plant features in real environments.

Keywords: Intelligent Transportation Systems, Performance Evaluation and Benchmarking, Collision Avoidance
Abstract: Validation of autonomous vehicles (AVs) is a critical task in their development and for their approval on public roads. Scenario-based testing is the state-of-the-art validation method and is recommended by international automotive regulators. While simulated execution of critical scenarios is essential, it cannot yet fully replace real-world testing, which however remains tedious, time-consuming, and resource-intensive. In this work, we propose an enhanced methodology using Augmented Reality to bridge the gap, providing an intermediate testing method that enables comprehensive real-world testing with reduced cost and improved realism. To demonstrate this methodology, we conducted tests in a controlled environment using six critical scenarios selected from road crash studies.

Keywords: Marine Robotics, Reinforcement Learning, Software, Middleware and Programming Environments
Abstract: This extended abstract presents MarineGym, a high-fidelity simulation framework designed to accelerate Reinforcement Learning (RL) training for Unmanned Underwater Vehicles (UUVs). Developed based on Isaac Sim, MarineGym leverages GPU acceleration to achieve a 10,000-fold performance increase compared to real-time, significantly reducing the training time for RL tasks to mere minutes. The framework includes modules for UUV dynamics, physical scene simulation, gym environments, and learning toolkits, ensuring accurate environmental replication and efficient parallel execution. Experiments conducted with the BlueROV2 Heavy UUV across four tasks—station-keeping, circle tracking, helical tracking, and lemniscate tracking demonstrated high precision and minimal error. This work presents a robust solution for advancing the development of intelligent UUVs and facilitating their deployment in real-world underwater applications.

Keywords: Space Robotics and Automation, Soft Robot Materials and Design, Soft Robot Applications
Abstract: In the confined environment of a space station, efficient utilization of available space is crucial. Traditional rigid robots, while effective in certain applications, can become obstacles themselves, limiting their utility in such restricted spaces. This paper proposes a novel solution: a soft robot capable of adapting its form to the task. Our hypothesis is that a soft robot, which can morph its shape as needed, will provide significant advantages in maneuvering within the tight quarters of a space station. The inherent flexibility and adaptability of soft robots enable them to navigate around obstacles and operate without becoming obstructions themselves. In this paper, we explore the design and potential applications of a soft robot for intra-vehicular use. One primary application is cargo handling within space stations. Through this study we aim to highlight how soft robots can revolutionize operations within space stations.

Keywords: Human-Centered Robotics, Human-Centered Automation, Haptics and Haptic Interfaces
Abstract: Interaction between humans and their environment has been a key factor in the evolution and the expansion of intelligent species. Since humans started to build their living environment, it has been static and passive until very recently. Here we present methods to design and build an artificial environment through interactive robotic surfaces. This is a physical platform-based environment that can communicate with users via force transduction, shape morphing and stiffness adaptivity in a bidirectional manner. Any part of the world can be represented by an assembly of surfaces of different sizes, at different scales. Our generic approach to building modular and multi-scale origami robotic surfaces allows us to rethink current human-built settings: we can reconfigure any living spaces. Our work opens the next level of human-computer interaction which impacts human-machine interaction as well as human-human communication. Here, we describe the scalability of such hardware platform, architecture format, and system requirements.

Keywords: Constrained Motion Planning, Motion and Path Planning, Collision Avoidance
Abstract: This research presents an online path planner for Unmanned Aerial Vehicles (UAVs) that can handle dynamic obstacles and UAV motion constraints, including maximum curvature and desired orientations. Our proposed planner uses a NURBS path representation and a Differential Evolution algorithm, incorporating concepts from the Velocity Obstacle approach in a constraint function. Initial results show that our approach is feasible and provides a foundation for future extensions to three-dimensional (3D) environments.

Keywords: Telerobotics and Teleoperation, Motion and Path Planning
Abstract: Virtual fixtures assist human operators in teleoperation settings by constraining their actions. This extended abstract introduces a novel virtual fixture formulation emph{on surfaces} for tactile robotics tasks. Unlike existing methods, our approach constrains the behavior based on the position on the surface and generalizes it over the surface by considering the distance (metric) on the surface. Our method works directly on possibly noisy and partial point clouds collected via a camera. Given a set of regions on the surface together with their desired behaviors, our method diffuses the behaviors across the entire surface by taking into account the surface geometry. We demonstrate our method's ability in two simulated experiments (i) to regulate contact force magnitude or tangential speed based on surface position and (ii) to guide the robot to targets while avoiding restricted regions defined on the surface. All source codes, experimental data, and videos are available as open access at https://sites.google.com/view/diffusion-virtual-fixtures.

Keywords: Telerobotics and Teleoperation, Force and Tactile Sensing, Haptics and Haptic Interfaces
Abstract: Telerobotics enables humans to overcome spatial constraints and physically interact with the environment in remote locations. However, the sensory feedback provided by the system to the user is often purely visual, limiting the user's dexterity in manipulation tasks. This work addresses this issue by equipping the robot's end-effector with high-resolution visuotactile GelSight sensors. Using low-cost MANUS-Gloves, we provide the user with haptic feedback about forces acting at the points of contact in the form of vibration signals. We employ two different methods for estimating these forces; one based on estimating the movement of markers on the sensor surface and one deep-learning approach. Additionally, we integrate our system into a virtual-reality teleoperation pipeline in which a human user controls both arms of a Tiago robot while receiving visual and haptic feedback. Lastly, we present a novel setup to evaluate normal force, shear force, and slip. We believe that integrating haptic feedback is a crucial step towards dexterous manipulation in teleoperated robotic systems.

Keywords: Human Factors and Human-in-the-Loop, Human-Centered Robotics, Intention Recognition
Abstract: Human-In-The-Loop (HITL) motion control strategies for robot hands based on surface electromyography (sEMG) are typically characterized by the usage of machine learning for decoding user's grasping intentions. However, significant challenges related to unsolved complexities and inaccuracies still hinder the achievement of truly stable sEMG-based human-robot interfaces. Existing literature has explored both supervised and unsupervised machine learning methods for the regression of sEMG signals into continuous robot hand motions; however, we observe how these methods exhibit structural limitations in the (i) precision of the regression output and/or (ii) reliability of simultaneous control of multiple grasping motions. In this paper, we propose the adoption of properly designed minimally supervised machine learning approaches as a way to bypass such limitations. Furthermore, we discuss the need for minimal supervision to allow exploiting machine learning to deal with nonlinear fitting requirements and imprecise labelling that unavoidably arise with physiological measurements such as the sEMG signal. Finally, we introduce two possible implementations of minimally supervised regression approaches for surface sEMG-based control of robot hands using neural networks (NN). The first approach uses a differentiable version of the Dynamic Time Warping (DTW) similarity (soft-DTW divergence) as loss function for NN training. The second approach combines Non-Negative Matrix Factorization (NMF) with NN for realizing self-supervised regression. Offline evaluations of the two methods involving a group of subjects are reported, showing unprecedented promising results.

Keywords: Marine Robotics, Field Robots, Sensor Fusion
Abstract: Ocean exploration has become a crucial challenge for understanding our planet. At MBARI, we seek to develop and deploy a cost-effective, portable, and energy-efficient autonomous underwater vehicle (AUV). These platforms should be able to operate effectively at short range in unfamiliar, rugged, and complex terrains, including reefs, canyon walls, and hydrothermal vents, for tasks such as mapping and monitoring the seafloor. We have been developing and testing advanced estimation and control methods on our research platform, the MOLA 6-DOF AUV, to address many challenges that have limited these platforms’ broader applications. We are working on a robust 6-DOF controller to navigate accurately through deep, rugged terrains at low altitudes fed with a low-cost and compact multimodal sensors package. We aim to lay the groundwork for new perception systems, which will be based on cameras, structural light, acoustic sensors, and autonomous simultaneous localization and mapping (SLAM) navigation. Our goal is to offer a more accessible and versatile autonomous platform for the scientific exploration of the oceans.

Keywords: Marine Robotics
Abstract: Bathymetry mapping of static water bodies, such as lakes, is crucial for sustainable ecosystem management, yet traditional single-beam echosounder (SBE) methods often yield high mapping errors, and multi-beam surveys are prohibitively expensive. Building upon our previous work with the OAS- GPUCB algorithm, which used an adaptive sampling approach with Gaussian Process Upper Confidence Bound (GPUCB), we now propose an advanced framework leveraging Thompson Sampling and multi-agent systems. This new approach promises to enhance sampling efficiency and accuracy by distributing tasks among multiple Autonomous Surface Vehicles (ASVs) equipped with SBEs. Preliminary simulations of OAS-GPUCB on actual lake bathymetry maps and initial real-world experi- ments indicate that this novel approach can potentially achieve less than 5% bathymetry error, reduce travel distance by over 60% compared to the lawn-mower method, and by over 95% compared to traditional GPUCB methods. We intend to improve these results by implementing Thompson sampling with a multi-agent system for a larger bathymetry mapping. This paper outlines the theoretical foundations, expected benefits, and future research directions for implementing Thompson Sampling and multi-agent systems in bathymetry mapping.

Keywords: Planning under Uncertainty, Localization, Collision Avoidance
Abstract: Self-localization is a fundamental job in robotics that lowers navigation error and facilitates downstream, high-level tasks. For many high-level tasks, high navigation error leads to a high probability of failure. For example, high navigation errors could cause a drone to mislabel scientific data that it collected from the environment. Thus, a robot must localize frequently to reduce failure in such cases. However, a robot may want to seldom localize during other cases. For instance, selectively localizing may be tolerable when localization is costly (such as with resource constrained robots), especially when navigating in sparsely cluttered environments or when large deviations along a path are tolerable. In this study, we propose a method that helps a robot determine ``when to localize'' to 1) minimize the number of such actions and 2) not exceed the probability of failing a task. We formulate our method using a constrained Partially Observable Markov Decision-making Process (constrained POMDP) and use the Cost-Constrained POMCP solver to plan the robot's actions. The CC-POMCP solver helps the robot simulate the probability of failing to determine when it should move toward its destination or localize to prevent failure (such as obstacle collision). Our preliminary results evaluated the proposed method in numerical experiments with multiple baselines.

Keywords: Intelligent Transportation Systems
Abstract: Abstract— This study evaluates the performance of various 2D Simultaneous Localization and Mapping (SLAM) techniques with Visual SLAM (VSLAM) under differing lighting conditions using the HiWonder Educational Ros Robot, equipped with 2D Lidar and a depth camera. Experiments were conducted in a controlled indoor environment with lighting levels set at approximately 110 Lux and 10 Lux mimicking a realworld scenario while driving. The SLAM techniques tested included GMapping, Hector, Karto, and the 3D VSLAM method Real Time Appearance Based (RTAB), each subjected to four trials per lighting condition. Performance was assessed based on the Structural Similarity Index Measure (SSIM), distance error, and average time to reach the target. Hector emerged as the most optimized technique across all parameters.

Keywords: 2D SLAM, VSLAM, lighting conditions, SSIM, distance error, average time.

Keywords: Localization, SLAM, Computer Vision for Automation
Abstract: We present SIM-Sync, a certifiably optimal algorithm that estimates camera trajectory and 3D scene structure directly from multiview image keypoints. The key enabler of SIM-Sync is a pretrained depth prediction network. Given a graph with nodes representing monocular images taken at unknown camera poses and edges containing pairwise image keypoint correspondences, SIM-Sync first uses a pretrained depth prediction network to lift the 2D keypoints into 3D scaled point clouds, where the scaling of the per-image point cloud is unknown due to the scale ambiguity in monocular depth prediction. SIM-Sync then seeks to synchronize jointly the unknown camera poses and scaling factors (i.e., over the 3D similarity group) by minimizing the sum of the Euclidean distances between edge-wise scaled point clouds. The SIM-Sync formulation, despite being nonconvex, allows for the design of an efficient, certifiably optimal solver that is almost identical to the SE-Sync algorithm. Particularly, after solving the translations in closed-form, the remaining optimization over the rotations and scales can be written as a quadratically constrained quadratic program, for which we apply Shor’s semidefinite relaxation. We demonstrate the empirical tightness and practical usefulness of SIM-Sync in both simulated and real experiments, and investigate the impact of graph structure and sparsity.

Keywords: Soft Robot Materials and Design, Soft Robot Applications, Grasping
Abstract: This paper describes a new type of soft robotic grippers with variable stiffness to grasp efficiently a wide variety of objects. The gripper includes two sections, a pneumatic chamber for actuation and a tipping section for adjusting stiffness and shape adaptation. The tipping section consists of Magneto-rheological (MR) fluid, a magnetic source, and a tactile sensor. The MR fluid is responsible for changing the stiffness and it is solidified using the magnetic source which is a combination of magnetic elastomer, an electromagnet, and a permanent magnet. The tactile sensor is embedded within the soft finger to detect contact with the object and to trigger the magnetic source. Five experiments have been conducted to evaluate the gripper’s performance, stiffness, and success rate. Results indicate that the proposed soft gripper is an effective design that can ensure robust grasping of a wide variety of objects. Furthermore, the study demonstrates that our design can change stiffness in less than one millisecond while increasing the applied force by approximately 2.5 times. In addition, by matching the fingertip to the shape of the object, the gripper exhibits a high success rate (more than 85%) ensuring reliable and secure grasping when the MR fluid solidifies.

Keywords: Multi-Robot Systems, Agent-Based Systems, Control Architectures and Programming
Abstract: This paper proposes a follower-stabilizing strategy for leader-waypoint-follower nonholonomic vehicle formation control. The follower-stabilizing strategy consists of the follower-stabilizing area and the follower's velocity correction term. When the follower crosses the waypoint generated by the leader, its steering is susceptible to oscillation. To overcome this issue, the follower-stabilizing area around the waypoint is proposed such that the follower imitates the leader's heading angle as it enters the follower-stabilizing area. At the same time, the velocity correction term drives the follower to track the waypoint stably. The follower-stabilizing strategy is validated by various simulations, which confirm that the follower-stabilizing area effectively reduces steering oscillation and that the velocity correction term verifies stable waypoint tracking. Additionally, GPS and AHRS-based small-scale vehicles are built to demonstrate the proposed method by experiment. The CTRA model-based Kalman filter is designed for GPS and AHRS to estimate the agents' states, such as the heading angle and linear velocities.

Keywords: Reinforcement Learning, Force and Tactile Sensing
Abstract: Humans have exceptional tactile sensing capabil- ities, which they can leverage to solve challenging, partially observable tasks that cannot be solved from visual observation alone. Research in tactile sensing attempts to unlock this new input modality for robots. Lately, these sensors have become cheaper and, thus, widely available. But, how to integrate them into control loops is still an active area of research, with central challenges being partial observability and the contact-rich nature of manipulation tasks. In this study, we propose to use Reinforcement Learning to learn an end-to-end policy, mapping directly from tactile sensor readings to actions. Specifically, we use Dreamer-v3 on a challenging, partially observable robotic insertion task with a Franka Research 3, both in simulation and on a real system. For the real setup, we built a robotic platform capable of resetting itself fully autonomously, allowing for extensive training runs without human supervision. Our initial results show that Dreamer is capable of utilizing tactile inputs to solve robotic manipulation tasks in simulation and reality. Further, we find that providing the robot with tactile feedback generally improves task performance, though, in our setup, we do not yet include other sensing modalities. In the future, we plan to utilize our platform to evaluate a wide range of other Reinforcement Learning algorithms on tactile tasks.

Keywords: Representation Learning, Incremental Learning, Machine Learning for Robot Control
Abstract: Reasoning about distance is indispensable for establishing or avoiding contact in manipulation tasks. To this end, we present an incremental approach for learning implicit representations of signed distance using piecewise polynomial basis functions. Starting from an arbitrary prior shape, our method gradually constructs a continuous and smooth distance representation from incoming surface point clouds, without the need to store training data. It offers a high degree of modularity through interpretable hyperparameters that can be used to influence the behavior and performance of the underlying model. We assess the accuracy of our model on a diverse set of household objects and compare it to neural network and Gaussian process counterparts. Benefits of the incrementally learned basis function representation are further highlighted in a physical experiment.

Keywords: Robust/Adaptive Control, Aerial Systems: Mechanics and Control
Abstract: We consider a zero-sum finite horizon linear-quadratic differential game. Suboptimal state-feedback controls of the players in this game are derived. This derivation is based on the approximate solution of the corresponding Riccati matrix differential equation by the method of artificial parameter. The theoretical results are illustrated by the approximate solution of the problem of pursuit-evasion engagement between two flying vehicles.

Keywords: Localization, Mapping
Abstract: In long-term and large-scale robotic missions, maintaining accurate pose estimation is critical, and loop closures achieved through place recognition play a key role in mitigating drift. However, achieving real-time performance, especially in resource-constrained platforms and multi-robot systems, remains challenging despite computational progress. Traditional methods of selecting keyframes often lead to either redundant information retention or overlooking relevant data. To tackle these issues, we present some preliminary results on a keyframe selection strategy for LiDAR-based place recognition, focusing on minimizing redundancy and preserving information in the descriptor space. Through the preliminary evaluation, we explore the impact of various approaches on both learning-based and handcrafted descriptors and demonstrate how constant interval sampling can degrade performance, whereas our proposed optimized approach not only maintains but in some cases enhances performance.

Keywords: Multi-Robot Systems, Probabilistic Inference, Networked Robots
Abstract: Multiple robots can effectively acquire information from an environment and transmit it to a static base station. However, communication may not be possible everywhere in the environment, so robots must coordinate when and where to rendezvous with disconnected teammates. Such coordination presents challenges since knowing how long it will take to explore requires information about the environment, which is typically what the robots are acquiring. We propose a method to estimate disconnected robots’ state and use observed lack of communication to refine our state estimation, allowing robots to plan for rendezvous or parallel exploration by predicting where teammates are likely to be. We show up to 87% improved performance for exploration tasks against a baseline approach that performs no such predictions.

Keywords: Soft Sensors and Actuators, Soft Robot Materials and Design
Abstract: Abstract is attached as pdf file.

Keywords: Multi-Robot Systems, Cooperating Robots, Motion and Path Planning
Abstract: Multi-robot system is desired to achieve global costefficiency by fully

utilizing individual advantages. For example, leveraging unmanned aerial vehicles’ (UAVs’) agility for inspection and ground vehicles’ (UGVs’) capability for heavy-duty tasks. Coordinating the motion of multiple robots smartly is complex and it becomes more challenging when

robots in the swarm are not equipped with sufficient sensors for environmental perception (e.g. in our case only one robot has a depth camera).

In this letter, we propose a tightly coupled systematic framework to navigate a swarm composed of UAV and UGVs in unknown scenes. The system

has only one depth camera with a field of view for environmental perception. We fully explore the cooperation between robots by proposing a Sequential Exploration and Aiding Localization (SEAL) planning strategy for the UAV and a Collision-Adaptive Trajectory (CAT)

optimization for UGVs. The UAV assists UGVs’ localization with relative

pose estimation and own global localization, meanwhile, it focuses on exploration to provide UGVs with abundant environmental information. The UGV team can navigate safely and autonomously in obstacle-rich environments and even maintain formations with only the wheel odometer and UAV’s assistance using CAT optimization. Our method is validated both in simulations and real-world experiments indoors and outdoors.

Keywords: Medical Robots and Systems, Autonomous Agents, Visual Servoing
Abstract: Cutting is a challenging area in the field of autonomous robotics but is especially interesting for applications such as surgery. One large challenge is the lack of simulations for cutting with surgical robots that can transfer to the real robot. In this work, we create a surgical robotic simulation of rope cutting with realistic visual and physics behavior using the da Vinci Research Kit (dVRK). We learn a cutting policy purely from simulation and sim2real transfer our learned models to real experiments by leveraging Domain Randomization. We find that cutting with surgical instruments such as the EndoWrist Round Tip Scissors comes with certain challenges such as deformations, cutting forces along the jaw, fine positioning, and tool occlusion. We overcome these challenges by designing a reward function that promotes successful cutting behavior through fine positioning of the jaws directly from image inputs. Policies are transferred using a custom sim2real pipeline based on a modular teleoperation framework for identical execution in simulation and the real robot. We achieve a 97.5% success rate in real cutting experiments with our 2D model and a 90% success rate in 3D after the sim2real transfer of our model. We showcase the need for Domain Randomization and a specialized reward function to achieve successful cutting behavior across different material conditions through optimal fine positioning. Our experiments cover varying rope thicknesses and tension levels and show that our final policy can successfully cut the rope across different scenarios by learning entirely from our simulation. Further project information is available at https://medcvr.utm.utoronto.ca/tase2024-cutrope.html

Keywords: Multi-Robot Systems, Robotics in Hazardous Fields, Path Planning for Multiple Mobile Robots or Agents
Abstract: Multi-agent patrolling is a key problem in a variety of domains such as intrusion detection, area surveillance, and policing, which involves repeated visits by a group of agents to specified points in an environment. While the problem is well-studied, most works do not provide performance guarantees and either do not consider agent attrition or impose significant communication requirements to enable adaptation. In this work, we present the Adaptive Heuristic-based Patrolling Algorithm, which is capable of adaptation to agent loss using minimal communication by taking advantage of Voronoi partitioning, and which meets guaranteed performance bounds. Additionally, we provide new centralized and distributed mathematical programming formulations of the patrolling problem, analyze the properties of Voronoi partitioning, and finally, show the value of our adaptive heuristic algorithm by comparison with various benchmark algorithms using physical robots and simulation based on the Robot Operating System (ROS) 2.

Keywords: Semantic Scene Understanding, Uncertainty Estimation, Deep Learning in Robotics and Automation, Autonomous Vehicle Navigation
Abstract: Knowing when a trained segmentation model is encountering data that is different to its training data is important. Understanding and mitigating the effects of this play an important part in their application from a performance and assurance perspective -- this being a safety concern in applications such as AVs. This work presents a segmentation network that can detect errors caused by challenging test domains without any additional annotation in a single forward pass. As annotation costs limit the diversity of labelled datasets, we use easy-to-obtain, uncurated and unlabelled data to learn to perform uncertainty estimation by selectively enforcing consistency over data augmentation. To this end, a novel segmentation benchmark based on the SAX Dataset is used, which includes labelled test data spanning three autonomous-driving domains, ranging in appearance from dense urban to off-road. The proposed method, named GammaSSL, consistently outperforms uncertainty estimation and OoD techniques on this difficult benchmark -- by up to 10.7% in area under the ROC curve and 19.2% in area under the PR curve in the most challenging of the three scenarios.

Keywords: Localization, Range Sensing, Multi-Robot Systems
Abstract: Ultra-wideband (UWB) transceivers, or tags, are often fixed at known locations to estimate the position and attitude of moving robots equipped with UWB tags. This note lays the foundation for range-based anchor-free relative localization between robots. In this note, centralized estimators in the form of a sliding-window filter and a Gaussian-sum filter address the localization of multi-robot systems in the presence of observability issues. A mathematically-motivated two-UWB-tag approach is featured that overcomes motion-based constraints for localization. In addition, cost functions are derived to balance between observability and used-defined formations in path planning tasks. Practical considerations concerning UWB ranging protocols, calibration, and information maximizing strategies are provided. Finally, sophisticated decentralized algorithms that consider cross-correlational terms between individual estimators are presented, drastically outperforming naive implementations.

Keywords: Physically Assistive Devices, Physical Human-Robot Interaction, Human Detection and Tracking
Abstract: Walking aid robots have been developed for elderly people or patients facing difficulties while walking. However, most of them are designed only for flat ground, must rely on handles, and have drawbacks such as oscillation and lack of stability. The primary goal of this research is to improve the transparency and safety of a walking aid robot, addressing both flat and sloping terrains. To achieve this goal, we propose a novel variable admittance control strategy for an omnidirectional mobile platform by combining human motion recognition and a slope-adaptive approach. We design a vision system with a wide-angle camera to capture skeletal whole body information at a close distance to recognize the walking direction. Accordingly, the damping values of the admittance controller are varied. In addition, these parameters are varied with respect to the slope angle of the ground, which is detected by the platform. We validated the controller performance with eleven healthy subjects performing two experiments on both flat and sloping terrains. Three admittance controllers are compared, with fixed parameters, variable damping by Cartesian velocity, and variable damping by walking direction. Experimental results show the advantages of the variable admittance control based on walking direction, which ensures high transparency and smoothness on both flat and sloping terrains.

Keywords: Humanoid Robot Systems, Developmental Robotics, Robot Companions
Abstract: The iCub project was initiated in 2004 by Giorgio Metta, Giulio Sandini, and David Vernon to create a robotic platform for embodied cognition research. The project received funding from the EU Commission and involved an international consortium coordinated by the University of Genoa, consisting of eleven research partners and a company.

The main goals of the project were to design a humanoid robot, named iCub, to create a community by leveraging on open-source licensing, and implement several basic elements of artificial cognition and developmental robotics.

The iCub is a small humanoid robot with 53 degrees of freedom, sensors for vision, touch, sound, and proprioception, and a modular software system based on YARP – a middleware. The iCub can perform tasks such as crawling, walking, and object manipulation. The consortium pioneered a variety of techniques as for example deep neural networks for sensory processing and motor control.

The Italian Institute of Technology (IIT) joined the iCub project in 2007 and provided the large resources and expertise to improve the iCub development to excel in the field of humanoid robotics. The iCub project has evolved over the years and has produced several versions of the iCub, the R1 robot and ergoCub. More than 50 iCub have been built and used worldwide for various research projects.

Keywords: SLAM, Localization, Range Sensing
Abstract: In this paper, we propose a continuous-time lidar-inertial odometry (CT-LIO) system named SLICT2, which promotes two main insights. One, contrary to conventional wisdom, CT-LIO algorithm can be optimized by linear solvers in only a few iterations, which is more efficient than commonly used nonlinear solvers. Two, CT-LIO benefits more from the correct association than the number of iterations. Based on these ideas, we implement our method with a customized solver where the feature association process is performed immediately after each incremental step, and the solution can converge within a few iterations. Our implementation can achieve real-time performance with a high density of control points while yielding competitive performance in highly dynamical motion scenarios. We demonstrate the advantages of our method by comparing with other existing state-of-the-art CT-LIO methods. For the benefits of the community, the source code will be released at https://github.com/brytsknguyen/slict.

Keywords: Collision Avoidance, Force Control, Motion Control
Abstract: Evaluating and updating the obstacle avoidance velocity for an autonomous robot in real-time ensures robustness against noise and disturbances. A passive damping controller can obtain the desired motion with a torque-controlled robot, which remains compliant and ensures a safe response to external perturbations. Here, we propose a novel approach for designing the passive control policy. Our algorithm complies with obstacle-free zones while transitioning to increased damping near obstacles to ensure collision avoidance. This approach ensures stability across diverse scenarios, effectively mitigating disturbances. Validation on a 7DoF robot arm demonstrates superior collision rejection capabilities compared to the baseline, underlining its practicality for real-world applications. Our obstacle-aware damping controller represents a substantial advancement in secure robot control within complex and uncertain environments.

Keywords: Vision-Based Navigation, Recognition, Cognitive Modeling
Abstract: Visual-based recognition, e.g., image classification, object detection, etc., is a long-standing challenge in computer vision and robotics communities. Concerning the roboticists, since the knowledge of the environment is a prerequisite for complex navigation tasks, visual place recognition is vital for most localization implementations or re-localization and loop closure detection pipelines within simultaneous localization and mapping (SLAM). More specifically, it corresponds to the system’s ability to identify and match a previously visited location using computer vision tools. Towards developing novel techniques with enhanced accuracy and robustness, while motivated by the success presented in natural language processing methods, researchers have recently turned their attention to vision-language models, which integrate visual and textual data.

Keywords: Aerial Systems: Perception and Autonomy, Data Sets for Robotic Vision, Computer Vision for Automation
Abstract: Visual detection of micro aerial vehicles (MAVs) has received increasing research attention in recent years due to its importance in many applications. However, the existing approaches based on either appearance or motion features of MAVs still face challenges when the background is complex, the MAV target is small, or the computation resource is limited. In this paper, we propose a global-local MAV detector that can fuse both motion and appearance features for MAV detection under challenging conditions. This detector first searches MAV targets using a global detector and then switches to a local detector which works in an adaptive search region to enhance accuracy and efficiency. Additionally, a detector switcher is applied to coordinate the global and local detectors. A new dataset is created to train and verify the effectiveness of the proposed detector. This dataset contains more challenging scenarios that can occur in practice. Extensive experiments on three challenging datasets show that the proposed detector outperforms the state-of-the-art ones in terms of detection accuracy and computational efficiency. In particular, this detector can run with near real-time frame rate on NVIDIA Jetson NX Xavier, which demonstrates the usefulness of our approach for real-world applications.

Keywords: Robot Safety, Motion Control
Abstract: We leverage energy-based control barrier functions (CBFs) as a way to limit the kinetic energy of a robot. In particular, the proposed algorithm implements a safety filter that transforms the nominal impedance torque command into the closest torque command which limits the total kinetic energy of the robot. We present experimental results on a 7-DoF torque-controlled manipulator.

Keywords: Aerial Systems: Mechanics and Control, Compliance and Impedance Control, Robot Safety
Abstract: Maintaining stability after catching an object is essential for the flight safety of unmanned aerial manipulators (UAMs). Current research on UAM catching stationary or moving targets neglects the impact of the target and the introduced center of mass (COM) displacement. This paper proposes an approach to absorb the impact during the capture of fast-flying heavy targets by UAMs, ultimately reducing UAM movement. An angular momentum theorem-based fast target mass estimation algorithm is proposed to estimate the target mass within the limited catching time. With the estimated target mass, an adaptive impedance controller for a brushless motor-based arm and a compensation algorithm for the UAM attitude controller are proposed. An innovative aspect of our approach is utilizing the estimated target mass to adjust the impedance controller and compensate for COM displacement, which enables effective absorption of the target's impact and reduces UAM movement. We validate our proposed approach through simulation and real-world experiments. The results demonstrate that our approach significantly reduces UAM body displacement after capturing fast-flying heavy targets, enhancing UAM's flight safety. A video showcasing our experiments can be accessed at: https://youtu.be/k28Xirbjuek

Keywords: Aerial Systems: Mechanics and Control, Manipulation Planning, Modeling, Control, and Learning for Soft Robots
Abstract: This extended abstract introduces a framework for planning and controlling an aerial robot whose task is to shape a flexible deformable linear object during contact-free flight or while interacting with a rigid environment.

Keywords: SLAM, Localization, Sensor Fusion
Abstract: This letter presents Switch-SLAM, switching-based LiDAR-inertial-visual SLAM for degenerate environments, designed to tackle the challenges in degenerate environments for LiDAR and visual SLAM. Switch-SLAM achieves high robustness and accuracy by utilizing a switching structure that transitions from LiDAR to visual odometry when degeneration of LiDAR odometry is detected. To efficiently detect degeneration, Switch-SLAM incorporates a non-heuristic degeneracy detection method that does not require heuristic tuning and demonstrates generalizability across various environments. Switch-SLAM is evaluated on diverse datasets containing both LiDAR and visual odometry degeneracy scenarios. The experimental results highlight the accurate and robust localization by the proposed method in multiple challenging environments with either LiDAR or visual SLAM degeneracy.

Keywords: Aerial Systems: Mechanics and Control, Mobile Manipulation, Task and Motion Planning
Abstract: This extended abstract proposes an optimal planning of smooth non-stop trajectories for fixed-wing UAVs carrying a motionless load via cables. The minimized cost function is related to energy consumption, and the optimization takes into account the system constraints such as maximum and minimum forward speed and banking angle boundaries.

Keywords: Multi-Robot Systems, Aerial Systems: Perception and Autonomy, Computer Vision for Automation
Abstract: The temporal and spatial relationships among various onboard sensors are crucial for ensuring the accuracy of visual observations carried out by the multiple unmanned aerial vehicles (multi-UAV) system. To leverage the integration within the multi-UAV system, we propose a buddy calibration method focused on calibrating the relative time offset and rotation parameters between the airborne camera and flight controller. We designate another UAV as the target and perform 6D pose estimation to calculate the relative pose between the camera mounted on the host UAV and the target UAV. Subsequently, we use the pose estimation results and the parameters recorded by the sensors on both UAVs to establish a comprehensive temporal and spatial calibration model. Finally, a specific method is developed to effectively solve the proposed model. Our approach was validated using two-UAV flight experiments. The results demonstrate a significant improvement, with leader attitude estimation errors decreasing to approximately less than 1 degree post-calibration compared to pre-calibration. Furthermore, positioning errors for dynamic and static targets were substantially reduced over 42.59% and 53.36%, respectively.

Keywords: Robot Safety, Control Architectures and Programming
Abstract: In this paper we show the effect of safety-critical control implemented with control barrier functions (CBFs) on the power balance of physical systems. The presented results will provide novel tools to design CBFs inducing desired energetic behaviors of the closed-loop system, including nontrivial damping injection effects and non-passive control actions, effectively injecting energy in the system in a controlled manner.

Keywords: Simulation and Animation, Planning under Uncertainty, Computational Geometry
Abstract: Physics simulations are crucial for domains like animation and robotics, yet they are limited to deterministic simulations with precise knowledge of initial conditions. We introduce a surrogate model for simulating rigid bodies with positional uncertainty (Gaussian) and use a non-uniform sphere hierarchy for object approximation. Our model outperforms traditional sampling-based methods by several orders of magnitude in efficiency while achieving similar outcomes.

Keywords: Failure Detection and Recovery, Mechanism Design
Abstract: Impact-aware control allows robots to exploit intentional impacts with the environment to speed up loco-manipulation tasks. Picking and placing heavy boxes with cycle times of a few seconds with a humanoid upper body is an example application. Execution inaccuracies are to be expected. Some of these failures may catastrophically damage the robot's hardware. A lightweight and compact overload clutch is presented that avoids hardware damage by decoupling upon severe impact.

Keywords: Aerial Systems: Perception and Autonomy, Distributed Robot Systems, Deep Learning for Visual Perception
Abstract: In this paper, we present a novel UAV-based object detection hierarchical federated learning framework. It addresses complexities in cloud and edge federated learning by using a hierarchical structure with cloud and edge servers. The cloud server accesses a large dataset, while edge servers facilitate rapid updates with local clients. Models are aggregated centrally for a unified UAV network knowledge. Evaluation on UAVDT and VisDrone datasets shows superior performance over conventional cloud-based federated learning, especially with non-uniform data. This study highlights hierarchical federated learning's potential for improving UAV-based object detection in challenging real-world scenarios.

Keywords: Modeling, Control, and Learning for Soft Robots, Probability and Statistical Methods, Flexible Robotics
Abstract: This extended abstract introduces a novel method for continuous state estimation of continuum robots. We formulate the estimation problem as a factor graph optimization problem using a novel Gaussian-process prior that is parameterized over both arclength and time. We use this to introduce the first continuous-time-and-space state estimation method for continuum robots.

Keywords: Aerial Systems: Mechanics and Control, Aerial Systems: Applications, Underactuated Robots
Abstract: In this work we consider the new concept of re-targeting commercially available under-actuated aerial vehicles to higher dimensional tasks, which is useful, for example, in the context of Aerial Physical Interaction (APhI). The re-targeting of the vehicle is achieved by 1)~input augmentation of the commercial vehicle with an add-on module which includes additional rotors and 2)~a controller which is able to let the platform attain full-pose tracking by commanding the additional rotors inputs and reinterpreting the available high-level commands (the standard four pilot commands) provided by the internal built-in controller. We present the mathematical formulation of the concept which is validated using numerical simulations.

Keywords: Soft Robot Applications, Space Robotics and Automation, Cooperating Robots
Abstract: We discuss a novel approach to manipulation and transport of satellites for Low Earth Orbit service operations and present our simulation results. Our approach is based on cooperative manipulation principles and leverages on a Reynolds boids' collective coordinated movement. The implementation exploits Belief Space Planning (BSP) to enforce the Reynolds' flock rules and the compliance of the satellite's exterior to perform grasping and transport of satellites from a lower to an higher orbit.

Keywords: Aerial Systems: Mechanics and Control
Abstract: In this work we propose an improved predictor of the thrust force produced by propellers of a Multi-rotor UAV in the presence of cross interference from other propellers placed at non-standard relative positions and orientations. For each propeller force, the predictor includes both the intrinsic kinematic parameters related to the propeller placements in the multi-rotor structure, and the spinning rate of all the propellers of the UAV. The prediction error is experimentally assessed by fitting and testing the predictor with force sensor measurements of multiple tests with up to eight propellers arranged on the vertexes of a cuboid structure and spanning different configurations of the adjacent propellers, tilted from 0 to 50 degrees. Results show that the use of five parameters is sufficient to have a prediction error that is 32% of the error shown by the 1-parameter state-of-the-art predictor which is agnostic to cross-interference. We report briefly our method, the results, and discuss future UAV research directions.

Keywords: Multi-Robot Systems, Autonomous Agents, Cooperating Robots
Abstract: Robot swarms can be tasked with a variety of sensing and inspection applications. In a previous work, we presented simulation studies of a swarm of miniaturized vibration-sensing robots that localize vibration sources over a 2.5D surface and employed a Niche-PSO search algorithm in the swarm. This work extends our simulation and algorithmic framework by (i) directly integrating the vibration simulation data within the robots' simulation environment in Webots robotic simulator and by (ii) extending the search algorithm to allow a variable number of robots participate in niches.

Keywords: Modeling, Control, and Learning for Soft Robots, Calibration and Identification, Soft Robot Applications
Abstract: The development of hardware and control systems for conventional rigid-link robots is facilitated by various existing robotic simulators that host extensive libraries of sensors and actuators. Soft robot development, however, is not supported by similar simulation tools where a soft device model is integrated within a full-fledged robotic simulator. In this work, we develop a lightweight open-source digital twin of a commercially available soft gripper. We use a Rigid-Link-Discretization modeling (RLD) approach within the robotic simulator Webots. Using Particle Swarm Optimization (PSO), we identify the parameters of the RLD model based on the kinematics and dynamics of the physical system and show the efficacy of our modeling approach.

Keywords: Multi-Robot Systems, Industrial Robots, Sensor-based Control
Abstract: This study presents the concept of a highly flexible robotic cell for mechanical testing of components. Two high-duty six-axis industrial robots, equipped with external sensors, ensure precise loading during the test process. An automotive component is used to demonstrate the capabilities of this multi-robot-based component testing.

Keywords: Cooperating Robots, Multi-Robot Systems, Reinforcement Learning
Abstract: Cooperative Navigation, akin to flocking behaviors in nature, offers significant advantages for multi-robot systems as it leads to more optimal resource allocation and robustness. While vanilla flocking rules are computationally efficient due to their straightforward evaluative nature, they are prone to issues like deadlocks and local minima. In contrast, Cooperative Navigation strategy like formation control are typically computationally expensive. In this paper, we propose to novel framework to integrate a learning-based Global Subgoal Planner (GSP) together with vanilla flocking rules. The interpretable framework is shown to alleviate local-minima issues, enabling faster collective navigation towards the target

Keywords: Force and Tactile Sensing, Soft Sensors and Actuators, Computer Vision for Automation
Abstract: Humans can classify deformable materials according to their hardness similarity, but existing robotic approaches focus on hardness recognition or absolute hardness prediction. In this work, we investigate hardness similarity detection using a vision-based tactile sensor (VBTS) and evaluate three methods: optical flow features and support vector machine (SVM) classifier, DINOv2 features and SVM classifier, and convolutional long short term memory (ConvLSTM) network trained with categorical cross-entropy loss. To evaluate these methods, we created a dataset of over 200 videos by pressing a GelSight Mini sensor, attached to a Franka-Panda robot, on five silicone objects of varying hardness, and also conducted a human-participant study showing humans achieved 80.25% average accuracy in hardness similarity detection. The three methods achieved average accuracies of 77.66%, 67.00%, and 70.00% with 15 samples per object, demonstrating that a VBTS can effectively classify objects based on hardness similarity.

Keywords: Force and Tactile Sensing, Soft Sensors and Actuators, Haptics and Haptic Interfaces
Abstract: The tactile or electronic skin (e-Skin) has been explored in recent years to endow robots with human like interaction capabilities. Significant advances have been made in this field by developing various types of touch sensors as well as by mimicking some of the morphological features such as fingerprints. Yet, in comparison with biological skin, the tactile skin research appears at its infancy as the former is not just about a few types of receptors, but it is a system with synergistic integration of distributed sensors, energy, and computing elements embedded in soft materials. Biological skin contains distributed molecular energy sources and processes data through embedded computing (e.g. processing of tactile data at the point of contact). However, unlike touch sensors, the distributed energy and distributed computing have not received much attention in e-skin. The touch sensors data is still processed and computed mainly using energy intensive von Neumann architecture. Herein, we present tactile skin with printed nanowires (NWs) based flexible synaptic transistors for in-skin (near-sensor) analog computing (i.e., processing of tactile data at the point of contact). In a complementary set of works, we have also demonstrated distributed energy harvesters (i.e. photovoltaics (PVs)) based multimodal tactile skin, with PV acting as touch sensors, object imagers while powering other components on robotics platform. The energy generated during the sensors’ sleep cycle can also be stored for longer and reliable operation of robots. These works show the exciting avenues for future autonomous robots that can “see” and “feel” through energy generating tactile skin.

Keywords: Multi-Robot Systems, Reinforcement Learning, Autonomous Agents
Abstract: This paper presents a navigation and obstacle avoidance policy network for multi-robot systems using deep reinforcement learning. The network design starts with a dual-robot system. By considering nonholonomic constraints and priority order, reinforcement learning trains the network to adhere to the kinematics of mobile robots, enabling collision avoidance and navigation. This study introduces an innovative expansion architecture. By incorporating the social force model, the architecture facilitates the constructive expansion of the dual-robot policy network to multi-robot scenarios with moderate computational cost. Although the network is trained in an open environment, it can be applied to generalized map environments by using virtual robots to simulate static obstacles. Simulations and indoor experiments validate the feasibility and performance of the proposed multi-robot navigation and obstacle avoidance policy network.

Keywords: Humanoid Robot Systems, Dexterous Manipulation, Multifingered Hands
Abstract: The wheeled humanoid neoDavid is with its 52 Degrees of freedom, 95 brushless dc motors, 184 position and 3 force sensors, and a control frequency of 3 kHz one of the most complex humanoid robots worldwide. All finger joints can be controlled individually, giving the system exceptional dexterity.

neoDavids Variable Stiffness Actuators (VSAs) enable very high performance in the tasks with fast collisions, highly energetic vibrations, or explosive motions, such as hammering, using power-tools, e.g. a drill-hammer, or throwing a ball.

The Elastic Structure Preserving (ESP) control concept was developed for compliant VSA robots. To the best of our knowledge, ESP is the world’s first, experimentally validated, (globally asymptotically stable and passive) motion tracking controller with link-side damping injection for VSA robots. Demanding manipulation tasks require grasp state knowledge. Our method to estimate the grasp state integrates information from proprioception and vision. Utilizing the estimated grasp state, the developed model-based controller realizes the compliant positioning of the object inside the hand.

neoDavid’s dexterous manipulation skills are further enabled through our M3T tracking framework, which incorporates visual feedback with forward kinematics and joint measurements. This allows us to simultaneously track the right arm as well as desired objects at real time and cope with severe object occlusions.

We present the first prototypes of VSAs and multiple generations of hands and body components. Keeping our long-term vision of a dexterous and robust service robot in mind, our focus changed from technological basics to skills and finally applications.

Keywords: Humanoid Robot Systems, Humanoid and Bipedal Locomotion, Whole-Body Motion Planning and Control
Abstract: Humanoid legged locomotion is versatile, but typically used for reaching nearby targets. Employing a personal transporter (PT) designed for humans, such as a Segway, offers an alternative for humanoids navigating the real world, enabling them to switch from walking to wheeled locomotion for covering larger distances, similar to humans. In this work, we develop control strategies that allow humanoids to operate PTs while maintaining balance. This control task differs from walking control but is equally challenging and demands quick reactions. Our controller is based on a stack of tasks quadratic program (QP) formulation, which accounts for contacts, stability, and bimanual manipulation constraints. We present experimental results of a REEM-C humanoid operating a Segway x2 SE.

Keywords: Reinforcement Learning, Software-Hardware Integration for Robot Systems, Software Tools for Robot Programming
Abstract: Sim2real, that is, the transfer of learned control policies from simulation to real world, is an area of growing interest in robotics due to its potential to efficiently handle complex tasks. The sim2real approach faces challenges due to mismatches between simulation and reality. These discrepancies arise from inaccuracies in modeling physical phenomena and asynchronous control, among other factors. To this end, we introduce EAGERx, a framework with a unified software pipeline for both real and simulated robot learning. It can support various simulators and aids in integrating state, action and time-scale abstractions to facilitate learning. EAGERx's integrated delay simulation, domain randomization features, and proposed synchronization algorithm contribute to narrowing the sim2real gap.We demonstrate (in the context of robot learning and beyond) the efficacy of EAGERx in accommodating diverse robotic systems and maintaining consistent simulation behavior. EAGERx is open source and its code is available at url{https://eagerx.readthedocs.io}.

Keywords: Soft Robot Materials and Design, Soft Sensors and Actuators, Biologically-Inspired Robots
Abstract: Despite the significant potential of soft robots for new applications, their widespread adoption in industrial settings is hindered by their susceptibility to damage. These clear limitations have driven the Brubotics group at VUB to develop self-healing polymers for soft robotics. These are elastomers that incorporate a Diels-Alder bond, enabling them to recover from macroscopic damages. These materials were used to create various soft robots, such as soft pneumatic grippers, which can fully recover from minor damages like punctures within minutes and from more significant damages, such as having one of their fingers cut in half, within a day. By applying heat the healing process can be accelerated. This was illustrated by embedding a heat cartridge in a soft gripper based on granular jamming, resulting in complete healing of severe damage within four hours at 70°C. In addition, a stretchable heater was made from a new self-healing electrically conductive composite and was embedded in a bending actuator. By applying current, Joule heating induces a local temperature increase at the damage site, enabling full recovery within a single hour. These composites were also utilized to develop stretchable piezoresistive sensors for self-healing soft robots, enabling them to track deformation, detect damage, and assess recovery. They regain their sensing performance after at 90°C for one hour. But making analytical or numerical models for these is challenging due to their nonlinear and time-dependent behavior. Nevertheless, we demonstrated that machine learning offers a solution, enabling the creation of neural network models for even more complex sensors, facilitating touch and damage localization as well as self-healing. By adjusting the polymer composition, highly flexible yet also very rigid self-healing polymers can be synthesized. Utilizing the same chemistry, these polymers can covalently bond together, forming strong interfaces, which allows to fabricate durable multi-material soft robots. Moreover, it has enabled the development of reconfigurable soft robots based on voxel-based designs. As such, Brubotics is at the forefront in the development of new sustainable materials and technologies for soft robotics.

Keywords: Bioinspired Robot Learning, Modeling, Control, and Learning for Soft Robots
Abstract: Complex locomotion patterns in animals and robots are usually driven by quasi-periodic oscillatory behaviors. Inspired by the dynamic coupling of moto-neurons and physical elasticity in animals, this work explores the possibility of generating these patterns by exciting physical oscillations in a soft snake by means of a low-level spiking neural mechanism. To achieve this goal, we introduce the Double Threshold Spiking neuron model with adjustable thresholds to generate varied output patterns in coupled mechanical oscillators. This neuron model, when applied to a soft robotic snake, enables distinct movements like turning or moving forward by simply altering the thresholds. Finally, we demonstrate that our approach, termed SpikingSoft, naturally pairs and integrates with reinforcement learning. The high-level agent only needs to adjust the two thresholds, thus strongly simplifying learning of reactive locomotion. Simulation results demonstrate that this architecture endows the soft snake robot with the capability of reaching target objectives with a substantially improved success rate, more quickly and smoothly compared to the vanilla reinforcement learning controllers acting directly in torque space.

Keywords: Motion and Path Planning, Reactive and Sensor-Based Planning, Service Robotics
Abstract: A common goal of robotics is to assist humans in performing repetitive tasks. Many tasks across different settings require complex tactile interactions with intricate curved surfaces, such as washing dishes, conducting medical inspections, and polishing in industrial environments. A shared challenge in all these tasks is achieving tactile coverage on curved surfaces. We address this problem with a human-inspired strategy that leverages visual and tactile feedback to create a closed-loop system. To achieve this, we collect point cloud data encoding both the surface and the coverage target using a camera attached to the end-effector. Then, we generate the coverage path in real-time by solving the ergodic control problem on point clouds using the Laplace-Beltrami eigenfunctions. Consequently, our approach can adapt to different surfaces, unmodeled dynamics, and changing coverage targets. We demonstrate the effectiveness of our method through real-world experiments, where we clean arbitrary hand-drawn targets on various curved surfaces.

Keywords: Continual Learning, Motion and Path Planning, Big Data in Robotics and Automation
Abstract: The accurate prediction of the trajectories of agents in the observed environment is crucial for the safe path planning of autonomous robotic systems. Forecasting the movements of vehicles and pedestrians enables intelligent systems to make informed control decisions. Machine learning has advanced trajectory prediction using rasterized bird's-eye-view maps, contextual scene information, and social dynamics. However, the collection and annotation of data for these systems are often expensive. Our research addresses this by examining the utility of trajectory data for active, semi-, or self-supervised learning, aiming to reduce annotation costs while maintaining model performance. Trajectory prediction methods in autonomous driving rely heavily on accurate perception of road infrastructure and agents. In order to accurately model trajectory behaviors, traditional machine learning approaches require extensive data annotation to provide ground truth scene perception as model input, including scene elements like lane markings, intersections, lights, signs, vehicles, and pedestrians, which are labor-intensive to label but provide valuable information about agent intention. To minimize annotation efforts to only data which is most informative to the learning system, active learning offers a solution by incrementally selecting and annotating data in a cost-effective manner. Our research builds on existing research in uncertainty-driven and diversity-driven active learning, emphasizing the phase transition in model performance with respect to data typicality.

Keywords: Reinforcement Learning, Safety in HRI, Intelligent Transportation Systems
Abstract: While Deep Reinforcement Learning (DRL) has emerged as a promising solution for intricate control tasks, the lack of explainability of the learned policies impedes its uptake in safety-critical applications, such as automated driving systems (ADS). Counterfactual (CF) explanations have recently gained prominence for their ability to interpret black-box Deep Learning (DL) models. CF examples are associated with minimal changes in the input, resulting in a complementary output by the DL model. Finding such alternations, particularly for high-dimensional visual inputs, poses significant challenges. Besides, the temporal dependency introduced by the reliance of the DRL agent action on a history of past state observations further complicates the generation of CF examples. To address these challenges, we propose using a saliency map to identify the most influential input pixels across the sequence of past observed states by the agent. Then, we feed this map to a deep generative model, enabling the generation of plausible CFs with constrained modifications centred on the salient regions. We evaluate the effectiveness of our framework in diverse domains, including ADS, Atari Pong, Pacman and space-invaders games, using traditional performance metrics such as validity, proximity and sparsity. Experimental results demonstrate that this framework generates more informative and plausible CFs than the state-of-the-art for a wide range of environments and DRL agents. In order to foster research in this area, we have made our datasets and codes publicly available at https://github.com/Amir-Samadi/SAFE-RL.

Keywords: Deep Learning Methods, Learning from Experience, Task and Motion Planning
Abstract: Learning from previous experience in motion planning has been shown to significantly reduce planning time in new problems. Prior work in experience based motion planning tends to use limited information, such as only task information, about the motion planning problem when querying a database for solutions. We propose querying a database of motion planning problems using validated point-clouds and task information. The solutions in our database are first clustered based on pairwise Dynamic Time Warping distances. We then build an attention-based model that predicts the cluster of solutions that a new motion planning problem most likely belongs to and only use the solutions in the predicted cluster as guidance when planning. We show this method significantly reduces the planning time, number of collision detections, and database query times across various environment distributions.

Keywords: Visual-Inertial SLAM, Vision-Based Navigation
Abstract: An event camera detects pixel-by-pixel brightness changes and outputs them as asynchronous events, mainly from geometric structures like edges. In this paper, we aim to utilize this line structure information with point features to achieve more accurate localization in indoor or human-made environments. To obtain more accurate line measurements, we propose a novel line detection method with coarse-to-fine motion compensation: detecting straight line measurements within a 2D set of event points, where camera motion is accurately compensated by IMU and contrast maximization. The extracted line features are used together with the point features in a multi-state constraint Kalman filter-based backend. The performance of the proposed method is validated on publicly available datasets and the results show that it improves the accuracy in terms of line detection and pose estimation.

Keywords: Motion and Path Planning, Constrained Motion Planning, Manipulation Planning
Abstract: Motion planning, and in particular in tight settings, is a key problem in robotics and manufacturing. One infamous example for a difficult, tight motion planning problem is the Alpha Puzzle. Many works deal with motion planning in tight scenarios and demonstrate solutions for the Alpha Puzzle in simulation. However, we have not yet seen a real-life robotic arm solving the puzzle in the physical world. The transition from simulation to a real-world solution requires dealing with various difficulties beyond the geometry of the problem, including finding a suitable grasp and robot placement, accounting for model inaccuracies, avoiding robot singularities and more. We present a first demonstration in the real world of an Alpha Puzzle solution with a Universal Robotics UR5e, using a solution path generated from our previous work. After manually placing the robot next to the puzzle pieces, the entire process is automatically planned and executed given the solution path and the robot's placement.

Keywords: Computer Vision for Transportation, Automation Technologies for Smart Cities, Intelligent Transportation Systems
Abstract: Even though a significant amount of work has been done to increase the safety of transportation networks, accidents still occur regularly. They must be understood as an unavoidable and sporadic outcome of traffic networks. We present the TUM Traffic Accident (TUMTraf-A) dataset, a collection of real-world highway accidents. It contains ten sequences of vehicle crashes at high-speed driving with 294,924 labeled 2D and 93,012 labeled 3D boxes and track IDs within 48,144 labeled frames recorded from four roadside cameras and LiDARs at 10 Hz. The dataset contains ten object classes and is provided in the OpenLABEL format. We propose Accid3nD, an accident detection model that combines a rule-based approach with a learning-based one. Experiments and ablation studies on our dataset show the robustness of our proposed method. The dataset, model, and code are available on our project website.

Keywords: Computer Vision for Transportation, Autonomous Vehicle Navigation, Intelligent Transportation Systems
Abstract: Recognizing a traffic accident is an essential part of any autonomous driving or road monitoring system. An accident can appear in a wide variety of forms, and understanding what type of accident is taking place may be useful to prevent it from reoccurring. We approach the problem of accident classification by likening a traffic scene to a graph, where objects such as cars can be represented as nodes, and relative distances and directions between them as edges. This work introduces Scene-Traffic-Graph Inference (STGi), a multi-stage, multimodal pipeline for accident classification. STGi achieves a balanced accuracy score of 57.77% on an (unbalanced) subset of the popular Detection of Traffic Anomaly (DoTA) benchmark, representing an increase of close to 5 percentage points from the case where scene graph information is not taken into account

Keywords: Motion and Path Planning, Task and Motion Planning, Deep Learning Methods
Abstract: A common standard in learning-based robotics is to train a value function model that estimates the cost-to-go between arbitrary starts and goals. Yet training such models, whether through reinforcement learning or supervised learning, can be prohibitively expensive and require a deep neural network to produce a global model of the implicit robot configuration space. In this work we showed that learning a surrogate of cost-to-go, namely the likelihood that a node on a search tree should be expanded, outperformed the baseline which directly predicted the cost-to-go. We trained our Attention-based models using sampling efficiency as an objective function and tested their performances in a set of environment distributions consisting of sequences of randomly placed narrow passages. Moreover, we showed robustness in results when using different input representations of the environments. These findings highlight the potential for more efficient planning using sampling efficiency-based models.

Keywords: Motion and Path Planning, Collision Avoidance
Abstract: As we inch closer to a future of humanoid robots, it has become apparent that we need fast motion planning for extremely high-DOF robots. The state of the art methods for high-DOF motion planning are generally sampling-based. Current attempts to parallelize sampling-based motion planning generally focus on a single avenue for parallelization, either focusing on GPU-based acceleration or traditional multi-core CPU-based approaches. This project, however, intends to use all forms of parallelism in order to maximally accelerate motion planning. The intention is to do this by performing the Spatial Subdivision Probabilistic Roadmap (PRM) algorithm on multiple CPU cores while having a GPU asynchronously perform collision detection queries, all while using modern software engineering approaches.

Keywords: Vision-Based Navigation, Aerial Systems: Perception and Autonomy, Deep Learning for Visual Perception
Abstract: The rapidly increasing capabilities of autonomous mobile robots promise to make them ubiquitous in the coming decade, enhancing efficiency and safety in novel applications such as disaster management, environmental monitoring, and bridge or agricultural inspection. To operate autonomously without constant human intervention, even in remote or hazardous areas, robots must sense, process, and interpret environmental data using only onboard sensing and computation. This capability is enabled by advancements in perception algorithms, allowing these robots to rely primarily on their perception capabilities for navigation tasks. However, tiny robot autonomy is hindered mainly by sensors, memory, and computing due to size, area, weight, and power constraints. This article presents a minimal perception framework, providing insights into designing a compact and efficient perception system for tiny autonomous robots from higher cognitive to lower sensor levels.

Keywords: Vision-Based Navigation, Legged Robots, Localization
Abstract: Visual-inertial odometry (VIO) is one of the useful methods for robot navigation. However, it is challenging for a quadruped robot due to textureless ground image by the low camera position, which degrades the performance of feature-based VIO, and gait-induced vibration, especially causing z-axis position errors. To address this issue, we propose a measurement model leveraging ground constraints. To construct the measurements, a process consisting of mesh clustering, ground plane fitting and tracking is proposed, and the ground measurements are utilized within the Multi-State Constraint Kalman Filter (MSCKF) with feature reprojection error. The proposed VIO is validated using an open-source dataset of a quadruped robot. The results show that the proposed VIO not only outperforms the feature-only MSCKF but also performs comparably to the state-of-the-art quadruped robot VIO without any encoder measurements.

Keywords: Reinforcement Learning, Machine Learning for Robot Control, Motion and Path Planning
Abstract: Assembly tasks, which involve connecting objects, represent some of the most frequently performed activities in both daily routine and industrial manufacturing processes. Assembly tasks are divided into two phases: search and insertion. In particular, the insertion phase has been extensively studied over the past few decades using various control methods. Recently, learning-based approaches have been reported that enable rapid insertion even into holes with clearances smaller than 0.1mm. However, even with a learning-based approach, achieving insertion to the target depth may not be possible depending on the structure of the hole or the small clearance within it. The insertion process is significantly influenced frictions, and unknown contact conditions. For example, when inserting into a hole with decreasing clearance, flexible control is necessary to achieve insertion to the target depth. In this paper, we propose a reinforcement learning-based directional action exploration algorithm for achieving precise insertion to the target depth, addressing variations in the internal structure of holes and small clearances. Our approach directs actions to guide the peg towards the target position, ensuring successful depth attainment. We trained our algorithm using the Isaac Gym simulator and tested the policy on a UR3e manipulator, demonstrating precise depth control during insertion.

Keywords: Intention Recognition, Prosthetics and Exoskeletons, Deep Learning Methods
Abstract: There is a large body of research using Electroencephalogram as a control method in existence. But little to no research as to the foundation of that control method in terms of statistical analysis and simple binary verification. This research would serve to remedy that by taking in EEG data from four electrodes during a simple button push experiment for statistical processing and analysis from an electrical engineering standing point as such to show that when processed and plugged into a machine learning classification algorithm, it is possible to map mental intention of motion to robotics movement in the simplest of terms using non-invasive consumer grade, affordable technology with applicable classification accuracy.