Keywords: Data Sets for Robot Learning, Imitation Learning, Deep Learning Methods
Abstract: Large, high-capacity models trained on diverse datasets have shown remarkable successes on efficiently tackling downstream applications. In domains from NLP to Computer Vision, this has led to a consolidation of pretrained models, with general pretrained backbones serving as a starting point for many applications. Can such a consolidation happen in robotics? Conventionally, robotic learning methods train a separate model for every application, every robot, and even every environment. Can we instead train ``generalist’’ cross-robot policy that can be adapted efficiently to new robots, tasks, and environments? In this paper, we provide datasets in standardized data formats and models to make it possible to explore this possibility in the context of robotic manipulation, alongside experimental results that provide an example of effective cross-robot policies. We assemble a dataset from 22 different robots collected through a collaboration between 21 institutions, demonstrating 527 skills (160266 tasks). We show that a high-capacity model trained on this data, which we call RT-X, exhibits positive transfer and improves the capabilities of multiple robots by leveraging experience from other platforms.

Keywords: Machine Learning for Robot Control, Learning from Demonstration, Big Data in Robotics and Automation
Abstract: We pursue the goal of developing robots that can interact zero-shot with generic unseen objects via a diverse repertoire of manipulation skills and show how passive human videos can serve as a rich source of data for learning such generalist robots. Unlike typical robot learning approaches which directly learn how a robot should act from interaction data, we adopt a factorized approach that can leverage large-scale human videos to learn how a human would accomplish a desired task (a human `plan'), followed by `translating’ this plan to the robot’s embodiment. Specifically, we learn a human `plan predictor’ that, given a current image of a scene and a goal image, predicts the future hand and object configurations. We combine this with a `translation’ module that learns a plan-conditioned robot manipulation policy, and allows following humans plans for generic manipulation tasks in a zero-shot manner with no deployment-time training. Importantly, while the plan predictor can leverage large-scale human videos for learning, the translation module only requires a small amount of in-domain data, and can generalize to tasks not seen during training. We show that our learned system can perform over 16 manipulation skills that generalize to 40 objects, encompassing 100 real-world tasks for table-top manipulation and diverse in-the-wild manipulation. https://homangab.github.io/hopman/

Keywords: Representation Learning, Sensorimotor Learning, Robot Audition
Abstract: Although pre-training on a large amount of data is beneficial for robot learning, current paradigms only perform large-scale pretraining for visual representations, whereas representations for other modalities are trained from scratch. In contrast to the abundance of visual data, it is unclear what relevant internet-scale data may be used for pretraining other modalities such as tactile sensing. Such pretraining becomes increasingly crucial in the low-data regimes common in robotics applications. In this paper, we address this gap by using contact microphones as an alternative tactile sensor. Our key insight is that contact microphones capture inherently audio-based information, allowing us to leverage large-scale audio-visual pretraining to obtain representations that boost the performance of robotic manipulation. To the best of our knowledge, our method is the first approach leveraging large-scale multisensory pre-training for robotic manipulation.

Keywords: Deep Learning Methods, Deep Learning in Grasping and Manipulation
Abstract: We present Self-Adaptive Robust Attention for Robotics Transformers (SARA-RT): a new paradigm for addressing the emerging challenge of scaling up Robotics Transformers (RT) for on-robot deployment. SARA-RT relies on the new method of fine-tuning proposed by us, called up-training. It converts pre-trained or already fine-tuned Transformer-based robotic policies of quadratic time complexity (including massive billion-parameter vision-language-action models or VLAs), into their efficient linear-attention counterparts maintaining high quality. We demonstrate the effectiveness of SARA-RT by speeding up: (a) the class of recently introduced RT-2 models, the first VLA robotic policies pre-trained on internet-scale data, as well as (b) Point Cloud Transformer (PCT) robotic policies operating on large point clouds. We complement our results with the rigorous mathematical analysis providing deeper insight into the phenomenon of SARA.

Keywords: Grasping, Force and Tactile Sensing, Grippers and Other End-Effectors
Abstract: Dexterous manipulation, especially of small daily objects, continues to pose complex challenges in robotics. This paper introduces the DenseTact-Mini, an optical tactile sensor with a soft, rounded, smooth gel surface and compact design equipped with a synthetic fingernail. We propose three distinct grasping strategies: tap grasping using adhesion forces such as electrostatic and van der Waals, fingernail grasping leveraging rolling/sliding contact between the object and fingernail, and fingertip grasping with two soft fingertips. Through comprehensive evaluations, the DenseTact-Mini demonstrates a lifting success rate exceeding 90.2% when grasping various objects, including items such as 1mm basil seeds, thin paperclips, and items larger than 15mm such as bearings. This work demonstrates the potential of soft optical tactile sensors for dexterous manipulation and grasping.

Keywords: Bimanual Manipulation, Constrained Motion Planning, Kinematics
Abstract: In order for a bimanual robot to manipulate an object that is held by both hands, it must construct motion plans such that the transformation between its end effectors remains fixed. This amounts to complicated nonlinear equality constraints in the configuration space, which are difficult for trajectory optimizers. In addition, the set of feasible configurations becomes a measure zero set, which presents a challenge to sampling-based motion planners. We leverage an analytic solution to the inverse kinematics problem to parametrize the configuration space, resulting in a lower-dimensional representation where the set of valid configurations has positive measure. We describe how to use this parametrization with existing motion planning algorithms, including sampling-based approaches, trajectory optimizers, and techniques that plan through convex inner-approximations of collision-free space.

Keywords: Deep Learning for Visual Perception, Deep Learning Methods, Visual Learning
Abstract: In order to navigate safely and reliably in novel environments, robots must estimate perceptual uncertainty when confronted with out-of-distribution (OOD) obstacles not seen in training data. We present a method to accurately estimate pixel-wise uncertainty in semantic segmentation without requiring real or synthetic OOD examples at training time. From a shared per-pixel latent feature representation, a classification network predicts a categorical distribution over semantic labels, while a normalizing flow estimates the probability density of features under the training distribution. The label distribution and density estimates are combined in a Dirichlet-based evidential uncertainty framework that efficiently computes epistemic and aleatoric uncertainty in a single neural network forward pass. Our method is enabled by three key contributions. First, we simplify the problem of learning a transformation to the training data density by starting from a fitted Gaussian mixture model instead of the conventional standard normal distribution. Second, we learn a richer and more expressive latent pixel representation to aid OOD detection by training a decoder to reconstruct input image patches. Third, we perform theoretical analysis of the loss function used in the evidential uncertainty framework and propose a principled objective that more accurately balances training the classification and density estimation networks. We demonstrate the accuracy of our uncertainty estimation approach under long-tail OOD obstacle classes for semantic segmentation in both off-road and urban driving environments.

Keywords: SLAM
Abstract: Neural implicit representations have emerged as a promising solution for addressing the challenges of Simultaneous Localization and Mapping (SLAM) problems in indoor scenes. This paper presents NGEL-SLAM, a low-latency global consistent SLAM system that utilizes neural implicit scene representation. To ensure global consistency, our system incorporates loop closure in the tracking module and maintains a global consistent map by representing the scene using multiple neural implicit fields and performing a quick adjustment to the loop closure. The fast convergence and rapid response to loop closure make our system a truly low-latency system that achieves global consistency. The neural implicit representation enables the rendering of high-fidelity RGB-D images with the extraction of explicit, dense, and interactive surfaces. Experiments were conducted on both synthetic and real-world datasets to evaluate the effectiveness of the proposed approach. The results demonstrate the achieved tracking and mapping accuracy and low-latency performance of our system.

Keywords: Visual Tracking, Deep Learning for Visual Perception, Computer Vision for Transportation
Abstract: 3D single object tracking (SOT) is an important and challenging task for the autonomous driving and mobile robotics. Most existing methods perform tracking between two consecutive frames while ignoring the motion patterns of the target over a series of frames, which would cause performance degradation in scenes with sparse points. To break through this limitation, we introduce Sequence-to-Sequence tracking paradigm and a tracker named SeqTrack3D to capture target motion across continuous frames. Unlike previous methods that primarily adopted three strategies: matching two consecutive point clouds, predicting relative motion, or utilizing sequential point clouds to address feature degradation, our SeqTrack3D combines both historical point clouds and bounding box sequences. This novel approach ensures robust tracking by leveraging location priors from historical boxes, even in scenes with sparse points. Extensive experiments conducted on large-scale datasets show that SeqTrack3D achieves new state-of-the-art performances, improving by 6.00% on NuScenes and 14.13% on Waymo dataset.

Keywords: Localization, Visual-Inertial SLAM, SLAM
Abstract: In this paper, we strongly advocate square-root covariance (instead of information) filtering for visual-inertial navigation, in particular on resource-constrained edge devices, because of its superior efficiency and numerical stability. Although Visual-Inertial Navigation Systems (VINS) have made tremendous progress in recent years, they still face resource stringency and numerical instability on embedded systems when imposing limited word length. To overcome these challenges, we develop an ultrafast and numerically-stable square-root filter (SRF)-based VINS algorithm (i.e., SR-VINS). The numerical stability of the proposed SR-VINS is inherited from the adoption of square-root covariance while the never-before-seen efficiency is largely enabled by the novel SRF update method that is based on our new permuted-QR (P-QR), which fully utilizes and properly maintains the upper triangular structure of the square-root covariance matrix. Furthermore, we choose a special ordering of the state variables which is amenable for (P-)QR operations in the SRF propagation and update and prevents unnecessary computation. The proposed SR-VINS is validated extensively through numerical studies, demonstrating that when the state-of-the-art (SOTA) filters have numerical difficulties, our SR-VINS has superior numerical stability, and remarkably, achieves efficient and robust performance on 32-bit single-precision float at a speed nearly twice as fast as the SOTA methods. We also conduct comprehensive real-world experiments to validate the efficiency, accuracy, and robustness of the proposed SR-VINS.

Keywords: Learning from Demonstration, Imitation Learning, Reinforcement Learning
Abstract: Real-world robotic tasks stretch over extended horizons and encompass multiple stages. Learning long-horizon manipulation tasks, however, is a long-standing challenge, and demands decomposing the overarching task into several manageable subtasks to facilitate policy learning and generalization to unseen tasks. Prior task decomposition methods require task-specific knowledge, are computationally intensive, and cannot readily be applied to new tasks. To address these shortcomings, we propose Universal Visual Decomposer (UVD), an off-the-shelf task decomposition method for visual long-horizon manipulation using pre-trained visual representations for robotic control. At a high level, UVD discovers subgoals by detecting phase shifts in the embedding space of the pre-trained representation. Operating purely on visual demonstrations without auxiliary information, UVD can effectively extract visual subgoals embedded in the videos, while incurring zero additional training cost on top of standard visuomotor policy training. Goal-conditioned policies learned with UVD-discovered subgoals exhibit significantly improved compositional generalization at test time to unseen tasks. Furthermore, UVD-discovered subgoals can be used to construct goal-based reward shaping that jump-starts temporally extended exploration for reinforcement learning. We extensively evaluate UVD on both simulation and real-world tasks, and in all cases, UVD substantially outperforms baselines across imitation and reinforcement learning settings on in-domain and out-of-domain task sequences alike, validating the clear advantage of automated visual task decomposition within the simple, compact UVD framework.

Keywords: Deep Learning for Visual Perception, Visual Learning, Computer Vision for Automation
Abstract: Given its wide application in robotics, point cloud registration is a widely researched topic. Conventional methods aim to find a rotation and translation that align two point clouds in 6 degrees of freedom (DoF). However, certain tasks in robotics, such as category-level pose estimation, involve non-uniformly scaled point clouds, requiring a 9DoF transform for accurate alignment. We propose HEGN, a novel equivariant graph neural network for 9DoF point cloud registration. HEGN utilizes equivariance to rotation, translation, and scaling to estimate the transformation without relying on point correspondences. Based on graph representations for both point clouds, we extract equivariant node features aggregated in their local, cross-, and global context. In addition, we introduce a novel node pooling mechanism that leverages the cross-context importance of nodes to pool the graph representation. By repeating the feature extraction and node pooling, we obtain a graph hierarchy. Finally, we determine rotation and translation by aligning equivariant features aggregated over the graph hierarchy. To estimate scaling, we leverage scale information contained in the vector norm of the equivariant features. We evaluate the effectiveness of HEGN through experiments with the synthetic ModelNet40 dataset and the real-world ScanObjectNN dataset. The results show the superior performance of HEGN in 9DoF point cloud registration and its competitive performance in conventional 6DoF point cloud registration.

Keywords: Motion and Path Planning, Reactive and Sensor-Based Planning, Field Robots
Abstract: Performing autonomous navigation in cluttered and unstructured terrains still remains a challenging task for legged and wheeled mobile robots. To accomplish such a task, online planners shall incorporate new terrain information perceived while the robot is moving within its environment. While hybrid mobility robots offer high flexibility in traversing challenging terrains by leveraging the advantages of both wheeled and legged locomotion, the effective hybrid planning of the mobility actions that transparently combine both modes of locomotion has not been extensively explored.

In this work, we present a hierarchical online hybrid primitive-based planner for autonomous navigation with wheeled-legged robots. The framework is handled by a Behavior Tree (BT) and it takes into account recovery methods to deal with possible failures during the execution of the navigation/mobility plan. The framework was evaluated in multiple irregular and heavily cluttered simulated environments randomly generated and in real-world trials, using the CENTAURO robot platform. With these experiments, we demonstrated autonomous capabilities without any human intervention, even in case of collision or planner failures.

Keywords: Motion and Path Planning, Machine Learning for Robot Control, AI-Based Methods
Abstract: Ensuring safety and meeting temporal specifications are critical challenges for long-term robotic tasks. Signal temporal logic (STL) has been widely used to systematically and rigorously specify these requirements. However, traditional methods of finding the control policy under those STL requirements are computationally complex and not scalable to high-dimensional or systems with complex nonlinear dynamics. Reinforcement learning (RL) methods can learn the policy to satisfy the STL specifications via hand-crafted or STL-inspired rewards, but might encounter unexpected behaviors due to ambiguity and sparsity in the reward. In this paper, we propose a method to directly learn a neural network controller to satisfy the requirements specified in STL. Our controller learns to roll out trajectories to maximize the STL robustness score in training. In testing, similar to Model Predictive Control (MPC), the learned controller predicts a trajectory within a planning horizon to ensure the satisfaction of the STL requirement in deployment. A backup policy is designed to ensure safety when our controller fails. Our approach can adapt to various initial conditions and environmental parameters. We conduct experiments on six tasks, where our method with the backup policy outperforms the classical methods (MPC, STL-solver), model-free and model-based RL methods in STL satisfaction rate, especially on tasks with complex STL specifications while being 10X-100X faster than the classical method

Keywords: Motion and Path Planning, Marine Robotics
Abstract: Many applications of path planning in time-varying flow fields, particularly in areas such as marine robotics and ship routing, can be modelled as instances of the time-varying shortest path (TDSP) problem. Although there are no known polynomial-time solutions to TDSP in general, our recent work has identified a tractable case where the flow is modelled as piecewise constant. Extending this method to allow for computational reuse in larger multi-query problems, however, requires additional thought. This paper shows that the piecewise-linear form of the cost function employed in previously work can be used to build an analogy of a shortest path tree, thereby enabling optimal concatenation of sub-problem solutions in the absence of an optimal substructure, and without uniform time discretisation. We present a framework for multi-query TDSP that finds an optimal path that passes through a defined sequence of waypoints and is computationally efficient. Performance comparison is provided in simulation that shows large (up to 100x) speedup compared to a naive approach. This result is significant for applications such as ship routing, where route evaluation is a desirable capability.

Keywords: Motion and Path Planning, Planning, Scheduling and Coordination
Abstract: We propose a new method called Clustering Topological PRM (CTopPRM) for finding multiple distinct paths in 3D cluttered environments. Finding such distinct paths is useful in many applications. Among others, using multiple distinct paths is necessary for optimization-based trajectory planners where found trajectories are restricted to only a single homotopy class of a given path. Distinct paths can also be used to guide sampling-based motion planning and thus increase the effectiveness of planning in environments with narrow passages. Graph-based representation called roadmap is a common representation for path planning and also for finding multiple distinct paths. Yet, challenging environments with multiple narrow passages require a densely sampled roadmap to capture the connectivity of the environment. Searching such a dense roadmap for multiple paths is computationally too expensive. The majority of existing methods construct only a sparse roadmap which, however, struggles to find all distinct paths in challenging environments. To this end, we propose the CTopPRM which creates a sparse graph by clustering an initially sampled dense roadmap. Such a reduced roadmap allows fast identification of homotopically distinct paths captured in the dense roadmap. We show, that compared to the existing methods the CTopPRM improves the probability of finding all distinct paths by almost 20%, during same run-time. The source code of our method is released as an open-source package.

Keywords: Motion and Path Planning, Optimization and Optimal Control, Collision Avoidance
Abstract: This paper presents a novel Stochastic Optimal Control (SOC) method based on Model Predictive Path Integral control (MPPI), named Stein Variational Guided MPPI (SVG-MPPI), designed to handle rapidly shifting multimodal optimal action distributions. While MPPI can find a Gaussian-approximated optimal action distribution in closed form, i.e., without iterative solution updates, it struggles with the multimodality of the optimal distributions. This is due to the less representative nature of the Gaussian. To overcome this limitation, our method aims to identify a target mode of the optimal distribution and guide the solution to converge to fit it. In the proposed method, the target mode is roughly estimated using a modified Stein Variational Gradient Descent (SVGD) method and embedded into the MPPI algorithm to find a closed-form ``mode-seeking'' solution that covers only the target mode, thus preserving the fast convergence property of MPPI. Our simulation and real-world experimental results demonstrate that SVG-MPPI outperforms both the original MPPI and other state-of-the-art sampling-based SOC algorithms in terms of path-tracking and obstacle-avoidance capabilities.

Keywords: Computational Geometry, Simulation and Animation, Motion and Path Planning
Abstract: This paper introduces a novel, lightweight method to solve the visibility problem for 2D grids. The proposed method evaluates the existence of lines-of-sight from a source point to all other grid cells in a single pass with no preprocessing and independently of the number and shape of obstacles. It has a compute and memory complexity of mathcal{O}(n), where n = n_{x}times n_{y} is the size of the grid, and requires at most ten arithmetic operations per grid cell. In the proposed approach, we use a linear first-order hyperbolic partial differential equation to transport the visibility quantity in all directions. In order to accomplish that, we use an entropy-satisfying upwind scheme that converges to the true visibility polygon as the step size goes to zero. This dynamic-programming approach allows the evaluation of visibility for an entire grid much faster than typical algorithms. We provide a practical application of our proposed algorithm by posing the visibility quantity as a heuristic and implementing a deterministic, local-minima-free path planner, setting apart the proposed planner from traditional methods. Lastly, we provide necessary algorithms and an open-source implementation of the proposed methods.

Keywords: Motion and Path Planning, Wheeled Robots
Abstract: In this paper, we propose a real-time clothoid tree-based path planning for self-driving robots. Clothoids, curves that exhibit linear curvature profiles, play an important role in road design and path planning due to their appealing properties. Nevertheless, their real-time applications face considerable challenges, primarily stemming from the lack of a closed-form clothoid expression. To address these challenges, we introduce two innovative techniques: 1) an efficient and precise clothoid approximation using the Gauss-Legendre quadrature; and 2) a data-efficient decoder for interpolating clothoid splines that leverages the symmetry and similarity of clothoids. These techniques are demonstrated with numerical examples. The clothoid approximation ensures an accurate and smooth representation of the curve, and the clothoid spline decoder effectively accelerates the clothoid tree exploration by relaxing the problem constraints and reducing the problem size. Both techniques are integrated into our path planning algorithm and evaluated in various driving scenarios.

Keywords: Motion and Path Planning, Multi-Robot Systems, Path Planning for Multiple Mobile Robots or Agents
Abstract: Path planning for multiple non-holonomic robots in continuous domains constitutes a difficult robotics challenge with many applications. Despite significant recent progress on the topic, computationally efficient and high-quality solutions are lacking, especially in lifelong settings where robots must continuously take on new tasks. In this work, we make it possible to extend key ideas enabling state-of-the-art (SOTA) methods for multi-robot planning in discrete domains to the motion planning of multiple Ackerman (car-like) robots in lifelong settings, yielding high-performance centralized and decentralized planners. Our planners compute trajectories that allow the robots to reach precise SE(2) goal poses. The effectiveness of our methods is thoroughly evaluated and confirmed using both simulation and real-world experiments.

Keywords: Motion and Path Planning, Energy and Environment-Aware Automation
Abstract: Continuous exploration without interruption is important in scenarios such as search and rescue and precision agriculture, where consistent presence is needed to detect events over large areas. Ergodic search already derives continuous trajectories in these scenarios so that a robot spends more time in areas with high information density. However, existing literature on ergodic search does not consider the robot's energy constraints, limiting how long a robot can explore. In fact, if the robots are battery-powered, it is physically not possible to continuously explore on a single battery charge. Our paper tackles this challenge, integrating ergodic search methods with energy-aware coverage. We trade off battery usage and coverage quality, maintaining uninterrupted exploration by at least one agent. Our approach derives an abstract battery model for future state-of-charge estimation and extends canonical ergodic search to ergodic search under battery constraints. Empirical data from simulations and real-world experiments demonstrate the effectiveness of our energy-aware ergodic search, which ensures continuous exploration and guarantees spatial coverage.

Keywords: Actuation and Joint Mechanisms, Compliant Joints and Mechanisms, Compliance and Impedance Control
Abstract: Series Elastic Actuator-based exoskeleton can offer precise torque control and transparency when interacting with human wearers. Accurate control of SEA-produced torques ensures the wearer’s voluntary motion and supports the implementation of multiple assistive paradigms. In this paper, a novel variable transmission series elastic actuator (VTSEA) is developed to meet torque-speed requirements in different exoskeleton-assisted locomotion modes, such as running, walking, sit-to-stand, and stand-to-sit. The VTSEA features a SEA-coupled variable transmission ratio adjusting mechanism and works between three discrete levels of transmission ratio depending on the user’s initiative. The proposed prototype can also improve transparency in human-robot interaction. Also, an accurate torque controller with inertial compensation is developed for the VTSEA via the singular perturbation theory, and its stability is proved. The feasibility of the proposed VTSEA prototype and the precise output torque performance of VTSEA are verified by experiments.

Keywords: Actuation and Joint Mechanisms, Compliant Joints and Mechanisms, Mechanism Design
Abstract: Actuation concepts such as Series Elastic Actuation (SEA), Parallel Elastic Actuation (PEA), and Biarticular Actuation (BA), which introduce elastic elements into the structure, have the potential to reduce the electrical energy consumption of a robot. This letter presents an optimization of the arrangement of springs for a 3 degrees of freedom robotic arm, with the aim of decreasing the electrical energy consumption for a given pick-and-place task. Through simulations and experimental validation, we show that the optimal configuration in terms of electrical energy consumption and complexity consists of rigid actuation on joint 1 and PEAs on joints 2 and 3. With this configuration, root mean square (RMS) and peak load torques for a specific pick-and-place task can be reduced respectively by up to 43% and 44% for joint 2, and by 15% and 21% for joint 3 compared to the configuration without springs.

Keywords: Actuation and Joint Mechanisms, Compliant Joints and Mechanisms, Mechanism Design
Abstract: The safer interaction between humans and robots is one of the challenges in robotics. To protect humans and robots from impact, researchers have developed many different soft robots, which incorporate mechanical springs into their joints. The forthcoming generation of soft robots necessitates adaptable joint stiffness to accommodate various tasks. Consequently, the development of variable stiffness joints (VSA) has become crucial. Among the prevalent approaches for stiffness adjustment, lever mechanisms have been implemented in numerous variable stiffness joints. Nonetheless, the integration of the lever technology into VSA often faces challenges in achieving a compact design. This paper introduces a mechanically compact design for a novel lever-cam-based variable stiffness joint.

Keywords: Actuation and Joint Mechanisms, Compliant Joints and Mechanisms, Mechanism Design
Abstract: Variable stiffness actuator (VSA) can imitate natural muscles in their compliance capbility, which can provide flexible adaptability for robots, improving the safety of robots interacting with the environment or human. This paper presents a new compact serial variable stiffness actuator ((SVSA-III)) with linear stiffness profile based on symmetrical variable lever arm mechanism. The stiffness motor is used to regulate the position of the pivot located on the Archimedean Spiral Relocation Mechanism (ASRM), so that the stiffness of the actuator can be adjusted (softening or hardening). By designing the lever length, the range of stiffness adjustment can change from 0.3Nm/degree to therotical infinity. Moreover, the continuous linear stiffness profile of the actuator can be customized by solving the transcendental equation of the relationship between the actuator stiffness and the rotation angle of the stiffness motor. SVSA-III has the advantages of compact structure, wide-range stiffness regulation, reduced control difficulty, and linear stiffness profile. Two experiments of step response and stiffness tracking have proved the high accuracy and fast response for both theoretical stiffness and position adjustment.

Keywords: Actuation and Joint Mechanisms, Compliant Joints and Mechanisms, Optimization and Optimal Control
Abstract: Elastic actuation taps into elastic elements' energy storage for dynamic motions beyond rigid actuation. While Series Elastic Actuators (SEA) and Variable Stiffness Actuators (VSA) are highly sophisticated, they do not fully provide control over energy transfer timing. To overcome this problem on the basic system level, the Bi-Stiffness Actuation (BSA) concept was recently proposed. Theoretically, it allows for full link decoupling, while simultaneously being able to lock the spring in the drive train via a switch-and-hold mechanism. Thus, the user would be in full control of the potential energy storage and release timing. In this work, we introduce an initial proof-of-concept of Bi-Stiffness-Actuation in the form of a 1-DoF physical prototype, which is implemented using a modular testbed. We present a hybrid system model, as well as the mechatronic implementation of the actuator. We corroborate the feasibility of the concept by conducting a series of hardware experiments using an open-loop control signal obtained by trajectory optimization. Here, we compare the performance of the prototype with a comparable SEA implementation. We show that BSA outperforms SEA 1) in terms of maximum velocity at low final times and 2) in terms of the movement strategy itself: The clutch mechanism allows the BSA to generate consistent launch sequences while the SEA has to rely on lengthy and possibly dangerous oscillatory swing-up motions.

Keywords: Actuation and Joint Mechanisms, Mechanism Design
Abstract: Recent trends in robotic actuation have highlighted the need for low-cost, high performance, and efficient gearing. We present an experimental study comparing pinwheel and non-pinwheel designs of cycloidal gearing. The open source designs are 3D-printable combined with off-the-shelf components, achieving a high performance-to-cost ratio. Extensive experimental data is presented, that compares two prototypes on run-in behaviour and a number of quantitative metrics including transmission error, play, friction, and stiffness. Furthermore, we assess overall actuator performance through position control experiments, and a 10-hour endurance test. The results show strong performance characteristics, and crucially, suggest that non-pinwheel designs of cycloidal gearing can be a lower complexity and cost alternative to classical pinwheel designs, while offering similar performance.

Keywords: Hydraulic/Pneumatic Actuators, Actuation and Joint Mechanisms, Modeling, Control, and Learning for Soft Robots
Abstract: A novel origami-inspired foldable pneumatic actuator is proposed in this letter to satisfy the comprehensive requirements in wearable assistive application. The pneumatic actuator combines the origami structure of designed Quadrangular-Expand pattern and the foldable pneumatic bellow. Integrated origami structure regulates the motion of actuator with high contraction ratio and enables accurate modeling. The origami framework also improves the strength of bearing negative pressure, and thus can provide bidirectional actuation. The workflow including design, fabrication and mathematic modeling of the pneumatic actuator is presented in detail. Based on the actuator modeling, the multi-objective optimization for parameters using Genetic Algorithm is then conducted to obtain the trade-off design. The verifications for static characteristics of output torque, as well as the dynamic characteristics of power density, mechanical efficiency and frequency response, have been conducted. In summary, the proposed actuator is powerful and energy-efficient.

Keywords: Hydraulic/Pneumatic Actuators
Abstract: A pneumatic linear actuator is presented and evaluated. Designed to operate in demanding environments such as MRI, it is developed to be used with two motion control modes: 1) a step-by-step mode with tooth-based gripping to ensure precision, 2) a continuous mode available locally for fine positioning. The actuator can also be disengaged to enable direct handling by an operator, for example for comanipulation. The design is presented. A prototype, developed in the medical context, is implemented and characterized. A specific step-by-step control sequence is then elaborated based on its characterization. Testing of the dual-mode actuation is finally described. The complementarity between the two motion modes and possible adaptations of the original design are discussed.

Keywords: Compliant Joints and Mechanisms, Actuation and Joint Mechanisms, Legged Robots
Abstract: Sitting down and standing up from a chair and, similarly, moving heavy objects up and down between factory lines are examples of cyclic tasks that require large forces but little to no net mechanical energy. Motor-driven artificial limbs and industrial robots can help humans do these tasks, but motors require energy to provide force even if they supply no net mechanical energy. Springs are energetically conservative mechanical elements useful for building robots that require no energy when performing cyclic tasks. However, conventional springs can be limited by their non-customizable force-deflection behavior -- for example, when they cannot meet the force demand despite storing enough energy to perform a cyclic task. Variable stiffness springs are a special type of spring with customizable force-deflection behavior, but most typical variable stiffness springs require energy to amplify force similar to motors. In this paper, we introduce a new type of variable stiffness spring design which is energetically conservative despite having a customizable force-deflection behavior. We present the theory of these springs and demonstrate their utility in performing a net-zero mechanical energy cost lifting task that requires force amplification and as such is not realizable using conventional springs.

Keywords: Deep Learning Methods, Kinematics
Abstract: In robotics and biomechanics, accurately determining joint parameters and computing the corresponding forward and inverse kinematics are critical yet often challenging tasks, especially when dealing with highly individualized and partly unknown systems. This paper unveils a cutting-edge kinematic optimizer, underpinned by an autoencoder-based architecture, to address these challenges. Utilizing a neural network, our approach simulates inverse kinematics, converting measurement data into joint-specific parameters during encoding, enabling a stable optimization process. These parameters are subsequently processed through a predefined, differentiable forward kinematics model, resulting in a decoded representation of the original data. Beyond offering a comprehensive solution to kinematics challenges, our method also unveils previously unidentified joint parameters. Real experimental data from knee and hand joints validate the optimizer's efficacy. Additionally, our optimizer is multifunctional: it streamlines the modeling and automation of kinematics and enables a nuanced evaluation of diverse modeling techniques. By assessing the differences in reconstruction losses, we illuminate the merits of each approach. Collectively, this preliminary study signifies advancements in kinematic optimization, with potential applications spanning both biomechanics and robotics.

Keywords: Dynamics, Mechanism Design, Kinematics, trajectory optimization
Abstract: Water jumping motion extends the robot's movement space and flexibility. However, the jumping performance is influenced by multiple

factors such as driving force, rowing trajectory and robot structure. The interaction force between the robot and water surface is complicated due to water deformation, and the difficulty of the water jumping increases with the robot's scale. This paper designs a miniature water jumping robot with rowing driving legs. The hydrodynamic model between driving legs and water is established based on the modified Wagner theory with consideration of water surface deformation. Particularly, the dynamic model of the robot for the whole

jumping process is also developed relate to multiple factors. Then the jumping performance is improved by optimizing the energy storage modality, rowing trajectory and supporting leg shapes through the theoretical analysis and experiments. The fabricated robot weights 91 g, and its length, width and height are 220 mm, 410 mm and 95 mm respectively. The maximum water jumping height and distance are 241 and

965 mm.

Keywords: Kinematics, Constrained Motion Planning
Abstract: In this paper, we present an iterative method to quickly traverse multi-dimensional paths considering jerk constraints. As a first step, we analyze the traversal of each individual path dimension. We derive a range of feasible target accelerations for each intermediate waypoint of a one-dimensional path using a binary search algorithm. Computing a trajectory from waypoint to waypoint leads to the fastest progress on the path when selecting the highest feasible target acceleration. Similarly, it is possible to calculate a trajectory that leads to minimum progress along the path. This insight allows us to control the traversal of a one-dimensional path in such a way that a reference path length of a multi-dimensional path is approximately tracked over time. In order to improve the tracking accuracy, we propose an iterative scheme to adjust the temporal course of the selected reference path length. More precisely, the temporal region causing the largest position deviation is identified and updated at each iteration. In our evaluation, we thoroughly analyze the performance of our method using seven-dimensional reference paths with different path characteristics. We show that our method manages to quickly traverse the reference paths and compare the required traversing time and the resulting path accuracy with other state-of-the-art approaches.

Keywords: Kinematics, Formal Methods in Robotics and Automation, Soft Robot Applications
Abstract: This letter presents an investigation into the mathematical relationships between the positions and orientations at the segment tips of a piecewise constant curvature (PCC) continuum robot with up to three segments. For one-segment, a reachability criterion is proposed, which simplifies the calculation of the neighboring orientation. For two-segments, a reachability criterion is proposed and the redundancy of its inverse kinematics solution is found, establishing a circle of tip locations. For three-segments, the redundancy of the inverse kinematics includes tips that lie on a sphere providing a closed-form solution to the inverse kinematics problem. These relationships are derived from the unique characteristics of the bisecting plane of a single segment. The degenerate cases for the solutions are also addressed. These outcomes stem from a specific PCC parametrization, with implications extending to the general PCC model. Note that this study is grounded solely in simulation.

Keywords: Kinematics, Nonholonomic Motion Planning, Motion and Path Planning
Abstract: We present an analytic solution to the 3D Dubins path problem for paths composed of an initial circular arc, a straight component, and a final circular arc. These are commonly called CSC paths. By modeling the start and goal configurations of the path as the base frame and final frame of an RRPRR manipulator, we treat this as an inverse kinematics problem. The kinematic features of the 3D Dubins path are built into the constraints of our manipulator model. Furthermore, we show that the number of solutions is not constant, with up to seven valid CSC path solutions even in non-singular regions. An implementation of solution is available at https://github.com/aabecker/dubins3D

Keywords: Kinematics, Optimization and Optimal Control, Parallel Robots
Abstract: Performance measures are essential to characterize a robot's ability to carry out manipulation tasks. Generally, these measures examine the system’s kinematic transformations from configuration to task space, but the Capacity margin, a polytope based kinetostatic index, provides additionally, both an accurate evaluation of the twist and wrench capacities of a robotic manipulator. However, this index is the minimum of a discontinuous scalar function leading to difficulties when computing gradients thereby rendering it unsuitable for online numerical optimization. In this letter, we propose a novel performance index using an approximation of the capacity margin. The proposed index is continuous and differentiable, characteristics that are essential for modelling smooth and predictable system behavior. We demonstrate the effectiveness both as a constraint and objective function for inverse kinematics optimization. Moreover, to show its practical use, two opposing robot architectures are chosen: (i) Serial robot - Universal Robot- UR5 (6-dof); Rethink Robotics- Sawyer Robot (7-dof) and (ii) Parallel manipulator - Cable Driven Parallel Robot to validate the results through both simulation and experiments. A visual representation of the performance index is also presented.

Keywords: Kinematics, Industrial Robots
Abstract: Robotic arms are highly common in various automation processes such as manufacturing lines. However, these highly capable robots are usually degraded to simple repetitive tasks such as pick-and-place. On the other hand, designing an optimal robot for one specific task consumes large resources of engineering time and costs. In this paper, we propose a novel concept for optimizing the fitness of a robotic arm to perform a specific task based on human demonstration. Fitness of a robot arm is a measure of its ability to follow recorded human arm and hand paths. The optimization is conducted using a modified variant of the Particle Swarm Optimization for the robot design problem. In the proposed approach, we generate an optimal robot design along with the required path to complete the task. The approach could reduce the time-to-market of robotic arms and enable the standardization of modular robotic parts. Novice users could easily apply a minimal robot arm to various tasks. Two test cases of common manufacturing tasks are presented yielding optimal designs and reduced computational effort by up to 92%.

Keywords: Kinematics, Learning from Demonstration
Abstract: Trajectory segmentation refers to dividing a trajectory into meaningful consecutive sub-trajectories. This paper focuses on trajectory segmentation for 3D rigid-body motions. Most segmentation approaches in the literature represent the body’s trajectory as a point trajectory, considering only its translation and neglecting its rotation. We propose a novel trajectory representation for rigid-body motions that incorporates both translation and rotation, and additionally exhibits several invariant properties. This representation consists of a geometric progress rate and a third-order trajectory-shape descriptor. Concepts from screw theory were used to make this representation time-invariant and also invariant to the choice of body reference point. This new representation is validated for a self-supervised segmentation approach, both in simulation and using real recordings of human-demonstrated pouring motions. The results show a more robust detection of consecutive sub-motions with distinct features and a more consistent segmentation compared to conventional representations. We believe that other existing segmentation methods may benefit from using this trajectory representation to improve their invariance.

Keywords: Kinematics, Modeling, Control, and Learning for Soft Robots
Abstract: A new, and extremely efficient, computational modeling paradigm is introduced here for specific finite elasticity problems that arise in the context of soft robotics. Whereas continuum mechanics is a very classical area of study that is broadly applicable throughout engineering, and significant effort has been devoted to the development of intricate constitutive models for finite elasticity, we show that for the most part, the isochoric (locally volume-preserving) constraint dominates behavior, and this can be built into closed-form kinematic deformation fields before even considering other aspects of constitutive modeling. We therefore focus on developing and applying primitive deformations that each observe this constraint. By composing a wide enough variety of such deformations, many of the most common behaviors observed in soft robots can be replicated. Case studies include isotropic objects subjected to different boundary conditions, a non-isotropic helically-reinforced tube, and a not-purely-kinematic scenario with gravity loading. We show that this method is at least 50 times faster than the ABAQUS implementation of the finite element method (FEM), and has speed comparable with the real-time FEM framework SOFA. Experiments show that both our method and ABAQUS have approximately 10% error relative to experimentally measured displacements, as well as to each other. And our method outperforms SOFA when the deformation is highly nonlinear.

Keywords: Multi-Robot Systems, Constrained Motion Planning, Cooperating Robots
Abstract: We propose a shared semantic map architecture to construct and configure Model Predictive Controllers (MPC) dynamically, that solve navigation problems for multiple robotic agents sharing parts of the same environment. The navigation task is represented as a sequence of semantically labeled areas in the map, that must be traversed sequentially, i.e. a route. Each semantic label represents one or more constraints on the robots’ motion behaviour in that area. The advantages of this approach are: (i) an MPC-based motion controller in each individual robot can be (re-)configured, at runtime, with the locally and temporally relevant parameters; (ii) the application can influence, also at runtime, the navigation behaviour of the robots, just by adapting the semantic labels; and (iii) the robots can reason about their need for coordination, through analyzing over which horizon in time and space their routes overlap. The paper provides simulations of various representative situations, showing that the approach of runtime configuration of the MPC drastically decreases computation time, while retaining task execution performance similar to an approach in which each robot always includes all other robots in its MPC computations.

Keywords: Multi-Robot Systems, Constrained Motion Planning, Reinforcement Learning
Abstract: This paper presents a decentralized cooperative motion planning approach for surface inspection of 3D structures which includes uncertainties like size, number, shape, position, using multi-robot systems (MRS). Given that most of existing works mainly focus on surface inspection of single and fully known 3D structures, our motivation is two-fold: first, 3D structures separately distributed in 3D environments are complex, therefore the use of MRS intuitively can facilitate an inspection by fully taking advantage of sensors with different capabilities. Second, performing the aforementioned tasks when considering uncertainties is a complicated and time-consuming process because we need to explore, figure out the size and shape of 3D structures and then plan surface-inspection path. To overcome these challenges, we present a meta-learning approach that provides a decentralized planner for each robot to improve the exploration and surface inspection capabilities. The experimental results demonstrate our method can outperform other methods by approximately 10.5%-27% on success rate and 70%-75% on inspection speed.

Keywords: Multi-Robot Systems, Motion and Path Planning, Distributed Robot Systems
Abstract: This paper proposes a decentralized trajectory planning framework for the collision avoidance problem of multiple micro aerial vehicles (MAVs) in environments with static and dynamic obstacles. The framework utilizes spatiotemporal occupancy grid maps (SOGM), which forecast the occupancy status of neighboring space in the near future, as the environment representation. Based on this representation, we extend the kinodynamic A* and the corridor-constrained trajectory optimization algorithms to efficiently tackle static and dynamic obstacles with arbitrary shapes. Collision avoidance between communicating robots is integrated by sharing planned trajectories and projecting them onto the SOGM. The simulation results show that our method achieves competitive performance against state-of-the-art methods in dynamic environments with different numbers and shapes of obstacles. Finally, the proposed method is validated in real experiments.

Keywords: Multi-Robot Systems, Path Planning for Multiple Mobile Robots or Agents
Abstract: We propose a decentralised multi-robot coordination algorithm that features a rich representation for encoding and communicating each robot’s intent. This representation for “intent messages” enables improved coordination behaviour and communication efficiency in difficult scenarios, such as those where there are unknown points of contention that require negotiation between robots. Each intent message is an adaptive policy that conditions on identified points of contention that conflict with the intentions of other robots. These policies are concisely expressed as behaviour trees via algebraic logic simplification, and are interpretable by robot teammates and human operators. We propose this intent representation in the context of the Dec-MCTS online planning algorithm for decentralised coordination. We present results for a generalised multi-robot orienteering domain that show improved plan convergence and coordination performance over standard Dec-MCTS enabled by the intent representation’s ability to encode and facilitate negotiation over points of contention.

Keywords: Multi-Robot Systems, Path Planning for Multiple Mobile Robots or Agents, Planning under Uncertainty
Abstract: This paper presents a method to reduce computations for stochastic dynamic games with game-theoretic belief space planning through partially propagating beliefs. Complex interactions in scenarios such as surveillance, herding, and racing can be modeled using game-theoretic frameworks in the belief space. Stochastic dynamic games can be solved to a local Nash Equilibrium using a game-theoretic belief space variant of an iterative Linear Quadratic Gaussian (iLQG). However, the scalability of this method suffers due to the large dimensionality of beliefs which the iLQG must propagate. We examine the utility of partial belief space propagation, which allows polynomial runtime to decrease. We validate our findings through simulations and hardware implementation.

Keywords: Multi-Robot Systems, Planning under Uncertainty, Localization
Abstract: Multi-agent multi-target tracking has a wide range of applications, including wildlife patrolling, security surveillance or environment monitoring. Such algorithms often make restrictive assumptions: the number of targets and/or their initial locations may be assumed known, or agents may be pre-assigned to monitor disjoint partitions of the environment, reducing the burden of exploration. This also limits applicability when there are fewer agents than targets, since agents are unable to continuously follow the targets in their fields of view. Multi-agent tracking algorithms additionally assume inter-agent synchronization of observations, or the presence of a central controller to coordinate joint actions. Instead, we focus on the setting of decentralized multi-agent, multi-target, simultaneous active search-and-tracking with asynchronous inter-agent communication. Our proposed algorithm DecSTER uses a sequential monte carlo implementation of the Probability Hypothesis Density filter for posterior inference combined with Thompson sampling for decentralized multi-agent decision making. We compare different action selection policies, focusing on scenarios where targets outnumber agents. In simulation, we demonstrate that DecSTER is robust to unreliable inter-agent communication and outperforms information-greedy baselines in terms of the Optimal Sub-Pattern Assignment (OSPA) metric for different numbers of targets and varying teamsizes.

Keywords: Multi-Robot Systems, Planning under Uncertainty, Reactive and Sensor-Based Planning
Abstract: We propose an autonomous exploration algorithm designed for decentralized multi-robot teams, which takes into account map and localization uncertainties of range-sensing mobile robots. Virtual landmarks are used to quantify the combined impact of process noise and sensor noise on map uncertainty. Additionally, we employ an iterative expectation-maximization inspired algorithm to assess the potential outcomes of both a local robot’s and its neighbors’ next-step actions. To evaluate the effectiveness of our framework, we conduct a comparative analysis with state-of-the-art algorithms. The results of our experiments show the proposed algorithm’s capacity to strike a balance between curbing map uncertainty and achieving efficient task allocation among robots.

Keywords: Multi-Robot Systems, Planning, Scheduling and Coordination, Task Planning
Abstract: This paper presents a novel approach to optimal multi-robot task allocation in heterogeneous teams of robots. When robots have heterogeneous capabilities and there are diverse objectives and constraints to comply with, computing optimal plans can become especially hard. Moreover, we increase the problem complexity by: 1) considering battery-limited robots that need to schedule recharges; 2) tasks that can be decomposed into multiple fragments; and 3) multi-robot tasks that need to be executed by a coalition synchronously. We define a new problem for heterogeneous multi-robot task allocation and formulate it as a Mixed-Integer Linear Program that includes all the aforementioned features. Then we use an off-the-shelf solver to show the type of optimal solutions that our planner can produce and assess its performance in random scenarios. Our method, which is released as open-source code, represents a first step to formalize and analyze a complex problem that has not been solved in the state of the art.

Keywords: Multi-Robot Systems, Planning, Scheduling and Coordination, Task Planning
Abstract: Most real-world Multi-Robot Task Allocation (MRTA) problems require fast and efficient decision-making, which is often achieved using heuristics-aided methods such as genetic algorithms, auction-based methods, and bipartite graph matching methods. These methods often assume a form that lends better explainability compared to an end-to-end (learnt) neural network based policy for MRTA. However, deriving suitable heuristics can be tedious, risky and in some cases impractical if problems are too complex. This raises the question: can these heuristics be learned? To this end, this paper particularly develops a Graph Reinforcement Learning (GRL) framework to learn the heuristics or incentives for a bipartite graph matching approach to MRTA. Specifically a Capsule Attention policy model is used to learn how to weight task/robot pairings (edges) in the bipartite graph that connects the set of tasks to the set of robots. The original capsule attention network architecture is fundamentally modified by adding encoding of robots' state graph, and two Multihead Attention based decoders whose output are used to construct a LogNormal distribution matrix from which positive bigraph weights can be drawn. The performance of this new bigraph matching approach augmented with a GRL-derived incentive is found to be at par with the original bigraph matching approach that used expert-specified heuristics, with the former offering notable robustness benefits. During training, the learned incentive policy is found to get initially closer to the expert-specified incentive and then slightly deviate from its trend.

Keywords: Computer Vision for Automation, Aerial Systems: Perception and Autonomy, Reactive and Sensor-Based Planning
Abstract: UAV-based intelligent data acquisition for 3D reconstruction and monitoring of infrastructure has experienced an increasing surge of interest due to recent advancements in image processing and deep learning-based techniques. View planning is an essential part of this task that dictates the information capture strategy and heavily impacts the quality of the 3D model generated from the captured data. Recent methods have used prior knowledge or partial reconstruction of the target to accomplish view planning for active reconstruction; the former approach poses a challenge for complex or newly identified targets while the latter is computationally expensive. In this work, we present Bag-of-Views (BoV), a fully appearance-based model used to assign utility to the captured views for both offline dataset refinement and online next-best-view (NBV) planning applications targeting the task of 3D reconstruction. With this contribution, we also developed the View Planning Toolbox (VPT), a lightweight package for training and testing machine learning-based view planning frameworks, custom view dataset generation of arbitrary 3D scenes, and 3D reconstruction. Through experiments which pair a BoV-based reinforcement learning model with VPT, we demonstrate the efficacy of our model in reducing the number of required views for high-quality reconstructions in dataset refinement and NBV planning.

Keywords: Computer Vision for Automation, Computer Vision for Manufacturing, Computer Vision for Transportation
Abstract: Dehazing and enhancing visibility in real-world hazy images pose significant challenges due to the physical complexity of haze, the variability in haze conditions, the tendency to capture more details in noisy scenes, and the risk of overexposure. Many existing single RGB image dehazing methods tend to perform well in synthetic hazy scenarios but struggle in real-world situations. It stems from the fact that these methods often rely solely on deep learning techniques or classical approaches. In addition, they neglect overall image quality improvement. To partially address these challenges, we introduce an innovative approach that harnesses the strengths of both modalities to dehaze and enhance visibility in a single real-world hazy RGB image. First, both Low-level and deep features are extracted, and then a pre-trained vector quantization GAN is employed to create a discrete codebook of well-detailed data patches. A decoder component, enhanced with a normalized module, effectively utilizes these high-quality features to produce clear results. Additionally, a controllable operation is introduced to improve feature matching. To further enhance dehazing and generalizability, the decoder's output undergoes a sequence of gamma-correction operations and generates a sequence of multi-exposure images that are combined to create a haze-free, visually pleasing, and higher-quality final image. The method effectively reduces haziness, enhances sharpness, preserves natural colors, and minimizes artifacts in challenging real-world scenarios. The proposed approach surpasses five SOTA methods in both qualitative and quantitative evaluations across three key metrics, utilizing two real-world and three synthetic hazy image datasets. Notably, it achieves a substantial improvement in real-world datasets over the second-best method, with gains of 0.5702 and 0.129 in FADE metrics for the RTTS and Fattal datasets, respectively.

Keywords: Computer Vision for Automation, Computer Vision for Manufacturing, Industrial Robots
Abstract: Fiducial markers, like AprilTag and ArUco, are extensively utilized in robotics applications within industrial environments, encompassing navigation, docking, and object grasping tasks. However, in contrast to controlled laboratory conditions, markers installed in factory grounds or equipment surfaces, often face challenges like damage or contamination. These issues can lead to compromised marker integrity, resulting in reduced detection reliability. To address this challenge, we propose a novel fiducial marker called CopperTag, which incorporates circular and square elements to create a robust occlusion-resistant pattern.

The CopperTag detection process relies on three fundamental steps: firstly, extracting all lines from the image; secondly, identifying corners; and lastly, searching for quadrilateral candidate regions using ellipses and nearby corners. The Reed-Solomon (RS) algorithm is utilized for both encoding and decoding the information content. This algorithm possesses the ability to recover corrupted messages in situations where CopperTag data is incomplete.

The experimental results illustrate that CopperTag exhibits superior robustness and accuracy in detection when compared to other state-of-the-art fiducial markers, even in scenarios with heavy occlusion. Moreover, CopperTag maintains an average processing time of 10ms per frame on a standard laptop, effectively meeting the real-time demands of robotics applications.

Keywords: Computer Vision for Automation, Computer Vision for Transportation, Deep Learning for Visual Perception
Abstract: A consistent spatial-temporal coordination across multiple agents is fundamental for collaborative perception, which seeks to improve perception abilities through information exchange among agents. To achieve this spatial-temporal alignment, traditional methods depend on external devices to provide localization and clock signals. However, hardware-generated signals could be vulnerable to noise and potentially malicious attack, jeopardizing the precision of spatial-temporal alignment. Rather than relying on external hardwares, this work proposes a novel approach: aligning by recognizing the inherent geometric patterns within the perceptual data of various agents. Following this spirit, we propose a robust collaborative perception system that operates independently of external localization and clock devices. The key module of our system,~emph{FreeAlign}, constructs a salient object graph for each agent based on its detected boxes and uses a graph neural network to identify common subgraphs between agents, leading to accurate relative pose and time. We validate emph{FreeAlign} on both real-world and simulated datasets. The results show that, the ~emph{FreeAlign} empowered robust collaborative perception system perform comparably to systems relying on precise localization and clock devices. We will release code related to this work.

Keywords: Computer Vision for Automation, Computer Vision for Transportation, Visual Learning
Abstract: When capturing images through the glass during rainy or snowy weather conditions, the resulting images often contain waterdrops adhered on the glass surface, and these waterdrops significantly degrade the image quality and perfor- mance of many computer vision algorithms. To tackle these limitations, we propose a method to reconstruct the clear 3D scene implicitly from multi-view images degraded by water- drops. Our method exploits an attention network to predict the location of waterdrops and then train a Neural Radiance Fields to recover the 3D scene implicitly. By leveraging the strong scene representation capabilities of NeRF, our method can render high-quality novel-view images with waterdrops removed. Extensive experimental results on both synthetic and real datasets show that our method is able to generate clear 3D scenes and outperforms existing state-of-the-art (SOTA) image adhesive waterdrop removal methods.

Keywords: Computer Vision for Automation, Deep Learning for Visual Perception, Data Sets for Robotic Vision
Abstract: Learning to interact with objects is significant for robots to integrate into human environments. When the interaction semantic is definite, manually guiding the manipulator is a commonly used method to teach robots how to interact with objects. However, the learning results are robot-dependent because the mechanical parameters are different for different robots, which means the learning process must be executed again. Moreover, during the manual guiding process, operators are responsible for recognizing the region being contacted and providing expert motion programming, which limits the robot's intelligence. To improve the degree of automation for robots interacting with objects, this paper proposes IRMT-Net (Interaction Region and Motion Trajectory prediction Network) to predict the interaction region and motion trajectory simultaneously based on images. IRMT-Net achieves state-of-the-art interaction region prediction results on Epic-kitchens dataset, generates reasonable motion trajectories and can support robot interaction in actual situations.

Keywords: Computer Vision for Automation, Deep Learning for Visual Perception, Localization
Abstract: Given the image collection of an object, we aim at building a real-time image-based pose estimation method, which requires neither its CAD model nor hours of object-specific training. Recent NeRF-based methods provide a promising solution by directly optimizing the pose from pixel loss between rendered and target images. However, during inference, they require long converging time, and suffer from local minima, making them impractical for real-time robot applications. We aim at solving this problem by marrying image matching with NeRF. With 2D matches and depth rendered by NeRF, we directly solve the pose in one step by building 2D-3D correspondences between target and initial view, thus allowing for real-time prediction. Moreover, to improve the accuracy of 2D-3D correspondences, we propose a 3D consistent point mining strategy, which effectively discards unfaithful points reconstruted by NeRF. Moreover, current NeRF-based methods naively optimizing pixel loss fail at occluded images. Thus, we further propose a 2D matches based sampling strategy to preclude the occluded area. Experimental results on representative datasets prove that our method outperforms state-of-the-art methods, and improves inference efficiency by 90x, achieving real-time prediction at 6 FPS.

Keywords: Computer Vision for Automation, Deep Learning for Visual Perception, Recognition
Abstract: Recognizing human action is one of the most critical factors in the visual perception of robots. Specifically, skeleton-based action recognition has been actively researched to enhance recognition performance at a lower cost. However, action recognition in occlusion situations, where body parts are not visible, is still challenging. We propose an occluded part-aware graph convolutional network (OP-GCN) to address this challenge using the optimal occluded body parts. The proposed model uses an occluded part detector to identify occluded body parts within a human skeleton. It is based on an autoencoder trained on a nonoccluded human skeleton and exploits the symmetry and angular information of the skeleton. Then, we select an optimal group constructed considering the occluded body parts. Each group comprises five sets of joint nodes, focusing on the body parts, excluding the occluded ones. Finally, to enhance interaction within the selected groups, we apply an interpart association module, considering the fusion of global and local elements. The experimental results reveal that the proposed model outperforms others on the occluded datasets. These comparative experiments demonstrate the effectiveness of the study in addressing the challenge of action recognition in occlusion situations. Our code is publicly available at https://github.com/MJ-Kor/OP-GCN.

Keywords: Computer Vision for Automation, Deep Learning for Visual Perception, RGB-D Perception
Abstract: Depth perception is crucial for a wide range of robotic applications. Multi-frame self-supervised depth estimation methods have gained research interest due to their ability to leverage large-scale, unlabeled real-world data. However, the self-supervised methods often rely on the assumption of a static scene and their performance tends to degrade in dynamic environments. To address this issue, we present Motion-Aware Loss, which leverages the temporal relation among consecutive input frames and a novel distillation scheme between the teacher and student networks in the multi-frame self-supervised depth estimation methods. Specifically, we associate the spatial locations of moving objects with the temporal order of input frames to eliminate errors induced by object motion. Meanwhile, we enhance the original distillation scheme in multi-frame methods to better exploit the knowledge from a teacher network. MAL is a novel, plug-and-play module designed for seamless integration into multi-frame self-supervised monocular depth estimation methods. Adding MAL into previous state-of-the-art methods leads to a reduction in depth estimation errors by up to 4.2% and 10.8% on KITTI and CityScapes benchmarks, respectively.

Keywords: Visual Servoing, Grasping, Humanoid Robot Systems
Abstract: This paper presents an image-based visual servoing scheme that can control robotic manipulators in 3D space using 2D stereo images without needing to perform stereo reconstruction. We use a stereo camera in an eye-to-hand configuration for controlling the robot to reach target positions by directly mapping image space errors to joint space actuation. We achieve convergence without a-priori knowledge of the target object, a reference 2D image, or 3D data. By doing so, we can reach targets in unstructured environments using high-resolution RGB images instead of utilizing relatively noisy depth data. We conduct several experiments on two different physical robots. The Panda 7DOF arm grasps a static target in 3D space, grasps a pitcher handle, and picks and places a box by determining the approach angle using 2D image features, demonstrating that this algorithm can be used for grasping practical objects in 3D space using only 2D image features for feedback. Our second platform, the Atlas humanoid robot, reaches a target from an unknown starting configuration, demonstrating that this controller achieves convergence to a target, even with the uncertainties introduced by walking to a new location. We believe that this algorithm is a step towards enabling intuitive interfaces that allow a user to initiate a grasp on an object by specifying a grasping point in a 2D image.

Keywords: Visual Servoing, Grasping, Underactuated Robots
Abstract: This paper presents a novel method to control an underactuated hand by using only a monocular camera, not using any internal sensors. In food factories, robots are required to handle a wide variety of foods without damaging them. To accomplish this, the use of underactuated hands is effective because they can adapt to various food shapes. However, if internal sensors such as tactile sensors and force sensors are used in the underactuated hands, it may cause a problem with hygiene and require complicated calibration. Moreover, if external sensors such as cameras are used, it is necessary to grasp foods without damaging them by using external information such as images. In our method, to tackle these problems, a camera is used as an external sensor. First, contact between the hand and the object is detected by using the contours of both, obtained from a camera image. Then, to avoid damaging the object, the following information is extracted from camera images and observed: the centroid of both the hand and object, the deformation of the object, and the occlusion rate of the hand. Furthermore, to prevent the object from dropping while the robotic arm is in motion, the distance between the centroid of the hand and the object is calculated. The experiments were conducted using twelve different food items.

Keywords: Visual Servoing, Manipulation Planning, Learning from Demonstration
Abstract: Learning-based manipulation policies from image inputs often show weak task transfer capabilities. In contrast, visual servoing methods allow efficient task transfer in high-precision scenarios while requiring only a few demonstrations. In this work, we present a framework that formulates the visual servoing task as graph traversal. Our method not only extends the robustness of visual servoing, but also enables multitask capability based on a few task-specific demonstrations. We construct demonstration graphs by splitting existing demonstrations and recombining them. In order to traverse the demonstration graph in the inference case, we utilize a similarity function that helps select the best demonstration for a specific task. This enables us to compute the shortest path through the graph. Ultimately, we show that recombining demonstrations leads to higher task-respective success. We present extensive simulation and real-world experimental results that demonstrate the efficacy of our approach.

Keywords: Visual Servoing, Motion Control
Abstract: Visual Servoing is an established approach for controlling robots using visual feedback. Most controllers in this domain generate velocity control signals to guide the cameras to desired positions and orientations. However, the dynamic characteristics of conventional visual servoing controllers may be unsatisfactory, and the velocity signal itself hinders the connection between the feature velocity model and the robot's dynamics. Consequently, research has explored models incorporating the second-order derivative of features and the robot's acceleration. The current state-of-the-art techniques mainly focus on image-based visual servoing, which deals with feature errors in the image domain. In this work, we propose an acceleration-based controller for the position-based visual servoing framework, which models the error in Cartesian space. Our approach involves extracting an acceleration control signal from the traditional velocity-based controller. To achieve this, we redefine the camera orientation using quaternions, generate new interaction matrices, and conduct comprehensive comparative experiments in simulated and real robot scenarios. We show that our method provides better dynamic properties in both image and Cartesian spaces, superior tracking performance, and less sensitivity to noise compared to velocity controllers.

Keywords: Visual Servoing, Visual Tracking, Sensor-based Control
Abstract: This paper presents a novel approach for tracking deformable contour targets using Unmanned Aerial Vehicles (UAVs). The proposed scheme combines image moments descriptor and event-triggered Nonlinear Model Predictive Control (NMPC) for efficient and accurate tracking. The deformable contour model allows adaptation to the evolving target's shape, while the proposed event-triggered scheme achieves improved computational efficiency and extended flight duration while generating new control sequences for the UAV. Real-world experiments validate the scheme, showcasing its robustness in handling complex scenarios. This approach holds promise for various applications, such as surveillance and autonomous navigation.

Keywords: Visual Servoing, Visual Tracking, Sensor-based Control, Manipulation of Deformable Objects
Abstract: In this paper, we propose a general unified tracking-servoing approach for controlling the shape of elastic deformable objects using robotic arms. Our approach works by forming a lattice around the object, binding the object to the lattice, and tracking and servoing the lattice instead of the object. This makes our approach have full control over the deformation of elastic deformable objects of any general form (linear, thin-shell, volumetric) in 3D space. Furthermore, it decouples the runtime complexity of the approach from the objects’ geometric complexity. Our approach is based on the As-Rigid-As-Possible (ARAP) deformation model. It requires no mechanical parameter of the object to be known and can drive the object toward desired shapes through large deformations. The inputs to our approach are the point cloud of the object’s surface in its rest shape and the point cloud captured by a 3D camera in each frame. Overall, our approach is more broadly applicable than existing approaches. We validate the efficiency of our approach through numerous experiments with elastic deformable objects of various shapes and materials (paper, rubber, plastic, foam).

Keywords: Visual Tracking
Abstract: Existing nighttime unmanned aerial vehicle (UAV) trackers follow an “Enhance-then-Track” architecture - first using a light enhancer to brighten the nighttime video, then employing a daytime tracker to locate the object. This separate enhancement and tracking fails to build an end-to-end trainable vision system. To address this, we propose a novel architecture called Darkness Clue-Prompted Tracking (DCPT) that achieves robust UAV tracking at night by efficiently learning to generate darkness clue prompts. Without a separate enhancer, DCPT directly encodes anti-dark capabilities into prompts using a darkness clue prompter (DCP). Specifically, DCP iteratively learns emphasizing and undermining projections for darkness clues. It then injects these learned visual prompts into a daytime tracker with fixed parameters across transformer layers. Moreover, a gated feature aggregation mechanism enables adaptive fusion between prompts and between prompts and the base model. Extensive experiments show state-of-the-art performance for DCPT on multiple dark scenario benchmarks. The unified end-to-end learning of enhancement and tracking in DCPT enables a more trainable system. The darkness clue prompting efficiently injects anti-dark knowledge without extra modules. Code is available at https://github.com/bearyi26/DCPT.

Keywords: Visual Tracking, Aerial Systems: Applications, Vision-Based Navigation
Abstract: Visual control enables quadrotors to adaptively navigate using real-time sensory data, bridging perception with action. Yet, challenges persist, including generalization across scenarios, maintaining reliability, and ensuring real-time responsiveness. This paper introduces a perception framework grounded in foundational models for universal object detection and tracking, moving beyond specific training categories. Integral to our approach is a multi-layered tracker integrated with the foundational detector, ensuring continuous target visibility, even when faced with motion blur, abrupt light shifts, and occlusions. Complementing this, we introduce a model-free controller tailored for resilient quadrotor visual tracking. Our system operates efficiently on limited hardware, relying solely on an onboard camera and an inertial measurement unit. Through extensive validation in diverse challenging indoor and outdoor environments, we demonstrate our system's effectiveness and adaptability. In conclusion, our research represents a step forward in quadrotor visual tracking, moving from task-specific methods to more versatile and adaptable operations.

Keywords: Visual Tracking, Aerial Systems: Perception and Autonomy, Human Detection and Tracking
Abstract: Multi-object tracking (MOT) on static platforms, such as by surveillance cameras, has achieved significant progress, with various paradigms providing attractive performances. However, the effectiveness of traditional MOT methods is significantly reduced when it comes to dynamic platforms like drones. This decrease is attributed to the distinctive challenges in the MOT-on-drone scenario: (1) objects are generally small in the image plane, often blurred, and frequently occluded, making them challenging to detect and recognize; (2) drones move and see objects from different angles, causing the unreliability of the predicted positions and feature embeddings of the objects. This paper proposes DroneMOT, which firstly proposes a Dual-Domain integrated Attention (DIA) module that considers the fast movements of drones to enhance the drone-based object detection and feature embedding for small-sized, blurred, and occluded objects. Then, an innovative Motion-Driven Association (MDA) scheme is introduced, considering the concurrent movements of both the drone and the objects. Within MDA, an Adaptive Feature Synchronization (AFS) technique is presented to update the object features seen from different angles. Additionally, a Dual Motion-based Prediction (DMP) method is employed to forecast the object positions. Finally, both the refined feature embeddings and the predicted positions are integrated to enhance the object association. Comprehensive evaluations on VisDrone2019-MOT and UAVDT datasets show that DroneMOT provides substantial performance improvements over the state-of-the-art in the domain of MOT on drones.

Keywords: Reinforcement Learning, Human-Robot Collaboration, Safety in HRI
Abstract: Deep reinforcement learning (RL) has shown promising results in robot motion planning with first attempts in human-robot collaboration (HRC). However, a fair comparison of RL approaches in HRC under the constraint of guaranteed safety is yet to be made. We, therefore, present human-robot gym, a benchmark suite for safe RL in HRC. We provide challenging, realistic HRC tasks in a modular simulation framework. Most importantly, human-robot gym is the first benchmark suite that includes a safety shield to provably guarantee human safety. This bridges a critical gap between theoretic RL research and its real-world deployment. Our evaluation of six tasks led to three key results: (a) the diverse nature of the tasks offered by human-robot gym creates a challenging benchmark for state-of-the-art RL methods, (b) by leveraging expert knowledge in form of an action imitation reward, the RL agent can outperform the expert, and (c) our agents negligibly overfit to training data.

Keywords: Reinforcement Learning, Collision Avoidance, Machine Learning for Robot Control
Abstract: Deploying a safe mobile robot policy in scenarios with human pedestrians is challenging due to their unpredictable movements. Current Reinforcement Learning-based motion planners rely on a single policy to simulate pedestrian movements and could suffer from the over-fitting issue. Alternatively, framing the collision avoidance problem as a multi-agent framework, where agents generate dynamic movements while learning to reach their goals, can lead to conflicts with human pedestrians due to their homogeneity.

To tackle this problem, we introduce an efficient method that enhances agent diversity within a single policy by maximizing an information-theoretic objective. This diversity enriches each agent's experiences, improving its adaptability to unseen crowd behaviors. In assessing an agent's robustness against unseen crowds, we propose diverse scenarios inspired by pedestrian crowd behaviors. Our behavior-conditioned policies outperform existing works in these challenging scenes, reducing potential collisions without additional time or travel.

Keywords: Reinforcement Learning, Collision Avoidance, Human-Centered Automation
Abstract: Reinforcement Learning (RL) has been widely used to create generalizable autonomous vehicles. However, they rely on fixed reward functions that struggle to balance values like safety and efficiency. How can autonomous vehicles balance different driving objectives and human values in a constantly changing environment? To bridge this gap, we propose an adaptive reward function that utilizes visual attention maps to detect pedestrians in the driving scene and dynamically switch between prioritizing safety or efficiency depending on the current observation. The visual attention map is used to provide spatial attention to the RL agent to boost the training efficiency of the pipeline. We evaluate the pipeline against variants of an occluded pedestrian crossing scenario in the CARLA Urban Driving simulator. Specifically, the proposed pipeline is compared against a modular setup that combines the well-established object detection model, YOLO, with a Proximal Policy Optimization (PPO) agent. The results indicate that the proposed approach can compete with the modular setup while yielding greater training efficiency. The trajectories collected with the approach confirm the effectiveness of the proposed adaptive reward function.

Keywords: Reinforcement Learning, Task and Motion Planning
Abstract: While reinforcement learning (RL) attracts increasing research attention, maximizing the return while keeping the agent safe at the same time remains an open problem. Motivated to address this challenge, this work proposes a new Fast and Safe Policy Optimization (FSPO) algorithm, which consists of three steps: the first step involves reward improvement update, the second step projects the policy to the neighborhood of the baseline policy to accelerate the optimization process, and the third step addresses the constraint violation by projecting the policy back onto the constraint set. Such a projection-based optimization can improve the convergence and learning performance. Unlike many existing works that require convex approximations for the objectives and constraints, this work exploits a first- order method to avoid expensive computations and high dimensional issues, enabling fast and safe policy optimization, especially for challenging tasks. Numerical simulation and physical experiments demonstrate that FSPO outperforms existing methods in terms of safety guarantees and task completion rate.

Keywords: Reinforcement Learning, Machine Learning for Robot Control
Abstract: Symmetry is a fundamental aspect of many real-world robotic tasks. However, current deep reinforcement learning (DRL) approaches can seldom harness and exploit symmetry effectively. Often, the learned behaviors fail to achieve the desired transformation invariances and suffer from motion artifacts. For instance, a quadruped may exhibit different gaits when commanded to move forward or backward, even though it is symmetrical about its torso. This issue becomes further pronounced in high-dimensional or complex environments, where DRL methods are prone to local optima and fail to explore regions of the state space equally. Past methods on encouraging symmetry for robotic tasks have studied this topic mainly in a single-task setting, where symmetry usually refers to symmetry in the motion, such as the gait patterns. In this paper, we revisit this topic for goal-conditioned tasks in robotics, where symmetry lies mainly in task execution and not necessarily in the learned motions themselves. In particular, we investigate two approaches to incorporate symmetry invariance into DRL – data augmentation and mirror loss function. We provide a theoretical foundation for using augmented samples in an on-policy setting. Based on this, we show that the corresponding approach achieves faster convergence and improves the learned behaviors in various challenging robotic tasks, from climbing boxes with a quadruped to dexterous manipulation.

Keywords: Reinforcement Learning, Task and Motion Planning
Abstract: This work focuses on the dual-arm object rearrangement problem abstracted from a realistic industrial scenario of Cartesian robots. The goal of this problem is to transfer all the objects from sources to targets with the minimum total completion time. To achieve the goal, the core idea is to develop an effective object-to-arm task assignment strategy for minimizing the cumulative task execution time and maximizing the dual-arm cooperation efficiency. One of the difficulties in the task assignment is the scalability problem. As the number of objects increases, the computation time of traditional offline-search-based methods grows strongly for computational complexity. Encouraged by the adaptability of reinforcement learning (RL) in long-sequence task decisions, we propose an online task assignment decision method based on RL, and the computation time of our method only increases linearly with the number of objects. Further, we design an attention-based network to model the dependencies between the input states during the whole task execution process to help find the most reasonable object-to-arm correspondence in each task assignment round. In the experimental part, we adapt some search-based methods to this specific setting and compare our method with them. Experimental result shows that our approach achieves outperformance over search-based methods in total execution time and computational efficiency, and also verifies the generalization of our method to different numbers of objects. In addition, we show the effectiveness of our method deployed on the real robot in the supplementary video.

Keywords: Reinforcement Learning, Learning from Demonstration, Robot Safety
Abstract: Safe Reinforcement Learning (RL) aims to find a policy that achieves high rewards while satisfying cost constraints. When learning from scratch, safe RL agents tend to be overly conservative, which impedes exploration and restrains the overall performance. In many realistic tasks, e.g. autonomous driving, large-scale expert demonstration data are available. We argue that extracting expert policy from offline data to guide online exploration is a promising solution to mitigate the conserveness issue. Large-capacity models, e.g. decision transformers (DT), have been proven to be competent in offline policy learning. However, data collected in real-world scenarios rarely contain dangerous cases (e.g., collisions), which makes it prohibitive for the policies to learn safety concepts. Besides, these bulk policy networks cannot meet the computation speed requirements at inference time on real-world tasks such as autonomous driving. To this end, we propose Guided Online Distillation (GOLD), an offline-to-online safe RL framework. GOLD distills an offline DT policy into a lightweight policy network through guided online safe RL training, which outperforms both the offline DT policy and online safe RL algorithms. Experiments in both benchmark safe RL tasks and real-world driving tasks based on the Waymo Open Motion Dataset (WOMD) demonstrate that GOLD can successfully distill lightweight policies and solve decision-making problems in challenging safety-critical scenarios.

Keywords: Reinforcement Learning, Engineering for Robotic Systems, Visual Learning
Abstract: Motivated by the challenge of achieving rapid learning in physical environments, this paper presents the development and training of a robotic system designed to navigate and solve a labyrinth game using model-based reinforcement learning techniques. The method involves extracting low-dimensional observations from camera images, along with a cropped and rectified image patch centered on the current position within the labyrinth, providing valuable information about the labyrinth layout. The learning of a control policy is performed purely on the physical system using model-based reinforcement learning, where the progress along the labyrinth's path serves as a reward signal. Additionally, we exploit the system's inherent symmetries to augment the training data. Consequently, our approach learns to successfully solve a popular real-world labyrinth game in record time, with only 5 hours of real-world training data.

Keywords: Reinforcement Learning, Path Planning for Multiple Mobile Robots or Agents
Abstract: This paper investigates the multi-agent cooperative exploration problem, which requires multiple agents to explore an unseen environment via sensory signals in a limited time. A popular approach to exploration tasks is to combine active mapping with planning. Metric

maps capture the details of the spatial representation, but are memory intensive and may vary significantly between scenarios, resulting in inferior generalization. Topological maps are a promising alternative as they consist only of nodes and edges with abstract but essential information and are less influenced by the scene structures. However, most existing topology-based exploration tasks utilize classical methods for planning, which are time-consuming and sub-optimal due to their handcrafted design. Deep reinforcement learning (DRL) has shown great potential for learning (near) optimal policies through fast end-to-end inference. In this paper, we propose Multi-Agent Neural Topological Mapping (MANTM) to improve exploration efficiency and generalization for multi-agent exploration tasks. MANTM mainly comprises a Topological Mapper and a novel RL-based Hierarchical Topological Planner (HTP). The Topological Mapper employs a visual encoder and distance-based heuristics to construct a graph containing main nodes and their corresponding ghost nodes. The HTP leverages graph

neural networks to capture correlations between agents and graph nodes in a coarse-to-fine manner for effective global goal selection. Extensi

Keywords: Reinforcement Learning, Transfer Learning, Deep Learning in Grasping and Manipulation
Abstract: In this work, we investigate the potential of improving multi-task training and leveraging it for transferring in the reinforcement learning setting. We identify several challenges towards this goal and propose a transferring approach with a parameter-compositional formulation. We investigate ways to improve the training of multi-task reinforcement learning, which serves as the foundation for transferring. Then we conduct a number of transferring experiments on various manipulation tasks. Experimental results demonstrate that the proposed approach can have improved performance in the multi-task training stage, and further show effective transferring in terms of both sample efficiency and performance.

Keywords: Reinforcement Learning, Representation Learning, Manipulation Planning
Abstract: Goal-Conditioned Reinforcement Learning (GCRL) can enable agents to spontaneously set diverse goals to learn a set of skills. Despite the excellent works proposed in various fields, reaching distant goals in temporally extended tasks remains a challenge for GCRL. Current works tackled this problem by leveraging planning algorithms to plan intermediate subgoals to augment GCRL. Their methods need two crucial requirements: (i) a state representation space to search valid subgoals, and (ii) a distance function to measure the reachability of subgoals. However, they struggle to scale to high-dimensional state space due to their non-compact representations. Moreover, they cannot collect high-quality training data through standard GC policies, which results in an inaccurate distance function. Both affect the efficiency and performance of planning and policy learning. In the paper, we propose a goal-conditioned RL algorithm combined with Disentanglement-based Reachability Planning (REPlan) to solve temporally extended tasks. In REPlan, a Disentangled Representation Module (DRM) is proposed to learn compact representations which disentangle robot poses and object positions from high-dimensional observations in a self-supervised manner. A simple REachability discrimination Module (REM) is also designed to determine the temporal distance of subgoals. Moreover, REM computes intrinsic bonuses to encourage the collection of novel states for training. We evaluate our REPlan in three vision-

Keywords: Representation Learning, Deep Learning Methods, Manipulation Planning
Abstract: Object-centric representation is an essential abstraction for forward prediction. Most existing forward models learn this representation through extensive supervision (e.g., object class and bounding box) although such ground-truth information is not readily accessible in reality. To address this, we introduce KINet (Keypoint Interaction Network) ---an end-to-end unsupervised framework to reason about object interactions based on a keypoint representation. Using visual observations, our model learns to associate objects with keypoint coordinates and discovers a graph representation of the system as a set of keypoint embeddings and their relations. It then learns an action-conditioned forward model using contrastive estimation to predict future keypoint states. By learning to perform physical reasoning in the keypoint space, our model automatically generalizes to scenarios with a different number of objects, novel backgrounds, and unseen object geometries. Experiments demonstrate the effectiveness of our model in accurately performing forward prediction and learning plannable object-centric representations for downstream robotic pushing manipulation tasks.

Keywords: Deep Learning Methods, Reinforcement Learning, Deep Learning in Grasping and Manipulation
Abstract: Current reinforcement learning algorithms struggle in sparse and complex environments, most notably in long-horizon manipulation tasks entailing a plethora of different sequences. In this work, we propose the Intrinsically Guided Exploration from Large Language Models (IGE-LLMs) framework. By leveraging LLMs as an assistive intrinsic reward, IGE-LLMs guides the exploratory process in reinforcement learning to address intricate long-horizon with sparse rewards robotic manipulation tasks. We evaluate our framework and related intrinsic learning methods in an environment challenged with exploration, and a complex robotic manipulation task challenged by both exploration and long-horizons. Results show IGE-LLMs (i) exhibit notably higher performance over related intrinsic methods and the direct use of LLMs in decision-making, (ii) can be combined and complement existing learning methods highlighting its modularity, (iii) are fairly insensitive to different intrinsic scaling parameters, and (iv) maintain robustness against increased levels of uncertainty and horizons.

Keywords: Reinforcement Learning, In-Hand Manipulation, Dexterous Manipulation
Abstract: Tactile information holds promise for enhancing the manipulation capabilities of multi-fingered robots. In tasks such as in-hand manipulation, where robots frequently switch between contact and non-contact states, it is important to address the partial observability of tactile sensors and to properly consider the history of observations and actions. Previous studies have shown that Recurrent Neural Network (RNN) can be used to learn latent representations for handling observation and action histories. However, this approach is usually combined with on-policy reinforcement learning (RL) and suffers from low sample efficiency. Integrating RNN with off-policy RL could enhance sample efficiency, but this often compromises stability and robustness, especially as the dimensions of observation and action increase. This paper presents a time-contrastive learning approach tailored for off-policy RL. Our method incorporates a temporal contrastive model and introduces a surrogate loss to extract task-related latent representations, enhancing the pursuit of the optimal policy. Simulations and real robot experiments demonstrate that our proposed method outperforms RNN-based approaches.

Keywords: Manipulation Planning, Deep Learning in Grasping and Manipulation, Dexterous Manipulation
Abstract: Multi-task learning of deformable object manipulation is a challenging problem in robot manipulation. Most previous works address this problem in a goal-conditioned way and adapt goal images to specify different tasks, which limits the multi-task learning performance and can not generalize to new tasks. Thus, we adapt language instruction to specify deformable object manipulation tasks and propose a learning framework. We first design a unified Transformer-based architecture to understand multi-modal data and output picking and placing action. Besides, we have applied the visible connectivity graph to tackle nonlinear dynamics and complex configuration of the deformable object. Both simulated and real experiments have demonstrated that the proposed method is effective and can generalize to unseen instructions and tasks. Compared with the state-of-the-art method, our method achieves higher success rates (87.2% on average) and has a 75.6% shorter inference time. We also demonstrate that our method performs well in real-world experiments. Supplementary videos can be found at https://sites.google.com/view/language-deformable.

Keywords: Reinforcement Learning, Grasping, Sensorimotor Learning
Abstract: Interactive grasping from clutter, akin to human dexterity, is one of the longest-standing problems in robot learning. Challenges stem from the intricacies of visual perception, the demand for precise motor skills, and the complex interplay between the two. In this work, we present Teacher-Augmented Policy Gradient (TAPG), a novel two-stage learning framework that synergizes reinforcement learning and policy distillation. After training a teacher policy to master the motor control based on object pose information, TAPG facilitates guided, yet adaptive, learning of a sensorimotor policy, based on object segmentation. We zero-shot transfer from simulation to a real robot by using Segment Anything Model for promptable object segmentation. Our trained policies adeptly grasp a wide variety of objects from cluttered scenarios in simulation and the real world based on human-understandable prompts. Furthermore, we show robust zero-shot transfer to novel objects. Videos of our experiments are available at url{https://maltemosbach.github.io/grasp_anything}.

Keywords: Reinforcement Learning, Integrated Planning and Learning, Deep Learning in Grasping and Manipulation
Abstract: We propose a new policy class, Composable Interaction Primitives (CIPs), specialized for learning sustained-contact manipulation skills like opening a drawer, pulling a lever, turning a wheel, or shifting gears. CIPs have two primary design goals: to minimize what must be learned by exploiting structure present in the world and the robot, and to support sequential composition by construction, so that learned skills can be used by a task-level planner. Using an ablation experiment in four simulated manipulation tasks, we show that the structure included in CIPs substantially improves the efficiency of motor skill learning. We then show that CIPs can be used for plan execution in a zero-shot fashion by sequencing learned skills. We validate our approach on real robot hardware by learning and sequencing two manipulation skills.

Keywords: Reinforcement Learning, Sensorimotor Learning, Imitation Learning
Abstract: Robotic systems that aspire to operate in uninstrumented real-world environments must perceive the world directly via onboard sensing. Vision-based learning systems aim to eliminate the need for environment instrumentation by building an implicit understanding of the world based on raw pixels, but navigating the contact-rich high-dimensional search space from solely sparse visual reward signals significantly exacerbates the challenge of exploration. The applicability of such systems is thus typically restricted to simulated or heavily engineered environments since agent exploration in the real-world without the guidance of explicit state estimation and dense rewards can lead to unsafe behavior and safety faults that are catastrophic. In this study, we isolate the root causes behind these limitations to develop a system, called MoDem-V2, capable of learning contact-rich manipulation directly in the uninstrumented real world. Building on the latest algorithmic advancements in model-based reinforcement learning (MBRL), demo-bootstrapping, and effective exploration, MoDem-V2 can acquire contact-rich dexterous manipulation skills directly in the real world. We identify key ingredients for leveraging demonstrations in model learning while respecting real-world safety considerations -- exploration centering, agency handover, and actor-critic ensembles. We empirically demonstrate the contribution of these ingredients in four complex visuo-motor manipulation problems in both simulation and the real world. To the best of our knowledge, our work presents the first successful system for demonstration-augmented visual MBRL trained directly in the real world. Visit sites.google.com/view/modemv2 for videos and more details.

Keywords: Motion and Path Planning, Collision Avoidance, Probabilistic Inference
Abstract: Collision checking is the primitive operation of motion planning that consumes most time. Machine learning algorithms have proven to accelerate collision checking. We propose CollisionGP, a Gaussian process-based algorithm for modeling a robot's configuration space and query collision checks. CollisionGP introduces a Pòlya-Gamma auxiliary variable for each data point in the training set to allow classification inference to be done exactly with a closed form expression. Gaussian processes provide a distribution as the output, obtaining a mean and variance for the collision check. The obtained variance is processed to reduce false negatives (FN). We demonstrate that CollisionGP can use GPU acceleration to process collision checks for thousands of configurations much faster than traditional collision detection libraries. Furthermore, we obtain better accuracy, TPR and TNR results than state-of-the-art learning-based algorithms using less support points, thus making our proposed method more sparse.

Keywords: Collision Avoidance, Planning under Uncertainty, Optimization and Optimal Control
Abstract: Autonomous mobile robots require predictions of human motion to plan a safe trajectory that avoids them. Because human motion cannot be predicted exactly, future trajectories are typically inferred from real-world data via learning-based approximations. These approximations provide useful information on the pedestrian’s behavior, but may deviate from the data, which can lead to collisions during planning. In this work, we introduce a joint prediction and planning framework, Partitioned Scenario Replay (PSR), that stores and partitions previously observed human trajectories, referred to as scenarios. During planning, scenarios observed in similar situations are reintroduced (or replayed) as motion predictions. By sampling real data and by building on scenario optimization and predictive control, the planner provides probabilistic collision avoidance guarantees in the real-world. Relying on this guarantee to remain safe, PSR can incrementally improve its prediction and planning performance online. We demonstrate our approach on a mobile robot navigating around pedestrians.

Keywords: Collision Avoidance, Deep Learning Methods, AI-Based Methods
Abstract: To explore safe interactions between a mobile robot and dynamic obstacles, this paper presents a comprehensive approach to collision-free navigation in dynamic indoor environments. The approach integrates multimodal motion predictions of dynamic obstacles with predictive control for obstacle avoidance. Multimodal Motion Prediction (MMP) is achieved by a deep-learning method that predicts multiple plausible future positions. By repeating the MMP for each time offset in the future, multi-time-step multimodal motion predictions are obtained. A nonlinear Model Predictive Control (MPC) solver utilizes the prediction outcomes to achieve collision-free trajectory tracking for the mobile robot. The proposed integration of multimodal motion prediction and trajectory tracking outperforms other non-deep-learning methods in complex scenarios. The approach enables safe interaction between the mobile robot and stochastic dynamic obstacles.

Keywords: Motion and Path Planning, Collision Avoidance, Constrained Motion Planning
Abstract: This paper proposes a GPU-accelerated optimization framework for collision avoidance problems where the controlled objects and the obstacles can be modeled as the finite union of convex polyhedra. A novel collision avoidance constraint is proposed based on scale-based collision detection and the strong duality of convex optimization. Under this constraint, the high-dimensional non-convex optimization problems of collision avoidance can be decomposed into several low-dimensional quadratic programmings (QPs) following the paradigm of alternating direction method of multipliers (ADMM). Furthermore, these low-dimensional QPs can be solved parallel with GPUs, significantly reducing computational time. High-fidelity simulations are conducted to validate the proposed method's effectiveness and practicality.

Keywords: Collision Avoidance, Human-Aware Motion Planning, Reinforcement Learning
Abstract: The latest robot navigation methods for dynamic environments assume that the states of obstacles, including their geometries and trajectories, are fully observable. While it's easy to obtain these states accurately in simulations, it's exceedingly challenging in the real world. Therefore, a viable alternative is to directly map raw sensor observations into robot actions. However, acquiring skills from high-dimensional raw observations demands massive neural networks and extended training periods. Furthermore, there are discrepancies between simulated and real environments that impede real-world implementations. To overcome these limitations, we propose a Learning framework for robot Navigation in Dynamic environments that uses sequential Normalized LiDAR (LNDNL) scans. We employ long-short-term memory (LSTM) to propagate historical environmental information from the sequential LiDAR observations. Additionally, we customize a LiDAR-integrated simulator to speed up sampling and normalize the geometry of real-world obstacles to match that of simulated objects, thereby bridging the sim-to-real gap. Our extensive comparisons with state-of-the-art baselines and real-world implementations demonstrate the potential of learning to navigate in dynamic environments using raw sensor observations and sim-to-real transfer.

Keywords: Collision Avoidance, Integrated Planning and Learning, Motion and Path Planning
Abstract: Collision Avoidance/Mitigation System (CAMS) for autonomous vehicles is a crucial technology that ensures the safety and reliability of autonomous driving systems. Conventional collision avoidance approaches struggle in complex and various scenarios by avoiding collisions based on rules for specific collision scenarios. This has led to learning-based methods using neural networks for adaptive collision avoidance. However, the approaches directly outputting control inputs through neural networks have drawbacks in interpretability and stability. To address these limitations, we propose a trajectory planning method for CAMS that combines deep reinforcement learning (DRL) and quintic polynomial (QP) trajectory planning. The proposed method determines the terminal state and confidence of the trajectory using DRL and plans a QP trajectory based on them. By utilizing the terminal state and confidence of the trajectory rather than direct control inputs as the output of the neural network, it generates a more realistic and continuous path. Moreover, this approach considers collision avoidance and mitigation in an integrated manner through the reward function of RL. Our experimental results demonstrate that the proposed method not only improves interpretability and stability compared to existing learning-based methods but also upholds performance in complex and various collision scenarios.

Keywords: Collision Avoidance, Planning under Uncertainty, Autonomous Vehicle Navigation
Abstract: It is challenging for the mobile robot to achieve autonomous and mapless navigation in the unknown environment with uneven terrain. In this study, we present a layered and systematic pipeline. At the local level, we maintain a tree structure that is dynamically extended with the navigation. This structure unifies the planning with the terrain identification. Besides, it contributes to explicitly identifying the hazardous areas on uneven terrain. In particular, certain nodes of the tree are consistently kept to form a sparse graph at the global level, which records the history of the exploration. A series of subgoals that can be obtained in the tree and the graph are utilized for leading the navigation. To determine a subgoal, we develop an evaluation method whose input elements can be efficiently obtained on the layered structure. We conduct both simulation and real-world experiments to evaluate the developed method and its key modules. The experimental results demonstrate the effectiveness and efficiency of our method. The robot can travel through the unknown uneven region safely and reach the target rapidly without a preconstructed map.

Keywords: Modeling, Control, and Learning for Soft Robots, Soft Sensors and Actuators, Grasping
Abstract: We tackle the problem of proprioception in soft robots, specifically soft grippers with tight packaging constraints, relying only on intrinsic sensors. While various sensing ap- proaches have been studied towards curvature estimation, we look into being able to sense local deformations. To accomplish this, we use a widely available, off-the-shelf resistive sensor and multi-tap this sensor, i.e. make multiple electrical connections onto the resistive layer of the sensor. This allows us to measure changes in resistance at multiple segments throughout the length of the sensor, providing improved resolution of local deformations in the soft body. These measurements inform a finite-element- method (FEM) based model to then estimate the shape of the soft body and the magnitude of an external force acting at a known arbitrary location. Our model-based approach estimates soft body deformation with approximately 3% average relative error and taking into account internal fluidic actuation, our estimate of external force disturbance has 11% relative error within a 5 N range. The combined sensing and modeling approach can be integrated into soft manipulation platforms to enable features such as identifying shape and material properties of an object being grasped. Such manipulators can benefit from the softness and compliance while being proprioceptive relying only on embedded sensing and not on external systems such as motion capture, which is essential for deployment in real-world scena

Keywords: Modeling, Control, and Learning for Soft Robots, Soft Sensors and Actuators, Learning from Demonstration
Abstract: Learning manipulation from demonstration is a key way for humans to teach complex tasks. However, this domain mainly focuses on kinetic teaching, and does not consider imitation of interaction forces which is essential for more contact rich tasks. We propose a framework that enables robotic imitation of contact from human demonstration using a wearable finger-tip sensor. By developing a multi-modal sensor (providing both force and contact location) and robotic collection of simple training data of different motion primitives (tapping, rotation and translation), an LSTM-based model can be used to replicate motion from tactile demonstration only. To evaluate this approach, we explore the performance on increasingly complex testing data generated by a robot, and also demonstrate the full pipeline from human demonstration via the sensor used as a wearable device. This approach of using tactile sensing as a means of inferring the required robot motion paves the way for imitation of more contact-rich tasks, and enables imitation of tasks where the demonstration and imitation is performed with different body-schema.

Keywords: Modeling, Control, and Learning for Soft Robots, Soft Sensors and Actuators, Motion Control, Dielectric elastomer actuators
Abstract: The continuous electromechanical deformation of dielectric elastomer actuators (DEAs) suffers from rate-dependent viscoelasticity, mechanical vibration and configuration dependency, making the generalized dynamic modeling and precise control elusive. In this work,

we present a generalized motion control framework for DEAs capable of accommodating different configurations, materials and degrees of freedom (DOFs). First, a generalized, control-enabling dynamic model is

developed for DEAs by taking both nonlinear electromechanical coupling,

mechanical vibration and rate-dependent viscoelasticity into consideration. Further, a state observer is introduced to predict the unobservable viscoelasticity. Then, an Enhanced Exponential Reaching Law based Sliding-Mode Controller (EERLSMC) is proposed to minimize the

viscoelasticity of DEAs. Its stability is also proved mathematically. The experimental results obtained for different DEAs (four configurations, two materials and multi-DOFs) demonstrate that our dynamic model can precisely describe their complex dynamic responses and the EERLSMC can achieve precise tracking control; verifying the generality of our framework.

Keywords: Modeling, Control, and Learning for Soft Robots, Soft Sensors and Actuators, Soft Robot Applications
Abstract: Soft continuum robots, fabricated from elastomeric materials, offer unparalleled flexibility and adaptability, making them ideal for applications such as minimally invasive surgery and inspections in constrained environments. With the miniaturization of imaging technologies and the development of novel control algorithms, these devices provide exceptional opportunities to visualize the internal structures of the human body. However, there are still challenges in accurately estimating external forces applied to these systems using current technologies. Adding additional sensors is challenging without compromising the softness of the device. This work presents a visual deformation-based force sensing framework for soft continuum robots. The core idea behind this work is that point loads lead to unique deformation profiles in an actuated soft-bodied robot. We introduce a Convolutional Neural Network-based tip force estimation method that utilizes arbitrarily placed camera images and actuation inputs to predict applied tip forces. Experimental validation was performed using the STIFF-FLOP robot, a pneumatically actuated soft robot developed for minimally invasive surgery. Our vision-based force estimation model demonstrated a sensing precision of 0.05 N in the XY plane during testing, with data collection and training taking only 70 minutes.

Keywords: Soft Robot Materials and Design, Soft Sensors and Actuators, Compliant Joints and Mechanisms
Abstract: Soft actuators with a function of variable stiffness are beneficial to the improvement of the adaptability of robots, expanding the application areas and environments. We propose a tendon-driven soft bending actuator that can change its stiffness using fiber jamming. The actuator is made of an elastomer tube filled with different types of fiber. The three types of fibers play different roles in maintaining the structure, variable stiffness by jamming, and fiber-optic shape sensing while sharing the same structure and materials, realizing a compact form factor of the entire structure. The stiffness of the actuator can be increased to higher than three times its original stiffness by jamming. In addition to jamming, the proposed actuator has a special function of shape sensing that estimates the tip location of the actuator based on image sensing from optical fibers packaged with the jamming fibers. The tip position sensing shows accuracies with errors of 3.1%, 3.0%, and 6.7% for the x, y, and z axes, respectively, using feature extraction and a deep neural network. The proposed actuator has two degrees of freedom (i.e., bending on two orthogonal planes) and is controlled by two tendons. When connected in series, multiple actuators form a soft robotic manipulator (i.e., arm), physically compliant or capable of delivering a relatively high force to the target objects.

Keywords: Soft Robot Materials and Design, Soft Sensors and Actuators, Mechanism Design
Abstract: The fields of soft and bio-inspired robotics promise to imbue synthetic systems with capabilities found in the natural world. However, many of these biological capabilities are yet to be realized. For example, vines in nature direct growth via localized responses embedded in the cells of vine body, allowing an organism without a central brain to successfully search for resources (e.g., light). Yet to date, vine-inspired robots have yet to show such localized embedded responsiveness. Here we present a vine-inspired robotic device with material-level responses embedded in its skin and capable of “growing” and steering toward either a light or heat stimulus. We present basic modeling of the concept, design details, and experimental results showing its behavior in response to infrared (IR) and visible light. Our simple design concept advances the capabilities of bio-inspired robots and lays the foundation for future “growing” robots that are capable of seeking light or heat, yet are extremely simple and low-cost. Potential applications include solar tracking, and in the future, firefighting smoldering fires. We envision using similar robots to find hot spots in hard- to-access environments, allowing us to put out potentially long- burning fires faster.

Keywords: Soft Robot Materials and Design, Soft Sensors and Actuators, Shape Optimization, Soft Robot Applications
Abstract: A homogeneous pneumatic soft robot may generate complex output motions using a simple input pressure, resulting from its morphological shape that locally deforms the soft material to different degrees by simultaneously tailoring the structural characteristics and orienting the input pressure. To date, design of the morphological shape (inverse problem) has not been fully addressed. This article outlines a geometry-mechanics-optimization integrated approach to automatically shaping a pneumatic soft actuator or robot that achieves the desired deformation behavior. Instead of constraining the robot's geometry within any predefined regular shape, we employ B-splines to allow generation of freeform boundary surfaces, and use nonlinear mechanical modelling and shape derivative based optimization to navigate the high-dimensional design space. Our design framework can readily regulate the surface quality during the morphological evolution, by imposing the geometric constraints in terms of the principal curvatures and the minimal distance between surfaces as penalty functions. The effect of external forces including the gravity and the interaction force at the end-effector is also taken int

Keywords: Soft Robot Materials and Design, Soft Sensors and Actuators, Soft Robot Applications
Abstract: Soft robots have shown remarkable distinct capabilities due to their high deformation. Recently increasing attention has been dedicated to developing fully soft robots to exploit their full potential, with a recognition that electronic powering may limit this achievement. Alternative powering sources compatible with soft robots have been identified such as combustion and chemical reactions. A further milestone to such systems would be to increase the controllability and responsiveness of their underlying reactions in order to achieve more complex behaviors for soft robots. In this paper, we present a thermally-activated reactor incorporating a biocompatible hydrogel valve that enables control of the biochemical reaction of sugar and yeast. The biochemical reaction is utilized to generate contained pressure, which in turn powers a fluidic soft actuator. Experiments were conducted to evaluate the response time of the hydrogel valves with three different crosslinker concentrations. Among the tested concentrations, we found that the lowest crosslinker concentration yielded the fastest response time of the valve at an ambient temperature of 50°C. We also evaluated the pressure generation capacity of the reactor, which can reach up to 0.22 bar, and demonstrated the thermo-responsive behavior of the reactor to trigger a biochemical reaction for powering a fluidic soft actuator. This work opens up the possibility to power and control tetherless and fully soft robots.

Keywords: Soft Robot Materials and Design, Soft Sensors and Actuators, Soft Robot Applications
Abstract: Designing information-rich and space-efficient sensors is a key challenge for soft robotics, and crucial for the development of safe soft robots. Sensing and understanding the environmental interactions with a minimal footprint is especially important in the medical context, where portability and unhindered patient/user movement is a priority, to move towards personalized and decentralized healthcare solutions. In this work, a pulsating fluidic soft sensor (PFS) capable of determining location, pressure and contact area of press events is shown. The sensor relies on spatio-temporal resistance changes driven by a pulsating conductive fluid. The sensor demonstrates good repeatability and distinction of single and multiple press events, detecting single indents of sizes greater than 1 cm, forces larger than 2 N, and various locations across the sensor, as well as multiple indents spaced 2 cm apart. Furthermore, the sensor is demonstrated in two applications to detect foot placement and grip location. Overall, the sensor represents an improvement towards minimizing electronic hardware, and cost of the sensing solution, without sacrificing the richness of the sensing information in the field of soft fluidic sensors.

Keywords: Semantic Scene Understanding, Cognitive Modeling, Perception for Grasping and Manipulation
Abstract: In our previous work, we designed a human-like white-box and causal generative model of perception NaivPhys4RP, essentially based on cognitive emulation to understand the past, the present and the future of the state of complex worlds from poor observations. In this paper, as recommended in that previous work, we first refine the theoretical model of NaivPhys4RP in terms of integration of variables as well as perceptual inference tasks to solve. Intuitively, the system is closed under the injection, update and dependency of variables. Then, we present a first implementation of NaivPhys4RP that demonstrates the learningless and safe recognition and 6D-Pose estimation of objects from poor sensor data (e.g., occlusion, transparency, poor-depth, in-hand). This does not only make a substantial step forward comparatively to classical perception systems in perceiving objects in these scenarios, but escape the burden of data-intensive learning and operate safely (transparency and causality — we fit sensor data into mentally constructed meaningful worlds). With respect to ChatGPT’s ambitions, it can imagine physico-realistic socio-physical scenes from texts, demonstrate understanding of these texts, and all these with no data- and resource-intensive learning.

Keywords: Semantic Scene Understanding, Deep Learning for Visual Perception, Visual Learning
Abstract: Robots require a semantic understanding of their surroundings to operate in an efficient and explainable way in human environments. In the literature, there has been an extensive focus on object labeling and exhaustive scene graph generation; less effort has been focused on the task of purely identifying and mapping large semantic regions. The present work proposes a method for semantic region mapping via embodied navigation in indoor environments, generating a high-level representation of the knowledge of the agent. To enable region identification, the method uses a vision-to-language model to provide scene information for mapping. By projecting egocentric scene understanding into the global frame, the proposed method generates a semantic map as a distribution over possible region labels at each location. This mapping procedure is paired with a trained navigation policy to enable autonomous map generation. The proposed method significantly outperforms a variety of baselines, including an object-based system and a pretrained scene classifier, in experiments in a photorealistic simulator.

Keywords: Semantic Scene Understanding, Deep Learning for Visual Perception, RGB-D Perception
Abstract: 3D visual grounding is a critical skill for household robots, enabling them to navigate, manipulate objects, and answer questions based on their environment. While existing approaches often rely on extensive labeled data or exhibit limitations in handling complex language queries, we propose LLM-Grounder, a novel zero-shot, open-vocabulary, Large Language Model (LLM)-based 3D visual grounding pipeline. LLM-Grounder utilizes an LLM to decompose complex natural language queries into semantic constituents and employs a visual grounding tool, such as OpenScene or LERF, to identify objects in a 3D scene. The LLM then evaluates the spatial and commonsense relations among the proposed objects to make a final grounding decision. Our method does not require any labeled training data and can generalize to novel 3D scenes and arbitrary text queries. We evaluate LLM-Grounder on the ScanRefer benchmark and demonstrate state-of-the-art zero-shot grounding accuracy. Our findings indicate that LLMs significantly improve the grounding capability, especially for complex language queries, making LLM-Grounder an effective approach for 3D vision-language tasks in robotics.

Keywords: Semantic Scene Understanding, Deep Learning for Visual Perception, Vision-Based Navigation
Abstract: Estimating the traversability of terrain should be reliable and accurate in diverse conditions for autonomous driving in off-road environments. However, learning-based ap- proaches often yield unreliable results when confronted with unfamiliar contexts, and it is challenging to obtain manual annotations frequently for new circumstances. In this paper, we introduce a method for learning traversability from images that utilizes only self-supervision and no manual labels, enabling it to easily learn traversability in new circumstances. To this end, we first generate self-supervised traversability labels from past driving trajectories by labeling regions traversed by the vehicle as highly traversable. Using the self-supervised labels, we then train a neural network that identifies terrains that are safe to traverse from an image using a one-class classification algorithm. Additionally, we supplement the limitations of self- supervised labels by incorporating methods of self-supervised learning of visual representations. To conduct a comprehensive evaluation, we collect data in a variety of driving environments and perceptual conditions and show that our method produces reliable estimations in various environments. In addition, the experimental results validate that our method outperforms other self-supervised traversability estimation methods and achieves comparable performances with supervised learning methods trained on manually labeled data.

Keywords: Semantic Scene Understanding, Deep Learning for Visual Perception, Deep Learning Methods
Abstract: Huge progress has been made in LiDAR 3D semantic segmentation, but there are still two under-explored imbalances on the radial axis: points are unevenly concentrated on the near side, and the distribution of foreground object instances is skewed to the near side. This leads the training of the model to favor semantics at the near side with the majority of points and object instances. Both the popular cylindrical and the neglected spherical voxelizations aim to address the problem of imbalanced point distribution by increasing the volume of voxels along the radial distance to include fewer near-side points in a smaller voxel and more far-side points in a bigger voxel. However, this causes a problem of the receptive field enlarging along the radial distance, which is not desirable in LiDAR point clouds since the size of an object is distance-independent. This can be addressed in cubic voxelization which has a fixed volume of voxels. Thus, we propose a new LiDAR 3D semantic segmentation network (Hi-VoxelNet) with Hybrid Voxelization that leverages the advantages of cubic, cylindrical, and spherical voxelizations for hybrid voxel feature learning. To address the radial imbalance of object instances, we propose a novel data augmentation technique termed as RadialMix that uses radial sample duplication to increase the number of distant foreground object instances and mixes the radial duplication with another point cloud for enriching the training samples. With the joint improvements of the radial imbalances, our method archives state-of-the-art performance on nuScenes and SemanticKITTI datasets, and consistently shows significant improvements along the radial distances. Our code is publicly available at https://github.com/jialeli1/lidarseg3d.

Keywords: Semantic Scene Understanding, Deep Learning for Visual Perception, Object Detection, Segmentation and Categorization
Abstract: Current state-of-the-art methods for panoptic segmentation require an immense amount of annotated training data that is both arduous and expensive to obtain posing a significant challenge for their widespread adoption. Concurrently, recent breakthroughs in visual representation learning have sparked a paradigm shift leading to the advent of large foundation models that can be trained with completely unlabeled images. In this work, we propose to leverage such task-agnostic image features to enable few-shot panoptic segmentation by presenting Segmenting Panoptic Information with Nearly 0 labels (SPINO). In detail, our method combines a DINOv2 backbone with lightweight network heads for semantic segmentation and boundary estimation. We show that our approach, albeit being trained with only ten annotated images, predicts high-quality pseudo-labels that can be used with any existing panoptic segmentation method. Notably, we demonstrate that SPINO achieves competitive results compared to fully supervised baselines while using less than 0.3% of the ground truth labels, paving the way for learning complex visual recognition tasks leveraging foundation models. To illustrate its general applicability, we further deploy SPINO on real-world robotic vision systems for both outdoor and indoor environments. To foster future research, we make the code and trained models publicly available at http://spino.cs.uni-freiburg.de.

Keywords: Semantic Scene Understanding, Deep Learning for Visual Perception, Visual Learning
Abstract: In the fields of robotic perception and computer vision, achieving accurate semantic segmentation of low-light or nighttime scenes is challenging. This is primarily due to the limited visibility of objects and the reduced texture and color contrasts among them. To address the issue of limited visibility, we propose a hierarchical gated convolution unit, which simultaneously expands the receptive field and restores edge texture. To address the issue of reduced texture among objects, we propose a dual closed-loop bipartite matching algorithm to establish a total loss function consisting of the unsupervised illumination enhancement loss and supervised intersection-over-union loss, thus enabling the joint minimization of both losses via the Hungarian algorithm. We thus achieve end-to-end training for a semantic segmentation network especially suitable for handling low-light scenes. Experimental results demonstrate that the proposed network surpasses existing methods on the Cityscapes dataset and notably outperforms state-of-the-art methods on both Dark Zurich and Nighttime Driving datasets.

Keywords: Semantic Scene Understanding, Autonomous Agents, Sensor Fusion
Abstract: The complementarity between camera and LiDAR data makes fusion methods a promising approach to improve 3D semantic segmentation performance. Recent transformer-based methods have also demonstrated superiority in segmentation. However, multimodal solutions incorporating transformers are underexplored and face two key inherent difficulties: over-attention and noise from different modal data. To overcome these challenges, we propose a Deformable Multimodal Representation Fusion (DefFusion) framework consisting mainly of a Deformable Representation Fusion Transformer and Dynamic Representation Augmentation Modules. The Deformable Representation Fusion Transformer introduces the deformable mechanism in multimodal fusion, avoiding over-attention and improving efficiency by adaptively modeling a 2D key/value set for a given 3D query, thus enabling multimodal fusion with higher flexibility. To enhance the 2D representation and 3D representation, the Dynamic Representation Enhancement Module is proposed to dynamically remove noise in the input representation via Dynamic Grouped Representation Generation and Dynamic Mask Generation. Extensive experiments validate that our model achieves the best 3D semantic segmentation performance on SemanticKITTI and NuScenes benchmarks.

Keywords: Semantic Scene Understanding, Continual Learning, SLAM
Abstract: Inventory monitoring in homes, factories, and retail stores relies on maintaining data despite objects being swapped, added, removed, or moved. We introduce Lifelong LERF, a method that allows a mobile robot with minimal compute to jointly optimize a dense language and geometric representation of its surroundings. Lifelong LERF maintains this representation over time by detecting semantic changes and selectively updating these regions of the environment, avoiding the need to exhaustively remap. Human users can query inventory by providing natural language queries and receiving a 3D heatmap of potential object locations. To manage the computational load, we use Fog-ROS2, a cloud robotics platform, to offload resource-intensive tasks. Lifelong LERF obtains poses from a monocular RGBD SLAM backend, and uses these poses to progressively optimize a Language Embedded Radiance Field (LERF) for semantic monitoring. Experiments with 3-5 objects arranged on a tabletop and a Turtlebot with a RealSense camera suggest that Lifelong LERF can persistently adapt to changes in objects with up to 91% accuracy.

Keywords: AI-Based Methods, Deep Learning in Grasping and Manipulation, Perception for Grasping and Manipulation
Abstract: Robotic manipulation requires accurate perception of the environment, which poses a significant challenge due to its inherent complexity and constantly changing nature. In this context, RGB image and point-cloud observations are two commonly used modalities in visual-based robotic manipulation, but each of these modalities have their own limitations. Commercial point-cloud observations often suffer from issues like sparse sampling and noisy output due to the limits of the emission-reception imaging principle. On the other hand, RGB images, while rich in texture information, lack essential depth and 3D information crucial for robotic manipulation. To mitigate these challenges, we propose an image-only robotic manipulation framework that leverages an eye-on-hand monocular camera installed on the robot's parallel gripper. By moving with the robot gripper, this camera gains the ability to actively perceive the object from multiple perspectives during the manipulation process. This enables the estimation of 6D object poses, which can be utilized for manipulation. While, obtaining images from more and diverse viewpoints typically improves pose estimation, it also increases the manipulation time. To address this trade-off, we employ a reinforcement learning policy to synchronize the manipulation strategy with active perception, achieving a balance between 6D pose accuracy and manipulation efficiency. Our experimental results in both simulated and real-world environments showcase the state-of-the-art effectiveness of our approach. We believe that our method will inspire further research on real-world-oriented robotic manipulation.

Keywords: Deep Learning in Grasping and Manipulation, RGB-D Perception, Reinforcement Learning
Abstract: Manipulating unseen articulated objects through visual feedback is a critical but challenging task for real robots. Existing learning-based solutions mainly focus on visual affordance learning or other pre-trained visual models to guide manipulation policies, which face challenges for novel instances in real-world scenarios. In this paper, we propose a novel part-guided 3D RL framework, which can learn to manipulate articulated objects without demonstrations. We combine the strengths of 2D segmentation and 3D RL to improve the efficiency of RL policy training. To improve the stability of the policy on real robots, we design a Frame-consistent Uncertainty-aware Sampling (FUS) strategy to get a condensed and hierarchical 3D representation. In addition, a single versatile RL policy can be trained on multiple articulated object manipulation tasks simultaneously in simulation and shows great generalizability to novel categories and instances. Experimental results demonstrate the effectiveness of our framework in both simulation and real-world settings.

Keywords: Deep Learning in Grasping and Manipulation, Reinforcement Learning, Mechanism Design
Abstract: We introduce MORPH, a method for co-optimization of hardware design parameters and control policies in simulation using reinforcement learning. Like most co-optimization methods, MORPH relies on a model of the hardware being optimized, usually simulated based on the laws of physics. However, such a model is often difficult to integrate into an effective optimization routine. To address this, we introduce a proxy hardware model, which is always differentiable and enables efficient co-optimization alongside a long-horizon control policy using RL. MORPH is designed to ensure that the optimized hardware proxy remains as close as possible to its realistic counterpart, while still enabling task completion. We demonstrate our approach on simulated 2D reaching and 3D multi-fingered manipulation tasks.

Keywords: Deep Learning in Grasping and Manipulation, Reinforcement Learning, Machine Learning for Robot Control
Abstract: Reinforcement learning solely from an agent's self-generated data is often believed to be infeasible for learning on real robots, due to the amount of data needed. However, if done right, agents learning from real data can be surprisingly efficient through re-using previously collected sub-optimal data. In this paper we demonstrate how the increased understanding of off-policy learning methods and their embedding in an iterative online/offline scheme ("collect and infer") can drastically improve data-efficiency by using all the collected experience, which empowers learning from real robot experience only. Moreover, the resulting policy improves significantly over the state of the art on a recently proposed real robot manipulation benchmark. Our approach learns end-to-end, directly from pixels, and does not rely on additional human domain knowledge such as a simulator or demonstrations.

Keywords: AI-Based Methods, Deep Learning in Grasping and Manipulation, Perception for Grasping and Manipulation
Abstract: Robotic affordances, providing information about what actions can be taken in a given situation, can aid robotic manipulation. However, learning about affordances requires expensive large annotated datasets of interactions or demonstrations. In this work, we argue that well-directed interactions with the environment can mitigate this problem and propose an information-based measure to augment the agent's objective and accelerate the affordance discovery process. We provide a theoretical justification of our approach and we empirically validate the approach both in simulation and real-world tasks. Our method, which we dub IDA, enables the efficient discovery of visual affordances for several action primitives, such as grasping, stacking objects, or opening drawers, strongly improving data efficiency in simulation, and it allows us to learn grasping affordances in a small number of interactions, on a real-world setup with a UFACTORY xArm 6 robot arm.

Keywords: Deep Learning in Grasping and Manipulation, Reinforcement Learning, Multifingered Hands
Abstract: Despite the prevalence of robotic manipulation tasks in various real-world applications of different requirements and needs, there has been a lack of focus on enhancing the adaptability of robotic grasping systems. Most of the current literature constructs models around a single gripper, succumbing to a tradeoff between gripper complexity and

generalizability. Adapting such models pre-trained on one type of gripper to another to work around the tradeoff is inefficient and not scalable, as it would require tremendous effort and computational cost to generate new datasets and relearn the grasping task. In this letter,

we propose a novel hybrid architecture for robot grasping that efficiently learns to adapt to different gripper designs. Our approach involves a three-step process that first obtains a rough grasp pose prediction from a parallel gripper model, then predicts an adaptive action using a convolutional neural network, and finally refines the predicted action with reinforcement learning. The proposed method shows

significant improvements in grasping performance compared to existing methods for both generated datasets and real-world scenarios, presenting a promising direction for improving the adaptability and flexibility of robotic manipulation systems.

Keywords: Deep Learning in Grasping and Manipulation, RGB-D Perception, Grasping
Abstract: Fast and robust object grasping in clutter is a crucial component of robotics. Most current works resort to the whole observed point cloud for 6-Dof grasp generation, ignoring the guidance information excavated

from global semantics, thus limiting high-quality grasp generation and real-time performance. In this work, we show that the widely used heatmaps are underestimated in the efficiency of 6-Dof grasp generation. Therefore, we propose an effective local grasp generator combined with grasp heatmaps as guidance, which infers in a global-to-local semantic-to-point way. Specifically, Gaussian encoding and the grid-based strategy are applied to predict grasp heatmaps as guidance to aggregate local points into graspable regions and provide global semantic information. Further, a novel non-uniform anchor sampling mechanism is designed to improve grasp accuracy and diversity.

Benefiting from the high-efficiency encoding in the image space and focusing on points in local graspable regions, our framework can perform high-quality grasp detection in real-time and achieve state-of-the-art results. In addition, real robot experiments demonstrate the effectiveness of our method with a success rate of 94% and a clutter completion rate of 100%.

Keywords: Deep Learning in Grasping and Manipulation, Transfer Learning, Visual Servoing
Abstract: Recently, several works achieve end-to-end visual servoing (VS) for robotic manipulation by replacing traditional controller with differentiable neural networks, but lose the ability to servo arbitrary desired poses. This letter proposes a differentiable architecture for arbitrary pose servoing: a hyper-network based neural controller (HPN-NC). To achieve this, HPN-NC consists of a hyper net and a low-level controller, where the hyper net learns to generate the parameters of the low-level controller and the controller uses the 2D keypoints error for control like traditional image-based visual servoing (IBVS). HPN-NC can complete 6 degree of freedom visual servoing with large initial offset. Taking advantage of the fully differentiable nature of HPN-NC, we provide a three-stage training procedure to servo real world objects. With self-supervised end-to-end training, the performance of the integrated model can be further improved in unseen scenes and the amount of manual annotations can be significantly reduced.

Keywords: Deep Learning in Grasping and Manipulation, Reinforcement Learning, Perception for Grasping and Manipulation
Abstract: We present a novel vision-based, 6-DoF grasping framework based on Deep Reinforcement Learning (DRL) that is capable of directly synthesizing continuous 6-DoF actions in cartesian space. Our proposed approach uses visual observations from an eye-in-hand RGB-D camera, and we mitigate the sim-to-real gap with a combination of domain randomization, image augmentation, and segmentation tools. Our method consists of an off-policy, maximum-entropy, Actor-Critic algorithm that learns a policy from a binary reward and a few simulated example grasps. It does not need any real-world grasping examples, is trained completely in simulation, and is deployed directly to the real world without any fine-tuning. The efficacy of our approach is demonstrated in simulation and experimentally validated in the real world on 6-DoF grasping tasks, achieving state-of-the-art results of an 86% mean zero-shot success rate on previously unseen objects, an 85% mean zero-shot success rate on a class of previously unseen adversarial objects, and a 74.3% mean zero-shot success rate on a class of previously unseen, challenging "6-DoF" objects.

Raw footage of real-world validation can be found at https://youtu.be/bwPf8Imvook

Keywords: Human-Centered Automation, Telerobotics and Teleoperation, Learning from Demonstration
Abstract: Most existing 6-DoF robot grasping solutions depend on strong supervision on grasp pose to ensure satisfactory performance, which could be laborious and impractical when the robot works in some restricted area. To this end, we propose a self-supervised 6-DoF grasp pose detection framework via an Augmented Reality (AR) teleoperation system that can efficiently learn human demonstrations and provide 6-DoF grasp poses without grasp pose annotations. Specifically, the system collects the human demonstration from the AR environment and contrastively learns the grasping strategy from the demonstration. For the real-world experiment, the proposed system leads to satisfactory grasping abilities and learning to grasp unknown objects within three demonstrations.

Keywords: Acceptability and Trust, Human-Robot Collaboration
Abstract: Collaboration between humans and robots is becoming increasingly crucial in our daily life. In order to accomplish efficient cooperation, trust recognition is vital, empowering robots to predict human behaviors and make trust-aware decisions. Consequently, there is an urgent need for a generalized approach to recognize human-robot trust. This study addresses this need by introducing an EEG-based method for trust recognition during human-robot cooperation. A human-robot cooperation game scenario is used to stimulate various human trust levels when working with robots. To enhance recognition performance, the study proposes an EEG Vision Transformer model coupled with a 3-D spatial representation to capture the spatial information of EEG, taking into account the topological relationship among electrodes. To validate this approach, a public EEG-based human trust dataset called EEGTrust is constructed. Experimental results indicate the effectiveness of the proposed approach, achieving an accuracy of 74.99% in slice-wise cross-validation and 62.00% in trial-wise cross-validation. This outperforms baseline models in both recognition accuracy and generalization. Furthermore, an ablation study demonstrates a significant improvement in trust recognition performance of the spatial representation. The source code and EEGTrust dataset are available at https://github.com/CaiyueXu/EEGTrust.

Keywords: Human-Centered Automation, Force and Tactile Sensing
Abstract: Machine learning and deep learning have been used extensively to classify physical surfaces through images and time-series contact data. However, these methods rely on human expertise and entail the time-consuming processes of data and parameter tuning. To overcome these challenges, we propose an easily implemented framework that can directly handle heterogeneous data sources for classification tasks. Our data-versus-data approach automatically quantifies distinctive differences in distributions in a high-dimensional space via kernel two-sample testing between two sets extracted from multimodal data (e.g., images, sounds, haptic signals). We demonstrate the effectiveness of our technique by benchmarking against expertly engineered classifiers for visual-audio-haptic surface recognition due to the industrial relevance, difficulty, and competitive baselines of this application; ablation studies confirm the utility of key components of our pipeline. As shown in our open-source code, we achieve 97.2% accuracy on a standard multi-user dataset with 108 surface classes, outperforming the state-of-the-art machine-learning algorithm by 6% on a more difficult version of the task. The fact that our classifier obtains this performance with minimal data processing in the standard algorithm setting reinforces the powerful nature of kernel methods for learning to recognize complex patterns.

Keywords: Acceptability and Trust, Human-Robot Teaming, Human-Robot Collaboration
Abstract: In complex tasks (beyond a single targeted controller) requiring robots to collaborate with multiple human users, two challenges arise: complex tasks are often composed of multiple behaviors which can only be evaluated as a collective (a meta-behavior) and user preferences often differ between individuals, yet successful interactions are expected across groups. To address these challenges, we formulate a set-wise preference learning problem, and validate a cost function that captures human group preferences for complex collaborative robotic tasks (cobotics). We develop a sparse optimization formulation to introduce a distinctiveness metric that aggregates individuals with similar preference profiles. Analysis of anonymized unlabelled preferences provides further insight into group preferences. Identification of the mode average most-preferred meta-behavior and minimum covariance bound allows us to analyze group cohesion. A user study with 43 participants is used to validate group preference profiles.

Keywords: Acceptability and Trust, Human-Centered Robotics, Human-Robot Collaboration
Abstract: In Human-Robot Interaction scenarios, human factors like trust can greatly impact task performance and interaction quality. Recent research has confirmed that perceived robot proficiency is a major antecedent of trust. By making robots aware of their capabilities, we can allow them to choose when to perform low-confidence actions, thus actively controlling the risk of trust reduction. In this paper, we propose Self-Confidence through Observed Novel Experiences (SCONE), a policy to learn self-confidence from experience using semantic action embeddings. Using an assistive cooking setting, we show that the semantic aspect allows SCONE to learn self-confidence faster than existing approaches, while also achieving promising performance in simple instructions following. Finally, we share results from a pilot study with 31 participants, showing that such a self-confidence-aware policy increases capability-based human trust.

Keywords: Human-Centered Robotics, AI-Based Methods, Reactive and Sensor-Based Planning
Abstract: This paper proposes an interactive navigation framework by using large language and vision-language models, allowing robots to navigate in environments with traversable obstacles. We utilize the large language model (GPT-3.5) and the open-set Vision-language Model (Grounding DINO) to create an action-aware costmap to perform effective path planning without fine-tuning. With the large models, we can achieve an end-to-end system from textual instructions like “Can you pass through the curtains to deliver medicines to me?”, to bounding boxes (e.g., curtains) with action-aware attributes. They can be used to segment LiDAR point clouds into two parts: traversable and untraversable parts, and then an action-aware costmap is constructed for generating a feasible path. The pre-trained large models have great generalization ability and do not require additional annotated data for training, allowing fast deployment in the interactive navigation tasks. We choose to use multiple traversable objects such as curtains and grasses for verification by instructing the robot to traverse them. Besides, traversing curtains in a medical scenario was tested. All experimental results demonstrated the proposed framework’s effectiveness and adaptability to diverse environments.

Keywords: Human-Centered Robotics, Brain-Machine Interfaces, Gesture, Posture and Facial Expressions
Abstract: In the past decade, there has been significant advancement in designing wearable neural interfaces for controlling neurorobotic systems, particularly bionic limbs. These interfaces function by decoding signals captured non-invasively from the skin's surface. Portable high-density surface electromyography (HD-sEMG) modules combined with deep learning decoding have attracted interest by achieving excellent gesture prediction and myoelectric control of prosthetic systems and neurorobots. However, factors like small electrode size and unstable electrode-skin contacts make HD-sEMG susceptible to pixel electrode drops. The sparse electrode-skin disconnections rooted in issues such as low adhesion, sweating, hair blockage, and skin stretch challenge the reliability and scalability of these modules as the perception unit for neurorobotic systems. This paper proposes a novel deep-learning model providing resiliency for HD-sEMG modules, which can be used in the wearable interfaces of neurorobots. The proposed 3D Dilated Efficient CapsNet model trains on an augmented input space to computationally `force' the network to learn channel dropout variations and thus learn robustness to channel dropout. The proposed framework maintained high performance under a sensor dropout reliability study conducted. Results show conventional models' performance significantly degrades with dropout and is recovered using the proposed architecture and the training paradigm.

Keywords: Human-Centered Automation, Intention Recognition, Human-Robot Collaboration
Abstract: In shared autonomy, human-robot handover for object delivery is crucial. Accurate robot predictions of human hand motion and intentions enhance collaboration efficiency. However, low prediction accuracy increases mental and physical demands on the user. In this work, we propose a system for predicting hand motion and intended target during human-robot handover using Inverse Reinforcement Learning (IRL). A set of feature functions were designed to explicitly capture users’ preferences during the task. The proposed approach was experimentally validated through user studies. Results indicate that the proposed method outperformed other state-of-the-art methods (PI-IRL, BP-HMT, RNNIK-MKF and CMk=5) with users feeling comfortable reaching upto 60% of the total distance to the target for handover with 90% target prediction accuracy. The target prediction accuracy reaches 99.9% when less than 20% of the task remains.

Keywords: Cognitive Control Architectures, Acceptability and Trust, Human Factors and Human-in-the-Loop
Abstract: Theory of Mind (ToM) is a fundamental cognitive architecture that endows humans with the ability to attribute mental states to others. Humans infer the desires, beliefs, and intentions of others by observing their behavior and, in turn, adjust their actions to facilitate better interpersonal communication and team collaboration. In this paper, we investigated trust-aware robot policy with the theory of mind in a multiagent setting where a human collaborates with a robot against another human opponent. We show that by only focusing on team performance, the robot may resort to the reverse psychology trick, which poses a significant threat to trust maintenance. The human's trust in the robot will collapse when they discover deceptive behavior by the robot. To mitigate this problem, we adopt the robot theory of mind model to infer the human's trust beliefs, including true belief and false belief (an essential element of ToM). We designed a dynamic trust-aware reward function based on different trust beliefs to guide the robot policy learning, which aims to balance between avoiding human trust collapse due to robot reverse psychology and leveraging its potential to boost team performance. The experimental results demonstrate the importance of the ToM-based robot policy for human-robot trust and the effectiveness of our robot ToM-based robot policy in multiagent interaction settings.

Keywords: Human Factors and Human-in-the-Loop
Abstract: 3D reconstruction based on monocular videos has attracted wide attention, and existing reconstruction methods usually work in a reconstruction-after-scanning manner. However, these methods suffer from insufficient data collection problems due to the lack of effective guidance for users during the scanning process, which affects reconstruction quality. We propose VIDAR, which visually guides users with the streaming incremental reconstructed mesh in data collection for monocular 3D reconstruction. We propose an incremental mesh extraction algorithm to achieve lossless fusion of streaming incremental mesh data via slice-style management for guidance quality. We also design an incremental mesh rendering algorithm to achieve precise memory reallocation by updating the buffer in a fill-in-the-blank pattern for guidance efficiency. Besides, we introduce several optimizations on data transmission and human-computer interaction to improve the overall system performance. The experiment results on real-world scenes show that VIDAR efficiently delivers high-quality visual guidance and outperforms the non-interactive data collection methods for scene reconstruction.

Keywords: Human Factors and Human-in-the-Loop, Prosthetics and Exoskeletons, Rehabilitation Robotics
Abstract: Rehabilitative robotics, particularly lower-limb exoskeletons (LLEs), have gained increasing importance in aiding patients regain ambulatory functions. One of the challenges in making these systems effective is the implementation of an assist-as-needed (AAN) control strategy that intervenes only when the patient deviates from the correct movement pattern. Equally crucial is the need for the LLE to exhibit "transparency" — minimising its interaction forces with the wearer to feel as natural as possible. This paper introduces a novel approach to transparency control based on a human-in-the-loop velocity optimisation framework. The proposed method employs torque data captured from past steps through a Series Elastic Actuator (SEA) to approximate the wearer's intended future movements and computes a corresponding transparent velocity trajectory. The velocity commands are complemented by an Adaptive Frequency Oscillator (AFO) based position controller that leverages the periodic nature of human gait and is modified with a force sensor for increased reactiveness to human gait variations. This approach is experimentally evaluated against a standard zero-torque controller with a stationary single-degree-of-freedom knee exoskeleton test platform in a proof-of-concept study. Preliminary results indicate that combining adaptive oscillators with interaction force sensing can improve transparency compared to the conventional zero-torque controller, using force readings for position control and torque measurements for velocity optimisation and control.

Keywords: Human Factors and Human-in-the-Loop, Human-Robot Collaboration, Robotics and Automation in Agriculture and Forestry
Abstract: This research contributes to the field of Human-Robot Collaboration (HRC) within dynamic and unstructured environments by extending the previously proposed Fuzzy State-Long Short-Term Memory (FS-LSTM) architecture to handle the uncertainty and irregularity inherent in real-world sensor data. Recognising the challenges posed by low-cost sensors, which are highly susceptible to environmental conditions and often fail to provide regular periodic readings, this paper introduces additional pre-processing blocks. These include two indirect Kalman filters and an additional LSTM network, which together enhance the input variables for the fuzzification process. The enhanced FS-LSTM approach is evaluated using real-world data, demonstrating its effectiveness in extracting meaningful information and accurately recognising human activities. This work underscores the potential of robotics in addressing global challenges, particularly in labour-intensive and hazardous tasks. By improving the integration of humans and robots in unstructured environments, this research contributes to the broader exploration of robotics in new societal applications, fostering connections and collaborations across diverse fields.

Keywords: Human Factors and Human-in-the-Loop, Intention Recognition
Abstract: Advanced Human-In-The-Loop (HITL) control strategies for robot hands based on surface electromyography (sEMG) are among major research questions in robotics. Due to intrinsic complexity and inaccuracy of labeling procedures, unsupervised regression of sEMG signals has been employed in literature, however showing several limitations in realizing multiple grasping motion control. In this work, we propose a novel Human-Robot interface (HRi) based on self-supervised regression of sEMG signals, combining Non-Negative Matrix Factorization (NMF) with Deep Neural Networks (DNN) in order to both avoid explicit labeling procedures and have powerful nonlinear fitting capabilities. Experiments involving 10 healthy subjects were carried out, consisting of an offline session for systematic evaluations and comparisons with traditional unsupervised approaches, and an online session for assessing realtime control of a wearable anthropomorphic robot hand. The offline results demonstrate that the proposed self-supervised regression approach overcame traditional unsupervised methods, even considering different robot hands with dissimilar kinematic structures. Furthermore, the subjects were able to successfully perform online control of multiple grasping motions of a real wearable robot hand, reporting for high reliability over repeated grasp-transportation-release tasks with different objects. Statistical support is provided along with experimental outcomes.

Keywords: Human Factors and Human-in-the-Loop, Physical Human-Robot Interaction, Human-Centered Robotics
Abstract: Handing objects to humans is an essential capability for collaborative robots. Previous research works on human-robot handovers focus on facilitating the performance of the human partner and possibly minimising the physical effort needed to grasp the object. However, altruistic robot behaviours may result in protracted and awkward robot motions, contributing to unpleasant sensations by the human partner and affecting perceived safety and social acceptance. This paper investigates whether transferring the psychological principle that "humans act coefficiently as a group" (i.e. simultaneously maximising the benefits of all agents involved) to human-robot cooperative tasks promotes a more seamless and natural interaction. Human-robot coefficiency is first modelled by identifying implicit indicators of human comfort and discomfort as well as calculating the robot energy consumption in performing the desired trajectory. We then present a reinforcement learning approach that uses the human-robot coefficiency score as reward to adapt and learn online the combination of robot interaction parameters that maximises such coefficiency. Results proved that by acting coefficiently the robot could meet the individual preferences of most subjects involved in the experiments, improve the human perceived comfort, and foster trust in the robotic partner.

Keywords: Human Factors and Human-in-the-Loop, Learning Categories and Concepts, Data Sets for Robotic Vision
Abstract: In the context of rapid advancements in industrial automation, vision-based robotic grasping plays an increasingly crucial role. In order to enhance visual recognition accuracy, the utilization of large-scale datasets is imperative for training models to acquire implicit knowledge related to the handling of various objects. Creating datasets from scratch is a time and labor-intensive process. Moreover, existing datasets often contain errors due to automated annotations aimed at expediency, making the improvement of these datasets a substantial research challenge. Consequently, several issues have been identified in the annotation of grasp bounding boxes within the popular Jacquard Grasp. We propose utilizing a Human-In-The-Loop(HIL) method to enhance dataset quality. This approach relies on backbone deep learning networks to predict object positions and orientations for robotic grasping. Predictions with Intersection over Union (IOU) values below 0.2 undergo an assessment by human operators. After their evaluation, the data is categorized into False Negatives(FN) and True Negatives(TN). FN are then subcategorized into either missing annotations or catastrophic labeling errors. Images lacking labels are augmented with valid grasp bounding box information, whereas images afflicted by catastrophic labeling errors are completely removed. The open-source tool Labelbee was employed for 53,026 iterations of HIL dataset enhancement, leading to the removal of 2,884 images and the incorporation of ground truth information for 30,292 images. The enhanced dataset, named the Jacquard V2 Grasping Dataset, served as the training data for a range of neural networks. We have empirically demonstrated that these dataset improvements significantly enhance the training and prediction performance of the same network, resulting in an increase of 7.1% across most popular detection architectures for ten iterations. This refined dataset will be accessible on Google Drive and Baidu Netdisk,

Keywords: Human Factors and Human-in-the-Loop, Design and Human Factors, Long term Interaction
Abstract: Assessing proficiency in small unmanned aerial vehicles (sUAVs) pilots is complex and not well understood, but increasingly important to employ these vehicles in serious jobs such as wildland firefighting and infrastructure inspection. The limited prior work with UAVs has focused on user training using modalities like simulators and VR and no performance assessments with line-of-sight UAVs. This paper presents a training methodology for novice pilots of sUAVs. We presented two studies: the Baseline study (21 participants) and the Training study (16 participants). Our work is of interest to sUAV operators, regulators, and companies developing this technologies to produce a more capable workforce capable of consistent, safe operations. We successfully utilized the method developed in cite{kunde2022recognizing} to assess user proficiency in flying UAVs. We presented a UAV pilot training schedule for novice users (in the Training study), and were able to determine the minimum training time necessary to observe performance gains and mitigate damage. Results indicate that task completions noticeably improved and crashes minimized by day 10 of training, with a training plateau observed by day 15.

Keywords: Human Factors and Human-in-the-Loop, Human-Robot Collaboration, Planning under Uncertainty
Abstract: Work related musculoskeletal disorders (WMSDs) are very common. Repetitive motion, which is often present in industrial work, is one of the main physical causes of WMSDs. It uses the same set of human joints repeatedly, which leads to localized joint fatigue. In this work, we present a framework to plan a policy of a collaborative robot that reduces the human fatigue in the long term, in highly repetitive co-manipulation tasks, while taking into account the uncertainty in the human postural reaction to the robot motion and the partial observability of the human fatigue state. We model the problem using continuous-state Partially Observable Markov Decision Process (POMDP), and use a physics-based digital human simulator to predict the fatigue cost of the possible robot actions. We then use an online planning algorithm to compute the optimal robot policy. We demonstrate our approach on a simulated experiment in which a robot repeatedly carries an object for the human to work on, and the object Cartesian pose needs to be optimized. We compare the policy generated with our approach with a random, a cyclic and a greedy (short-term optimization) policy, for different user profiles. We show that our approach outperforms the other policies on all tested scenarios.

Keywords: Force and Tactile Sensing, Deep Learning in Grasping and Manipulation, Product Design, Development and Prototyping
Abstract: Accurate perception of the surrounding environment stands as a primary objective for robots. Through tactile interaction, vision-based tactile sensors provide the capability to capture high-resolution and multi-modal surface information of objects, thereby facilitating robots in achieving more dexterous manipulations. However, the prevailing GelSight sensors entail intricate calibration procedures, posing challenges in their application on curved surfaces and requiring the maintenance of stable lighting conditions throughout experimentation. Additionally, constrained by shape and structure, current vision-based tactile sensors are predominantly applied to measurements within a limited area. In this study, we design a novel cylindrical vision-based tactile sensor that enables continuous and swift perception of large-scale object surfaces through rolling. To tackle the challenges posed by laborious calibration processes, we propose a self-supervised photometric stereo method based on deep learning, which eliminates pre-calibration requirements and enables the derivation of surface normals from a single image without relying on stable lighting conditions. Finally, we perform surface reconstruction from normal and point cloud registration on the multiple frames of images obtained by rolling the cylindrical sensor, resulting in large surface reconstruction. We compare our method with the representative lookup table method in the GelSight sensors. The results show that the proposed method enhances both reconstruction accuracy and robustness, thereby demonstrating the potential of the proposed sensor in large-scale surface reconstruction. Codes and mechanical structures are available at: https://github.com/ZhangZhiyuanZhang/GelRoller

Keywords: Force and Tactile Sensing, Deep Learning Methods, Simulation and Animation
Abstract: Data-driven methods have been successfully applied to images from vision-based tactile sensors to fulfill various manipulation tasks. Nevertheless, these methods remain inefficient because of the lack of methods for simulating the sensors. Relevant research on simulating vision-based tactile sensors generally focus on generating images without markers, owing to the challenges in accurately generating marker motions caused by elastomer deformation. This disallows access to tactile information deducible from markers. In this work, we propose a generative adversarial network (GAN)-based method to generate realistic marker-embedded tactile images in GelSight-like vision-based tactile sensors. We trained the proposed GAN model with an aligned real tactile and simulated depth image dataset obtained from deforming the sensor against various objects. This allows the model to translate simulated depth image sequences into RGB tactile images with markers. Furthermore, the generator in the proposed GAN allows the network to integrate the history of deformations from the depth image sequences to generate realistic marker motions during the normal and lateral sensor deformations. We evaluated and compared the positional accuracy of the markers and image similarity metrics of the images generated via our method with those from prior methods. The generated tactile images from the proposed model show a 28.3 % decrease in marker positional error and a 93.5 % decrease in the image similarity m

Keywords: Force and Tactile Sensing, In-Hand Manipulation, Perception for Grasping and Manipulation
Abstract: We introduce a novel approach that combines tactile estimation and control for in-hand object manipulation. By integrating measurements from robot kinematics and an image-based tactile sensor, our framework estimates and tracks object pose while simultaneously generating motion plans in a receding horizon fashion to control the pose of a grasped object. This approach consists of a discrete pose estimator that tracks the most likely sequence of object poses in a coarsely discretized grid, and a continuous pose estimator-controller to refine the pose estimate and accurately manipulate the pose of the grasped object. Our method is tested on diverse objects and configurations, achieving desired manipulation objectives and outperforming single-shot methods in estimation accuracy. The proposed approach holds potential for tasks requiring precise manipulation and limited intrinsic in-hand dexterity under visual occlusion, laying the foundation for closed-loop behavior in applications such as regrasping, insertion, and tool use. Please see https://sites.google.com/view/texterity for videos of real-world demonstrations.

Keywords: Force and Tactile Sensing, Intelligent and Flexible Manufacturing
Abstract: This work presents a novel shape evaluation and optimization approach for shape sensing, specifically targeting the constrained, irregular, and intricate spatial shapes of flexible bronchoscopes (FB) in human bronchial tree. The proposed evaluation criteria and optimization methods combine clinical significance related to bronchial anatomical structures and address issues related to singular points and discontinuities in traditional shape reconstruction models. Three-dimensional experiments were conducted within eight spatial complex configurations printed from a proportional bronchial model. The 3D experiment results demonstrate an average reduction of approximately 34.1% in shape reconstruction errors across all eight airway models compared to the traditional model, validating the effectiveness and feasibility.

Keywords: Force and Tactile Sensing, Multi-Contact Whole-Body Motion Planning and Control
Abstract: It is difficult for robots to retrieve objects in densely cluttered lateral access scenes with movable objects as jamming against adjacent objects and walls can inhibit progress. We propose the use of two action primitives---burrowing and excavating---that can fluidize the scene to un-jam obstacles and enable continued progress. Even when these primitives are implemented in an open loop manner at clock-driven intervals, we observe a decrease in the final distance to the target location. Furthermore, we combine the primitives into a closed loop hybrid control strategy using tactile and proprioceptive information to leverage the advantages of both primitives without being overly disruptive. In doing so, we achieve a 10-fold increase in success rate above the baseline control strategy and significantly improve completion times as compared to the primitives alone or a naive combination of them.

Keywords: Force and Tactile Sensing, Physical Human-Robot Interaction
Abstract: Iontronic tactile sensors are promising to measure spatial-temporal contact information with high performance. However, no suitable measuring method has been presented, due to issues with crosstalk and non-negligible equivalent resistance. Hence, this study presents an impedance-separating method, which does not require complex analog components. A general Quadri-Terminal Impedance Network (QTIN) model is introduced to reduce crosstalk, which has specific compatibility with the impedance-separating method. The precise ranges are measured, showing non-rectangle shapes suitable for the response of iontronic tactile sensors. A simple denoising method is provided to reduce initial array noise obviously. This work could benefit various scenarios, such as human-robot interaction and physiological information monitoring.

Keywords: Force and Tactile Sensing, Physical Human-Robot Interaction, Manipulation Planning
Abstract: Assistive robots have the potential to support disabled and elderly people in daily dressing activities. An intermediate stage of dressing is to manipulate the garment from a crumpled initial state to an unfolded configuration that facilitates robust dressing. Applying quasi-static grasping actions with vision feedback on garment unfolding usually suffers from occluded grasping points. In this work, we propose a dynamic manipulation strategy: tracing the garment edge until the hidden corner is revealed. We introduce a model-based approach, where a deep visual-tactile predictive model iteratively learns to perform servoing from raw sensor data. The predictive model is formalized as Conditional Variational Autoencoder with contrastive optimization, which jointly learns underlying visual-tactile latent representations, a latent garment dynamics model, and future predictions of garment states. Two cost functions are explored: the visual cost defined by garment corner positions guarantees the gripper to move towards the corner, while the tactile cost defined by garment edge poses prevents the garment from falling from the gripper. The experimental results demonstrate the improvement of our contrastive visual-tactile model predictive control over single sensing modality and baseline model learning techniques. The proposed method enables a robot to unfold back-opening hospital gowns and perform upper-body dressing.

Keywords: Force and Tactile Sensing, Representation Learning
Abstract: The rapidly evolving field of robotics necessitates methods that can facilitate the fusion of multiple modalities. Specifically, when it comes to interacting with tangible objects, effectively combining visual and tactile sensory data is key to understanding and navigating the complex dynamics of the physical world, enabling a more nuanced and adaptable response to changing environments. Nevertheless, much of the earlier work in merging these two sensory modalities has relied on supervised methods utilizing datasets labeled by humans. This paper introduces MViTac, a novel methodology that leverages contrastive learning to integrate vision and touch sensations in a self-supervised fashion. By availing both sensory inputs, MViTac leverages intra and inter-modality losses for learning representations, resulting in enhanced material property classification and more adept grasping prediction. Through a series of experiments, we showcase the effectiveness of our method and its superiority over existing state-of-the-art self-supervised and supervised techniques. In evaluating our methodology, we focus on two distinct tasks: material classification and grasping success prediction. Our results indicate that MViTac facilitates the development of improved modality encoders, yielding more robust representations as evidenced by linear probing assessments.

Keywords: Force and Tactile Sensing, Robot Safety, Multi-Contact Whole-Body Motion Planning and Control
Abstract: Using tactile signal is a natural way to perceive potential dangers and safeguard robots. One possible method is to use full-body tactile sensors on the robot and perform safety maneuvers when dangerous stimuli are detected. In this work, we proposed a method based on full-body tactile sensors that operates at three different levels of granularity to ensure that robot interacts with the environment safely. The results showed that our system dramatically reduced the overall collision chance compared with several baselines, and intelligently handled current collisions. Our proposed framework is generalizable to a wide variety of robots, enabling them to predict and avoid dangerous collisions and reactively handle accidental tactile stimuli.

Keywords: Legged Robots, Optimization and Optimal Control, Modeling, Control, and Learning for Soft Robots
Abstract: Quadrupeds deployed in real-world scenarios need to be robust to unmodelled dynamic effects. In this work, we aim to increase the robustness of quadrupedal periodic forward jumping (i.e., pronking) by unifying cutting-edge model-based trajectory optimization and iterative learning control. Using a reduced-order soft anchor model, the optimization-based motion planner generates the periodic reference trajectory. The controller then iteratively learns the feedforward control signal in a repetition process, without requiring an accurate full-body model. When enhanced by a continuous learning mechanism, the proposed controller can learn the control input without resetting the system at the end of each iteration. Simulations and experiments on a quadruped with parallel springs demonstrate that continuous jumping can be learned in a matter of minutes, with high robustness against various types of terrain.

Keywords: Legged Robots, Mechanism Design, Motion Control
Abstract: Roller skating with passive wheels on a quadrupedal robot is more efficient than traditional walking. However, the typical mammalian quadruped robot with 3-DoFs legs can only perform one dynamic roller skating gait and has difficulty achieving turning motion. To address this limitation, we designed a novel quadrupedal robot with each leg having 4-DoFs to enable various roller skating locomotion including Swizzling, Stroking, and trot-like gaits while easily achieving turning motions. We considered the geometrical characteristics of the passive wheel and used the Levenberg-Marquardt method in robot kinematics to improve precision for both roller skating kinematics and contact point position for the dynamics controller. The position of the robot foot and the yaw angle of the passive wheel are decoupled for motion planning of all proposed gaits. Our proposed kinematics with wheeled geometry was verified through experiments to have higher precision, while the feasibility of all proposed roller-skating gaits was confirmed during straight motion and turning motion with a small radius on our prototype robot. Finally, we discussed the mobility efficiency of different roller skating gaits which were found to be more efficient than walking.

Keywords: Humanoid and Bipedal Locomotion, Legged Robots, Mapping
Abstract: Humanoid robots possess the ability to perform complex tasks in challenging environments. However, they require a model of the surroundings in a representation that is sufficient enough for downstream tasks such as footstep planning. The maps generated by existing mapping algorithms are either sparse, insufficient for footstep planning, memory intensive, or too slow for dynamic humanoid behaviors. In this work, we develop a mapping algorithm that combines planar region measurements along with kinematic-inertial state estimates to build a dense but efficient map of bounded planar surfaces. We present novel algorithms for plane feature matching, tracking and registration for mapping within a factor graph framework. The generated map is not only memory efficient, but also offers higher reliability and speed in bipedal footstep planning, than was possible earlier. The complete algorithm is also demonstrated using a full-scale humanoid robot, Nadia, walking over both flat ground and rough terrain utilizing the generated terrain map.

Keywords: Legged Robots, Optimization and Optimal Control, Robust/Adaptive Control
Abstract: In order to perform highly dynamic and agile maneuvers, legged robots typically spend time in underactuated domains (e.g. with feet off the ground) where the system has limited command of its acceleration and a constrained amount of time before transitioning to a new domain (e.g. foot touchdown). Meanwhile, these transitions can instantaneously change the system’s state, possibly causing perturbations to be mapped arbitrarily far away from the target trajectory. These properties make it difficult for local feedback controllers to effectively recover from disturbances as the system evolves through underactuated domains and hybrid impact events. To address this, we utilize the fundamental solution matrix that characterizes the evolution of perturbations through a hybrid trajectory and its 2-norm, which represents the worst- case growth of perturbations. In this paper, the worst-case perturbation analysis is used to explicitly reason about the tracking performance of a hybrid trajectory and is incorporated in an iLQR framework to optimize a trajectory while taking into account the closed-loop convergence of the trajectory under an LQR tracking controller. The generated convergent trajectories recover more effectively from perturbations, are more robust to large disturbances, and use less feedback control effort than trajectories generated with traditional methods.

Keywords: Legged Robots, Optimization and Optimal Control, Whole-Body Motion Planning and Control
Abstract: Robot skateboarding is an unexplored and challenging task for legged robots. Accurately modeling the dynamics of dual floating bases and developing effective planning and control methods present significant complexities in accomplishing skateboarding behavior.This paper focuses on enabling the quadrupedal platform CyberDog2 to achieve dynamic balancing and acceleration on a skateboard. An optimization-based control pipeline is developed through careful derivation of the system's equations of motion, considering both the robot and skateboard dynamics. By accounting for system physical constraints, an advanced offline trajectory optimization method is employed to generate various acceleration trajectories, creating a motion library for the system. An online linear model predictive control with whole body control framework is used to track the generated trajectories and stablize the system in real-time. To validate its effectiveness , we conducted experiments in various scenarios. The quadrupedal robot successfully performed acceleration from a static state to various velocities and demonstrated the ability to balance and steer the skateboard.

Keywords: Legged Robots, Robotics and Automation in Construction, Constrained Motion Planning
Abstract: This study focuses on a layered, experience-based, multi-modal contact planning framework for agile quadrupedal locomotion over a constrained rebar environment. To this end, our hierarchical planner incorporates locomotion-specific modules into the high-level contact sequence planner and solves kinodynamically-aware trajectory optimization as the low-level motion planner. Through quantitative analysis of the experience accumulation process and experimental validation of the kinodynamic feasibility of the generated locomotion trajectories, we demonstrate that the experience planning heuristic offers an effective way of providing candidate footholds for a legged contact planner. Additionally, we introduce a guiding torso path heuristic at the global planning level to enhance the navigation success rate in the presence of environmental obstacles. Our results indicate that the torso-path guided experience accumulation requires significantly fewer offline trials to successfully reach the goal compared to regular experience accumulation. Finally, our planning framework is validated in both dynamics simulations and real hardware implementations on a quadrupedal robot provided by Skymul Inc.

Keywords: Legged Robots, Robust/Adaptive Control, Reinforcement Learning
Abstract: This paper proposes a learning-based adaptive propulsion control (APC) method for a quadruped robot integrated with thrusters in amphibious environments, allowing it to move efficiently in water while maintaining its ground locomotion capabilities. We designed the specific reinforcement learning method to train the neural network to perform the vector propulsion control. Our approach coordinates the legs and propeller, enabling the robot to achieve speed and trajectory tracking tasks in the presence of actuator failures and unknown disturbances. Our simulated validations of the robot in water demonstrate the effectiveness of the trained neural network to predict the disturbances and actuator failures based on historical information, showing that the framework is adaptable to changing environments and is suitable for use in dynamically changing situations. Our proposed approach is suited to the hardware augmentation of quadruped robots to create avenues in the field of amphibious robotics and expand the use of quadruped robots in various applications.

Keywords: Machine Learning for Robot Control, Legged Robots, Humanoid and Bipedal Locomotion
Abstract: In recent years, legged and wheeled-legged robots have gained prominence for tasks in environments predominantly created for humans across various domains. One significant challenge faced by many of these robots is their limited capability to navigate stairs, which hampers their functionality in multi-story environments. This study proposes a method aimed at addressing this limitation, employing reinforcement learning to develop a versatile controller applicable to a wide range of robots. In contrast to the conventional velocity-based controllers, our approach builds upon a position-based formulation of the RL task, which we show to be vital for stair climbing. Furthermore, the methodology leverages an asymmetric actor-critic structure, enabling the utilization of privileged information from simulated environments during training while eliminating the reliance on exteroceptive sensors during real-world deployment. Another key feature of the proposed approach is the incorporation of a boolean observation within the controller, enabling the activation or deactivation of a stair-climbing mode. We present our results on different quadrupeds and bipedal robots in simulation and showcase how our method allows the balancing robot Ascento to climb 15cm stairs in the real world, a task that was previously impossible for this robot.

Keywords: Passive Walking, Legged Robots
Abstract: In the natural world, benefited from the advantages of the spine, quadrupeds exhibiting extraordinary flexibility which allowing them to move efficiently on variable terrains. The previous researches have indicated the legged robots which efficiently utilizing their spine can achieve rapid and stable locomotion. However,within the field of legged robot dynamics, the design of the spine and understanding how it positively influences locomotion is unclear, which is significant for quadruped robot to achieve efficient and stable walking. In this study, we proposed a model formed by tensegrity spine and rimless wheel to represent quadrupeds, using passive dynamic walking as a method, which has been well-demonstrated for observing the inherent characteristics, exhibited the locomotion characteristic of the model proposed. By numerical simulation, we observed change trend of locomotion performance with the configurations of spine's shape, and found direction of spine design that have a positive impact on walking. These findings contribute to the design of spine structures in quadruped robots.

Keywords: Calibration and Identification, Sensor-based Control, Motion Control of Manipulators, Surgical Robotics: Laparoscopy
Abstract: Manipulation in tight environment is challenging but increasingly common in vision-guided robotic applications. The significantly reduced amount of available feedback (limited visual cues, field of view, robot motion space, etc.) hinders solving the hand-eye relationship accurately. In this paper, we propose a new generic approach for online camera-robot calibration that could deal with the least feedback input available in tight environment: an arbitrarily restricted motion space and a single feature point with unknown position for the robot end-effector. We introduce the interactive perception to generate prescribed but tunable robot motions to reveal high-dimensional sensory feedback, which is not obtainable from static images. We then define the interactive feature plane (IFP), whose spatial property corresponds to the robot-actuating trajectories. A depth-free adaptive controller is proposed based on image feedback, where the converged orientation of IFP directly harvests the data for solving the hand-eye relationship. Our algorithm requires neither external calibration sensors/objects nor

large-scale data acquisition process. Simulations demonstrate the va

Keywords: Calibration and Identification, SLAM, Sensor Fusion
Abstract: The system observability analysis is of practical importance, for example, due to its ability to identify the unobservable directions of the estimated state which can influence estimation accuracy and help develop consistent and robust estimators. Recent studies focused on analyzing the observability of the state of various multisensor systems with a particular interest in unobservable directions induced by degenerate motions. However, those studies mostly stay in the specific sensor domain without aiding to extend the understanding to other heterogeneous systems. To this end, in this work, we provide degenerate motion analysis on general local and global sensor-paired systems, offering insights applicable to a wide range of existing navigation systems. Our analysis includes 9 degenerate motion identification including 5 already identified in literature and 4 new motions with both synchronous and asynchronous sensor-pair cases. Comprehensive numerical studies are conducted to verify those identified motions, show the effect of degenerate motion on state estimation, and demonstrate the generalizability of our analysis on various multisensor systems.

Keywords: Force and Tactile Sensing, Force Control, Contact Modeling
Abstract: The sense of touch enables humans to perform many delicate tasks on deformable objects and/or in a vision-denied environment. For a robot to achieve similar desirable interactions, such as administering a swab test, tactile information sensed beyond the tool-in-hand is correspondingly crucial for contact state estimation and contact tracking control. In this paper, we propose a tactile-guided planning and control framework using GTac, a hetero-Geneous Tactile sensor tailored for interaction with deformable objects beyond the immediate contact area. The biomimetic GTac in use is an improved version optimised for readout linearity which provides reliability in contact state estimation and force tracking. While a tactile-based classification and manipulation process is designed to estimate and align to the contact angle between the tool and the environment, a Koopman operator-based optimal control scheme is proposed to address challenges in the nonlinear control arising from the interaction with the deformable object. Several experiments are conducted to verify the effectiveness of the proposed framework. The experimental results demonstrate that the proposed framework can achieve accurate contact angle estimation as well as excellent tracking performance and strong robustness in force control.

Keywords: Force and Tactile Sensing, Force Control, Robotics and Automation in Agriculture and Forestry
Abstract: Although the potential demand for force sensors in both robotics and automation is high, the complexity of their structure increases the number of manufacturing processes. As a result, the rising cost of sensors has hindered the practical application of force measurement and force control. In this study, a flexure element comprising a structure that is easier to cut and process than conventional ones, as well as holes through the side of a cuboid, is proposed to simplify the manufacturing of force sensors. To ensure the safety of the proposed sensor design, an approximate equation is derived to predict the maximum von Mises stress on the flexure element using design parameters. Subsequently, we clarified a way to attach the strain gauge in a position that improves sensitivity. The results of the actual prototype sensor based on the proposed method show that the maximum nonlinearity error and decoupling error in the other axes are 0.442 %R.O. and 0.660 %R.O., respectively, and the performance is comparable to that of conventional force sensors. Because the prototype has a difference in resolution between the axes, a method for improving the resolution isotropy without changing the difficulty of machining is also proposed. In addition, the validity of the proposed method is demonstrated using experiments. Consequently, a force sensor with the same level of performance was developed using the proposed method, and the cutting process was made easier compared to that of convention

Keywords: Force Control, Force and Tactile Sensing, Medical Robots and Systems
Abstract: Intraoperative contact sensing has the potential to reduce the risk of surgical errors and enhance manipulation capabilities for medical robots, particularly in contact force control. Current intrinsic force sensing (IFS) methods are limited in application to medical instruments with arbitrary shape, due to high computational time and reliance on precise surface equations. This study presents an ultra-fast IFS method that uses multiple planes to establish surface geometry descriptions. The method can reduce high-order contact mechanical models that need to be solved iteratively to a set of linear equations, and calculate contact location analytically. In addition, a robot motion control approach based on the contact sensing method is proposed to maintain stable contact force and regulate the probe's orientation for robotic ultrasound systems (RUSS). Experimental results show that the contact sensing method is robust to friction and can achieve a mean (±SD) displacement error of 1.04±0.43 mm in contact location with computational time less than 1 ms. The system has been evaluated on a phantom with sinusoidal motion. To the best of our knowledge, this is the first study to validate adaptiveness of RUSS under dynamic conditions. The results demonstrated that the system exhibits comparable manipulation capabilities to human operators with only force sensing, indicating a high level of adaptiveness.

Keywords: Force Control, Motion Control, Robust/Adaptive Control
Abstract: This paper presents a control framework for legged robots that enables self-perception and resistance to external disturbances. First, a novel proprioceptive-based disturbance estimator is proposed. Compared with other disturbance estimators, this estimator possesses notable advantages in terms of filtering foot-ground interaction noise and suppressing the accumulation of estimation errors. Additionally, our estimator is a fully proprioceptive-based estimator, eliminating the need for any exteroceptive devices or observers. Second, we present a hierarchical optimized whole-body controller (WBC), which takes into account the full body dynamics, the actuation limits, the external disturbances, and the interactive constraints. Finally, extensive experimental trials conducted on the point-foot biped robot BRAVER validate the capabilities of the proposed estimator and controller under various disturbance conditions.

Keywords: Industrial Robots, Force and Tactile Sensing, Calibration and Identification
Abstract: This paper is concerned with the contact force estimation problem of robot manipulators based on imperfect dynamic models of the manipulator and the contact force. To handle the imperfect dynamic information of the manipulator, a hybrid model, consisting of the nominal model and the residual dynamics, is established for the manipulator, and the Gaussian process regression (GPR) technique is employed to learn the mean and covariance of the residual dynamics. On this basis, a virtual measurement equation is established for contact force estimation and a Gaussian process adaptive disturbance Kalman filter (GPADKF) is developed where the variational Bayes technique is employed to achieve online identification of the noise statistics in the force dynamics. The GPADKF is capable of decoupling the contact force from residual dynamics and system noises, thereby reducing the dependence on accurate dynamic models of the manipulator and the contact force. Simulation and experimental results demonstrate that the proposed scheme outperforms the state-of-art methods.

Keywords: Medical Robots and Systems, Modeling, Control, and Learning for Soft Robots, Kinematics
Abstract: Concentric tube continuum robots utilize nested tubes, which are subject to a set of inequalities. Current approaches to account for inequalities rely on branching methods such as if-else statements. It can introduce discontinuities, may result in a complicated decision tree, has a high wall-clock time, and cannot be vectorized. This affects the behavior and result of downstream methods in control, learning, workspace estimation, and path planning, among others.

In this paper, we investigate a mapping to mitigate branching methods. We derive a lower triangular transformation matrix to disentangle the inequalities and provide proof for the unique existence. It transforms the interdependent inequalities into independent box constraints. Further investigations are made for sampling, control, and workspace estimation. Approaches utilizing the proposed mapping are at least 14 times faster (up to 176 times faster), generate always valid joint configurations, are more interpretable, and are easier to extend.

Keywords: Medical Robots and Systems, Motion Control, Control Architectures and Programming
Abstract: Millimeter-scale magnetic rotating swimmers have multiple potential medical applications. They could, for example, navigate inside the bloodstream of a patient toward an occlusion and remove it. Magnetic rotating swimmers have internal magnets and propeller fins with a helical shape. A rotating magnetic field applies torque on the swimmer and makes it rotate. The shape of the swimmer, combined with the rotational movement, generates a propulsive force. Visual feedback is suitable for in-vitro closed-loop control. However, in-vivo procedures will require different feedback modalities due to the opacity of the human body. In this paper, we provide new methods and tools that enable the 3D control of a magnetic swimmer using a 2D ultrasonography device attached to a robotic arm to sense the swimmer's position. We also provide an algorithm that computes the placement of the robotic arm and a controller that keeps the swimmer within the ultrasound imaging slice. The position measurement and closed-loop control were tested experimentally.

Keywords: Medical Robots and Systems, Motion Control
Abstract: In recent years, the area of Robot-Assisted Minimally Invasive Surgery (RAMIS) is standing on the the verge of a new wave of innovations. However, autonomy in RAMIS is still in a primitive stage. Therefore, most surgeries still require manual control of the endoscope and the robotic instruments, resulting in surgeons needing to switch attention between performing surgical procedures and moving endoscope camera. Automation may reduce the complexity of surgical operations and consequently reduce the cognitive load on the surgeon while speeding up the surgical process. In this paper, a hybrid robotic endoscope control system based on fusion model of natural language processing (NLP) and modified YOLO-V8 vision model is proposed. This proposed system can analyze the current surgical workflow and generate logs to summarize the procedure for teaching and providing feedback to junior surgeons. The user study of this system indicated a significant reduction of the number of clutching actions and mean task time, which effectively enhanced the surgical training.

Keywords: Medical Robots and Systems, Object Detection, Segmentation and Categorization, Simulation and Animation
Abstract: To date, endovascular surgeries are performed using the golden standard of Fluoroscopy, which uses ionising radiation to visualise catheters and vasculature. Prolonged Fluoroscopic exposure is harmful for the patient and the clinician, and may lead to severe post-operative sequlae such as the development of cancer. Meanwhile, the use of interventional Ultrasound has gained popularity, due to its well-known benefits of small spatial footprint, fast data acquisition, and produce higher tissue contrast images. However, ultrasound images are hard to interpret, and it is difficult to localise vessels, catheters, and guidewires within them. This work proposed a solution using an adaptation of a state-of-the-art machine learning architecture (Transformers) to detect and segment catheters in axial interventional Ultrasound image sequences. The network architecture was inspired by the Attention in Attention mechanism, temporal tracking networks, and introduced a novel 3D segmentation head that performs 3D deconvolution across time. In order to facilitate training of such deep learning networks, we introduced a new data synthesis pipeline that used physics-based catheter insertion simulations, along with a convolutional ray-casting ultrasound simulator to produce synthetic ultrasound images of endovascular interventions. The proposed method was validated on a hold-out validation dataset, thus demonstrated robustness to ultrasound noise and a wide range of scanning angles. It was also tested on data collected from silicon-based aorta phantoms, thus demonstrated its potential for translation from sim-to-real. This work represents a significant step towards safer and more efficient endovascular surgery using interventional ultrasound.

Keywords: Medical Robots and Systems, Parallel Robots, Mechanism Design
Abstract: This paper presents a 6-DOF hybrid robot for percutaneous needle intervention procedures. The new robot combines the advantages of both serial robots and parallel robots, featuring compactness, high accuracy, and small footprint while overcoming the problems of the high cost of serial robots and the small workspace and singularity issue of parallel robots. Besides, by analyzing the workspace of the robot, the equation is derived between the structure parameter and workspace to adjust the parameters of the robot to satisfy different working scenes. According to the experiment, the accuracy of the robot is related to the position, distance, and insertion angle. The result shows that the performance is better when working near the center workspace and away from the servos and the average error of the robot is 1.39mm. The phantom experiment of lumbar puncture validates its feasibility.

Keywords: Medical Robots and Systems, Robot Safety, Machine Learning for Robot Control
Abstract: Several robotic systems have been emerged in the recent past to enhance the precision of micro-surgeries such as retinal procedures. Significant advancements have recently been achieved to increase the precision of such systems beyond surgeon capabilities. However, little attention has been paid to the impact of non-predicted and sudden movements of the patient and the environment. Therefore, analyzing environmental motion and vibrations is crucial to ensuring the optimal performance and reliability of medical systems that require micron-level precision, especially in real-life scenarios.

To address this challenge, this paper introduces a novel environmental motion analysis system that employs a grid layout with distributed sensing nodes throughout the environment. This system effectively tracks undesired movements (motions) at designated locations and predicts upcoming motions using neural network-based approaches. The outcomes of our experiments exhibit promising prospects for real-time motion monitoring and prediction, which has the potential to form a solid basis for enhancing the automation, safety, integration, and overall efficiency of robot-assisted micro-surgeries.

Keywords: Medical Robots and Systems, Robust/Adaptive Control, Force Control
Abstract: A surgeon's physiological hand tremor can significantly impact the outcome of delicate and precise retinal surgery, such as retinal vein cannulation (RVC) and epiretinal membrane peeling. Robot-assisted eye surgery technology provides ophthalmologists with advanced capabilities such as hand tremor cancellation, hand motion scaling, and safety constraints that enable them to perform these otherwise challenging and high-risk surgeries with high precision and safety. Steady-Hand Eye Robot (SHER) with cooperative control mode can filter out surgeon's hand tremor, yet another important safety feature, that is, minimizing the contact force between the surgical instrument and sclera surface for avoiding tissue damage cannot be met in this control mode. Also, other capabilities, such as hand motion scaling and haptic feedback, require a teleoperation control framework. In this work, for the first time, we implemented a teleoperation control mode incorporated with an adaptive sclera force control algorithm using a PHANTOM Omni haptic device and a force-sensing surgical instrument equipped with Fiber Bragg Grating (FBG) sensors attached to the SHER 2.1 end-effector. This adaptive sclera force control algorithm allows the robot to dynamically minimize the tool-sclera contact force. Moreover, for the first time, we compared the performance of the proposed adaptive teleoperation mode with the cooperative mode by conducting a vessel-following experiment inside an eye phantom under a microscope.

Keywords: Medical Robots and Systems, Robust/Adaptive Control
Abstract: Motion scaling is essential to empower users to conduct precise manipulation during teleoperation for robot-assisted microsurgery (RAMS). A constant, small motion scaling ratio can enhance the precision of teleoperation but hinder the operator from quickly reaching distant targets. The concept of self-adaptive motion scaling has been proposed in previous work. However, previous frameworks required extensive manual tuning of core parameters, which significantly depends on prior knowledge and may potentially lead to non-optimal solutions. This paper presents a hybrid offline reinforcement learning and damping control approach to regulate the motion scaling ratio for different operations during offline training. This method can take user-specific characteristics into consideration and help them achieve better teleoperation performance. Comparisons are made with and without using the adaptive motion-scaling algorithm. Detailed user studies indicate that a suitable motion-scaling ratio can be obtained and adjusted online. The overall performance of the operators in terms of time cost for task completion is significantly improved, while the variance of average speed and the total distance for robot operation is reduced.

Keywords: Medical Robots and Systems, Soft Robot Applications
Abstract: Capsule endoscopes, predominantly serving diagnostic functions, provide lucid internal imagery but are devoid of surgical or therapeutic capabilities. Consequently, despite lesion detection, physicians frequently resort to traditional endoscopic or open surgical procedures for treatment, resulting in more complex, potentially risky interventions. To surmount these limitations, this study introduces a flexible capsule endoscope (FCE) design concept, specifically conceived to navigate the inherent volume constraints of capsule endoscopes whilst augmenting their therapeutic functionalities. The FCE’s distinctive flexibility originates from a conventional rotating joint design and the incision pattern in the flexible material. In vitro experiments validated the passive navigation ability of the FCE in rugged intestinal tracts. Further, the FCE demonstrates consistent reptile-like peristalsis under the influence of an external magnetic field, and possesses the capability for film expansion and disintegration under high-frequency electromagnetic stimulation. These findings illuminate a promising path toward amplifying the therapeutic capacities of capsule endoscopes without necessitating a size compromise.

Keywords: Performance Evaluation and Benchmarking
Abstract: We propose a new metric for robot state estimation based on the recently introduced text{SE}_2(3) Lie group definition. Our metric is related to prior metrics for SLAM but explicitly takes into account the linear velocity of the state estimate, improving over current pose-based trajectory analysis. This has the benefit of providing a single, quantitative metric to evaluate state estimation algorithms against, while being compatible with existing tools and libraries. Since ground truth data generally consists of pose data from motion capture systems, we also propose an approach to compute the ground truth linear velocity based on polynomial interpolation. Using Chebyshev interpolation and a pseudospectral parameterization, we can accurately estimate the ground truth linear velocity of the trajectory in an optimal fashion with best approximation error. We demonstrate how this approach performs on multiple robotic platforms where accurate state estimation is vital, and compare it to alternative approaches such as finite differences. The pseudospectral parameterization also provides a means of trajectory data compression as an additional benefit. Experimental results show our method provides a valid and accurate means of comparing state estimation systems, which is also easy to interpret and report.

Keywords: Performance Evaluation and Benchmarking, Data Sets for Robot Learning, Reinforcement Learning
Abstract: Offline reinforcement learning has emerged as a promising technology by enhancing its practicality through the use of pre-collected large datasets. Despite its practical benefits, most algorithm development research in offline reinforcement learning still relies on game tasks with synthetic datasets. To address such limitations, this paper provides autonomous driving datasets and benchmarks for offline reinforcement learning research. We provide 19 datasets, including real-world human driver's datasets, and seven popular offline reinforcement learning algorithms in three realistic driving scenarios. We also provide a unified decision-making process model that can operate effectively across different scenarios, serving as a reference framework in algorithm design. Our research lays the groundwork for further collaborations in the community to explore practical aspects of existing reinforcement learning methods. Dataset and codes can be found in https://sites.google.com/view/ad4rl.