Keywords: Data Sets for Robot Learning, Transfer Learning
Abstract: Several machine learning (ML)-based measurement systems have been proposed to estimate difficult-to-measure quantities from the values of distance sensor arrays. However, variations in sensor output characteristics (OCs) can lead to degradation in the estimation accuracy when transferring training data acquired from the original acquisition sensors to new target sensors. Moreover, acquiring training data from target sensors is time and labor intensive. We propose two methods to convert previously collected training data to reflect different OCs, enabling their repeated use. For evaluation, we use a device that estimates the relative position and orientation of vehicles based on the values of distance sensor arrays. The correction approach for the training data based on the OC data reduces the root-mean-square error (RMSE) by up to 23% compared with transferring training data. The augmentation approach transforms the training data into data that include different OCs using a mapping function constructed from a small batch of training data. Furthermore, a method for collecting a small batch of training data to achieve a higher OC conversion accuracy is demonstrated. The RMSE is reduced by up to 58% by the proposed method compared with transferring training data. The results of this study demonstrate the feasibility of the practical applications of ML-based measurement systems using distance sensor arrays, which may facilitate the development of simple and fast calibration methods.

Keywords: RGB-D Perception, Deep Learning for Visual Perception, Deep Learning Methods
Abstract: Object pose estimation using visual data is crucial for robotic interaction with the environment. Many existing instance-level methods are restricted by their requirements for 3D CAD models or multiple object views, which limits their flexibility and generalizability. Overcoming this limitation is critical to enhance the adaptability of pose estimation systems. In this work, a novel pipeline that leverages recent advances in reconstruction techniques is presented to address these challenges. To this end, Large Reconstruction Models (LRM) represent an advanced neural architecture capable of generating 3D object models from a limited set of views. Nevertheless, the resulting 3D models often lack relevant geometric and texture details due to insufficient input information. This research presents InstantPose, an innovative zero-shot instance-level pose estimation method that, building upon LRM, can determine the pose of unseen objects using as little as a single RGB-D query image. Extensive experiments demonstrate

that InstantPose achieves remarkable performance in object pose estimation on the YCB-V dataset, compared to methods conceived to rely on a geometrically perfect object's model. Furthermore, the 6D pose provided through the presented approach facilitates successful object grasping, highlighting its practical utility in robotic manipulation tasks.

Keywords: Reinforcement Learning, Embodied Cognitive Science, Machine Learning for Robot Control
Abstract: 机器人 快速掌握作技能是一项重大挑战， 受制于物理设备的寿命和安全 要求。目前，强化学习技术在 解决涉及丰富接触的动态、无结构问题 场景。然而，这些算法的收敛率通常为 由于机器人状态-动作映射的维度较高，速度较慢 空间以及广泛的初始政策搜索空间。与此同时， 大型语言模型（LLM）的进步赋予了这些模型 具有一定的逻辑推理能力，使他们能够接受 机器人初期阶段的主动目标导向行动 任务。这些模型可以隐式生成状态的特征和 揭示轨迹生成中的潜在模式。然而，复杂地说 涉及丰富接触场景的作性任务，LLM依然会失败 短暂。因此，整合了 的强大交互功能 &

Keywords: Collision Avoidance, Computational Geometry, Aerial Systems: Applications
Abstract: We propose an online iterative algorithm to optimize a convex cover to under-approximate the free space for autonomous navigation to delineate Safe Flight Corridors (SFC). The convex cover consists of a set of polytopes such that the union of the polytopes represents obstacle-free space, allowing us to find trajectories for robots that lie within the convex cover. In order to find the SFC that facilitates trajectory optimization, we iteratively find overlapping polytopes of maximum volumes that include specified waypoints initialized by a geometric or kinematic planner. Constraints at waypoints appear in two alternating stages of a joint optimization problem, which is solved by a method inspired by the Alternating Direction Method of Multipliers (ADMM) with partially distributed variables. We validate the effectiveness of our proposed algorithm using a range of parameterized environments and show its applications for two-stage motion planning.

Keywords: Optimization and Optimal Control, Motion and Path Planning, Collision Avoidance, Autonomous Embedded Robotics
Abstract: To efficiently deploy robotic systems in society, mobile robots need to autonomously and safely move through complex environments. Nonlinear model predictive control (MPC) methods provide a natural way to find a dynamically feasible trajectory through the environment without colliding with nearby obstacles. However, the limited computation power available on typical embedded robotic systems, such as quadrotors, poses a challenge to running MPC in real-time, including its most expensive tasks: constraints generation and optimization. To address this problem, we propose a novel hierarchical MPC scheme that consists of a planning and a tracking layer. The planner constructs a trajectory with a long prediction horizon at a slow rate, while the tracker ensures trajectory tracking at a relatively fast rate. We prove that the proposed framework avoids collisions and is recursively feasible. Furthermore, we demonstrate its effectiveness in simulations and lab experiments with a quadrotor that needs to reach a goal position in a complex static environment. The code is efficiently implemented on the quadrotor's embedded computer to ensure real-time feasibility. Compared to a state-of-the-art single-layer MPC formulation, this allows us to increase the planning horizon by a factor of 5, which results in significantly better performance.

Keywords: Human-Robot Collaboration, Physical Human-Robot Interaction, Reinforcement Learning
Abstract: Despite a large body of research on robot learning, it has not yet been thoroughly studied how collaborating humans and robots learn reciprocally. In such situations, both humans and robots continuously learn about each other and the task through interaction. This paper addresses the research question: “How can human-robot co-learning be facilitated in physically embodied collaborative tasks?”. First, we derived five requirements for successful human-robot co-learning from literature: shared goal, synchrony, interdependence, adaptability, and transparency. Based on these requirements, we designed a collaborative human-robot handover task and a robot Q-learning method. In an evaluation with six human participants co-learning was indeed found to emerge in the hand-over task. Particularly, for three of the human-robot dyads, our designed setup proved to facilitate co-learning in a way that met all five requirements. The task and robot learning method presented in this paper demonstrate how human-robot co-learning can be enabled in physically embodied tasks.

Keywords: Deep Learning for Visual Perception, Computer Vision for Transportation, Object Detection, Segmentation and Categorization
Abstract: Semantic segmentation is crucial for autonomous navigation in off-road environments, enabling precise classification of surroundings to identify traversable regions. However, distinctive factors inherent to off-road conditions, such as source-target domain discrepancies and sensor corruption from rough terrain, can result in distribution shifts

that alter the data differently from the trained conditions. This often

leads to inaccurate semantic label predictions and subsequent failures in navigation tasks. To address this, we propose ST-Seg, a novel framework that expands the source distribution through style expansion (SE) and texture regularization (TR). Unlike prior methods that implicitly apply generalization within a fixed source distribution, ST-Seg offers an intuitive approach for distribution shift. Specifically, SE broadens domain coverage by generating diverse realistic styles, augmenting the limited style information of the source domain. TR stabilizes local texture representation affected by style-augmented learning through a deep texture manifold. Experiments across various distribution-shifted target domains demonstrate the effectiveness of ST-Seg, with substantial improvements over existing methods. These results highlight the robustness of ST-Seg, enhancing the real-world applicability of semantic segmentation for off-road navigation.

Keywords: Aerial Systems: Mechanics and Control, Motion Control, Optimization and Optimal Control
Abstract: Accurate trajectory tracking is crucial in aerial robotics. Optimal control methods such as Nonlinear Model Predictive Control (NMPC) are able to track trajectories exploiting the full nonlinear dynamics while

respecting constraints. However, the NMPC model-based nature makes it sensitive to mismatches among nominal and real models. A common workaround to mitigate the effects of model uncertainties is to implement an inner-loop controller which robustifies the NMPC outer-loop. However, this inner-loop is usually based on purely feedback-based controllers such as PID or Incremental Nonlinear Dynamic

Inversion (INDI), which do not allow to consider any constraint (such as limited actuation) or optimization criteria. In contrast, in this work we propose an optimization-based inner-loop controller inspired by

Time Delay Control (TDC), that, thanks to a Quadratic Program (QP) formulation, is able to respect constrains and can thus preserve stability in presence of input saturation and model mismatches. Furthermore, thanks to the use of acceleration feedback, the knowledge of inertial parameters is not required by the proposed inner-loop which

therefore makes it even more robust against model uncertainties. The overall architecture is validated on a fully-actuated hexarotor under model mismatches and aggressive trajectories. The experiments clearly show that our QP-based inner-loop improves the NMPC tracking performance while preserving the stability in conditions where a non-optimal (and more classical) inner-loop controllers would fail.

Keywords: Reactive and Sensor-Based Planning, Collision Avoidance, Motion Control
Abstract: Deployment of robots in dynamic environments requires reactive trajectory generation. While optimization-based methods, such as Model Predictive Control focus on constraint verificaction, Geometric Fabrics offer a computationally efficient way to generate trajectories that include all avoidance behaviors if the environment can be represented as a set of object primitives. Obtaining such a representation from sensor data is challenging, especially in dynamic environments. In this paper, we integrate implicit environment representations, such as Signed Distance Fields and Free Space Decomposition into the framework of Geometric Fabrics. In the process, we derive how numerical gradients can be integrated into the push and pull operations in Geometric Fabrics. Our experiments reveal that both, ground robots and robotic manipulators, can be controlled using these implicit representations. Moreover, we show that, unlike the explicit representation, implicit representations can be used in the presence of dynamic obstacles without further considerations. Finally, we demonstrate our methods in the real-world, showing the applicability of our approach in practice.

Keywords: Imitation Learning, Deep Learning in Grasping and Manipulation, Learning from Demonstration
Abstract: Inferring object motion representations from observations enhances the performance of robotic manipulation tasks. This paper introduces a new paradigm for robot imitation learning that generates action sequences by reasoning about object motion from visual observations.We propose MBA, a novel module that employs two cascaded diffusion processes for object motion generation and robot action generation under object motion guidance. MBA first predicts the future pose sequence of the object based on observations, then uses this sequence as a condition to guide robot action generation. Designed as a plug-and-play component, MBA can be flexibly integrated into existing robotic manipulation policies with diffusion action heads. Extensive experiments in both simulated and real-world environments demonstrate that our approach substantially improves the performance of existing policies across a wide range of manipulation tasks. Project page: https://selen-suyue. github.io/MBApage/

Keywords: Intelligent Transportation Systems, Visual Tracking, Motion Control
Abstract: The evolution of Advanced Driver Assistance Systems (ADAS) has increased the need for robust and generalizable algorithms for multi-object tracking. Traditional statistical model-based tracking methods rely on predefined motion models and assumptions about system noise distributions. Although computationally efficient, they often lack adaptability to varying traffic scenarios and require extensive manual design and parameter tuning. To address these issues, we propose a novel 3D multi-object tracking approach for vehicles, HybridTrack, which integrates a data-driven Kalman Filter (KF) within a tracking-by-detection paradigm. In particular, it learns the transition residual and Kalman gain directly from data, which eliminates the need for manual motion and stochastic parameter modeling. Validated on the real-world KITTI dataset, HybridTrack achieves 82.72% HOTA accuracy, significantly outperforming state-of-the-art methods. We also evaluate our method under different configurations, achieving the fastest processing speed of 112 FPS. Consequently, HybridTrack eliminates the dependency on scene-specific designs while improving performance and maintaining real-time efficiency.

Keywords: Robotics and Automation in Agriculture and Forestry, Agricultural Automation
Abstract: Recent advancements in visual odometry systems have improved autonomous navigation, yet challenges persist in complex environments like forests, where dense foliage, variable lighting, and repetitive textures compromise the accuracy of feature correspondences. To address these challenges, we introduce ForestGlue. ForestGlue enhances the SuperPoint feature detector through four configurations - grayscale, RGB, RGB-D, and stereo-vision inputs - optimised for various sensing modalities. For feature matching, we employ LightGlue or SuperGlue, both of which have been retrained using synthetic forest data. ForestGlue achieves comparable pose estimation accuracy to baseline LightGlue and SuperGlue models, yet require only 512 keypoints, just 25% of the 2048 keypoints used by baseline models, to achieve an LO-RANSAC AUC score of 0.745 at a 10 degree threshold. With a 1/4 of the keypoints required, ForestGlue has the potential to reduce computational overhead whilst being effective in dynamic forest environments, making it a promising candidate for real-time deployment on resource-constrained platforms such as drones or mobile robotic platforms. By combining ForestGlue with a novel transformer based pose estimation model, we propose ForestVO that estimates relative camera poses using the 2D pixel coordinates of matched features between frames. On challenging TartanAir forest sequences, ForestVO achieves an average relative pose error (RPE) of 1.09 m and kitti_score of 2.33%, outperforming direct-based methods such as DSO in dynamic scenes, while maintaining competitive performance with TartanVO despite being a significantly lighter model trained on only 10% of the dataset. This work establishes an end-to-end deep learning pipeline tailored for visual odometry in forested environments, leveraging forest-specific training data to optimise feature correspondence and pose estimation for improved accuracy and robustness in autonomous navigation systems.

Keywords: Field Robots, Vision-Based Navigation, Deep Learning for Visual Perception
Abstract: The increasing use of robots in unstructured environments necessitates the development of effective perception and navigation strategies to enable field robots to successfully perform their tasks. In particular, it is key for such robots to understand where in their environment they can and cannot travel---a task known as traversability estimation. However, existing geometric approaches to traversability estimation may fail to capture nuanced representations of traversability, whereas vision-based approaches typically either involve manually annotating a large number of images or require robot experience. In addition, existing methods can struggle to address domain shifts as they typically do not learn during deployment. To this end, we propose a human-in-the-loop (HiL) method for traversability estimation that prompts a human for annotations as-needed. Our method uses a foundation model to enable rapid learning on new annotations and to provide accurate predictions even when trained on a small number of quickly-provided HiL annotations. We extensively validate our method in simulation and on real-world data, and demonstrate that it can provide state-of-the-art traversability prediction performance.

Keywords: Rehabilitation Robotics, Medical Robots and Systems, Human-Centered Robotics, Energy Regeneration
Abstract: Knee pain is prevalent in over 20% of the population, limiting the mobility of those affected. In turn, isokinetic dynamometers and robots have been used to facilitate rehabilitation for those still capable of ambulation. However, there are at most only a few wearable robots capable of delivering isokinetic training for bedridden patients. Here, we developed a wearable robot that provides bedside isokinetic training by utilizing a variable stiffness actuator and dynamic energy regeneration. The efficacy of this device was validated in a study involving six subjects with debilitating knee injuries. During two courses of rehabilitation over a total of three weeks, the average peak torque, average torque, and average work produced by their affected knees increased significantly by 81.0%, 101.4%, and 117.6%, respectively. Furthermore, the device’s energy regeneration features were found capable of extending its operating time to 198 days under normal usage, representing a 57.8% increase over the same device without regeneration. These results suggest potential methodologies for delivering isokinetic joint rehabilitation to bedridden patients in areas with limited infrastructure.

Keywords: Grippers and Other End-Effectors, Soft Sensors and Actuators, Hydraulic/Pneumatic Actuators
Abstract: Robotic wrists play a crucial role in enhancing the dexterity and stability of robotic end-effectors. Existing rigid robotic wrists tend to be complex and lack flexibility, while soft robotic wrists often struggle with limited load-bearing capacity and lower accuracy. Human wrists feature multi-degrees of freedom and variable stiffness, which help human hands to accomplish daily tasks. This study presents an innovative anthropomorphic soft robotic wrist, VarWrist, equipped with a fiber jamming variable stiffness module, enabling stiffness adjustment through vacuuming. VarWrist consists of three parallel bellows, utilizing a positive-negative pneumatic actuation strategy to mimic human wrist motion. In addition, the trajectory equation of the rotation center was fitted through modeling. We developed a prototype of VarWrist and assessed its performance. Results indicate that the soft wrist surpasses the motion range of human wrists, achieving flexion (81.9°), extension (78.5°), ulnar deviation (70.5°), and radial deviation (70.5°). The bending motion trajectory showed a 73% increase in similarity to human motion compared to fixed-axis rotation, with VarWrist exhibiting a significant range of variable stiffness (resting state: 206%, working state: 155%). Demonstration experiments confirm that this wrist facilitates a dexterous hand in completing grasping tasks that would be unattainable by the hand alone.

Keywords: Imitation Learning, AI-Based Methods, Data Sets for Robot Learning
Abstract: Robotics has long sought to develop robots capable of completing previously unseen long-horizon tasks. Hierarchical approaches offer a pathway for achieving this goal by executing skill combinations arranged by a task planner, with each visuomotor skill pre-trained using a specific imitation learning (IL) algorithm. However, even in simple long-horizon tasks like skill chaining, hierarchical approaches often struggle due to a problem we identify as Observation Space Shift (OSS), where the sequential execution of preceding skills causes shifts in the observation space, disrupting the performance of subsequent individually trained skill policies. To understand OSS and evaluate its impact on long-horizon tasks, we introduce BOSS (a Benchmark for Observation Space Shift). BOSS comprises three distinct challenges: "Single Predicate Shift," "Accumulated Predicate Shift," and "Skill Chaining," each designed to assess a different aspect of OSS's negative effect. We evaluated several recent popular IL algorithms on BOSS, including three Behavioral Cloning methods and the Visual Language Action model OpenVLA. Even on the simplest challenge, we observed average performance drops of 67%, 35%, 34%, and 54%, respectively, when comparing skill performance with and without OSS. Additionally, we investigate three potential solutions, including using frozen robotics-specific vision encoders, switching to 3D pointcloud-based inputs, and applying data augmentation to expand visual diversity. Our results show that none of these approaches are sufficient to resolve OSS. The project page is: https://boss-benchmark.github.io/

Keywords: Art and Entertainment Robotics, Social HRI, Robot Companions
Abstract: The rise in prevalence of AI-enabled technologies (from voice assistants to social robots) has not yet been accompanied by an analogous mastery of computer-mediated humor. Although humans often use jokes to repair interactions and navigate uncomfortable scenarios, social robots in similar roles typically fall short at reading the room and adapting behavior according to sensed social contexts and reactions. We pursued two studies to gain clearer evidence about adaptive robot joking's influence (compared to hardcoded repartee or no robot banter). The first study (N = 48, between-subjects design) examined in-person one-on-one human-robot interactions across the three conditions. The results indicated that adaptive repartee by robots tended to increase perceived warmth, competence, comfort, social closeness feelings, and humorousness, and that human behavioral responses varied significantly between conditions, with any repartee leading to significant gains over no repartee. The second study used an online video-based survey with a within-subjects design (N = 99) to examine the same conditions. This follow-up effort showed significant gains in perceived competence and anthropomorphism for any type of repartee, although this banter also made the robot more discomforting. Our work can help practitioners who are interested in applying playful banter to enhance robot charm and success.

Keywords: Localization, SLAM, Sensor Fusion
Abstract: Camera localization within LiDAR maps has gained significant attention due to its potential for accurate positioning with low-cost and lightweight sensors compared to LiDAR-based systems. However, existing methods often prioritize localization accuracy, sometimes compromising efficiency, which can limit their suitability for real-time applications. To address these issues, we propose I2D-LocX, a lightweight monocular camera localization framework with three branches, establishing pixel-level and feature-level constraints to enhance localization performance without increasing model complexity. Specifically, the main branch generates a flow map to represent pixel-point displacements. One auxiliary branch shares the same input as the main branch and employs an additional decoder to evaluate the confidence of the flow map. The other auxiliary branch leverages a zero-flow generated from the displacement-free input to guide feature matching, thereby enhancing localization robustness. Notably, both auxiliary branches share parameters with the main branch and are omitted during inference, ensuring computational efficiency. Extensive experiments on benchmark datasets, including KITTI-Odometry, Argoverse, Waymo, and nuScenes, show that I2D-LocX can achieve centimeter-level localization accuracy with about 37 millisecond inference time, greatly improving the localization performance for real-world applications.

Keywords: Reinforcement Learning, Modeling, Control, and Learning for Soft Robots
Abstract: In soft robotics, actuators using both positive and negative pressures are notable for their high payload-to-weight ratios and wide operating ranges, but they require separate power sources. A single-pump system generating dual pressures presents a promising solution, though addressing pressure fluctuations due to coupled dynamics remains a challenge. In this work, we propose a reinforcement learning (RL)-based controller capable of tracking both pressures over a wide range. To facilitate RL training, we built a simulator that models not only airflow dynamics but also the pump’s kinematics and the electromagnetic behavior of pneumatic components. Our controller employs Model-Predicted Observation (MPObs) to predict future input effects and mitigate nonlinearities, and uses a Conditioning for Action Policy Smoothness (CAPS)-based action smoothing to reduce abrupt input changes. Experimental results show that the proposed RL controller achieves root-mean-square errors (RMSEs) of 0.6935 kPa (positive) and 0.2646 kPa (negative), outperforming the Disturbance Observer (DOB)-based approach. Ablation studies confirm the synergistic effect of MPObs and CAPS, underscoring their importance in control. Furthermore, robustness tests with external loads from 0 to 20 kg demonstrate a maximum RMSE of 0.7906 kPa (positive) and 0.1186 kPa (negative), indicating strong robustness. This study verifies that our proposed RL-based controller overcomes the nonlinear challenges of pneumatic power sources and highlights its potential for future stand-alone systems in field applications.

Keywords: Marine Robotics, Deep Learning Methods, Deep Learning for Visual Perception
Abstract: This work proposes a method for learning features from a batch of 2D sonar images to predict a multi-view point-cloud for achieving a dense 3D-reconstruction. In comparison to vision-based sensors, acoustics are considered a reliable sensing modality in underwater environments. The output of sonars is a 2D image which is unable to represent the scanned scene in all three dimensions. Estimation of this missing information, known as the elevation angle, is the key to performing 3d-reconstruction from acoustic images. One of the approaches is to predict a depth-map from the 2D sonar image, and transforming it into a point-cloud. In this paper, this idea is further improved into learning features from a batch of 2D acoustic images and predicting multiple depthmaps of the scanned object which covers it from different viewpoints. For training the deep learning model, and due to the lack of datasets from real environments, data was generated synthetically. For reducing the simulation-to-real gap, a Cycle-GAN was trained on real images for transferring the realistic style into the synthetically generated images. The conducted experiments in simulation showed that the proposed method is able to perform dense 3D reconstruction. The approach was then further tested in a real environment using an underwater vehicle, which accurately 3d-reconstructed the scanned objects achieving an average chamfer distance error of 0.06 meters when compared to a laser-scanned ground-truth.

Keywords: Swarm Robotics, Multi-Robot Systems, Failure Detection and Recovery
Abstract: An active approach to fault tolerance is essential for robot swarms to achieve long-term autonomy. Previous efforts have focused on responding to spontaneous electro-mechanical faults and failures. However, many faults occur gradually over time. This work argues that the principles of predictive mainte- nance, in which potential faults are resolved before they hinder the operation of the swarm, offer a promising means of achieving long-term fault tolerance. This is a novel approach to swarm fault tolerance, which is shown to give a comparable or improved performance when tested against a reactive approach in almost all cases tested.

Keywords: SLAM, Range Sensing, Aerial Systems: Perception and Autonomy
Abstract: We propose a graph SLAM algorithm for sparse range sensing that incorporates a soft Manhattan world utilizing landmark-landmark constraints. Sparse range sensing is necessary for tiny robots that do not have the luxury of using heavy and expensive sensors. Existing SLAM methods dealing with sparse range sensing lack accuracy and accumulate drift error over time due to limited access to data points. Algorithms that cover this flaw using structural regularities, such as the Manhattan world (MW), have shortcomings when mapping real-world environments that do not coincide with the rules. We propose SoMaSLAM, a 2D graph SLAM designed for tiny drones with sparse range sensing. Our approach effectively maps sparse range data without enforcing strict structural regularities and maintains an adaptive graph. We implement the MW assumption as soft constraints, which we refer to as a soft Manhattan world. We propose novel soft landmark-landmark constraints to incorporate the soft MW into graph SLAM. Through extensive evaluation, we demonstrate that our proposed SoMaSLAM method improves localization accuracy on diverse datasets and is flexible enough to be used in the real world. We plan to release our code and dataset on our project page https://SoMaSLAM.github.io/.

Keywords: Motion and Path Planning, Collision Avoidance, Integrated Planning and Control
Abstract: This work presents a motion planning framework for robotic manipulators that computes collision-free paths directly in image space. The generated paths can then be tracked using vision-based control, eliminating the need for an explicit robot model or proprioceptive sensing. At the core of our approach is the construction of a roadmap entirely in image space. To achieve this, we explicitly define sampling, nearest-neighbor selection, and collision checking based on visual features rather than geometric models. We first collect a set of image-space samples by moving the robot within its workspace, capturing keypoints along its body at different configurations. These samples serve as nodes in the roadmap, which we construct using either learned or predefined distance metrics. At runtime, the roadmap generates collision-free paths directly in image space, removing the need for a robot model or joint encoders. We validate our approach through an experimental study in which a robotic arm follows planned paths using an adaptive vision-based control scheme to avoid obstacles. The results show that paths generated with the learned-distance roadmap achieved 100% success in control convergence, whereas the predefined image-space distance roadmap enabled faster transient responses but had a lower success rate in convergence.

Keywords: Haptics and Haptic Interfaces, Telerobotics and Teleoperation, Perception for Grasping and Manipulation
Abstract: Robotic telemanipulation—the human-guided manipulation of remote objects—plays a pivotal role in several applications, from healthcare to operations in harsh environments. While visual feedback from cameras can provide valuable information to the human operator, haptic feedback is essential for accessing specific

object properties that are difficult to be perceived by vision, such as

stiffness. For the first time, we present a participant study demonstrating that operators can perceive the stiffness of remote objects during real-world telemanipulation with a dexterous robotic hand, when haptic feedback is generated from tactile sensing fingertips. Participants were tasked with squeezing soft objects by teleoperating a

robotic hand, using two methods of haptic feedback: one based solely on the measured contact force, while the second also includes the squeezing displacement between the leader and follower devices. Our results demonstrate that operators are indeed capable of discriminating objects of different stiffness, relying on haptic feedback alone and without any visual feedback. Additionally, our findings suggest that the displacement feedback component may enhance discrimination with objects of similar stiffness.

Keywords: Legged Robots, Machine Learning for Robot Control, Optimization and Optimal Control
Abstract: This paper presents a novel approach that combines the advantages of both model-based and learning-based frameworks to achieve robust locomotion. The residual modules are integrated with each corresponding part of the model-based framework, a footstep planner and dynamic model designed using heuristics, to complement performance degradation caused by a model mismatch. By utilizing a modular structure and selecting the appropriate learning-based method for each residual module, our framework demonstrates improved control performance in environments with high uncertainty, while also achieving higher learning efficiency compared to baseline methods. Moreover, we observed that our proposed methodology not only enhances control performance but also provides additional benefits, such as making nominal controllers more robust to parameter tuning. To investigate the feasibility of our framework, we demonstrated residual modules combined with model predictive control in a real quadrupedal robot. Despite uncertainties beyond the simulation, the robot successfully maintains balance and tracks the commanded velocity.

Keywords: Mechanism Design, Tendon/Wire Mechanism, Soft Robot Materials and Design
Abstract: Thoracostomy involves draining fluid from the pleural cavity using chest tubes. This medical intervention is currently performed manually by inserting a hollow flexible tube, risking damage to vital organs, including the lungs, diaphragm, spleen, and mediastinum, due to the lack of control over the tube’s path inside the patient’s body. Inspired by snake-like structures, continuum robots are particularly well-suited to address the challenges encountered during thoracostomy. Taking advantage of their slender shape, they can nest inside the tubes and guide them from within without requiring further incision. However, available continuum robots are not suitable for this application due to geometrical and payload requirements. In this paper, a novel design is presented, leveraging a multi-backbone structure with asymmetrical rolling joints to enhance payload capacity and dexterity while maintaining the slender shape of the robot. A static modeling approach is proposed to estimate the configuration of the robot given the force applied to the robot, including the effects of friction and gravity often neglected for these robots. Two prototypes were 3D-printed, allowing for after-use disposal due to their cost-effectiveness, thereby preventing cross-contamination. Stiffness and position error were evaluated for the prototypes, demonstrating a modeling accuracy of 2.25%.

Keywords: Task and Motion Planning, Industrial Robots, Robotics and Automation in Construction
Abstract: The heat transfer tubes of the steam generator are critical components of the nuclear power system and require regular inspection to ensure safety. The SG-Climbot, a quadruped heat transfer tube inspection robot, is equipped with a guiding device capable of simultaneously aligning with and inspecting two heat transfer tubes. Furthermore, The guiding device must execute hundreds of pose configuration transformations to complete a localized coverage inspection, thereby presenting challenges to the robot’s efficient autonomous planning. This letter presents a planning framework for the SG-Climbot’s localized coverage inspection task.The framework consists of four planning levels: pair planning, position and orientation planning for the guiding device, inspection sequence planning, and time-optimal trajectory planning. A maximum matching algorithm suitable for robotic arms equipped with dual execution devices to perform tasks has been proposed, achieving the optimal pairing of heat transfer tubes and reducing inspection time by over 48 minutes (18.32% improvement). In addition, we analyze the impact of various Traveling Salesman Problem (TSP) solving algorithms on sequence planning issues that require reaching numerous nodes within short operation times, reducing the arm operating time by 33.20 s (6.99% improvement). Finally, the effectiveness of the proposed planning algorithm was validated through simulations and experiments.

Keywords: Manipulation Planning, Data Sets for Robot Learning, Deep Learning Methods
Abstract: In this paper, we present the tidiness score-guided Monte Carlo tree search (TSMCTS), a novel framework designed to address the tabletop tidying up problem using only an RGB-D camera. We address two major problems for tabletop tidying up problem: (1) the lack of public datasets and benchmarks, and (2) the difficulty of specifying the goal configuration of unseen objects. We address the former by presenting the tabletop tidying up (TTU) dataset, a structured dataset collected in simulation. Using this dataset, we train a vision-based discriminator capable of predicting the tidiness score. This discriminator can consistently evaluate the degree of tidiness across unseen configurations, including real-world scenes. Addressing the second problem, we employ Monte Carlo tree search (MCTS) to find tidying trajectories without specifying explicit goals. Instead of providing specific goals, we demonstrate that our MCTS-based planner can find diverse tidied configurations using the tidiness score as a guidance. Consequently, we propose TSMCTS, which integrates a tidiness discriminator with an MCTS-based tidying planner to find optimal tidied arrangements. TSMCTS has successfully demonstrated its capability across various environments, including coffee tables, dining tables, office desks, and bathrooms. The TTU dataset and code will be publicly available.

Keywords: Aerial Systems: Mechanics and Control, Legged Robots, Dynamics, Hybrid Locomotion
Abstract: Mobile robots have revolutionized various fields, offering solutions for manipulation, environmental monitoring, and exploration. However, payload capacity remains a limitation. This paper presents a novel thrust-based robotic hopper capable of carrying payloads up to 9 times its own weight while maintaining agile mobility over less structured terrain. The 220 gram robot carries up to 2 kg while hopping—--a capability that bridges the gap between high-payload ground robots and agile aerial platforms. Key advancements that enable this high-payload capacity include the integration of bidirectional thrusters, allowing for both upward and downward thrust generation to enhance energy management while hopping. Additionally, we present a refined model of dynamics that accounts for heavy payload conditions, particularly for large jumps. To address the increased computational demands, we employ a neural network compression technique, ensuring real-time onboard control. The robot's capabilities are demonstrated through a series of experiments, including leaping over a high obstacle, executing sharp turns with large steps, as well as performing simple autonomous navigation while carrying a 730 g LiDAR payload. This showcases the robot's potential for applications such as mobile sensing and mapping in challenging environments.

Keywords: Representation Learning, Learning from Demonstration
Abstract: Real-time motion generation -- which is essential for achieving reactive and adaptive behavior -- under kinodynamic constraints for high-dimensional systems is a crucial yet challenging problem. We address this with a two-step approach: offline learning of a lower-dimensional trajectory manifold of task‑relevant, constraint‑satisfying trajectories, followed by rapid online search within this manifold. Extending the discrete‑time Motion Manifold Primitives (MMP) framework, we propose Differentiable Motion Manifold Primitives (DMMP), a novel neural network architecture that encodes and generates continuous‑time, differentiable trajectories, trained using data collected offline through trajectory optimizations, with a strategy that ensures constraint satisfaction -- absent in existing methods. Experiments on dynamic throwing with a 7‑DoF robot arm demonstrate that DMMP outperforms prior methods in planning speed, task success, and constraint satisfaction.

Keywords: Object Detection, Segmentation and Categorization, Computer Vision for Automation, AI-Based Methods
Abstract: Monocular 3D object detection has gained attention for its cost-efficiency and simpler setup compared to multi-sensor systems. In this task, accurate depth estimation is crucial for precise object localization, yet extracting sufficient depth cues from a single image remains inherently challenging. Moreover, when occlusions occur, structural cues become limited, making precise object localization even more difficult. To address these problems, we propose MonoKey, a keypoint-based monocular 3D object detection method robust to occlusion. MonoKey leverages 2D keypoints due to their suitability for recovering occluded regions. The occlusion-robust 2D keypoint detection approach estimates keypoints and reconstructs occluded ones by using prior information. The frequency-based global-local depth predictor estimates 3D cues using fast Fourier convolution to incorporate both global and local context. These 3D cues and keypoints are then fused in a 3D detection decoder. Additionally, relational graph refinement adjusts initial bounding boxes for improved localization. Experimental results indicate that MonoKey outperforms the existing monocular 3D object detection methods. The source code is available at https://anonymous.4open.science/r/MonoKey-B72B.

Keywords: Motion and Path Planning, Task and Motion Planning, Simulation and Animation
Abstract: In recent decades, mobile robot motion planning has seen significant advancements. Both search-based and sampling-based methods have demonstrated capabilities to find feasible solutions in complex scenarios. Mainstream path planning algorithms divide the map into occupied and free spaces, considering only planar movement and ignoring

the ability of mobile robots to traverse obstacles in the z-direction. Additionally, paths generated often have numerous bends, requiring additional smoothing post-processing. In this work, a fast, and direct motion planning method based on continuous curvature integration that takes into account the robot's obstacle-crossing ability under different parameter settings is proposed. This method generates smooth paths directly with pseudo-constant velocity and limited curvature, and

performs curvature-based speed planning in complex 2.5-D terrain-based environment (take into account the ups and downs of the terrain), eliminating the subsequent path smoothing process and enabling the robot to track the path generated directly. The proposed method is also

compared with some existing approaches in terms of solution time, path length, memory usage and smoothness under multiple scenarios. The proposed method is vastly superior to the average performance of state-of-the-art (SOTA) methods, especially in terms of the self-defined S_2 smoothness (mean angle of steering). Furthermore, simulations and experiments are conducted on our self-designed wheel-legged robot with 2.5-D traversability. These results demonstrate

the effectiveness and superiority of the proposed approach in several representative environments. The implementation of this work is available at https://github.com/SkelonChan/GPCC_curvature_planning.

Keywords: Multi-Robot Systems, Sensor Networks, Networked Robots
Abstract: This paper presents a novel approach to range-based distributed cooperative localization (DCL) for robot swarms in GPS-denied environments, relying solely on inter-robot range measurements, specifically addressing the limitations of current methods in noisy and sparse settings where the geometric non-rigidity of the sensing graph creates flipping (suboptimal) effects in the localization outcomes. We propose a robust multilayered localization framework (DCL-Sparse) that utilizes distributed 1-hop shadow edges (S1-Edge) to address the non-rigidity problem and improve localization convergence in sparse and noisy sensing graphs. Our approach leverages the advantages of distributed localization methods, enhancing scalability and adaptability in large robot networks. We establish theoretical conditions for the new S1-Edge that ensure solutions exist even in the presence of noise, thereby validating the effectiveness of the new shadow edge localization. Extensive simulation and real-world experiments confirm the superior performance of our method compared to state-of-the-art techniques, resulting in a reduction of up to 93% in the localization error in DCL. These experiments demonstrate substantial improvements in localization accuracy and robustness to sparse graphs. DCL-Sparse increases the localizability of large multi-robot and sensor networks, offering a powerful tool for high-performance and reliable operations in challenging large-scale environments.

Keywords: Planning under Uncertainty, Autonomous Agents
Abstract: Online planning under uncertainty remains a critical challenge in robotics and autonomous systems. While tree search techniques are commonly employed to construct partial future trajectories within computational constraints, most existing methods discard information from previous planning sessions considering continuous spaces. This study presents a novel, computationally efficient approach that leverages historical planning data in current decision-making processes. We provide theoretical foundations for our information reuse strategy and introduce an algorithm based on Monte Carlo Tree Search (MCTS) that implements this approach. Experimental results demonstrate that our method significantly reduces computation time while maintaining high performance levels. Our findings suggest that integrating historical planning information can substantially improve the efficiency of online decisionmaking in uncertain environments, paving the way for more responsive and adaptive autonomous systems.

Keywords: Wearable Robots, Mechanism Design, Optimization and Optimal Control, Rehabilitation Robotics
Abstract: Supernumerary robotic limbs (SRL) offer substantial potential in both the rehabilitation of hemiplegic patients and the enhancement of functional capabilities for healthy individuals. Designing a general-purpose SRL device is inherently challenging, particularly when

developing a unified theoretical framework that meets the diverse functional requirements of both upper and lower limbs. In this paper, we propose a MOO design theory that integrates grasping workspace similarity, walking workspace similarity, bracing force for STS movements, and overall mass and inertia. To facilitate rapid and stable

convergence of the model to high-dimensional irregular Pareto fronts, we introduce a multi-subpopulation correction firefly algorithm. The optimized solution is utilized to redesign the prototype for experimentation to meet specified requirements. Six healthy participants and two hemiplegia patients participated in real experiments. Compared to the pre-optimization results, the average grasp success rate improved by 7.2%, while muscle activity during walking and STS tasks decreased by an average of 12.7% and 25.1%, respectively, following the optimization.

Keywords: Human Factors and Human-in-the-Loop, Acceptability and Trust, Human-Robot Collaboration
Abstract: Industry 5.0 focuses on human-centric collaboration between humans and robots, prioritizing safety, comfort, and trust. This study introduces a data-driven framework to assess trust using behavioral indicators. The framework employs a Preference-Based Optimization algorithm to generate trust-enhancing trajectories based on operator feedback. This feedback serves as ground truth for training machine learning models to predict trust levels from behavioral indicators. The framework was tested in a chemical industry scenario where a robot assisted a human operator in mixing chemicals. Machine learning models classified trust with over 80% accuracy, with the Voting Classifier achieving 84.07% accuracy and an AUC-ROC score of 0.90. These findings underscore the effectiveness of data-driven methods in assessing trust within human-robot collaboration, emphasizing the valuable role behavioral indicators play in predicting the dynamics of human trust.

Keywords: Soft Robot Applications, Hydraulic/Pneumatic Actuators, Human-Robot Collaboration
Abstract: This paper introduces a high-response artificial muscle actuator using dimethyl ether combustion (HADEC), which is a novel method to enhance the responsiveness and force output of pneumatic actuators. The HADEC system integrates a McKibben-type artificial muscle filled with a

combustible mixture of dimethyl ether (DME) and air, and it is ignited to generate rapid fluid pressure through combustion. This approach achieves force, displacement, and response speeds comparable to those of biological muscles while maintaining the simplicity and low-cost structure of McKibben-type actuators. The system provides instantaneous

force generation without the need for complex mechanisms such as latches or brakes. DME, which is an environment- friendly fuel, ensures

minimal emissions. Experimental results validate the effectiveness of HADEC in improving responsiveness, and the findings suggest superior force generation, faster response times, and high-frequency operability

compared to that of conventional pneumatic actuators. Further, the paper discusses the potential for repeated actuation and highlights the

benefits of HADEC in various robotic applications that require rapid and significant force.

Keywords: Soft Robot Materials and Design, Hydraulic/Pneumatic Actuators
Abstract: Soft pneumatic actuators (SPA) made from elastomeric materials can provide large strain and large force. The behavior of locally strain-restricted hyperelastic materials under inflation has been investigated thoroughly for shape reconfiguration, but requires further investigation for trajectories involving external force. In this work we model force-pressure-height relationships for a concentrically strain-limited class of soft pneumatic actuators and demonstrate the use of this model to design SPA response for object lifting. We predict relationships under different loadings by solving energy minimization equations and verify this theory by using an automated test rig to collect rich data for n=22 Ecoflex 00-30 membranes. We collect data using an active learning pipeline to efficiently model the design space. We show that this learned model outperforms the theory-based model and a naive regression. We use our model to optimize membrane design for different lift tasks and compare this performance to other designs. These contributions represent a step towards understanding the natural response for this class of actuator and embodying intelligent lifts in a single-pressure input actuator system.

Keywords: Optimization and Optimal Control, Constrained Motion Planning
Abstract: We present FilterDDP, a differential dynamic programming algorithm for solving discrete-time, optimal control problems (OCPs) with nonlinear equality constraints. Unlike prior methods based on merit functions or the augmented Lagrangian class of algorithms, FilterDDP uses a step filter in conjunction with a line search to handle equality constraints. We identify two important design choices for the step filter criteria which lead to robust numerical performance: 1) we use the Lagrangian instead of the cost in the step acceptance criterion and, 2) in the backward pass, we perturb the value function Hessian. Both choices are rigorously justified, for 2) in particular by a formal proof of local quadratic convergence. In addition to providing a primal-dual interior point extension for handling OCPs with both equality and inequality constraints, we validate FilterDDP on three contact implicit trajectory optimisation problems which arise in robotics.

Keywords: Multifingered Hands, Force and Tactile Sensing
Abstract: Tactile sensing allows robots to gather detailed geometric information about objects through physical interaction, complementing vision-based approaches. However, efficiently acquiring useful tactile data remains challenging due to the time-consuming nature of physical contact and the need to strategically choose contact locations that maximize information gain while minimizing physical interactions. This paper studies how different contact modes affect object shape reconstruction using a tactile-enabled dexterous gripper. We compare three contact interaction modes: grasp-releasing, sliding induced by finger-grazing, and palm-rolling. These contact modes are combined with an information-theoretic exploration framework that guides subsequent sampling locations using a shape completion model. Our results show that the improved tactile sensing efficiency of finger-grazing and palm-rolling translates into faster convergence in shape reconstruction, requiring 34% fewer physical interactions while improving reconstruction accuracy by 55%. We validate our approach using a UR5e robot arm equipped with an Inspire-Robots Dexterous Hand, showing robust performance across primitive object geometries.

Keywords: Deep Learning for Visual Perception, Computer Vision for Medical Robotics, Computer Vision for Automation
Abstract: Stereo disparity estimation is crucial for obtaining depth information in robot-assisted minimally invasive surgery (RAMIS). While current deep learning methods have made significant advancements, challenges remain in achieving an optimal balance between accuracy, robustness, and inference speed. To address these challenges, we propose the StereoMamba architecture, which is specifically designed for stereo disparity estimation in RAMIS. Our approach is based on a novel Feature Extraction Mamba (FE-Mamba) module, which enhances long-range spatial dependencies both within and across stereo images. To effectively integrate multi-scale features from FE-Mamba, we then introduce a novel Multidimensional Feature Fusion (MFF) module. Experiments against the state-of-the-art on the ex-vivo SCARED benchmark demonstrate that StereoMamba achieves superior performance on EPE of 2.64 px and depth MAE of 2.55 mm, the second-best performance on Bad2 of 41.49% and Bad3 of 26.99%, while maintaining an inference speed of 21.28 FPS for a pair of high-resolution images (1280*1024), striking the optimum balance between accuracy, robustness, and efficiency. Furthermore, by comparing synthesized right images, generated from warping left images using the generated disparity maps, with the actual right image, StereoMamba achieves the best average SSIM (0.8970) and PSNR (16.0761), exhibiting strong zero-shot generalization on the in-vivo RIS2017 and StereoMIS datasets.

Keywords: Constrained Motion Planning, Motion and Path Planning, Task and Motion Planning
Abstract: Motion profile optimization is a powerful optimization technique that allows to reduce the energy consumption of robotic systems by changing the temporal profile of the joint position setpoints. However, despite the extensive exploration of these techniques for robotic systems following constrained paths, many existing methodologies rely on complex optimization processes or a larger number of design parameters. This paper introduces a novel approach that leverages the Chebyshev basis to optimize the motion along a fixed geometric path, thereby achieving a measured torque difference of -15% while requiring only a limited number of design parameters. By employing the Chebyshev basis, the formulation leads to a smooth objective function and enables the definition of linear inequality constraints that accurately enclose the feasible design space. This unique combination of features not only simplifies the optimization problem but also enhances the probability of locating the global optimum, particularly illustrated in the two-dimensional case. The methodology is established in a generic and model-independent manner, setting a promising direction for future research in motion profile optimization for constrained-path robotic systems.

Keywords: Aerial Systems: Applications, Task and Motion Planning, Aerial Systems: Perception and Autonomy
Abstract: Terrestrial-aerial bimodal vehicles, which integrate the high mobility of aerial robots with the long endurance of ground robots, offer significant potential for autonomous exploration. Given the inherent energy and time constraints in practical exploration tasks, we present a hierarchical framework for the bimodal vehicle to utilize its flexible locomotion modalities for exploration. Beginning with extracting environmental information to identify informative regions, we generate a set of potential bimodal viewpoints. To adaptively manage energy and time constraints, we introduce an extended Monte Carlo Tree Search approach that strategically optimizes both modality selection and viewpoint sequencing. Combined with an improved bimodal vehicle motion planner, we present a complete bimodal energy- and time-aware exploration system. Extensive simulations and deployment on a customized real-world platform demonstrate the effectiveness of our system.

Keywords: Vision-Based Navigation, Semantic Scene Understanding, Wheeled Robots
Abstract: Object-Goal Navigation in dynamic environments remains challenging because many existing approaches rely primarily on reactive mapping and lack the ability to retain historical experience or establish structured memory associations. To address this limitation, we introduce MemClaw-RAG, an embodied multimodal framework. MemClaw-RAG includes three main components: (1) a Memory Graph Retrieval (MGR) module that leverages multimodal knowledge graphs to support semantic association and target retrieval; (2) a SelfClaw cognitive module that manages skill scheduling and task execution through memory-aware reasoning; and (3) a Hybrid Adaptive Locomotion Policy (HALP) based on deep reinforcement learning that enables efficient locomotion for wheeled-legged robots across different terrain conditions.On Habitat benchmarks, MemClaw-RAG achieves a Success Rate (SR) of 0.81 and a Success-weighted Path Length (SPL) of 0.51 on the Gibson and HM3D datasets. In the more challenging multi-layer environments of MP3D, the proposed method achieves an SR of 0.76 and an SPL of 0.48, outperforming several representative memory-based and end-to-end navigation approaches. Real-world deployment on a Unitree wheeled-legged robot further demonstrates the practicality of the system, achieving an average per-step inference latency of 55 ms on a Jetson Orin platform while maintaining stable navigation behavior in dynamic indoor environments.

Keywords: Vision-Based Navigation, Performance Evaluation and Benchmarking

Keywords: Space Robotics and Automation, Planning, Scheduling and Coordination
Abstract: The Dynamic Targeting (DT) mission concept is an approach to satellite observation in which a lookahead sensor gathers information about the upcoming environment and uses this information to intelligently plan observations. Previous work has shown that DT has the potential to increase the science return across several applications. However, DT mission concepts must address challenges such as the limited spatial extent of onboard lookahead data and instrument mobility, data throughput, and onboard computation constraints. In this work, we show how the performance of DT systems can be improved by using supplementary data streamed from geostationary satellites that provide lookahead information up to 35 minutes ahead of time rather than the 1 minute latency from an onboard lookahead sensor. While there is a greater volume of geostationary data, the search space for observation planning explodes exponentially with the size of the horizon. To address this, we introduce a hierarchical planning approach in which the geostationary data is used to plan a long-term observation blueprint in polynomial time, then the onboard lookahead data is leveraged to refine that plan over short-term horizons. We compare the performance of our approach to that of traditional DT planners relying on onboard lookahead data across four different problem instances: three cloud avoidance variations and a storm hunting scenario. We show that our hierarchical planner outperforms the traditional DT planners by up to 41% and examine the features of the scenarios that affect the performance of our approach. We demonstrate that incorporating geostationary satellite data is most effective for dynamic problem instances in which the targets of interest are sparsely distributed throughout the overflight.

Keywords: Autonomous Vehicle Navigation, Computer Vision for Automation, Vision-Based Navigation
Abstract: Accurately predicting how agents move in dynamic scenes is essential for safe autonomous driving. State-of-the-art motion forecasting models rely on datasets with manually annotated or post-processed trajectories. However, building these datasets is costly, generally manual, hard to scale, and lacks reproducibility. They also introduce domain gaps that limit generalization across environments. We introduce PPT (Pretraining with Pseudo-labeled Trajectories), a simple and scalable pretraining framework that uses unprocessed and diverse trajectories automatically generated from off-the-shelf 3D detectors and tracking. Unlike data annotation pipelines aiming for clean, single-label annotations, PPT is a pretraining framework embracing off-the-shelf trajectories as useful signals for Learning robust representations. With optional finetuning on a small amount of labeled data, models pretrained with PPT achieve strong performance across standard benchmarks, particularly in low-data regimes, and in cross-domain, end-to- end, and multi-class settings. PPT is easy to implement and improves generalization in motion forecasting.

Keywords: Surveillance Robotic Systems, Deep Learning for Visual Perception, Data Sets for Robotic Vision
Abstract: Accurate 3D pose estimation of drones is essential for security and surveillance systems. However, existing methods often rely on prior drone information such as physical sizes or 3D meshes. At the same time, current datasets are small-scale, limited to single models, and collected under constrained environments, which makes reliable validation of generalization difficult. We present DroneKey++, a prior-free framework that jointly performs keypoint detection, drone classification, and 3D pose estimation. The framework employs a keypoint encoder for simultaneous keypoint detection and classification, and a pose decoder that estimates 3D pose using ray-based geometric reasoning and class embeddings. To address dataset limitations, we construct 6DroneSyn, a large-scale synthetic benchmark with over 50K images covering 7 drone models and 88 outdoor backgrounds, generated using 360-degree panoramic synthesis. Experiments show that DroneKey++ achieves MAE 17.34° and MedAE 17.1° for rotation, MAE 0.135 m and MedAE 0.242 m for translation, with inference speeds of 19.25 FPS (CPU) and 414.07 FPS (GPU), demonstrating both strong generalization across drone models and suitability for real-time applications. The dataset is available at [link].

Keywords: Physical Human-Robot Interaction, Human-Robot Collaboration, Aerial Systems: Applications
Abstract: This paper presents an Adaptive Gain Nonlinear Observer (AGNO) for estimating the external interaction wrench (forces and torques) in human-UAV physical interaction for assistive payload transportation. The proposed AGNO uses the full nonlinear dynamic model to achieve an accurate and robust wrench estimation without relying on dedicated force-torque sensors. A key feature of this approach is the explicit consideration of the non-constant inertia matrix, which is essential for aerial systems with asymmetric mass distribution or shifting payloads. A comprehensive dynamic model of a cooperative transportation system composed of two quadrotors and a shared payload is derived, and the stability of the observer is rigorously established using Lyapunov-based analysis. Simulation results validate the effectiveness of the proposed observer in enabling intuitive and safe human-UAV interaction. Comparative evaluations demonstrate that the proposed AGNO outperforms an Extended Kalman Filter (EKF) in terms of estimation root mean square errors (RMSE), particularly for torque estimation under nonlinear interaction conditions. This approach reduces system weight and cost by eliminating additional sensing hardware, enhancing practical feasibility.

Keywords: Motion Control, Reinforcement Learning
Abstract: Drifting, characterized by controlled vehicle motion at high sideslip angles, is crucial for safely handling emergency scenarios at the friction limits. While recent reinforcement learning approaches show promise for drifting control, they struggle with the significant simulation-to-reality gap, as policies that perform well in simulation often fail when transferred to physical systems. In this paper, we present a reinforcement learning framework with GPU-accelerated parallel simulation and systematic domain randomization that effectively bridges the gap. The proposed approach is validated on both simulation and a custom-designed and open-sourced 1/10 scale Individual Wheel Drive (IWD) RC car platform featuring independent wheel speed control. Experiments across various scenarios from steady-state circular drifting to direction transitions and variable-curvature path following demonstrate that our approach achieves precise trajectory tracking while maintaining controlled sideslip angles throughout complex maneuvers in both simulated and real-world environments.

Keywords: Deep Learning for Visual Perception, Visual Servoing, Imitation Learning
Abstract: Effective robotic manipulation relies on a precise understanding of 3D scene geometry, and one of the most straightforward ways to acquire such geometry is through multi-view observations. Motivated by this, we present GP3—a 3D geometry-aware robotic manipulation policy that leverages multi-view input. GP3 employs a spatial encoder to infer dense spatial features from RGB observations, which enable the estimation of depth and camera parameters, leading to a compact yet expressive 3D scene representation tailored for manipulation. This representation is fused with language instructions and translated into continuous actions via a lightweight policy head. We further introduce G-FiLM, which applies language-conditioned FiLM only to cross-view global attention. Comprehensive experiments demonstrate that GP3 consistently outperforms state-of-the-art methods on simulated benchmarks. Furthermore, GP3 transfers effectively to realworld robots in depth-challenging scenes with only minimal fine-tuning. These results highlight GP3 as a practical, sensoragnostic solution for geometry-aware robotic manipulation.

Keywords: Vision-Based Navigation, AI-Based Methods, AI-Enabled Robotics

Keywords: Machine Learning for Robot Control, Simulation and Animation, Optimization and Optimal Control
Abstract: Generative control policies have recently unlocked major progress in robotics. These methods produce action sequences via diffusion or flow matching, with training data provided by demonstrations. But existing methods come with two key limitations: they require expert demonstrations, which can be difficult or costly to obtain, and they are limited to relatively slow, quasi-static tasks. In this paper, we leverage a tight connection between sampling-based predictive control and generative modeling to address these issues. In particular, we introduce generative predictive control, a supervised learning framework for tasks with fast dynamics that are easy to simulate but difficult to demonstrate. We show how trained flow-matching policies can be warm-started at inference time, maintaining temporal consistency and enabling high-frequency feedback. We believe that generative predictive control offers a complementary approach to existing behavior cloning methods, and hope that it will pave the way toward generalist policies that extend beyond quasi-static demonstration-oriented tasks.

Keywords: Telerobotics and Teleoperation, Human-Robot Collaboration, Dexterous Manipulation
Abstract: Collecting demonstrations through human teleoperation is an effective approach for learning complex manipulation skills. However, challenges such as morphology gaps, control latency, and limited feedback make high-quality data collection costly and inefficient. In this paper, we introduce Neural Teleoperation, a shared-autonomy system that integrates human guidance with a robust grasping policy using a learning-based policy switcher. This hybrid framework allows users to focus on high-level planning while delegating fine-grained control to an autonomous policy when needed. Our system supports both immersive VR devices and lightweight 6-DoF controllers, making dexterous hand teleoperation more accessible. Real-world experiments across six manipulation tasks show that Neural Teleop increases success rates and reduces demonstration collection time compared to state-of-the-art baselines.

Keywords: Mechanism Design, Actuation and Joint Mechanisms, Product Design, Development and Prototyping
Abstract: Robotic hands, prosthetics, and hand exoskeletons struggle with replicating the natural dexterity of human hands: the mechanical intelligence of our muscles can be hardly replicated with rigid actuators, while soft mechanisms compromise precision. Underactuated hand mechanisms represent a trade-off between these extremes. However, single-motor solutions, while robust and compact, generally actuate a maximum of four fingers or present critical differences in force transmission between the fingers. Here, we propose a design for a single-input, five-output differential gearbox that delivers balanced transmission thanks to a unique asymmetrical layout. This feature enables adaptive grasp control through mechanical intelligence only, providing the user with a reliable, safe, and lightweight solution for tendon-driven hand mechanisms. A preliminary 3D-printed prototype is presented to demonstrate the concept.

Keywords: AI-Enabled Robotics, Imitation Learning, Engineering for Robotic Systems
Abstract: This paper presents RoboMatch, a novel unified teleoperation platform for mobile manipulation with an auto-matching network architecture, designed to tackle long-horizon tasks in dynamic environments. Our system enhances teleoperation performance, data collection efficiency, task accuracy, and operational stability. The core of RoboMatch is a cockpit-style control interface that enables synchronous operation of the mobile base and dual arms, significantly improving control precision and data collection. Moreover, we introduce the Proprioceptive-Visual Enhanced Diffusion Policy (PVE-DP), which leverages Discrete Wavelet Transform (DWT) for multi-scale visual feature extraction and integrates high-precision IMUs at the end-effector to enrich proprioceptive feedback, substantially boosting fine manipulation performance. Furthermore, we propose an Auto-Matching Network (AMN) architecture that decomposes long-horizon tasks into logical sequences and dynamically assigns lightweight pre-trained models for distributed inference. Experimental results demonstrate that our approach improves data collection efficiency by over 20%, increases task success rates by 20–30% with PVE-DP, and enhances long-horizon inference performance by approximately 40% with AMN, offering a robust solution for complex manipulation tasks. Project website: https://robomatch.github.io

Keywords: Formal Methods in Robotics and Automation, Learning and Adaptive Systems, Probability and Statistical Methods, Active Inference
Abstract: When a robot autonomously performs a complex task, it frequently must balance competing objectives while maintaining safety. This becomes more difficult in uncertain environments with stochastic outcomes. Enhancing transparency in the robot’s behavior and aligning with user preferences are also crucial. This paper introduces a novel framework for multi-objective reinforcement learning that ensures safe task execution, optimizes trade-offs between objectives, and adheres to user preferences. The framework has two main layers: a multi objective task planner and a high-level selector. The planning layer generates a set of optimal trade-off plans that guarantee satisfaction of a temporal logic task. The selector uses active inference to decide which generated plan best complies with user preferences and aids learning. Operating iteratively, the framework updates a parameterized learning model based on collected data. Case studies and benchmarks on both manipulation and mobile robots show that our framework outperforms other methods and (i) learns multiple optimal trade-offs, (ii) adheres to a user preference, and (iii) allows the user to adjust the balance between (i) and (ii).

Keywords: Aerial Systems: Perception and Autonomy, Intelligent Transportation Systems, Aerial Systems: Applications
Abstract: Accurate trajectory prediction can improve General Aviation safety in non-towered terminal airspace, where high traffic density increases accident risk. We present ASCENT, a lightweight transformer-based model for multimodal 3D aircraft trajectory forecasting, which integrates domain-aware 3D coordinate normalization and parameterized predictions. ASCENT employs a transformer-based motion encoder and a query-based decoder, enabling the generation of diverse maneuver hypotheses with low latency. Experiments on the TrajAir and TartanAviation datasets demonstrate that our model outperforms prior baselines, as the encoder effectively captures motion dynamics and the decoder aligns with structured aircraft traffic patterns. Furthermore, ablation studies confirm the contributions of the decoder design, coordinate-frame modeling, and parameterized outputs. These results establish ASCENT as an effective approach for real-time aircraft trajectory prediction in non-towered terminal airspace.

Keywords: Field Robots, Aerial Systems: Applications
Abstract: This paper presents a reinforcement learning-based quadrotor navigation method that leverages efficient differentiable simulation, novel loss functions, and privileged information to navigate around large obstacles. Prior learning-based methods perform well in scenes that exhibit narrow obstacles, but struggle when the goal location is blocked by large walls or terrain. In contrast, the proposed method utilizes time-of-arrival (ToA) maps as privileged information and a yaw alignment loss to guide the robot around large obstacles. The policy is evaluated in photo-realistic simulation environments containing large obstacles, sharp corners, and dead-ends. Our approach achieves an 86% success rate and outperforms baseline strategies by 34%. We deploy the policy onboard a custom quadrotor in outdoor cluttered environments both during the day and night. The policy is validated across 20 flights, covering 589 meters without collisions at speeds up to 4 m/s.

Keywords: Motion and Path Planning, Integrated Planning and Learning, Deep Learning Methods
Abstract: Diffusion-based planners have achieved generalization comparable to classical planners by leveraging inference-time optimization through guidance. However, their limited ability to capture environmental variations often constrains their responsiveness in unseen settings. In addition, the diversity-consistency trade-off inherent in guidance has remained unresolved. In this work, we propose Prior-Constrained Explorative Guidance (PCEG), a novel approach that gathers environmental information through local exploration and prevents guided samples from converging prematurely to similar solutions by leveraging a trajectory prior. The collected information is included in the guidance via stochastic gradient estimation, while a succinct parameter scheduling strategy enables latent optimization driven by environmental signals without significant computational overhead. Furthermore, during the modal-seeking stages of the reverse diffusion process, we employ a Gaussian Process (GP) to enforce dynamics-informed priors, effectively constraining the exploration region of each sample and thereby enhancing solution diversity. Across diverse benchmarks including 7-degree-of-freedom (7-DoF) robot-arm manipulation, PCEG substantially improves success rate by up to 30 percentage points compared to competitive diffusion planners without compromising trajectory quality, even in scenarios involving unseen obstacles. Real-world experiments further validate these findings, showcasing the generation of smooth, collision-free trajectories in novel environments. The project page is available at https://rml-unist.github.io/PCEG/.

Keywords: Learning from Demonstration, Imitation Learning
Abstract: This paper introduces DynaFlow, a novel framework that embeds a differentiable simulator directly into a flow matching model. By generating trajectories in the action space and mapping them to dynamically feasible state trajectories via the simulator, DynaFlow ensures all outputs are physically consistent by construction. This end-to-end differentiable architecture enables training on state-only demonstrations, allowing the model to simultaneously generate physically consistent state trajectories while inferring the underlying action sequences required to produce them. We demonstrate the effectiveness of our approach through quantitative evaluations and showcase its real-world applicability by deploying the generated actions onto a physical Go1 quadruped robot. The robot successfully reproduces diverse gait present in the dataset, executes long-horizon motions in open-loop control and translates infeasible kinematic demonstrations into dynamically executable, stylistic behaviors. These hardware experiments validate that DynaFlow produces deployable, highly effective motions on real-world hardware from state-only demonstrations, effectively bridging the gap between kinematic data and real-world execution.

Keywords: Integrated Planning and Control, Machine Learning for Robot Control, Simulation and Animation
Abstract: Diffusion models offer powerful generative capabilities for robot trajectory planning, yet their practical deployment on robots is hindered by a critical bottleneck: reliance on imitation learning from expert demonstrations. This paradigm is often impractical for specialized robots where data is scarce and creates an inefficient, theoretically suboptimal training pipeline. To overcome this, we introduce PegasusFlow, a hierarchical rolling-denoising framework that enables direct and parallel sampling of trajectory score gradients from environmental interaction, completely bypassing the need for expert data. Our core innovation is a novel sampling algorithm, Weighted Basis Function Optimization (WBFO), which leverages spline basis representations to achieve superior sample efficiency and faster convergence compared to traditional methods like MPPI. The framework is embedded within a scalable, asynchronous parallel simulation architecture that supports massively parallel rollouts for efficient data collection. Extensive experiments on trajectory optimization and robotic navigation tasks demonstrate that our approach, particularly Action-Value WBFO (AVWBFO) combined with a reinforcement learning warm-start, significantly outperforms baselines. In a challenging barrier-crossing task, our method achieved a 100% success rate and was 18% faster than the next-best method, validating its effectiveness for complex terrain locomotion planning. https://masteryip.github.io/pegasusflow.github.io/

Keywords: Computer Vision for Transportation, Deep Learning for Visual Perception, Multi-Robot Systems
Abstract: Collaborative perception aims to extend sensing coverage and improve perception accuracy by sharing information among multiple agents. However, due to differences in viewpoints and spatial positions, agents often acquire heterogeneous observations. Existing intermediate fusion methods primarily focus on aligning similar features, often overlooking the perceptual diversity among agents. To address this limitation, we propose CoBEVMoE, a novel collaborative perception framework that operates in the Bird’s Eye View (BEV) space and incorporates a Dynamic Mixture-of-Experts (DMoE) architecture. In DMoE, each expert is dynamically generated based on the input features of a specific agent, enabling it to extract distinctive and reliable cues while attending to shared semantics. This design allows the fusion process to explicitly model both feature similarity and heterogeneity across agents. Furthermore, we introduce a Dynamic Expert Metric Loss (DEML) to enhance inter-expert diversity and improve the discriminability of the fused representation. Extensive experiments on the OPV2V and DAIR-V2X-C datasets demonstrate that CoBEVMoE achieves state-of-the-art performance. Specifically, it improves the IoU for Camera-based BEV segmentation by +1.5% on OPV2V and the AP@0.5 for LiDAR-based 3D object detection by 3.0% on DAIR-V2X-C, verifying the effectiveness of expert-based heterogeneous feature modeling in multi-agent collaborative perception. The source code will be made available.

Keywords: Deep Learning for Visual Perception, Visual Learning, Computer Vision for Automation
Abstract: Robotic surgical systems rely heavily on high-quality visual feedback for precise teleoperation; yet, surgical smoke from energy-based devices significantly degrades endoscopic video feeds, compromising the human-robot interface and surgical outcomes. This paper presents RGA-Net (Reciprocal Gating and Attention-fusion Network), a novel deep learning framework specifically designed for smoke removal in robotic surgery workflows. Our approach addresses the unique challenges of surgical smoke-including dense, non-homogeneous distribution and complex light scattering-through a hierarchical encoder-decoder architecture featuring two key innovations: (1) a Dual-Stream Hybrid Attention (DHA) module that combines shifted window attention with frequency-domain processing to capture both local surgical details and global illumination changes, and (2) an Axis-Decomposed Attention (ADA) module that efficiently processes multi-scale features through factorized attention mechanisms. These components are connected via reciprocal cross-gating blocks that enable bidirectional feature modulation between encoder and decoder pathways. Extensive experiments on the DesmokeData and LSD3K surgical datasets demonstrate that RGA-Net achieves superior performance in restoring visual clarity suitable for robotic surgery integration. Our method enhances the surgeon-robot interface by providing consistently clear visualization, laying a technical foundation for alleviating surgeons' cognitive burden, optimizing operation workflows, and reducing iatrogenic injury risks in minimally invasive procedures. These practical benefits could be further validated through future clinical trials involving surgeon usability assessments. The proposed framework represents a significant step toward more reliable and safer robotic surgical systems through computational vision enhancement.

Keywords: Multi-Robot Systems
Abstract: Collaborative planning under operational constraints is an essential capability for heterogeneous robot teams tackling complex large-scale real-world tasks. Unmanned Aerial Vehicles (UAVs) offer rapid environmental coverage, but flight time is often limited by energy constraints, whereas Unmanned Ground Vehicles (UGVs) have greater energy capacity to support long-duration missions, but movement is constrained by traversable terrain. Individually, neither can complete tasks such as environmental monitoring. Effective UAV-UGV collaboration therefore requires energy-constrained multi-UAV task planning, traversability-constrained multi-UGV path planning, and crucially, synchronized concurrent co-planning to ensure timely in-mission recharging. To enable these capabilities, we propose Collaborative Planning with Concurrent Synchronization (CoPCS), a learning-based approach that integrates a heterogeneous graph transformer for operationally constrained task encoding with a transformer decoder for joint, synchronized co-planning that enables UAVs and UGVs to act concurrently in a coordinated manner. CoPCS is trained end-to-end under a unified imitation learning paradigm. We conducted extensive experiments to evaluate CoPCS in both robotic simulations and physical robot teams. Experimental results demonstrate that our method provides the novel multi-robot capability of synchronized concurrent co-planning and substantially improves team performance. More details of this work are available on the project website: https://hcrlab.gitlab.io/project/CoPCS.

Keywords: Soft Sensors and Actuators, Perception for Grasping and Manipulation, Force and Tactile Sensing
Abstract: Universal jamming grippers excel at grasping unknown objects due to their compliant bodies. Traditional tactile sensors can compromise this compliance, reducing grasping performance. We present acoustic sensing as a form of morphological sensing, where the gripper's soft body itself becomes the sensor. A speaker and microphone are placed inside the gripper cavity, away from the deformable membrane, fully preserving compliance. Sound propagates through the gripper and object, encoding object properties, which are then reconstructed via machine learning. Our sensor achieves high spatial resolution in sensing object size (2.6 mm error) and orientation (0.6 deg error), remains robust to external noise levels of 80 dBA, and discriminates object materials (up to 100% accuracy) and 16 everyday objects (85.6% accuracy). We validate the sensor in a realistic tactile object sorting task, achieving 53 minutes of uninterrupted grasping and sensing, confirming the preserved grasping performance. Finally, we demonstrate that disentangled acoustic representations can be learned, improving robustness to irrelevant acoustic variations.

Keywords: Robotics and Automation in Agriculture and Forestry, Mapping

Keywords: Deep Learning in Grasping and Manipulation, Perception for Grasping and Manipulation, Grasping
Abstract: We introduce Hand-Objectemph{(HO)GraspFlow}, an affordance-centric approach that retargets a single RGB with hand-object interaction (HOI) into multi-modal executable parallel jaw grasps without explicit geometric priors on target objects. Building on existing learning-based hand reconstruction and the vision foundation model, we synthesize SE(3) grasp poses with denoising flow matching (FM), conditioned on the following three complementary cues: RGB foundation features as visual semantics, HOI contact reconstruction, and taxonomy-aware prior on grasp types. Our approach demonstrates high fidelity in grasp synthesis without explicit HOI contact input or object geometry, while maintaining strong contact and taxonomy recognition. Another controlled comparison shows that emph{HOGraspFlow} consistently outperforms diffusion-based variants (emph{HOGraspDiff}), achieving high distributional fidelity and more stable optimization in SE(3). We demonstrate a reliable, object-agnostic grasp synthesis from human demonstrations in real-world experiments, where an average success rate of over 83% is achieved. Code: https://github.com/YitianShi/HOGraspFlow

Keywords: Bimanual Manipulation, Imitation Learning, Deep Learning in Grasping and Manipulation
Abstract: Contact-rich bimanual manipulation involves precise coordination of two arms to change object states through strategically selected contacts and motions. Due to the inherent complexity of these tasks, acquiring sufficient demonstration data and training policies that generalize to unseen scenarios remains a largely unresolved challenge. Building on recent advances in planning through contacts, we introduce Planning-Guided Diffusion Policy Learning (LIDE), an approach that effectively learns to solve contact-rich bimanual manipulation tasks by leveraging model-based motion planners to generate demonstration data in high-fidelity physics simulation. Through efficient planning in randomized environments, our approach generates large-scale and high-quality synthetic motion trajectories for tasks involving diverse objects and transformations. We then train a task-conditioned diffusion policy via behavior cloning using these demonstrations. To reduce the sim-to-real gap, we propose a set of designs in feature extraction, action prediction, and data augmentation that enable learning robust prediction of smooth action sequences and generalization to unseen scenarios. Through experiments in both simulation and the real world, we demonstrate that our approach can enable a bimanual robotic system to effectively manipulate objects of diverse geometries, dimensions, and physical properties.

Keywords: Imitation Learning, Manipulation Planning, Deep Learning in Grasping and Manipulation
Abstract: Policy learning often encounters difficulties in long-horizon tasks. Subgoal-conditioned policies address long-horizon problems by decomposing them into manageable segments, but they usually struggle with identifying informative subgoals. To address this limitation, we propose PDS (planning with dual subgoal), an architecture that learns short-horizon and low-variance subgoals in embedding space, ensuring the planning both reachable and consistent. We begin by analyzing the impact of horizon and consistency on the performance of subgoal-conditioned policies. We evaluate the performance of commonly used subgoal definitions (time-based, visual-based, and language-based) in tasks with different lengths. Subsequently, we demonstrate that our approach, which predicts and conditions on dual subgoals, improves success rates and enhances stability across diverse tasks in simulation and real-world.

Keywords: Grasping, Deep Learning in Grasping and Manipulation, Perception for Grasping and Manipulation
Abstract: Grasping is a fundamental robot skill, yet despite significant research advancements, learning-based 6-DOF grasping approaches are still not turnkey and struggle to generalize across different embodiments and in-the-wild settings. We build upon the recent success on modeling the object-centric grasp generation process as an iterative diffusion process. Our proposed framework, GraspGen, consists of a Diffusion-Transformer architecture that enhances grasp generation, paired with an efficient discriminator to score and filter sampled grasps. We introduce a novel and performant on-generator training recipe for the discriminator. To scale GraspGen to both objects and grippers, we release a new simulated dataset consisting of over 53 million grasps. We demonstrate that GraspGen outperforms prior methods in simulations with singulated objects across different grippers, achieves state-of-the-art performance on the FetchBench benchmark for grasping in clutter, and performs well on a real robot with noisy visual observations.

Keywords: Safety in HRI, Touch in HRI, Physical Human-Robot Interaction
Abstract: This paper introduces the CareBot-H Robot, a humanoid nursing robot designed to perform patient transfer tasks in confined environments. The robot is equipped with biomimetic arms that replicate human arm size and function, and distributed tactile sensors that enhance operational safety during physical contact. To achieve stable and anthropomorphic motion, a trajectory deformation algorithm is proposed. The method comprises an offline phase, where expert demonstrations are encoded into prior trajectories using a Variational Autoencoder (VAE), and an online phase, where a tactile-informed Zero-Moment Point (ZMP) model enables real-time trajectory adjustment. Experimental validation with human participants demonstrates that the proposed approach outperforms manual teleoperation, producing smoother and more efficient transfer trajectories while significantly reducing deviations between actual and ideal ZMP. These results indicate that the CareBot-H achieves reliable and safe patient transfer performance, offering practical potential for deployment in real-world nursing care scenarios.

Keywords: Motion and Path Planning, Climbing Robots, Nonholonomic Motion Planning
Abstract: Pipeline inspection is essential for maintaining the safety of critical infrastructure, but manual inspection is dangerous and inefficient, and existing robotic solutions struggle to handle curved and constrained surfaces. Traditional planning methods are either computationally expensive or prone to redundancy and discretization artifacts. To address these challenges, this paper proposes a centerline-aligned Frenet graph framework for surface-based path planning in pipeline environments. By embedding the pipeline surface into a structured two-dimensional manifold passing through the pipeline's central axis, the framework enables efficient heuristic search while maintaining geometric consistency. By combining quadratic programming with kinematic limits, an initial geodesic constrained path is generated and optimized, resulting in a smooth and executable trajectory. Extensive experiments on pipelines with sharp bends, intersections, and real-world pipeline environments demonstrate significant improvements in computational efficiency, path quality, and robustness compared to traditional methods.

Keywords: Aerial Systems: Perception and Autonomy, Omnidirectional Vision, Collision Avoidance
Abstract: Detecting and estimating distances to power lines is a challenge for both human UAV pilots and autonomous systems, which increases the risk of unintended collisions. We present a mmWave radar–based perception system that provides spherical sensing coverage around a small UAV for robust power line detection and avoidance. The system integrates multiple compact solid-state mmWave radar modules to synthesize an omnidirectional field of view while remaining lightweight. We characterize the sensing behavior of this omnidirectional radar arrangement in power line environments and develop a robust detection-and-avoidance algorithm tailored to that behavior. Field experiments on real power lines demonstrate reliable detection at ranges up to 10 m, successful avoidance maneuvers at flight speeds upwards of 10 m/s, and detection of wires as thin as 1.2 mm in diameter. These results indicate the approach’s suitability as an additional safety layer for both autonomous and manual UAV flight.

Keywords: Autonomous Vehicle Navigation, Deep Learning for Visual Perception, Recognition

Keywords: Agent-Based Systems, AI-Based Methods, Human Factors and Human-in-the-Loop
Abstract: The integration of human expertise into reinforcement learning has gained increasing attention as a means to improve sample efficiency and stability. Current approaches often depend on pre-collected expert demonstrations or virtual reality setups, which are costly to generate and difficult to adapt to dynamic training conditions. In this work, a framework is introduced that augments deep reinforcement learning with real-time demonstrations provided through mixed reality interaction. A structured robotic pick-and-place task serves as the benchmark, where a robot must execute sequential phases of grasping, transporting, and releasing an object. Expert guidance is delivered via mixed reality annotations, which are converted into reference trajectories and injected into the learning process whenever performance falls below a predefined threshold. A modified replay buffer accommodates both agent-generated and expert-generated transitions, allowing controlled sampling with a dynamically adjusted expert-to-agent ratio. Training in the real workspace through mixed reality reduces the simulation-to-reality gap considerably, as confirmed by experiments on a physical robot platform. Experimental evaluation demonstrates that the proposed framework accelerates policy convergence, ensures stability under noisy feedback, and achieves strong generalization to unseen task configurations. These findings highlight the potential of demonstration-augmented reinforcement learning through mixed reality as a data-efficient and robust approach to robot training in real-world scenarios.

Keywords: Deep Learning for Visual Perception, Object Detection, Segmentation and Categorization, Semantic Scene Understanding
Abstract: Embodied intelligence relies on accurately segmenting objects actively involved in interactions. Action-based video object segmentation addresses this by linking segmentation with action semantics, but it depends on large-scale annotations and prompts that are costly, inconsistent, and prone to multimodal noise such as imprecise masks and referential ambiguity. To date, this challenge remains unexplored. In this work, we take the first step by studying action-based video object segmentation under label noise, focusing on two sources: textual prompt noise (category flips and within-category noun substitutions) and mask annotation noise (perturbed object boundaries to mimic imprecise supervision). Our contributions are threefold. First, we introduce two types of label noises for the action-based video object segmentation task. Second, we build up the first action-based video object segmentation under a label noise benchmark ActiSeg-NL and adapt six label-noise learning strategies to this setting, and establish protocols for evaluating them under textual, boundary, and mixed noise. Third, we provide a comprehensive analysis linking noise types to failure modes and robustness gains, and we introduce a Parallel Mask Head Mechanism (PMHM) to address mask annotation noise. Qualitative evaluations further reveal characteristic failure modes, including boundary leakage and mislocalization under boundary perturbations, as well as occasional identity substitutions under textual flips. Our comparative analysis reveals that different learning strategies exhibit distinct robustness profiles, governed by a foreground-background trade-off where some achieve balanced performance while others prioritize foreground accuracy at the cost of background precision. These results establish a clear sensitivity profile of action-based video object segmentation to imperfect annotations and set a benchmark for studying noise-robust learning in embodied perception.

Keywords: Prosthetics and Exoskeletons, Wearable Robotics, Physical Human-Robot Interaction
Abstract: This paper presents the design, control, and experimental validation of a lightweight hip exoskeleton for walking assistance. By integrating quasi-direct drive actuators, single-piece stainless steel frames, and passive revolute joints, the device achieves a high torque-to-mass ratio while maintaining a compact and lightweight structure. A delayed output feedback control strategy synchronizes assistive torque with the gait cycle by actively leading the wearer's hip motion, with user studies identifying a consistent optimal phase difference across participants and walking speeds, eliminating repeated calibration. Surface electromyography validates the assistance, demonstrating substantial reductions in activation of the vastus medialis and vastus lateralis at the optimal time delay. Power analysis further confirms that this setting maximizes positive power transfer while minimizing resistive effects. The proposed exoskeleton delivers physiologically meaningful and energetically efficient hip assistance suitable for everyday mobility support.

Keywords: Simulation and Animation, Computer Vision for Transportation, Deep Learning for Visual Perception
Abstract: This paper focuses on scene reconstruction under nighttime conditions in autonomous driving simulation. Recent methods based on Neural Radiance Fields (NeRFs) and 3D Gaussian Splatting (3DGS) have achieved photorealistic modeling in autonomous driving scene reconstruction, but they primarily focus on normal-light conditions. Low-light driving scenes are more challenging to model due to their complex lighting and appearance conditions, which often causes performance degradation of existing methods. To address this problem, this work presents a novel approach that integrates physically based rendering into 3DGS to enhance nighttime scene reconstruction for autonomous driving. Specifically, our approach integrates physically based rendering into composite scene Gaussian representations and jointly optimizes Bidirectional Reflectance Distribution Function (BRDF) based material properties. We explicitly model diffuse components through a global illumination module and specular components by anisotropic spherical Gaussians. As a result, our approach improves reconstruction quality for outdoor nighttime driving scenes, while maintaining real-time rendering. Extensive experiments across diverse nighttime scenarios on two real-world autonomous driving datasets, including nuScenes and Waymo, demonstrate that our approach outperforms the state-of-the-art methods both quantitatively and qualitatively.

Keywords: Field Robots, Search and Rescue Robots
Abstract: Mass-casualty incidents demand rapid and accurate triage, but the scale and acuity of injuries often overwhelm available medical personnel. To address this, we present a system that enables ground and aerial robots to localize and assess casualties using non-contact sensors, including color and thermal cameras, millimeter wave radar, and microphones. Injury and vital sign measurements from modality-specific classifiers are fused using a probabilistic model that captures correlations between injury states and supports distributed, asynchronous evidence accumulation. We validate the system through a series of timed mass-casualty field experiments using custom-built drones and Boston Dynamics Spot ground robots customized for robotic medical triage, demonstrating reliable estimation of casualty states and robustness to noisy conditions and sensor drop out.

Keywords: Networked Robots, Software, Middleware and Programming Environments
Abstract: The Robot Operating System (ROS) is widely adopted in the robotics community, powering applications from self-driving vehicles to industrial automation. ROS 2 utilizes the Data Distribution Service (DDS) middleware for decentralized communication, making it inherently susceptible to reconnaissance and exploitation attacks. Previous research has examined the security implications of DDS implementations but has not systematically distinguished ROS 2 nodes from standalone DDS deployments, a critical distinction that significantly influences the execution and outcome of cyberattacks. This paper presents the first systematic fingerprinting framework designed specifically for ROS 2, demonstrating how DDS-based metadata leakage can facilitate precise identification and targeted exploitation of robotic systems. Through controlled experiments and an Internet-wide scan of DDS deployments, we identify extensive metadata exposure across actively supported ROS 2 implementations. Despite existing security solutions such as Secure ROS 2 (SROS2), deployments using default configurations remain vulnerable, highlighting the need for enhanced metadata obfuscation, stricter network access policies, and deployment of real-time anomaly detection mechanisms to strengthen the security posture of ROS 2 systems.

Keywords: Localization, Software Tools for Benchmarking and Reproducibility, SLAM
Abstract: Event-based localization research and datasets are a rapidly growing area of interest, with a tenfold increase in the cumulative total number of published papers on this topic over the past 10 years. Whilst the rapid expansion in the field is exciting, it brings with it an associated challenge: a growth in the variety of required code and package dependencies as well as data formats, making comparisons difficult and cumbersome for researchers to implement reliably. To address this challenge, we present Event-LAB: a new and unified framework for running several event-based localization methodologies across multiple datasets. Event-LAB is implemented using the Pixi package and dependency manager, that enables a single command-line installation and invocation for combinations of localization methods and datasets. To demonstrate the capabilities of the framework, we implement two common event-based localization pipelines: Visual Place Recognition (VPR) and Simultaneous Localization and Mapping (SLAM). We demonstrate the ability of the framework to systematically visualize and analyze the results of multiple methods and datasets, revealing key insights such as the association of parameters that control event collection counts and window sizes for frame generation to large variations in performance. The results and analysis demonstrate the importance of fairly comparing methodologies with consistent event image generation parameters. Our Event-LAB framework provides this ability for the research community, by contributing a streamlined workflow for easily setting up multiple conditions.

Keywords: Grippers and Other End-Effectors, Multifingered Hands
Abstract: Robotic grasping requires flexible reconfiguration to handle diverse objects and tasks. This paper proposes a monorail-like reconfiguration framework for robotic grippers, inspired by train–rail relationships, that generates diverse finger layouts. The proposed framework unifies two complementary forms: dynamic reconfiguration, in which finger units move along an arbitrary non-circular track defined by the palm shape (palm track) to change the finger layout, and modular reconfiguration, in which the palm track shape and the number of fingers are modified to alter the achievable finger layout space. We developed a prototype gripper system that embodies the proposed framework and experimentally validated its unified reconfiguration capability. Dynamic reconfiguration with the S-shaped palm achieved seven distinct finger layouts with successful object grasping, while on-the-fly modular reconfiguration expanded the achievable finger layout space, enabling rapid adaptation to different grasping tasks. This work establishes a new design principle for reconfigurable grippers toward highly adaptive and versatile robotic grasping.

Keywords: Learning from Demonstration, Integrated Planning and Learning, Manipulation Planning
Abstract: While skill-centric approaches leverage foundation models to enhance generalization in compositional tasks, they often rely on fixed skill libraries, limiting adaptability to new tasks without manual intervention. To address this, we propose Uni-Skill, a Unified Skill-centric framework that supports skill-aware planning and facilitates automatic skill evolution. Unlike prior methods that restrict planning to predefined skills, Uni-Skill requests for new skill implementations when existing ones are insufficient, ensuring adaptable planning with self-augmented skill library. To support automatic implementation of diverse skills requested by the planning module, we construct SkillFolder, a VerbNet-inspired repository derived from large-scale unstructured robotic videos. SkillFolder introduces a hierarchical skill taxonomy that captures diverse skill descriptions at multiple levels of abstraction. By populating this taxonomy with large-scale, automatically annotated demonstrations, Uni-Skill shifts the paradigm of skill acquisition from inefficient manual annotation to efficient offline structural retrieval. Retrieved examples provide semantic supervision over behavior patterns and fine-grained references for spatial trajectories, enabling few-shot skill inference without deployment-time demonstrations. Comprehensive experiments in both simulation and real-world settings verify the state-of-the-art performance of Uni-Skill over existing VLM-based skill-centric approaches, highlighting its advanced reasoning capabilities and strong zero-shot generalization across a wide range of novel tasks.

Keywords: SLAM, Mapping, Visual Tracking
Abstract: The challenge of dynamic scenes has long been one of the core issues in the application and generalization of SLAM systems. Traditional visual SLAM systems often rely on depth sensors and prior camera parameters, making it difficult to correct dynamic challenges from arbitrary input images while simultaneously constructing dense maps. Recently, neural network-based methods for two-view point cloud prediction have gained attention, and SLAM systems such as DUST3R and MAST3R have emerged based on this approach. However, these systems face challenges when applied to dynamic scenes and cannot directly use traditional methods for correction, such as semantic masking or optical flow segmentation. To address this issue, we propose MASTD3R-SLAM, a SLAM method specifically designed for dynamic scenes that supports arbitrary video inputs. The method combines fused mask-based processing with coarse-to-fine pointmap alignment and optimization to achieve point cloud–to–pose re-mapping correction, and further performs Gaussian rendering to remove rendering artifacts and suppress dynamic mapping interference. Compared to the original baseline, our approach improves tracking ATE accuracy by more than 90% and successfully restores the correct 3D map.

Keywords: Imitation Learning, Learning from Demonstration, Deep Learning Methods
Abstract: We propose the first framework that leverages physically-based inverse rendering for novel lighting generation on existing real-world human demonstrations of robotic manipulation tasks. Specifically, inverse rendering decomposes the first frame in each demonstration into geometric (surface normal, depth) and material (albedo, roughness, metallic) properties, which are then used to render appearance changes under different lighting sources. To improve efficiency and maintain consistency across each generated sequence, we fine-tune Stable Video Diffusion on robot execution videos for temporal lighting propagation. We evaluate our framework by measuring the visual quality of the generated sequences, assessing its effectiveness in improving the imitation learning policy performance (38.75%) under six unseen real-world lighting conditions, and conducting ablation studies on individual modules of the proposed framework. We further showcase three downstream applications enabled by the proposed framework: background generation, object texture generation and distractor positioning.

Keywords: Path Planning for Multiple Mobile Robots or Agents, Multi-Robot Systems, Motion and Path Planning
Abstract: In warehouse environments, corridor conflicts often lead to traffic congestion, which result in many Multi-Agent Path Finding (MAPF) algorithms failing to find solutions within a reasonable time limit. Previous works have studied using corridor reasoning techniques or incorporating guidance, such as highways, to address this problem. However, these approaches often either encounter timeouts or yield low-quality solutions. In this work, based on Conflict-Based Search (CBS), we propose a technique called Reversible Lanes, specifically designed to address corridor conflicts by imposing a new constraint that forces agents to use bypasses for conflict resolution. Our approach is motivated by three key observations from prior research: (1) in warehouse maps, the overhead associated with maintaining solution optimality via corridor reasoning technique is often disproportionate to the benefits gained; (2) the fixed nature of manually designed highways exhibits a lack of adaptability, leading to poor solution quality on certain instances; and (3) the structural properties of warehouse layouts render direct bypass usage feasible and incur minimal additional costs. Theoretically, we demonstrate the feasibility of our algorithm by analyzing its relationship to both corridor reasoning techniques and highways. Experimentally, the results show that our algorithm provides a more effective approach for resolving corridor conflicts compared to these existing methods, achieving a superior trade off between solution quality and computational efficiency by finding near-optimal solutions with reduced runtime.

Keywords: Modeling, Control, and Learning for Soft Robots, Soft Robot Applications, Collision Avoidance
Abstract: A cable-driven continuum robot with high redundancy is capable of performing the tip trajectory tracking task while simultaneously satisfying additional safety constraints, such as joint limits or external obstacles in the environment. To address these challenges, efficient motion planning methods are required. This paper proposes a quadratic programming based method in conjunction with convex polytopes based distance computation. Our methodology integrates safety constraints based on the robots' posture states, thus enabling barriers evasion in dynamic situations. Simulation outcomes demonstrate effective trajectory tracking in the presence of various objects and provide a comprehensive performance evaluation based on the generated robot state. Finally, real-word experiment was conducted on a prototype of a three-segment cable-driven continuum manipulator, which confirmed the efficacy of the proposed obstacle avoidance approach. The approach is versatile and can be adapted to similar multiple segments cable-driven continuum robotic systems by designing the robot parameters, enabling the success of tip trajectory tracking tasks under complex obstacle conditions.

Keywords: Parallel Robots, Mechanism Design, Kinematics
Abstract: This paper introduces a novel design for a robotic hand based on parallel mechanisms. The proposed hand uses a triple-symmetric Bricard linkage as its reconfigurable palm, enhancing adaptability to objects of varying shapes and sizes. Through topological and dimensional synthesis, the mechanism achieves a well-balanced degree of freedom and link configuration suitable for reconfigurable palm motion, balancing dexterity, stability, and load capacity. Furthermore, kinematic analysis is performed using screw theory and closed-loop constraints, and performance is evaluated based on workspace, stiffness, and motion/force transmission efficiency. Finally, a prototype is developed and tested through a series of grasping experiments, demonstrating the ability to perform stable and efficient manipulation across a wide range of objects. The results validate the effectiveness of the design in improving grasping versatility and operational precision, offering a promising solution for advanced robotic manipulation tasks.

Keywords: Surgical Robotics: Planning, Collision Avoidance, Robust/Adaptive Control
Abstract: Crown preparation aims to create an optimal foundation for durable and functional restoration by reshaping the tooth with a cutting tool. Robotic crown preparation has emerged as a promising approach to overcome the inherent limitations of manual procedures, yet challenges remain in achieving efficient cutting path generation, collision-free orientation adjustment and precise cutting path following, since the oral cavity is a confined space with the target tooth tightly surrounded by other teeth. This paper introduces a novel, in-situ automated robotic full crown preparation system comprising (1) Preoperative Path Planning: generating high-efficiency universal cutting paths based on tooth morphological features; (2) Intraoral Collision Avoidance: optimizing the cutting tool's orientation within the constrained oral cavity; (3) MPC-Based Adaptive Control: modulating the path-following feed rate using model predictive control (MPC) according to intraoperative force feedback. The proposed system was thoroughly validated on a human head phantom targeting a permanent tooth to simulate a real clinical scenario, yielding an average root-mean-square (RMS) error (tooth shape after preparation) of 0.17 mm and an overall mean execution time of 347.77 s, achieving a 74.2% improvement in cutting efficiency over state-of-the-art methods. A comparative evaluation against conventional dental guides further demonstrates its technical feasibility and significant potential for clinical translation.

Keywords: Medical Robots and Systems, Collision Avoidance, Dual Arm Manipulation
Abstract: During surgery, scrub nurses are required to frequently deliver surgical instruments to surgeons, which can lead to physical fatigue and decreased focus. Robotic scrub nurses provide a promising solution that can replace repetitive tasks and enhance efficiency. Existing research on robotic scrub nurses relies on predefined paths for instrument delivery, which limits their generalizability and poses safety risks in dynamic environments. To address these challenges, we present a collision-free dual-arm surgical assistive robot capable of performing instrument delivery. A vision-language model is utilized to automatically generate the robot's grasping and delivery trajectories in a zero-shot manner based on surgeons' instructions. A real-time obstacle minimum distance perception method is proposed and integrated into a unified quadratic programming framework. This framework ensures reactive obstacle avoidance and self-collision prevention during the dual-arm robot's autonomous movement in dynamic environments. Extensive experimental validations demonstrate that the proposed robotic system achieves an 83.33% success rate in surgical instrument delivery while maintaining smooth, collision-free movement throughout all trials. The project page and source code are available at https://give-me-scissors.github.io/.

Keywords: Machine Learning for Robot Control, Reinforcement Learning, Learning from Demonstration
Abstract: Despite the recent advancements of vision-language-action (VLA) models on a variety of robotics tasks, they suffer from critical issues such as poor generalizability to unseen tasks, due to their reliance on behavior cloning exclusively from successful rollouts. Furthermore, they are typically fine-tuned to replicate demonstrations collected by experts under different settings, thus introducing distribution bias and limiting their adaptability to diverse manipulation objectives, such as efficiency, safety, and task completion. To bridge this gap, we introduce GRAPE: Generalizing Robot Policy via Preference Alignment. Specifically, GRAPE aligns VLAs on a trajectory level and implicitly models reward from both successful and failure trials to boost generalizability to diverse tasks. Moreover, GRAPE breaks down complex manipulation tasks to independent stages and automatically guides preference modeling through customized spatiotemporal constraints with keypoints proposed by a large vision-language model. Notably, these constraints are flexible and can be customized to align the model with varying objectives, such as safety, efficiency, or task success. We evaluate GRAPE across a diverse array of tasks in both real-world and simulated environments. Experimental results demonstrate that GRAPE enhances the performance of state-of-the-art VLA models, increasing success rates on in-domain and unseen manipulation tasks by 51.79% and 58.20%, respectively. Additionally, GRAPE can be aligned with various objectives, such as safety and efficiency, reducing collision rates by 37.44% and rollout step-length by 11.15%, respectively.

Keywords: Robot Safety, Representation Learning, Deep Learning for Visual Perception

Keywords: Deep Learning in Grasping and Manipulation, Dexterous Manipulation, Learning from Demonstration
Abstract: We propose DemoDiffusion, a simple method for enabling robots to perform manipulation tasks by imitating a single human demonstration, without requiring task-specific training or paired human-robot data. Our approach is based on two insights. First, the hand motion in a human demonstration provides a useful prior for the robot’s end-effector trajectory, which we can convert into a rough open-loop robot motion trajectory via kinematic retargeting. Second, while this retargeted motion captures the overall structure of the task, it may not align well with plausible robot actions in-context. To address this, we leverage a pre-trained generalist diffusion policy to modify the trajectory, ensuring it both follows the human motion and remains within the distribution of plausible robot actions. Unlike approaches based on online reinforcement learning or paired human-robot data, our method enables robust adaptation to new tasks and scenes with minimal effort. In real-world experiments across 8 diverse manipulation tasks, DemoDiffusion achieves 83.8% average success rate, compared to 13.8% for the pre-trained policy and 52.5% for kinematic retargeting, succeeding even on tasks where the pre-trained generalist policy fails entirely. Project page: https://demodiffusion.github.io/

Keywords: SLAM, Localization
Abstract: Single sensor (visual or LiDAR) simultaneous localization and mapping (SLAM) is fragile in the complex environment, which makes visual-LiDAR fusion a mainstream in SLAM research. However, most existing fusion methods omit explicit modeling of feature uncertainties and do not quantify each feature's constraint strength on each degree of freedom (DoF) of the 6-DoF pose, thereby hindering the full exploitation of the complementary information across different sensors. In this paper, a tightly coupled visual-LiDAR SLAM method termed UCA-SLAM is proposed, which integrates the closed-form uncertainty propagation and the DoF-wise constraint analysis. Specifically, UCA-SLAM maintains uncertainties for visual map points and LiDAR voxel planes, and computes DoF-wise constraint strength for each feature. In the front-end tracking, the DoF-wise constraints of features are comprehensively analyzed, which provides an adaptive fusion mechanism for pose estimation, and an explicit uncertainty propagation from feature measurements to the 6-DoF pose is derived. The resultant feature and pose uncertainties are then used to weight the cost function in local bundle adjustment (BA) optimization of UCA-SLAM to improve the accuracy of the system. Extensive experiments conducted on public datasets and in real-world environments demonstrate that UCA-SLAM outperforms state-of-the-art visual-LiDAR fusion SLAM methods. UCA-SLAM is open-sourced to benefit the community.

Keywords: Mapping, Computer Vision for Automation, SLAM
Abstract: Visual SLAM algorithms achieve significant improvements through the exploration of 3D Gaussian Splatting (3DGS) representations, particularly in generating high-fidelity dense maps. However, they depend on a static environment assumption and experience significant performance degradation in dynamic environments. This paper presents GGD-SLAM, a framework that employs a generalizable motion model to address the challenges of localization and dense mapping in dynamic environments—without predefined semantic annotations or depth input. Specifically, the proposed system employs a First-In-First-Out (FIFO) queue to manage incoming frames, facilitating dynamic semantic feature extraction through a sequential attention mechanism. This is integrated with a dynamic feature enhancer to separate static and dynamic components. Additionally, to minimize dynamic distractors' impact on the static components, we devise a method to fill occluded areas via static information sampling and design a distractor-adaptive Structure Similarity Index Measure (SSIM) loss tailored for dynamic environments, significantly enhancing the system's resilience. Experiments conducted on real-world dynamic datasets demonstrate that the proposed system achieves state-of-the-art performance in camera pose estimation and dense reconstruction in dynamic scenes.

Keywords: Perception for Grasping and Manipulation, Sensorimotor Learning
Abstract: Most robot manipulation focuses on changing the kinematic state of objects: picking, placing, opening, or rotating them. However, a wide range of real-world manipulation tasks involve a different class of object state change—such as mashing, spreading, or slicing—where the object’s physical and visual state evolve progressively without necessarily changing its position. We present SPARTA, the first unified framework for the family of object state change manipulation tasks. Our key insight is that these tasks share a common structural pattern: they involve spatially-progressing, object-centric changes that can be represented as regions transitioning from an actionable to a transformed state. Building on this insight, SPARTA integrates spatially progressing object change segmentation maps, a visual skill to perceive actionable vs. transformed regions for specific object state change tasks, to generate a) structured policy observations that strip away appearance variability, and b) dense rewards that capture incremental progress over time. These are leveraged in two SPARTA policy variants: reinforcement learning for fine-grained control without demonstrations or simulation; and greedy control for fast, lightweight deployment. We validate SPARTA on a real robot for three challenging tasks across 10 diverse real-world objects, achieving significant improvements in training time and accuracy over sparse rewards and visual goal-conditioned baselines. Our results highlight progress-aware visual representations as a versatile foundation for the broader family of object state manipulation tasks. More information at https://vision.cs.utexas.edu/projects/sparta-robot

Keywords: Learning from Demonstration, Imitation Learning, Dexterous Manipulation
Abstract: Achieving human-like dexterous manipulation is essential for general-purpose robots but remains a challenge. Recent advances in Vision-Language-Action (VLA) models offer the potential to learn flexible skills from demonstration data. However, training effective VLAs requires a large amount of high-quality data, which is difficult to obtain: fully manual teleoperation cognitively overloads human operators, while automated planning produces unnatural motions and lacks data diversity. We present a Shared Autonomy framework: a human operator teleoperates the arm for global motion, while an autonomous DexGrasp-VLA policy, as an AI Copilot, generates force-adaptive actions for a five-finger hand with tactile feedback -- drastically reducing human effort and enabling efficient collection of high-quality demonstrations. Using these data, we train an end-to-end VLA policy with a novel Arm-Hand Feature Enhancement module -- shared representations are conjunct with separate arm and hand latent features, representing the distinct dynamics of macro and micro movements, leading to more robust and natural coordination of arm-hand motions. Our Corrective Teleoperation can further refine the policy with failure-recovery demonstrations via human intervention. Experiments show our approach efficiently generates high-quality data and learns policies with a high success rate and natural behaviors. The trained arm-hand VLA policy is effectively generalized to both seen and unseen objects, with a success rate of around 90% in more than 50 diverse objects.

Keywords: Medical Robots and Systems, Computer Vision for Medical Robotics, Human-Robot Collaboration
Abstract: Surgical robotics have revolutionized medical procedures by offering enhanced precision and reduced complications. However, vitreoretinal surgery still relies heavily on manual techniques, where surgeons manage both a surgical tool and a light pipe, complicating operations and potentially affecting outcomes. To improve efficiency and outcomes while reducing workloads on surgeons, a novel vision-based robot-assisted system with advanced surgical scene understanding ability is proposed. The system automatically positions a light pipe held by a specialized surgical robot through optimization-based visual collaborative control. By identifying target areas for automatic illumination, the system allows surgeons to focus on surgical tasks and supports more complex surgeries such as three-arm procedures. Besides, the system enhances surgical safety by detecting surgical activities and dangerous areas and issuing alerts accordingly. Postoperatively, the system records tool trajectories and detected activities, providing data for surgical reports, skill evaluation, and training. Experiments prove the effectiveness of the control system, visual algorithm, and overall collaborative system.

Keywords: Deep Learning Methods, Range Sensing
Abstract: Despite its resilience in adverse weather, millimeter-wave (mmWave) radar yields sparse and noisy point clouds that limit its perception and localization performance. Diffusion models have recently gained attention for enhancing millimeter-wave radar in perception tasks due to their strong denoising and generative capabilities. Yet, the enhanced radar point cloud is still far from expected due to a lack of texture information and errors caused by inherent sensor–model mismatch between LiDAR and radar. In this paper, we propose an adaptive vision-aided radar data enhancement method based on a conditional diffusion model for denoising and densifying radar point clouds. The pipeline decomposes mmWave radar into depth and BEV views, fuses the depth view with synchronized images, and uses the fused features together with BEV tokens to condition the diffusion model. LiDAR is used only for training supervision, but not for inference. Extensive experiments demonstrate that our proposed method produces dense and geometrically consistent radar point clouds, validating the effectiveness of the introduced vision-aid for radar enhancement. Notably, our method even works well in scenarios under visual occlusions. The accurate odometry and high-fidelity map reconstruction using enhanced radar point cloud highlights the great potential of our method for other downstream tasks in robotics and autonomous driving.

Keywords: Object Detection, Segmentation and Categorization
Abstract: Recently, deep learning–based methods for road crack segmentation have achieved promising performance, particularly in robotic vision applications such as automated inspection and maintenance. However, most frequency-domain methods employ a decoupled processing strategy, overlooking the dynamic modulation mechanism between high- and low-frequency components, which constrains the model's effectiveness in detecting cracks within complex environments. Moreover, existing methods suffer from low information fidelity during feature transmission, where critical encoder details are progressively lost in the decoder, making it difficult to reconstruct complete crack structures. To address these issues, we propose a Bidirectional Frequency-domain Modulation Progressive Fusion Network (BFMPF-Net). Specifically, we propose a Bidirectional Frequency-domain Modulation Enhancement (BFME) module that effectively exploits bidirectional modulation between high- and low-frequency components and learns the spatial weights of high-frequency features to attenuate noise and preserve crack edge details, thereby improving the performance of crack segmentation. Furthermore, the Progressive Guidance Fusion module serves as another core component of our framework. It leverages the spatial prior provided by the original low-resolution image to guide feature refinement via stepwise optimization from coarse contours to fine edges, thereby ensuring the integrity of crack segmentation. Evaluation on three publicly available datasets—CrackTree260, CrackLS315, and Crack760—affirms the superior segmentation accuracy of the proposed BFMPF-Net compared to current mainstream methods.

Keywords: Localization, SLAM, Autonomous Vehicle Navigation
Abstract: Implicit representations for LiDAR-based Simultaneous Localization and Mapping (SLAM) offer significant advantages in storage efficiency and expressive power over traditional explicit maps. However, a critical limitation for implicit SLAM is their deterministic nature, which prevents the quantification of prediction uncertainty in sparse or noisy conditions. Furthermore, the accuracy of the underlying Signed Distance Field (SDF) is often compromised by systematic errors arising from the angular dependency of LiDAR measurements, where oblique incident angles lead to biased distance estimations and degrade map quality. To address these challenges, this paper introduces a framework that enhances the robustness and accuracy of implicit LiDAR SLAM by integrating uncertainty estimation and an adaptive sampling strategy. We propose a neural network-based approach to learn and predict SDF uncertainty, which is then effectively incorporated into both localization and mapping processes. Concurrently, to mitigate incident angle-induced errors, we develop an adaptive sampling scheme that weights LiDAR rays based on surface normal information. Validation on public datasets and a custom experimental platform demonstrates that our approach outperforms baseline methods in terms of localization, mapping accuracy, and robustness.

Keywords: Mobile Manipulation, Imitation Learning, Wheeled Robots
Abstract: In whole-body mobile manipulation, existing teleoperation systems often suffer from high complexity and cost, while imitation learning approaches are frequently limited by insufficient modeling of long-horizon action sequences and inadequate fusion of multi-receptive-field visual features. These constraints significantly hinder the collection of high-quality demonstration data and the effective transfer of complex robotic skills. To address these challenges, this paper proposes an integrated exoskeleton-VR teleoperation system that enables single-operator whole-body control of mobile manipulators with basic force feedback, substantially reducing the cost of data collection while improving demonstration quality. Furthermore, we introduce MIMO, an encoder–decoder imitation learning framework, which incorporates an Efficient Context Modeling Network (ECM-Net) based on linear-complexity temporal modeling to mitigate error accumulation in long-horizon tasks, and a Multi-Receptive Field Fusion Network (MRF-Net) that employs dual-path attention to achieve precise alignment between multi-scale visual cues and motion phases. Real-world experiments on a mobile manipulator demonstrate that MIMO consistently outperforms state-of-the-art baselines across multiple whole-body mobile manipulation tasks, confirming its effectiveness in long-horizon, fine-grained robotic control.

Keywords: Computer Vision for Automation, Deep Learning for Visual Perception
Abstract: Recent advances in collaborative perception systems have led to significant improvements in 3D object detection performance. While widely deployed LiDAR and camera systems often experience performance degradation under adverse weather conditions, weather-robust 4D radar offers a promising alternative to address this challenge. However, effectively fusing 4D radar measurements with degraded LiDAR data remains a critical challenge. In this work, we decompose the weather-induced degradation in LiDAR perception into feature attenuation requiring enhancement and feature contamination requiring suppression, based on the underlying physical interactions. Building upon this decomposition, we propose a dual-branch network to handle each degradation pattern in a specialized manner. One branch focuses on enhancement based on spatial and channel attention, guided by 4D radar cues. The other branch focuses on suppression based on intra-modal structural consistency and cross-modal consistency. To achieve adaptive branch integration, we propose a dynamic decision network to generate a decision weight map for each branch and capture the complex interaction between branches. To validate the effectiveness of our method, we conduct extensive experiments on V2X-R, the only publicly available collaborative LiDAR-4D radar dataset. Extensive experimental results demonstrate that our method achieves improvements of 3.65% and 10.80% in mAP@0.7 under fog and snow conditions, respectively, outperforming previous state-of-the-art approaches.

Keywords: Learning from Demonstration, Reinforcement Learning, Dexterous Manipulation
Abstract: This work presents DemoBot, a learning framework that enables a dual-arm, multi-finger robotic system to acquire complex manipulation skills from a single unannotated RGB-D video demonstration. The method extracts structured motion trajectories of both hands and objects from raw video data. These trajectories serve as motion priors for a novel reinforcement learning (RL) pipeline that learns to refine them through contact-rich interactions, thereby eliminating the need to learn from scratch. To address the challenge of learning long-horizon manipulation skills, we introduce: (1) Temporal-segment based RL to enforce temporal alignment of the current state with demonstrations; (2) Success-Gated Reset strategy to balance the refinement of readily acquired skills and the exploration of subsequent task stages; and (3) Event-Driven Reward curriculum with adaptive thresholding to guide the RL learning of high-precision manipulation. The novel video processing and RL framework successfully achieved long-horizon synchronous and asynchronous bimanual assembly tasks, offering a scalable approach for direct skill acquisition from human videos.

Keywords: Motion and Path Planning, Perception-Action Coupling, Contact Modeling
Abstract: Box/cabinet scenarios pose with stacked objects significant challenges for robotic motion due to visual occlusions and constrained free space. Traditional collision-free trajectory planning methods often fail when no collision-free paths exist, and may even lead to catastrophic collisions caused by invisible objects. To overcome these challenges, we propose an operational aware interactive motion planner (PaiP) a real-time closed-loop planning framework utilizing multimodal tactile perception. This framework autonomously infers object interaction features by perceiving motion effects at interaction interfaces. These interaction features are incorporated into grid maps to generate operational cost maps. Building upon this representation, we extend sampling-based planning methods to interactive planning by optimizing both path cost and operational cost. Experimental results demonstrate that PaiP achieves robust motion in narrow spaces. Project page: https://travelers-lab.github.io/PaiP/

Keywords: Legged Robots, Multi-Contact Whole-Body Motion Planning and Control, Humanoid and Bipedal Locomotion
Abstract: We present a sampling-based model predictive control (MPC) framework that enables emergent locomotion without relying on handcrafted gait patterns or predefined contact sequences. Our method discovers diverse motion patterns, ranging from trotting to galloping, robust standing policies, jumping, and handstand balancing, purely through the optimization of high-level objectives.

Building on model predictive path integral (MPPI), we propose a cubic Hermite spline parameterization that operates on position and velocity control points. Our approach enables contact-making and contact-breaking strategies that adapt automatically to task requirements, requiring only a limited number of sampled trajectories.

This sample efficiency enables real-time control on standard CPU hardware, eliminating the GPU acceleration typically required by other state-of-the-art MPPI methods.

We validate our approach on the Go2 quadrupedal robot, demonstrating a range of emergent gaits and basic jumping capabilities. In simulation, we further showcase more complex behaviors, such as backflips, dynamic handstand balancing and locomotion on a Humanoid, all without requiring reference tracking or offline pre-training.

Keywords: Humanoid and Bipedal Locomotion, Legged Robots, Reinforcement Learning
Abstract: Humanoid Whole-Body Controllers trained with reinforcement learning (RL) have recently achieved remarkable performance, yet many target a single robot embodiment. Variations in dynamics, degrees of freedom (DoFs), and kinematic topology still hinder a single policy from commanding diverse humanoids. Moreover, obtaining a generalist policy that not only transfers across embodiments but also supports richer behaviors—beyond simple walking to squatting, leaning—remains especially challenging. In this work, we tackle these obstacles by introducing EAGLE, an iterative generalist-specialist distillation framework that produces a single unified policy that controls multiple heterogeneous humanoids without per-robot reward tuning. During each cycle, embodiment-specific specialists are forked from the current generalist, refined on their respective robots, and new skills are distilled back into the generalist by training on the pooled embodiment set. Repeating this loop until performance convergence produces a robust Whole-Body Controller validated on robots such as Unitree H1, G1, and Fourier N1. We conducted experiments on five different robots in simulation environments and four in real-world settings. Through quantitative evaluations, EAGLE achieves high tracking accuracy and robustness compared to other methods, marking a step toward scalable, fleet-level humanoid control.

Keywords: Computer Vision for Medical Robotics, Surgical Robotics: Laparoscopy, Vision-Based Navigation
Abstract: Accurate depth estimation plays a critical role in the navigation of endoscopic surgical robots, forming the foundation for 3D reconstruction and safe instrument guidance. Fine-tuning pretrained models heavily relies on endoscopic surgical datasets with precise depth annotations. While existing self-supervised depth estimation techniques eliminate the need for accurate depth annotations, their performance degrades in environments with weak textures and variable lighting, leading to sparse reconstruction with invalid depth estimation. Depth completion using sparse depth maps can mitigate these issues and improve accuracy. Despite the advances in depth completion techniques in general fields, their application in endoscopy remains limited. To overcome these limitations, we propose EndoDDC, an endoscopy depth completion method that integrates images, sparse depth information with depth gradient features, and optimizes depth maps through a diffusion model, addressing the issues of weak texture and light reflection in endoscopic environments. Extensive experiments on two publicly available endoscopy datasets show that our approach outperforms state-of-the-art models in both depth accuracy and robustness. This demonstrates the potential of our method to reduce visual errors in complex endoscopic environments. Our code will be released at https://github.com/yinheng-lin/EndoDDC.

Keywords: Motion and Path Planning, Nonholonomic Motion Planning, Logistics
Abstract: The Dubins Moving Target Traveling Salesman Problem with Obstacles (Dubins MT-TSP-O) seeks an obstacle-free trajectory for an agent with a fixed speed and minimum turning radius that intercepts several moving targets. To tackle this NP-hard problem, we introduce the Lazy Iterated Random Generalized TSP (Lazy IRG) algorithm. Each iteration of Lazy IRG samples a set of possible interception points in space-time along the trajectories of the targets. Lazy IRG then manages the high computational cost of motion planning by alternating between two steps: first, it optimistically selects a sequence of interception points by solving a Generalized TSP (GTSP) assuming an obstacle-free world; second, it searches for obstacle-free trajectories between consecutive points in the sequence using an obstacle-aware RRT-Connect planner. If a trajectory is not found, Lazy IRG solves the GTSP again; otherwise, Lazy IRG enters its next iteration and samples new interception points. By deferring expensive collision-checking, our method efficiently focuses computational effort on the most promising solutions. Numerical results show that Lazy IRG finds significantly lower-cost solutions within a 1-minute time budget compared to the existing IRG-PGLNS algorithm.

Keywords: Mapping, RGB-D Perception, Deep Learning for Visual Perception
Abstract: Active scene reconstruction aims to autonomously recover the fine-grained appearance and structural details of a complex unknown scenes. Existing approaches based on 2D topological or voxel-based abstractions often scale poorly to large environments and rely heavily on handcrafted features and heuristic rules, limiting scalability and robustness. To address these challenges, using a RGB-D camera on a mobile robot, we present a graph-based planning framework by integrating skeleton-derived topology, Bird’s-Eye-View (BEV)-augmented graph inference, and offline Reinforcement Learning (RL) for policy optimization. The 3D skeleton graph captures full spatial connectivity, overcoming the limitations of 2D representations. BEV-augmented graph inference enriches node embeddings with semantic context, avoiding handcrafted feature design. The offline RL approach replaces heuristic planning with data-driven decision-making, while an additional Maximum Mean Discrepancy (MMD) term mitigates distributional shift before and after feature injection, improving stability. Extensive simulation results validate the efficacy of the proposed method. Real-world experiments demonstrate the zero-shot transferability of the learned policy, highlighting its potential for scalable, fine-grained scene reconstruction.

Keywords: Perception for Grasping and Manipulation, Deep Learning for Visual Perception
Abstract: Robotic manipulation requires explicit or implicit knowledge of the robot's joint positions. Precise proprioception is standard in high-quality industrial robots but is often unavailable in inexpensive robots operating in unstructured environments. In this paper, we ask: to what extent can a fast, single-pass regression architecture perform visual proprioception from a single external camera image, available even in the simplest manipulation settings? We explore several latent representations, including CNNs, VAEs, ViTs, and bags of uncalibrated fiducial markers, using fine-tuning techniques adapted to the limited data available. We evaluate the achievable accuracy through experiments on an inexpensive 6-DoF robot.

Keywords: Dual Arm Manipulation, Optimization and Optimal Control, Collision Avoidance
Abstract: The paper presents a new approach for constructing a library of optimal trajectories for two robotic manipulators, Two-Arm Optimal Control and Avoidance Library (TOCALib). The optimization takes into account kinodynamic and other constraints within the FROST framework. The novelty of the method lies in the consideration of collisions using the DCOL method, which allows obtaining symbolic expressions for assessing the presence of collisions and using them in gradient-based optimization control methods. The proposed approach is applicable for complex bimanual manipulations that require precision. In this paper we tested TOCALib on Mobile Aloha robot, as an example. The approach can be extended to other bimanual robots, as well as to gait control of bipedal robots. It can also be used to construct training data for machine learning tasks for manipulation.

Keywords: Service Robotics, Sensor-based Control, Deep Learning for Visual Perception
Abstract: Healthcare robotics has been gaining traction as a key area of research focused on enhancing human wellness. This paper explores the use of intelligent robots in the beauty industry, specifically within the context of photorejuvenation-based cosmetic dermatology, aimed at improving facial skin aesthetics. The beauty industry, traditionally labor-intensive, is experiencing a critical shortage of skilled beauticians, highlighting the opportunity for robotic technologies to meet this demand. However, integrating robots into cosmetic procedures presents unique challenges, particularly in tasks requiring high precision, such as laser pulse delivery and thermal dose management. This study addresses these challenges by introducing a deep learning approach for trajectory generation in laser path planning and a model-based control strategy for thermal dose regulation. Our empirical results demonstrate that the presented healthcare robots can deliver effective photorejuvenation treatments, suggesting a promising future for increased automation in cosmetic services.

Keywords: Reinforcement Learning, Legged Robots, Sensorimotor Learning
Abstract: Wheeled-legged hybrid robots have generated growing interest in the research community due to the need for more efficient and versatile locomotion. Most recent research has focused on active wheels, but passive wheeled systems have great potential in improving energy efficiency. However, skating remains highly complex due to the difficulties of balancing dynamic motion, managing wheel-ground interactions, achieving precise torque control for smooth rolling, and adapting to unpredictable terrain while maintaining stability. We present an end-to-end model-free reinforcement learning approach that enables quadrupedal robots to skate efficiently, achieving agile and robust locomotion on both flat and rough terrain. Our skating-specific policy and sim-to-real pipeline are validated on a physical quadruped across diverse terrains with varying roughness, slopes, and features, consistently demonstrating controlled and efficient traversal. The robot achieves velocities up to 1.5 m/s with a cost of transport 40.9% lower than the skating state of the art and 70.9% lower than standard legged locomotion. These results establish skating as a feasible and efficient alternative mode of urban locomotion for quadrupedal robots, setting a foundation for future wheeled-legged research.

Keywords: Visual Tracking
Abstract: Object pose tracking is a fundamental and essential task for robotics to perform tasks in the home and industrial settings. The most commonly used sensors to do so are RGB-D cameras, which can hit limitations in highly dynamic environments due to motion blur and frame-rate constraints. Event cameras have remarkable features such as high temporal resolution and low latency, which make them a potentially ideal vision sensors for object pose tracking at high speed. Even so, there are still only few works on 6D pose tracking with event cameras. In this work, we take advantage of the high temporal resolution and propose a method that uses both a propagation step fused with a pose correction strategy. Specifically, we use 6D object velocity obtained from event-based optical flow for pose propagation, after which, a template-based local pose correction module is utilized for pose correction. Our learning-free method has comparable performance to the state-of-the-art algorithms, and in some cases out performs them for fast-moving objects. The results indicate the potential for using event cameras in highly-dynamic scenarios where the use of deep network approaches are limited by low update rates.

Keywords: Autonomous Agents, Agent-Based Systems, AI-Based Methods
Abstract: Autonomous racing has become increasingly pop-ular in both academia and industry as a testbed for pushing general autonomous driving modules, such as perception, plan-ning, and control, to their limits. Although traditional control approaches can generate optimal control sequences at the edge of the racing vehicles’ physical controllability, they are highly sensitive to the accuracy of modeling parameters, such as tire model coefficients. Meanwhile, end-to-end learning methods are susceptible to distributional shifts, leading to unpredictable and irreversible failures. To address these challenges, this work introduces a physics-constrained imitation learning (PCIL) framework that effectively leverages the advantages of deep learning techniques and knowledge-driven strategies. Specifically, a fallback strategy would be automatically triggered when the vehicle states exceed predefined physical constraints. Meanwhile, the data from the knowledge-driven strategy will be augmented into the original dataset, and repeated re-training using an aggregated dataset could progressively improve PCIL. A series of simulations and real-world shadow testing are conducted at the Yas Marina circuit, and experimental results demonstrate superior performance compared to state-of-the-art methods, which suggests that it provides a promising solution for real-world autonomous racing.

Keywords: Semantic Scene Understanding, Data Sets for Robot Learning, Computer Vision for Automation
Abstract: Human assistance in robotics spans around several tasks such as navigation, object manipulation, and placement, where a key challenge is selecting target destinations that align with human intentions or preferences. We focus on this challenge in the context of Virtual Placement (VP), the task of identifying all plausible target locations given scene context and human-centric constraints. This differs from traditional placement tasks that typically focus on a single, predefined target location. The VP problem is complex, as it requires both global and local reasoning about the scene's geometry, semantics, and plausibility. To address this gap, we introduce {bf Assistant Placement Aria}, the first benchmark to explore diverse aspects of VP, including global, local, and human-centric constraints. It contains both synthetic and real indoor scenes annotated for three tasks: (i)~2D Panel Placement, (ii)~Sitting Suggestion, and (iii)~TV Placement. Each scene includes 2D images, a 3D point cloud, and a textual description of the objects within the scene. By contributing this benchmark, we aim to encourage further research in this underexplored and challenging field that is critically dependent on relevant data. We also evaluate several foundation models for object detection and segmentation on our benchmark.

Keywords: Transfer Learning, Learning from Demonstration, Representation Learning
Abstract: Given a demonstration, a robot should be able to generalize a skill to any object it encounters—but existing approaches to skill transfer often fail to adapt to objects with unfamiliar shapes. Motivated by examples of improved transfer from compositional modeling, we propose a method for improving transfer by decomposing objects into their constituent semantic parts. We leverage data-efficient generative shape models to accurately transfer interaction points from the parts of a demonstration object to a novel object. We autonomously construct an objective to optimize the alignment of those points on skill-relevant object parts. Our method generalizes to a wider range of object geometries than existing work, and achieves successful one-shot transfer for a range of skills and objects from a single demonstration, in both simulated and real environments.

Keywords: Performance Evaluation and Benchmarking, Intelligent Transportation Systems, Software Tools for Benchmarking and Reproducibility
Abstract: Planner evaluation in closed-loop simulation often uses rule-based traffic agents, whose simplistic and passive behavior can hide planner deficiencies and bias rankings. Widely used IDM agents simply follow a lead vehicle and cannot react to vehicles in adjacent lanes, hindering tests of complex interaction capabilities. We address this issue by integrating the state-of-the-art learned traffic agent model SMART into nuPlan. Thus, we are the first to evaluate planners under more realistic conditions and quantify how conclusions shift when narrowing the sim-to-real gap. Our analysis covers 14 recent planners and established baselines and shows that IDM-based simulation overestimates planning performance: nearly all scores deteriorate. In contrast, many planners interact better than previously assumed and even improve in multi-lane, interaction-heavy scenarios like lane changes or turns. Methods trained in closed-loop demonstrate the best and most stable driving performance. However, when reaching their limits in augmented edge-case scenarios, all learned planners degrade abruptly, whereas rule-based planners maintain reasonable basic behavior. Based on our results, we suggest SMART-reactive simulation as a new standard closed-loop benchmark in nuPlan and release the SMART agents as a drop-in alternative to IDM. Code, models, and scripts will be released upon acceptance.

Keywords: Swarm Robotics, Cooperating Robots, Reinforcement Learning
Abstract: In this work, we investigate cooperative strategies for an evader drone team of various sizes using multi-agent reinforcement learning in a multi-agent pursuit-evasion scenario. The objective of the evader team is to reach a goal with minimal velocity while not colliding with the pursuer team. The objective of the pursuer team is to defend the goal by catching evaders before they reach it. In this environment, we allow the pursuer to have superior control authority compared to the evader such that reaching the goal is challenging for the evader in a one-on-one scenario. The proposed strategy for an evader is to team up with an ally to lead pursuers into a collision with each other instead of intercepting the evader. We design policies using multi-agent proximal policy optimization, an actor-critic reinforcement learning method, and investigate how the learned strategy changes when we vary the size of the pursuer and evader teams. Finally, we demonstrate the learned policy's sim-to-real capabilities through a hardware demonstration.

Keywords: Motion and Path Planning, Representation Learning
Abstract: Trajectory prediction for traffic agents is critical for safe autonomous driving. However, achieving effective zero-shot generalization in previously unseen domains remains a significant challenge. Motivated by the consistent nature of kinematics across diverse domains, we aim to incorporate domain-invariant knowledge to enhance zero-shot trajectory prediction capabilities. The key challenges include: 1) effectively extracting domain-invariant scene representations, and 2) integrating invariant features with kinematic models to enable generalized predictions. To address these challenges, we propose a novel generalizable Physics-guided Causal Model (PCM), which comprises two core components: a Disentangled Scene Encoder, which adopts intervention-based disentanglement to extract domain-invariant features from scenes, and a CausalODE Decoder, which employs a causal attention mechanism to effectively integrate kinematic models with meaningful contextual information. Extensive experiments on real-world autonomous driving datasets demonstrate our method's superior zero-shot generalization performance in unseen cities, significantly outperforming competitive baselines. The source code is released at https://github.com/ZY-Zong/Physics-guided-Causal-Model.

Keywords: Human-Aware Motion Planning, Machine Learning for Robot Control, AI-Enabled Robotics
Abstract: Robust robot planning in dynamic, human-centric environments remains challenging due to multimodal uncertainty, the need for real-time adaptation, and safety requirements. Optimization-based planners enable explicit constraint handling but can be sensitive to initialization and struggle in dynamic settings. Learning-based planners capture multimodal solution spaces more naturally, but often lack reliable constraint satisfaction. In this paper, we introduce a unified generation-refinement framework that combines reward-guided conditional flow matching (CFM) with model predictive path integral (MPPI) control. Our key idea is a bidirectional information exchange between generation and optimization: reward-guided CFM produces diverse, informed trajectory priors for MPPI refinement, while the optimized MPPI trajectory warm-starts the next CFM generation step. Using autonomous social navigation as a motivating application, we demonstrate that the proposed approach improves the trade-off between safety, task performance, and computation time, while adapting to dynamic environments in real-time. The source code is publicly available at https://cfm-mppi.github.io.

Keywords: Deep Learning in Grasping and Manipulation, Imitation Learning, Learning from Demonstration
Abstract: A key requirement for generalist robots is compositional generalization—the ability to combine atomic skills to solve complex, long-horizon tasks. While prior work has primarily focused on synthesizing a planner that sequences pre-learned skills, robust execution of the individual skills themselves remains challenging, as visuomotor policies often fail under distribution shifts induced by scene composition. To address this, we introduce a scene graph-based representation that focuses on task-relevant objects and relations, thereby mitigating sensitivity to irrelevant variation. Building on this idea, we develop a scene-graph skill learning framework that integrates graph neural networks with diffusion-based imitation learning, and further combine “focused” scene-graph skills with a vision-language model (VLM) based task planner. Experiments in both simulation and real-world manipulation tasks demonstrate substantially higher success rates than state-of-the-art baselines, highlighting improved robustness and compositional generalization in long-horizon tasks.

Keywords: Constrained Motion Planning, Bimanual Manipulation, Representation Learning
Abstract: The objective of constrained motion planning is to connect start and goal configurations while satisfying task-specific constraints. Motion planning becomes inefficient or infeasible when the configurations lie in disconnected regions, known as essentially mutually disconnected components (EMDs). Constraints further restrict feasible space to a lower-dimensional submanifold, while redundancy introduces additional complexity because a single end-effector pose admits infinitely many inverse kinematic solutions that may form discrete self-motion manifolds. This paper addresses these challenges by learning a connectivity-aware representation for selecting start and goal configurations prior to planning. Joint configurations are embedded into a latent space through multi-scale manifold learning across neighborhood ranges from local to global, and clustering generates pseudo-labels that supervise a contrastive learning framework. The proposed framework provides a connectivity-aware measure that biases the selection of start and goal configurations in connected regions, avoiding EMDs and yielding higher success rates with reduced planning time. Experiments on various manipulation tasks showed that our method achieves 1.9 times higher success rates and reduces the planning time by a factor of 0.43 compared to baselines.

Keywords: Assembly, Reinforcement Learning, Continual Learning
Abstract: Simulation-based learning has enabled policies for precise, contact-rich tasks (e.g., robotic assembly) to reach high success rates (~80%) under high levels of observation noise and control error. Although such performance may be sufficient for research applications, it falls short of industry standards and makes policy chaining exceptionally brittle. A key limitation is the high variance in individual policy performance across diverse initial conditions. We introduce Refinery, an effective framework that bridges this performance gap, robustifying policy performance across initial conditions. We propose Bayesian Optimization-guided fine-tuning to improve individual policies, and Gaussian Mixture Model-based sampling during deployment to select initializations that maximize execution success. Using Refinery, we improve mean success rates by 10.98% over state-of-the-art methods in simulation-based learning for robotic assembly, reaching 91.51% in simulation and comparable performance in the real world. Furthermore, we demonstrate that these fine-tuned policies can be chained to accomplish long-horizon, multi-part assembly—successfully assembling up to 8 parts without requiring explicit multi-step training.

Keywords: Optimization and Optimal Control, Robust/Adaptive Control, Motion Control
Abstract: This paper provides a new repetitive control framework for robot manipulators with periodic reference and disturbance signals. We first take the inverse dynamics (ID) approach to a robot manipulator to transform its nonlinear input/output behavior into an equivalent linear time-invariant (LTI) system, for which the conventional repetitive control strategy is employed. To facilitate an optimal controller synthesis and an associated stability analysis, we next derive the so-called delay-feedback system. We then provide two linear matrix inequality (LMI)-based optimal controller synthesis procedures for minimizing the H_infty and the generalized H_2 norms from the disturbance to the tracking error, respectively. We next established operator-theoretic stability assertions in terms of the monodromy operator. In particular, a necessary and sufficient condition for the exponential stability of the delay-feedback system is derived for the case without external disturbances and we show that the delay-feedback system is input-to-state stable if it is exponentially stable. Finally, experiment comparisons are given to demonstrate the overall developed arguments.

Keywords: Wheeled Robots, Field Robots, Dynamics
Abstract: Modeling environmental forces remains a critical challenge in the design and control of robots operating on granular terrain. In pushing locomotion, propulsion is generated by displacing a large number of particles; however, the resulting terrain deformation complicates accurate real-time force prediction. Most existing resistive force models do not explicitly account for these deformation effects. To address this limitation, we develop a force model that incorporates motion-induced terrain deformation for pushing motions in granular media. A wheel lug is adopted as a representative element. We first investigate translational motion using the discrete element method (DEM) to characterize terrain deformation under different velocity directions. The analysis identifies dominant deformation patterns, which are embedded in the force model. Building on this analysis, we examine the rotational motion of a single lug through experiments, DEM simulations, and model predictions. The results demonstrate that the proposed model accurately captures force responses across varying velocity directions, exhibiting closer agreement with DEM and experiments than conventional approaches. This work advances real-time force modeling for robot-granular terrain interactions and highlights the potential of deformation-integrated models in deformable environments.

Keywords: Intelligent Transportation Systems, Cooperating Robots, Data Sets for Robot Learning
Abstract: We introduce M3CAD, a comprehensive benchmark designed to advance research in generic cooperative autonomous driving. M3CAD comprises 204 sequences with 30,000 frames. Each sequence includes data from multiple vehicles and different types of sensors, e.g., LiDAR point clouds, RGB images, and GPS/IMU, supporting a variety of autonomous driving tasks, including object detection and tracking, mapping, motion forecasting, occupancy prediction, and path planning. This rich multimodal setup enables M3CAD to support both single-vehicle and multi-vehicle cooperative autonomous driving research. To the best of our knowledge, M3CAD is the most complete benchmark specifically designed for cooperative, multi-task autonomous driving research. To test its effectiveness, we use M3CAD to evaluate both state-of-the-art single-vehicle and cooperative driving solutions, setting baseline performance results. Since most existing cooperative perception methods focus on merging features but often ignore network bandwidth requirements, we propose a new multi-level fusion approach which adaptively balances communication efficiency and perception accuracy based on the current network conditions. We release M3CAD, along with the baseline models and evaluation results, to support the development of robust cooperative autonomous driving systems. All resources will be made publicly available on our project webpage.

Keywords: Reinforcement Learning, Multi-Robot Systems, Task and Motion Planning
Abstract: Hierarchical Reinforcement Learning (HRL) is a potent paradigm for addressing long-horizon sequential decision-making in swarm confrontation. However, its strategic capabilities are often bottlenecked by high-level policies that struggle to reason over the dynamic, variable-sized observations of other agents. To address this, we introduce a novel decentralized HRL framework featuring a Transformer-based strategic policy. The Transformer's self-attention mechanism is uniquely suited to capture complex spatio-temporal relationships among a varying number of entities, enabling robust long-horizon task allocation. This high-level strategy is then translated by a low-level policy into collision-free navigation. In complex swarm confrontation scenarios, our method significantly outperforms established baselines, achieving win rates of up to 81%. Beyond this performance, the learned policies exhibit strong zero-shot generalization to larger swarms, offer decision-making interpretability via the attention mechanism, and foster the autonomous emergence of complex cooperative tactics. This work provides a blueprint for scalable, strategically sophisticated, and interpretable multi-agent systems.

Keywords: Deep Learning Methods, Deep Learning in Grasping and Manipulation, Perception for Grasping and Manipulation
Abstract: Enabling robots to execute novel manipulation tasks zero-shot is a central goal in robotics. Most existing methods assume in-distribution tasks or rely on fine-tuning with embodiment-matched data, limiting transfer across platforms. We present NovaFlow, an autonomous manipulation framework that converts a task description into an actionable plan for a target robot without any demonstrations. Given a task description, NovaFlow synthesizes a video using a video generation model and distills it into 3D actionable object flow using off-the-shelf perception modules. From the object flow, it computes relative poses for rigid objects and realizes them as robot actions via grasp proposals and trajectory optimization. For deformable objects, this flow serves as a tracking objective for model-based planning with a particle-based dynamics model. By decoupling task understanding from low-level control, NovaFlow naturally transfers across embodiments. We validate on rigid, articulated, and deformable object manipulation tasks using a table-top Franka arm and a Spot quadrupedal mobile robot, and achieve effective zero-shot execution without demonstrations or embodiment-specific training.

Keywords: Biologically-Inspired Robots, Neural and Fuzzy Control, Climbing Robots

Keywords: Multi-Robot Systems, Imitation Learning, Cooperating Robots
Abstract: As robots become more integrated in society, their ability to coordinate with other robots and humans on multi-modal tasks (those with multiple valid solutions) is crucial. Such behaviors can be learned from expert demonstrations via imitation learning (IL), but when expert demonstrations are multi-modal, standard IL approaches usually average across modes or collapse to a single mode, preventing effective coordination. Being inspired by diffusion models’ ability to capture complex multi-modal trajectory distributions in single-agent settings, we develop a diffusion-based framework for coordinated multi-modal behavior in multi-agent systems. However, existing multi-agent diffusion approaches typically require a centralized planner or explicit communication among agents. This assumption can fail in real-world scenarios where robots must operate independently or with agents like humans that they cannot directly communicate with. Therefore, we propose MIMIC-D, a Centralized Training, Decentralized Execution paradigm for multi-modal multi-agent IL via diffusion. We jointly train all agents' policies with full information, but execute using only local information to achieve implicit coordination. In simulation and hardware experiments, our method exhibits robust multi-modal coordination behavior in various tasks and environments, improving upon state-of-the-art baselines.

Keywords: Semantic Scene Understanding, Deep Learning for Visual Perception, Deep Learning Methods
Abstract: Semantic understanding of 3D scenes is essential for robots to operate effectively and safely in complex environments. Existing methods for semantic scene reconstruction and semantic-aware novel view synthesis often rely on dense multi-view inputs and require scene-specific optimization, limiting their practicality and scalability in real-world applications. To address these challenges, we propose SemGS, a feed-forward framework for reconstructing generalizable semantic fields from sparse image inputs. SemGS uses a dual-branch architecture to extract color and semantic features, where the two branches share shallow CNN layers, allowing semantic reasoning to leverage textural and structural cues in color appearance. We also incorporate a camera-aware attention mechanism into the feature extractor to explicitly model geometric relationships between camera viewpoints. The extracted features are decoded into dual-Gaussians that share geometric consistency while preserving branch-specific attributes, and further rasterized to synthesize semantic maps under novel viewpoints. Additionally, we introduce a regional smoothness loss to enhance semantic coherence. Experiments show that SemGS achieves state-of-the-art performance on benchmark datasets, while providing rapid inference and strong generalization capabilities across diverse synthetic and real-world scenarios.

Keywords: Deep Learning for Visual Perception, Recognition, Visual Learning
Abstract: With the rise of pre-trained models in the 3D point cloud domain for a wide range of real-world applications, adapting them to downstream tasks has become increasingly important. However, conventional full fine-tuning methods are computationally expensive and storage-intensive. Although prompt tuning has emerged as an efficient alternative, it often suffers from overfitting, thereby compromising generalization capability. To address this issue, we propose Prototypical Point-level Prompt Tuning (P3T), a parameter-efficient prompt tuning method designed for pre-trained 3D vision-language models (VLMs). P3T consists of two components: 1) Point Prompter, which generates instance-aware point-level prompts for the input point cloud, and 2) Text Prompter, which employs learnable prompts into the input text instead of hand-crafted ones. Since both prompters operate directly on input data, P3T enables task-specific adaptation of 3D VLMs without sacrificing generalizability. Furthermore, to enhance embedding space alignment, which is key to fine-tuning 3D VLMs, we introduce a prototypical loss that reduces intra-category variance. Extensive experiments demonstrate that our method matches or outperforms full fine-tuning in classification and few-shot learning, and further exhibits robust generalization under data shift in the cross-dataset setting. The code is available at https://github.com/gyjung975/P3T.

Keywords: Computer Vision for Automation, RGB-D Perception, Deep Learning Methods
Abstract: Underwater and in-air environments exhibit distinct imaging characteristics, which should be carefully considered and effectively exploited for accurate depth estimation. In this work, we analyze the effectiveness of wavelength-dependent attenuation for underwater depth estimation and show that it is helpful but insufficient to perform depth estimation independently. Therefore, we propose a fast underwater monocular depth estimation network that incorporates underwater light absorption difference (ULAD) as supplementary information. Compared with methods that rely solely on RGB input, the proposed approach provides more accurate depth predictions. In our network, RGB and ULAD features are extracted by MobileNetV4 and fused using FusionMamba, followed by decoding and refinement with a micro Vision Transformer. The network is trained on the USOD10K dataset and evaluated on both its test set and the FLSea dataset. Experimental results demonstrate that our method achieves more accurate depth estimation and higher efficiency compared with other lightweight networks. Furthermore, Compared with existing state-of-the-art fast underwater depth estimation methods, our network further reduces the number of parameters by 10% and improves inference speed by 43%.

Keywords: Vision-Based Navigation, Autonomous Vehicle Navigation, Deep Learning for Visual Perception
Abstract: Vision-Language Navigation (VLN) presents a unique challenge for Large Vision-Language Models (VLMs) due to their inherent architectural mismatch: VLMs are primarily pretrained on static, disembodied vision-language tasks, which fundamentally clash with the dynamic, embodied, and spatially-structured nature of navigation. Existing large-model-based methods often resort to converting rich visual and spatial information into text, forcing models to implicitly infer complex visual-topological relationships or limiting their global action capabilities. To bridge this gap, we propose TagaVLM (Topology-Aware Global Action reasoning), an end-to-end framework that explicitly injects topological structures into the VLM backbone. To introduce topological edge information, Spatial Topology Aware Residual Attention (STAR-Att) directly integrates it into the VLM’s self-attention mechanism, enabling intrinsic spatial reasoning while preserving pretrained knowledge. To enhance topological node information, an Interleaved Navigation Prompt strengthens node-level visual-text alignment. Finally, with the embedded topological graph, the model is capable of global action reasoning, allowing for robust path correction. On the R2R benchmark, TagaVLM achieves state-of-the-art performance among large-model-based methods, with a Success Rate (SR) of 51.09% and SPL of 47.18 in unseen environments, outperforming prior work by 3.39% in SR and 9.08 in SPL. This demonstrates that, for embodied spatial reasoning, targeted enhancements on smaller open-source VLMs can be more effective than brute-force model scaling. The code can be found on our project page: https://apex-bjut.github.io/Taga-VLM/.

Keywords: Motion and Path Planning, Semantic Scene Understanding, Field Robots
Abstract: Aerial imagery provides essential global context for autonomous navigation, enabling route planning at scales inaccessible to onboard sensing. We address the problem of generating global costmaps for long-range planning directly from satellite imagery when entities and mission-specific traversal rules are expressed in natural language at test time. This setting is challenging since mission requirements vary, terrain entities may be unknown at deployment, and user prompts often encode compositional traversal logic. Existing approaches relying on fixed ontologies and static cost mappings cannot accommodate such flexibility. While foundation models excel at language interpretation and open-vocabulary perception, no single model can simultaneously parse nuanced mission directives, locate arbitrary entities in large-scale imagery, and synthesize them into an executable cost function for planners. We therefore propose OVerSeeC, a zero-shot modular framework that decomposes the problem into Interpret–Locate–Synthesize: (i) an LLM extracts entities and ranked preferences, (ii) an open-vocabulary segmentation pipeline identifies these entities from high-resolution imagery, and (iii) the LLM uses user's natural language preferences and masks to synthesize executable costmap code. Empirically, OVerSeeC handles novel entities, respects ranked and compositional preferences, and produces routes consistent with human-drawn trajectories across diverse regions, demonstrating robustness to distribution shifts. This shows that modular composition of foundation models enables open-vocabulary, preference-aligned costmap generation for scalable, mission-adaptive global planning.

Keywords: Dexterous Manipulation, AI-Enabled Robotics, Reinforcement Learning
Abstract: In this work, we study Compositional Dexterous Functional Object Manipulation (CD-FOM): tasks such as aiming and actuating a spray bottle on a plant or a glue gun on wood, which require both actuating an object's internal mechanism and controlling its pose to apply the object's function to the environment. These tasks pose significant challenges for robots due to the demanding integration of semantic understanding—of the object's function, actuation mode, and application area—with intricate physical dexterity—to manage grasp stability, movement trajectory, and actuation. We introduce CoDex, a zero-demonstration framework that autonomously discovers CD-FOM manipulation strategies. CoDex uses vision–language models (VLMs) to infer semantic constraints from the task and scene. These constraints guide analytic constrained optimization to generate a short list of functional grasp candidates that can be efficiently refined with reinforcement learning to generate full grasp–move–actuate policies transferrable from simulation to the real world. We evaluate CoDex on a 7-DoF robot arm with a 16-DoF multi-fingered hand across six CD-FOM tasks involving previously unseen objects with internal mechanisms (spray bottles, hot glue guns, air dusters, flashlights, pepper grinders) and their application to unseen target objects, showcasing its ability to autonomously discover and execute complex, physically viable dexterous behaviors without human demonstrations. More information at https://robin-lab.cs.utexas.edu/CoDex/.

Keywords: Computer Vision for Transportation, Intelligent Transportation Systems, Deep Learning for Visual Perception
Abstract: Current state-of-the-art autonomous vehicles could face safety critical situations when their local sensors are occluded by large objects on the road nearby. Vehicle-to-vehicle (V2V) cooperative autonomous driving is proposed to address this problem. More recent work further adopts a new approach that applies Multimodal Large Language Models (MLLMs) for cooperative autonomous driving due to its potential multimodal understanding and reasoning abilities. However, graph-of-thoughts reasoning frameworks have not been considered for prior research on V2V cooperative autonomous driving. In this paper, we propose a novel graph-of-thoughts framework specifically designed for MLLM-based cooperative autonomous driving. Our graph-of-thoughts includes our proposed novel ideas of occlusion-aware perception and planning-aware prediction. We curate the V2V-GoT-QA dataset and develop the V2V-GoT model for training and testing the cooperative driving graph-of-thoughts. Our experimental results show that our proposed method outperforms other baselines in cooperative perception, prediction, and planning tasks. Our code and dataset are released to facilitate open-source research at https://eddyhkchiu.github.io/v2vgot.github.io/.

Keywords: Deep Learning Methods, Whole-Body Motion Planning and Control, Dual Arm Manipulation
Abstract: Recent Vision-Language-Action (VLA) models show potential to generalize across embodiments but struggle to quickly align with a new robot’s action space when high-quality demonstrations are scarce, especially for bipedal humanoids. We present TrajBooster, a cross-embodiment framework that leverages abundant wheeled-humanoid data to boost bipedal VLA. Our key idea is to use end-effector trajectories as a morphology-agnostic interface. TrajBooster (i) extracts 6D dual-arm end-effector trajectories from real-world wheeled humanoids, (ii) retargets them in simulation to Unitree G1 with a whole-body controller trained via a heuristic-enhanced harmonized online DAgger to lift low-dimensional trajectory references into feasible high-dimensional whole-body actions, and (iii) forms heterogeneous triplets that couple source vision/language with target humanoid-compatible actions to post-pre-train a VLA, followed by only 10 minutes of teleoperation data collection on the target humanoid domain. Deployed on Unitree G1, our policy achieves beyond-tabletop household tasks, enabling squatting, cross-height manipulation, and coordinated whole-body motion with markedly improved robustness and generalization. Results show that TrajBooster allows existing wheeled-humanoid data to efficiently strengthen bipedal humanoid VLA performance, reducing reliance on costly same-embodiment data while enhancing action space understanding and zero-shot skill transfer capabilities.

Keywords: Intelligent Transportation Systems, AI-Based Methods, Deep Learning for Visual Perception
Abstract: Autonomous driving has advanced significantly with the integration of large Vision-Language Models (VLMs), which excel in understanding and analyzing driving data. However, existing VLMs face challenges, particularly in terms of latency, which is crucial for real-time driving tasks. While shrinking the model size can reduce latency, it also limits the model's ability to handle both regular and corner cases effectively. To address this challenge, we propose the Curriculum Learning-based Knowledge Distillation (CLKD) framework. CLKD enhances student model performance through three key innovations: (1) integration of a Mixture-of-Experts (MoE) architecture to preserve model expressiveness; (2) Hardness-explored at Two Granularities (H2G), which dynamically identifies easy and difficult samples at both instance and feature levels; and (3) Progressive Release Distillation strategy that gradually reduces reliance on the teacher model, thereby fostering the student’s autonomy and improving its generalization capability in complex driving scenarios. In real-world data experiments, CLKD has achieved a twofold increase in speed compared to existing approaches while maintaining comparable performance.

Keywords: Mechanism Design, Motion Control, Robust/Adaptive Control
Abstract: This paper introduces a suspension-integrated tilting mechanism for narrow-track mobility robot platforms, offering a novel means of achieving commanded body tilt without departing from conventional suspension layouts. The architecture provides a two-degree-of-freedom roll path with inherent passive self-centering, thereby reconciling mechanical simplicity with enhanced motion capability. A linearized dynamic model forms the foundation for a dual-layer control scheme, consisting of a reference generator and a high-bandwidth tracking loop. The proposed approach is rigorously evaluated through hardware-in-the-loop simulations that combine a real-time driving simulator with a physical hardware bench. Relative to a non-tilting baseline, the platform demonstrates substantial reductions in perceived lateral acceleration and lateral load transfer, signifying improved ride comfort and rollover stability. These findings establish suspension-integrated tilting with model-based control as a compelling pathway toward safe and stable next-generation mobility robot platforms.

Keywords: Datasets for Human Motion
Abstract: Robots are increasingly being deployed in public spaces such as shopping malls, sidewalks, and hospitals, where safe and socially aware navigation depends on anticipating how pedestrians respond to their presence. However, existing datasets rarely capture the full spectrum of robot-induced reactions, e.g., avoidance, neutrality, attraction, which limits progress in modeling these interactions. In this paper, we present the Pedestrian-Robot Interaction (PeRoI) dataset that captures pedestrian motions categorized into attraction, neutrality, and repulsion across two outdoor sites under three controlled conditions: no robot present, with stationary robot, and with moving robot. This design explicitly reveals how pedestrian behavior varies across robot contexts, and we provide qualitative and quantitative comparisons to established state-of-the-art datasets. Building on these data, we propose the Neural Robot Social Force Model (NeuRoSFM), an extension of the Social Force Model that integrates neural networks to augment inter-human dynamics with learned components and explicit robot-induced forces to better predict pedestrian motion in vicinity of robots. We evaluate NeuRoSFM by generating trajectories on multiple real-world datasets. The results demonstrate improved modeling of pedestrian-robot interactions, leading to better prediction accuracy, and highlight the value of our dataset and method for advancing socially aware navigation strategies in human-centered environments.

Keywords: Collision Avoidance, Robot Safety, Reinforcement Learning

Keywords: Compliant Joints and Mechanisms, Force Control, Mechanism Design
Abstract: Accurate torque estimation in robotic actuators with harmonic drives is challenging due to nonlinear hysteresis and efficiency losses, often necessitating external torque sensors. This paper presents a learning-based torque estimation method that leverages encoder-derived features and mechanical compliance to enhance estimation accuracy without additional sensors. An actuator design incorporating a compliant helical tube provides deformation features that are effectively modeled using a Long Short-Term Memory (LSTM) network. Unlike conventional calibration or parametric approaches, the proposed framework captures nonlinear, history-dependent behaviors across varying operating conditions. Experimental evaluations demonstrate that compliant tubes significantly improve estimation accuracy compared with designs using stiffer or even rigid tubes, enabling more robust generalization under different torques, impedance modes, and stiffness levels. These results highlight the importance of co-designing actuator compliance and deep learning models to achieve reliable and compact torque estimation for harmonic drive actuators.

Keywords: Legged Robots, Force and Tactile Sensing
Abstract: Legged robots rely on accurate ground interaction awareness to traverse variable terrains, such as slippery surfaces. Existing slip detection methods often rely on kinematics and proprioception, which lack the sensitivity to detect early-stage slips that occur prior to catastrophic instability. Thus, this paper presents SlipSense, a novel framework for online force-based slip detection using a custom lightweight sensorized foot for quadrupeds to detect slip. The framework integrates a multimodal sensor design with a LSTM-based model to infer ground reaction forces and detect slip-indicative anomalies during locomotion. The proposed framework is deployed on a Unitree Go1 quadruped to demonstrate blind online slip detection over a slippery terrain. Our method detects early-stage slips down to an average displacement of 24.1 ± 6.4mm with an overall accuracy of 85.9%. This represents a 3.3-fold finer detection resolution and a 24% relative accuracy improvement over a standard kinematic baseline that uses foot velocity inferred through state estimation. The work in this paper serves as a foundation for force-aware gait adaptation in legged robotic locomotion, allowing future controllers to estimate terrain friction and adjust constraints, thus improving the overall stability of the system.

Keywords: Machine Learning for Robot Control, Dynamics, Robust/Adaptive Control
Abstract: This paper presents a novel Koopman operator formulation for Euler–Lagrangian dynamics that employs an implicit generalized momentum-based state space representation, which decouples a known linear actuation channel from state-dependent dynamics and makes the system more amenable to linear Koopman modeling. By leveraging this structural separation, the proposed formulation only requires to learn the unactuated dynamics rather than the complete actuation-dependent system, thereby significantly reducing the number of learnable parameters, improving data efficiency, and lowering overall model complexity. In contrast, conventional explicit formulations inherently couple inputs with the state-dependent terms in a nonlinear manner, making them more suitable for bilinear Koopman models, which are more computationally expensive to train and deploy. Notably, the proposed scheme enables the formulation of linear models that achieve superior prediction performance compared to conventional bilinear models while remaining substantially more efficient. To realize this framework, we present two neural network architectures that construct Koopman embeddings from actuated or unactuated data, enabling flexible and efficient modeling across different tasks. Robustness is ensured through the integration of a linear Generalized Extended State Observer (GESO), which explicitly estimates disturbances and compensates for them in real-time. The combined momentum-based Koopman and GESO framework is validated through comprehensive trajectory tracking simulations and experiments on robotic manipulators, demonstrating superior accuracy, robustness, and learning efficiency relative to state-of-the-art alternatives.

Keywords: Intelligent Transportation Systems, Deep Learning for Visual Perception, Object Detection, Segmentation and Categorization
Abstract: Self-supervised pre-training with masked autoencoders has shown promise for 3D perception, yet most approaches treat LiDAR point clouds in a geometry-agnostic manner. In this paper, we introduce Re-MAE, a geometry-aware self-supervised learning framework for LiDAR-based 3D object detection that explicitly encodes core properties of LiDAR point clouds: occlusion, distance-driven sparsity, and occupied-empty voxel structure. Re-MAE rethinks the geometric characteristics of LiDAR point clouds from the perspectives of "what to learn" and "how to learn", and introduces three components: (i) Geometry-Aware Masking, which realistically simulates occlusions in LiDAR scans and enables learning complete object representations from partial observations; (ii) Reconstruction-Contextual BCE loss, which effectively guides a multi-scale occupancy prediction task to mitigate distance-dependent point sparsity and the strong occupied-empty voxel imbalance, improving detection of both large vehicles and small, distant pedestrians; and (iii) Realistic Object Augmentation, a label-free foreground augmentation strategy that promotes object-centric representation learning and yields consistent gains across categories. Experiments on ONCE and Waymo Open Dataset validate the effectiveness of Re-MAE, delivering 2.83 mAP and 1.53 L2 mAP respectively over baselines. These results demonstrate that explicitly incorporating the geometric characteristics of LiDAR point clouds enhances the effectiveness of self-supervised learning. The code will be released.

Keywords: Sensor-based Control, Computer Vision for Manufacturing, Engineering for Robotic Systems
Abstract: Robotic sole deburring is a key, yet underexplored, challenge in footwear automation, where the deformable nature of rubber, variability of burrs, and diversity of sole geometries make automation difficult. Existing deburring approaches typically rely on CAD models or large training datasets, and often lack the ability to adapt online during execution. This paper presents a CAD-free, vision-guided framework for robotic deburring of shoe soles that integrates: (i) defect detection using the Segment Anything Model 2 without sole-specific training; (ii) motion planning for burr removal; and (iii) motion execution combining Forward Dynamics Compliance Control with online vision-based path tracking. The framework was validated on a UR5e robot equipped with a custom vacuum gripper. Results demonstrate a 95% success rate across soles of varying sizes, colors, and shapes. By eliminating CAD dependence, ensuring robust online correction, and maintaining compatibility with existing industrial deburring machines, this work provides a scalable step toward robotic finishing solutions in footwear manufacturing.

Keywords: Object Detection, Segmentation and Categorization, Deep Learning for Visual Perception, Computer Vision for Transportation
Abstract: Classical autonomous driving systems connect perception and prediction modules via hand-crafted bounding-box interfaces, limiting information flow and propagating errors to downstream tasks. Recent research aims to develop end-to-end models that jointly address perception and prediction; however, they often fail to fully exploit the synergy between appearance and motion cues, relying mainly on short-term visual features. We follow the idea of ''looking backward to look forward'', and propose MASAR, a novel fully differentiable framework for joint 3D detection and trajectory forecasting compatible with any transformer-based 3D detector. MASAR employs an object-centric spatio-temporal mechanism that jointly encodes appearance and motion features. By predicting past trajectories and refining them using guidance from appearance cues, MASAR captures long-term temporal dependencies that enhance future trajectory forecasting. Experiments conducted on the nuScenes dataset demonstrate MASAR's effectiveness, showing improvements of over 20% in minADE and minFDE while maintaining robust detection performance. Code and models are available at link.

Keywords: Micro/Nano Robots, Automation at Micro-Nano Scales, Medical Robots and Systems
Abstract: Magnetic microrobots hold great promise for biomedical applications. However, achieving flexible magnetic field adjustment with a magnetic actuation system (MAS) to actuate diverse microrobots remains a significant challenge. In this work, we propose an Electromagnetic-Permanent Magnet Actuation (EPMA) system that generates controllable magnetic field variations to enable microrobot actuation for diverse tasks, including microrobotic actuation, microswarm pattern transformation and targeted delivery. Automatic ellipsoid calibration of the Hall sensors enables real-time magnetic field orientation measurement with an error under 3◦. Experimental results demonstrate the microrobot's actuation performance in four distinct scenarios, with a rotation frequency of 0.5 Hz. Furthermore, by adjusting the dynamic magnetic field, we achieve microswarm pattern reconfiguration under static conditions as well as targeted delivery in fluidic environments at a flow speed of 52 mm/s and a rotation frequency of 4 Hz. This study presents a hybrid MAS for the microrobotic actuation in diverse environments by controllable dynamic magnetic fields.

Keywords: Integrated Planning and Learning, Motion and Path Planning, Search and Rescue Robots
Abstract: Autonomous exploration in structured and complex indoor environments remains a challenging task, as existing methods often struggle to appropriately model unobserved space and plan globally efficient paths. To address these limitations, we propose GUIDE, a novel exploration framework that synergistically combines global graph inference with diffusionbased decision-making. We introduce a region-evaluation global graph representation that integrates both observed environmental data and predictions of unexplored areas, enhanced by a region-level evaluation mechanism to prioritize reliable structural inferences while discounting uncertain predictions. Building upon this enriched representation, a diffusion policy network generates stable, foresighted action sequences with significantly reduced denoising steps. Extensive simulations and real-world deployments demonstrate that GUIDE consistently outperforms state-of-the-art methods, achieving up to 18.3% faster coverage completion and a 34.9% reduction in redundant movements.

Keywords: Domestic Robotics
Abstract: Robots operating in everyday environments must understand fine-grained human actions, intentions, and contextual cues from broad views where people occupy only small regions, a capability unmet by current systems. While open-vocabulary action recognition methods remain limited to assigning predefined labels, and vision-language models (VLMs) face an inherent trade-off between informational richness and factual fidelity in their outputs, neither approach achieves the deep semantic interpretation required for reliable human-robot interaction. We propose Gold Points Sniper (GPS), a novel framework that empowers lightweight VLMs with self-guided multimodal reasoning capabilities for fine-grained human action understanding. Our approach comprises three key modules: Gold Points Extractor trains VLMs to identify critical action-relevant details, Selective Socratic Ques- tioner validates and refines these details through selective self-questioning, and Semantic Entailment Evaluator quantitatively assesses factual consistency using semantic entailment classification. Extensive experiments on our curated instruction-tuning dataset based on the CAP benchmark demonstrate that GPS-enhanced lightweight VLMs achieve substantial performance improvements, with some models reaching performance comparable to proprietary GPT-4o while maintaining superior factual accuracy. Our work establishes a reliable foundation for fine-grained action understanding in domestic robotics, enabling robots to safely interpret human behavior through information-dense yet factually grounded descriptions. Source code, training configurations, annotation prompts, and dataset details are released at https://github.com/Haodi-Liu/GPS-Gold-Point-Sniper.

Keywords: Force and Tactile Sensing, Dexterous Manipulation, Telerobotics and Teleoperation
Abstract: Deformable objects often appear in unstructured configurations. Tracing deformable objects helps bringing them into extended states and facilitating the downstream manipulation tasks. Due to the requirements for object-specific modeling or sim-to-real transfer, existing tracing methods either lack generalizability across different categories of deformable objects or struggle to complete tasks reliably in real world. To address this, we propose a novel visual-tactile imitation learning method to achieve one-dimensional (1D) and two-dimensional (2D) deformable object tracing with a unified model. Our method is designed from both local and global perspectives based on visual and tactile sensing. Locally, we introduce a weighted loss that emphasizes actions maintaining contact near the center of the tactile image, improving fine-grained adjustment. Globally, we propose a tracing task loss that helps the policy to regulate task progression. On the hardware side, to compensate for the limited features extracted from visual information, we integrate tactile sensing into a low-cost teleoperation system considering both the teleoperator and the robot. Extensive ablation and comparative experiments on diverse 1D and 2D deformable objects demonstrate the effectiveness of our approach, achieving an average success rate of 80% on seen objects and 65% on unseen objects. Demos, code and datasets are available at https://sites.google.com/view/vitac-tracing.

Keywords: Soft Robot Materials and Design
Abstract: Autonomous and mobile soft robots require internal oscillators, similar to a biological heart, to generate rhythmic motions. However, existing soft oscillators typically have fixed operational parameters and suffer from an inherent coupling between control input and power output, limiting their versatility and adaptability. This paper addresses this challenge by introducing a new design paradigm: a soft, multi-port, bistable oscillator whose core nonlinear energy landscape can be continuously and actively tuned on-the-fly. Our approach, based on mechanically reconfiguring the physical constraints of a bistable elastomeric structure, achieves a decoupling of kinematics (frequency) from dynamics (output pressure). We demonstrate this principle in two modes: first, active programming, where we continuously modulate the oscillator’s coupled frequency-amplitude relationship in real-time under a constant power input. Secondly, we demonstrate passive adaptation, where an autonomous walker powered by our oscillator exhibits physical intelligence. By physically interacting with a confined environment, the walker autonomously and instantaneously adapts its gait from a low-frequency, large-amplitude mode to a high-frequency, small-amplitude mode. This work provides a new pathway for creating adaptive, intelligent soft robots that can autonomously respond to their physical world without any electronic computation.

Keywords: Task and Motion Planning, Task Planning
Abstract: Task and Motion Planning (TAMP) integrates high-level task planning with low-level motion feasibility, but existing methods are costly in long-horizon problems due to excessive motion sampling. While LLMs provide commonsense priors, they lack 3D spatial reasoning and cannot ensure geometric or dynamic feasibility. We propose a kinodynamic TAMP planner based on a hybrid state tree that uniformly represents symbolic and numeric states during planning, enabling task and motion decisions to be jointly decided. Kinodynamic constraints embedded in the TAMP problem are verified by an off-the-shelf motion planner and physics simulator, and a VLM guides exploring a TAMP solution and backtracks the search based on visual rendering of the states. Experiments on the simulated domains and in the real world show 32.14% ∼ 1166.67% increased average success rates compared to traditional and LLM-based TAMP planners and reduced planning time on complex problems, with ablations further highlighting the benefits of VLM backtracking. More details are available at https://graphics.ewha.ac.kr/kinodynamicTAMP/.

Keywords: Calibration and Identification, Probability and Statistical Methods, Sensor Fusion
Abstract: Reliable state estimation hinges on accurate specification of sensor noise covariances, which weigh heterogeneous measurements. In practice, these covariances are difficult to identify due to environmental variability, front-end preprocessing, and other reasons. We address this by formulating noise covariance estimation as a bilevel optimization that, from a Bayesian perspective, factorizes the joint likelihood of so-called odometry and supervisory measurements, thereby balancing information utilization with computational efficiency. The factorization converts the nested Bayesian dependency into a chain structure, enabling efficient parallel computation: at the lower level, an invariant extended Kalman filter with state augmentation estimates trajectories, while a derivative filter computes analytical gradients in parallel for upper-level gradient updates. The upper level refines the covariance to guide the lower-level estimation. Experiments on synthetic and real-world datasets show that our method achieves higher efficiency than existing baselines.

Keywords: AI-Based Methods, Localization, Deep Learning for Visual Perception
Abstract: Precise localization with respect to a set of objects of interest enables mobile robots to perform various tasks. With the rise of edge devices capable of deploying deep neural networks (DNNs) for real-time inference, it stands to reason to use artificial intelligence (AI) for the extraction of object-specific, semantic information from raw image data, such as the object class and the relative six degrees of freedom (6-DoF) pose. However, fusing such AI-based measurements in an Extended Kalman Filter (EKF) requires quantifying the DNNs' uncertainty and outlier rejection capabilities.

This paper presents the benefits of reformulating the measurement equation in AI-based, object-relative state estimation. By deriving an EKF using the direct object-relative pose measurement, we can decouple the position and rotation measurements, thus limiting the influence of erroneous rotation measurements and allowing partial measurement rejection. Furthermore, we investigate the performance and consistency improvements for state estimators provided by replacing the fixed measurement covariance matrix of the 6-DoF object-relative pose measurements with the predicted aleatoric uncertainty of the DNN.

Keywords: Robotics and Automation in Agriculture and Forestry, Agricultural Automation, Force and Tactile Sensing
Abstract: Robotic navigation in dense, cluttered environments such as agricultural canopies presents significant challenges due to physical and visual occlusion caused by leaves and branches. Traditional vision-based or model-dependent approaches often fail in these settings, where physical interaction without damaging foliage and branches is necessary to reach a target. We present a novel reactive controller that enables safe navigation for a robotic arm in a contact-rich, cluttered, deformable environment using end-effector position and real-time tactile feedback. Our proposed framework's interaction strategy is based on a trade-off between minimizing disturbance by maneuvering around obstacles and pushing through them to move towards the target. We show that over 35 trials in 3 experimental plant setups with an occluded target, the proposed controller successfully reached the target in all trials without breaking any branch, and outperformed the established control strategy for dense foliage in reliability and adaptability. This work lays the foundation for safe, adaptive interaction in cluttered, contact-rich deformable environments, enabling future agricultural tasks such as pruning and harvesting in plant canopies.

Keywords: Force and Tactile Sensing
Abstract: Vision-based tactile sensors (VBTS) face an inherent trade-off in tactile skin design. Opaque ink markers enable accurate force and tangential displacement estimation but occlude geometric features essential for object and texture classification. Conversely, markerless skins preserve surface details yet provide limited capability for tangential motion estimation. Existing approaches, including UV illumination and learning-based virtual marker transfer, increase hardware complexity or computational cost. We present a novel tactile skin with translucent, tinted markers balancing the modes of marker and markerless for VBTS. This design enables concurrent tangential displacement tracking, force estimation, and preservation of surface geometry. It integrates directly with the GelSight sensor family, requiring no additional hardware and minimal software modification. Experimental evaluation demonstrates that translucent skin improves overall sensing performance relative to both opaque-marker and markerless configurations. It achieves 99.17% accuracy in object classification, 93.51% in texture classification, 97% point retention in tangential displacement tracking, and a 66% reduction in total force error. These results indicate that translucent skin substantially mitigate the traditional trade-off between marker-based and markerless tactile sensing, thereby expanding the applicability of multi-modal VBTS in tactile robotics.

Keywords: Medical Robots and Systems, Machine Learning for Robot Control, Optimization and Optimal Control
Abstract: This paper introduces a learning-based control framework for a soft robotic actuator system designed to modulate intracranial pressure (ICP) waveforms, which is essential for studying cerebrospinal fluid dynamics and pathological processes underlying neurological disorders. A two-layer framework is proposed to safely achieve a desired ICP waveform modulation. First, a model predictive controller (MPC) with a disturbance observer is used for offset-free tracking of the system’s motor position reference trajectory under safety constraints. Second, to address the unknown nonlinear dependence of ICP on the motor position, we employ a Bayesian optimization (BO) algorithm used for online learning of a motor position reference trajectory that yields the desired ICP modulation. The framework is experimentally validated using a test bench with a brain phantom that replicates realistic ICP dynamics in vitro. Compared to a previously employed proportional-integral-derivative controller, the MPC reduces mean and maximum motor position reference tracking errors by 83 % and 73 %, respectively. In less than 20 iterations, the BO algorithm learns a motor position reference trajectory that yields an ICP waveform with the desired mean and amplitude.

Keywords: Multi-Robot Systems, Cooperating Robots, Agent-Based Systems
Abstract: This paper presents a novel control strategy for multi-agent shepherding of non-cohesive targets in obstacle-rich environments. Unlike previous approaches that assume cohesive flocking behavior, our method handles targets that interact only with nearby herders through repulsive forces and exhibit no inter-target coordination. Each herder employs a hybrid control policy that combines direct goal-oriented steering with obstacle-tangent maneuvering, enabling targets to circumnavigate obstacles while being guided toward a goal region. The herder dynamics integrate three key behaviors: return-to-goal motion when idle, target steering with adaptive directional control, and obstacle avoidance using both normal and tangential force components. Numerical simulations demonstrate superior performance compared to existing shepherding methods, achieving higher target confinement rates in cluttered environments. Experimental validation using TurtleBot4 herders and Osoyoo target robots in an indoor arena confirms the practical effectiveness of the proposed approach.

Keywords: Human Performance Augmentation, Brain-Machine Interfaces, Human Factors and Human-in-the-Loop
Abstract: Efficient brain functional training with rehabilitation robots has been an important and challenging topic in the human-machine interaction (HMI) field. Adjusting the interaction and gaming behaviors between human and machine to effectively activate the brain’s functional behavior is still a substantial challenge. In this paper, we take the visual-attention training as an example, and propose a novel human-machine co-gaming interaction framework by integrating a dual-task gaming paradigm and a human–machine gaming strategy. It has a remarkable capability of effectively utilizing the gaming characteristics of HMI behaviors and tasks, to effectively and precisely activate the human’s active attention and passive attention for training. Specifically, we design a gaze-driven dual-task gaming paradigm to co-activate the active and passive attention-network competition for systematically engaging human visual-attention allocation and training. We further develop a reinforcement-learning-based human–machine gaming strategy to adjust the task parameters for improving the attention training efficiency. Consequently, we conduct an experiment study with 8 healthy participants, by jointly analyzing participants’ EEG and eye-tracking data through the training process. Results show that our method can achieve improvement of brain engagement by an average of 15.6% over the widely-employed staircase strategy.

Keywords: Aerial Systems: Applications, Aerial Systems: Mechanics and Control, Aerial Systems: Perception and Autonomy
Abstract: Aerial manipulation requires force-aware capabilities to enable safe and effective grasping and physical interaction. Previous works often rely on heavy, expensive force sensors unsuitable for typical quadrotor platforms, or perform grasping without force feedback, risking damage to fragile objects. To address these limitations, we propose a novel force-aware grasping framework incorporating six low-cost, sensitive skin-like tactile sensors. We introduce a magnetic-based tactile sensing module that provides high-precision three-dimensional force measurements. We eliminate geomagnetic interference through a reference Hall sensor and simplify the calibration process compared to previous work. The proposed framework enables precise force-aware grasping control, allowing safe manipulation of fragile objects and real-time weight measurement of grasped items. The system is validated through comprehensive real-world experiments, including balloon grasping, dynamic load variation tests, and ablation studies, demonstrating its effectiveness in various aerial manipulation scenarios. Our approach achieves fully onboard operation without external motion capture systems, significantly enhancing the practicality of force-sensitive aerial manipulation. The supplementary video is available at: https://www.youtube.com/watch?v=mbcZkrJEf1I.

Keywords: Multi-Robot Systems, Path Planning for Multiple Mobile Robots or Agents, Reinforcement Learning
Abstract: This paper investigates the multi-robot efficient search (MuRES) problem in uncertain topological networks. One unique characteristic of the studied problem is that the topology of the underlying network is uncertain, posing great challenges to canonical MuRES solutions which presumes a fixed network topology. To address the challenge, this paper proposes the STructure-Adaptive Graph-Encoded policy gradient (STAGE) algorithm for moving target search. STAGE comprises two main components: (1) the bi-scale graph attention network (GAT) encoder, which fuses a k-hop local GAT with a distance-augmented long-range GAT to enable the encoder to capture both local and long-range network structural changes; and (2) the entropy-regularized counterfactual policy gradient module, which employs a structure-aware centralized critic to estimate both the team returns and the network structure information, and train the decentralized actors via counterfactual marginalization with entropy regularization. Extensive simulation results and physical experiment demonstrate the feasibility and superiority of STAGE for solving MuRES in uncertain topological environments.

Keywords: Medical Robots and Systems, Surgical Robotics: Planning, Computer Vision for Automation
Abstract: Retinal endolaser photocoagulation (REPC) is a repetitive intraocular surgical procedure that could greatly benefit from automation and distance-based control, improving both efficiency and safety. This work presents a robotic system designed for automated REPC, utilizing instrument-integrated optical coherence tomography (iiOCT) to facilitate real-time distance measurements. The system employs intraoperative spherical and ellipsoidal retinal models to convert 2D laser patterns into 3D arrangements, which are further refined through a control loop that incorporates online feedback. Ex vivo experiments in porcine eyes demonstrated clinical-level accuracy, with lateral and axial errors of 44 micrometers and 29 micrometers, respectively. Additionally, the proposed mapping technique produced patterns with greater equidistance than baseline methods. This system showcases the potential to automate repetitive surgical tasks while maintaining the surgeon's control over critical decision-making processes in ophthalmic surgery.

Keywords: Touch in HRI, Social HRI
Abstract: Physical touch, such as handshakes, plays a critical role in human-robot interaction (HRI), influencing perceived naturalness and social presence. This study investigates how arm compliance, hand grip strength, and motion synchrony jointly affect the subjective quality of a human-robot handshake. We implemented a fully actuated, tactile-sensorized humanoid hand mounted on a manipulator arm and designed a compliant, oscillatory handshake controller with adaptive synchronization. Sixteen participants experienced handshakes across a 2x2x2 factorial design, varying arm compliance, grip strength, and synchrony. Objective kinematic analysis revealed significant main and interaction effects across all factors. At the same time, subjective ratings showed clear preferences for a weaker grip and greater arm compliance, with synchrony exerting minimal influence on perceived naturalness. These results highlight a perceptual hierarchy in HRI: foundational haptic properties exert the strongest influence on user experience, while advanced kinematic adjustments have limited impact when basic comfort is lacking. This insight provides concrete guidance for designing robotic handshakes that feel more human-like and pleasant.

Keywords: Legged Robots, Whole-Body Motion Planning and Control, Optimization and Optimal Control
Abstract: We demonstrate the surprising real-world effectiveness of a very simple approach to whole-body model- predictive control (MPC) of quadruped and humanoid robots: the iterative linear-quadratic regulator (iLQR) algorithm with MuJoCo dynamics and finite-difference approximated derivatives. Building upon the previous success of model-based behavior synthesis and control of locomotion and manipulation tasks with MuJoCo in simulation, we show that these policies can easily generalize to the real world with few sim-to-real considerations. Our baseline method achieves real-time MPC while leveraging whole-body dynamics collision detection on a variety of hardware experiments, including dynamic quadruped locomotion, quadruped walking on two legs, and full-sized humanoid bipedal locomotion. Additionally, our GUI system enables users to interactively update robot behavior in real-time on the robot hardware, making task-specific objective parameter tuning easy and intuitive. Our code is available at:https://johnzhang3.github.io/mujoco_ilqr

Keywords: Autonomous Vehicle Navigation, Intelligent Transportation Systems
Abstract: End-to-end autonomous driving has been greatly advanced in recent years. However, most of existing work focuses on small vehicles (e.g., cars). Driving articulated trucks, such as tractor-trailers, still remains less being explored. The underactuated nature and extended wheelbase of tractor-trailers pose considerable driving challenges, especially when navigating narrow roads. For example, when a left-hand-drive tractor-trailer makes a right turn on a two-way two-lane narrow road, the tractor usually needs to encroach some spaces in the opposing lane. Otherwise, the trailer may have insufficient spaces to turn right and strike curbside objects. To provide a solution to this problem, we employ deep reinforcement learning to train an end-to-end autonomous driving policy with a trailer-aware reward function. Through planar rigid-body kinematics analysis, we locate the reference points on the tractor and the trailer. We also build a tractor-trailer model for CARLA. Experimental results demonstrate the effectiveness and superiority of our method in CARLA.

Keywords: SLAM, Optimization and Optimal Control, Planning under Uncertainty
Abstract: Graph-based SLAM models robot poses as vertices and relative-pose measurements (odometry and loop-closures) as edges. Odometry edges are always kept to preserve connectivity, while loop-closure edges reduce drift but cannot all be stored due to memory or computation limits. Our challenge is to decide online which closures to keep under a strict budget, when the full set of measurements cannot be stored or centralized. Prior work instead addresses an offline problem that assumes access to the complete pose graph and optimizes a log-determinant (D-optimality) surrogate. In the online regime, an additional difficulty arises because the odometry backbone grows over time and the utility of each loop-closure changes as the graph evolves. We formulate this problem as streaming submodular maximization with a time-varying log-determinant objective. We propose a one-pass preemptive greedy policy that operates with exactly k memory slots for loop-closures. We show that, under arbitrary arrival order, it achieves a uniform constant-factor guarantee on the log-determinant improvement beyond an odometry-only baseline, relative to the hindsight-optimal size-k solution. On benchmark data, the proposed method closely matches offline greedy despite the conservative bound, showing that principled streaming selection can recover most of the benefit of loop-closures while respecting resource limits.

Keywords: Deep Learning for Visual Perception, Visual Learning, RGB-D Perception
Abstract: Understanding object affordances is essential for enabling robots to perform purposeful and fine-grained interactions in diverse and unstructured environments. However, existing approaches either rely on retrieval, which is fragile due to sparsity and coverage gaps, or on large-scale models, which frequently mislocalize contact points and mispredict post-contact actions when applied to unseen categories, thereby hindering robust generalization. We introduce Retrieval-Augmented Affordance Prediction (RAAP), a framework that unifies affordance retrieval with alignment-based learning. By decoupling static contact localization and dynamic action direction, RAAP transfers contact points via dense correspondence and predicts action directions through a retrieval-augmented alignment model that consolidates multiple references with dual-weighted attention. Trained on compact subsets of DROID and HOI4D with as few as tens of samples per task, RAAP achieves consistent performance across unseen objects and categories, and enables zero-shot robotic manipulation in both simulation and the real world. Project website: https://github.com/SEU-VIPGroup/RAAP.

Keywords: Force and Tactile Sensing, Simulation and Animation, Deep Learning Methods
Abstract: Simulating optical tactile sensors presents significant challenges due to their high deformability and intricate optical properties. To address these issues and enable a physically accurate simulation, we propose DOT-Sim: Differentiable Optical Tactile Simulation. Unlike prior simulators that rely on simplified models of deformable sensors, DOT-Sim accurately captures the physical behavior of soft sensors by modeling them as elastic materials using the Material Point Method (MPM). DOT-Sim enables rapid calibration of optical tactile sensor simulation using a small number of demonstrations within minutes, which is substantially faster than existing methods. Compared to current baselines, our approach supports much larger and non-linear deformations. To handle the optical aspect, we propose a novel approach to simulating optical responses by learning a residual image relative to the real-world idle state. We validate the physical and visual realism of our method through a series of zero-shot sim-to-real tasks. Our experiments show that DOT-Sim (1) accurately replicates the physical dynamics of a DenseTact optical tactile sensor in reality, (2) generates realistic optical outputs in contact-rich scenarios, and (3) enables direct deployment of simulation-trained classifiers in the real world, achieving 85% classification accuracy on challenging objects and 90% accuracy in embedded tumor-type detection, and (4) allows precise trajectory following with policy trained from demonstrations in simulation with an average error of less than 0.9 mm.

Keywords: Medical Robots and Systems, Sensor-based Control, Surgical Robotics: Steerable Catheters/Needles
Abstract: Robot-assisted minimally invasive neurosurgeries have shown great promise in enabling lower invasiveness and faster patient recovery times. However, performing such surgeries remains challenging, mainly due to the use of rigid surgical tools and limited accessibility to deep-seated brain structures. Employing robotically steerable tools could address these challenges, as these devices, being relatively more dexterous, can gain access to different regions in the brain. The autonomous control of these tools could further enable manipulation with higher precision and lower procedural time, facilitating less fatigue for surgeons. In this paper, we present a control strategy for the precise manipulation of a robotically steerable endoscopic cannula (RSEC). The proposed control architecture uses a combination of inverse kinematics, endoscopic imaging, and electromagnetic tracking feedback to perform task-space control of the RSEC in real-time. A joint angle estimation algorithm is proposed to estimate the bending angles of the RSEC using an endoscopic camera. The tip-position RMSE value of the RSEC when bending the proximal and distal joints, obtained using the proposed control strategy, was 0.7 mm. The results indicate that the proposed method can be used to achieve position control of the RSEC with sub-mm accuracy.

Keywords: Probability and Statistical Methods, Optimization and Optimal Control, Failure Detection and Recovery
Abstract: Cyber-physical robotic systems are vulnerable to false data injection attacks (FDIAs), in which an adversary corrupts sensor signals while evading residual-based passive anomaly detectors such as the chi-squared test. Such stealthy attacks can induce substantial end-effector deviations without triggering alarms. This paper studies the resilience of redundant manipulators to stealthy FDIAs and advances the architecture from passive monitoring to active defence. We formulate a closed-loop model comprising a feedback-linearized manipulator, a steady-state Kalman filter, and a chi-squared-based anomaly detector. Building on this passive monitoring layer, we propose an active control-level defence that attenuates the control input through a monotone function of an anomaly score generated by a novel actuation-projected, measurement-free state predictor. The proposed design provides probabilistic guarantees on nominal actuation loss and preserves closed-loop stability. From the attacker perspective, we derive a convex QCQP for computing one-step optimal stealthy attacks. Simulations on a 6-DOF planar manipulator show that the proposed defence significantly reduces attack-induced end-effector deviation while preserving nominal task performance in the absence of attacks.