Keywords: Physically Assistive Devices, Physical Human-Robot Interaction
Abstract: Sit-to-stand (STS) transfer is a fundamental but challenging movement that plays a vital role in older adults’ daily activities. The decline in muscular strength and coordination ability can result in difficulties performing STS and, therefore, the need for mobility assistance by humans or assistive devices. Robotics rollators are being developed to provide active mobility assistance to older adults, including STS assistance. In this paper, we consider the robotic walker SkyWalker, which can provide active STS assistance by moving the handles upwards and forward to bring the user to a standing configuration. In this context, it is crucial to monitor if the user is performing the STS and adapt the rollator’s control accordingly. To achieve this, we utilized a standard vision-based method for estimating the human pose during the STS movement using Mediapipe pose tracking. Since estimating a user’s state from extreme proximity to the camera is challenging, we compared the pose identification results from Mediapipe to ground truth data obtained from Vicon marker-based motion capture to assess accuracy and reliability of the STS motion. The fourteen kinematic features critical for accurate pose estimation were selected based on literature review and the specific requirements of our robot’s STS method. By employing these features, we have implemented a phase classification system that enables the SkyWalker to classify the user’s STS phase in real-time. The selected kinematics from vision-based human state estimation method and trained classifier can be furthermore generalized to other types of motion support, including adaptive STS path planning and emergency stops for safety insurance during STS.

Keywords: SLAM, Visual-Inertial SLAM, Visual Learning
Abstract: Despite advancements in self-supervised monocular depth estimation, challenges persist in dynamic scenarios due to the dependence on assumptions about a static world. In this paper, we present Manydepth2, to achieve precise depth estimation for both dynamic objects and static backgrounds, all while maintaining computational efficiency. To tackle the challenges posed by dynamic content, we incorporate optical flow and coarse monocular depth to create a pseudo-static reference frame. This frame is then utilized to build a motion-aware cost volume in collaboration with the vanilla target frame. Furthermore, to improve the accuracy and robustness of the network architecture, we propose an attention-based depth network that effectively integrates information from feature maps at different resolutions by incorporating both channel and non-local attention mechanisms. Compared to methods with similar computational costs, Manydepth2 achieves a significant reduction of approximately five percent in root-mean-square error for self-supervised monocular depth estimation on the KITTI-2015 dataset. The code could found https://github.com/kaichen-z/Manydepth2.

Keywords: SLAM, Localization, Sensor Fusion
Abstract: Unlike standard camera that relies on exposure to obtain output frame by frame, event camera only output an event when the change of brightness intensity in a pixel exceed a threshold, and the outputs of different pixels are independent to each other. Benefited from its bio-inspired design, event camera has the advantages of low latency and

high dynamic range. The researches on multi-sensor fusion with event camera are few so far. In this paper, we propose FAST-LIEO, a framework

for fast and real-time LiDAR-inertial-event odometry. The framework tightly fuses LiDAR and event camera measurements without any feature extraction or matching. Besides, our system supports both LIEO and LIEVO (extended with RGB camera fusion). We design a novel EIO subsystem For LiDAR-event fusion. The EIO subsystem maintains a semi-dense event map and estimates the state by aligning the event representation to map. The semi-dense event

map is built from LiDAR points by utilizing the edge information and temporal information provided by event representations. Besides testing

our method on public benchmark dataset, we also collected real-world data by utilizing our sensor suite and conducted experiments on our self-captured dataset. The experiment results show the high robustness and accuracy of our method in challenging conditions with high real-time ability. To the best of our knowledge, our FAST-LIEO is the first system that can tightly fuse LiDAR, IMU, event camera and standard camera measurements in simultaneously localization and mapping. The source code of FAST-LIEO and our dataset are available at:

https://github.com/wsjpla/FAST-LIEO.

Keywords: Autonomous Vehicle Navigation, Human-Aware Motion Planning, Reinforcement Learning
Abstract: Robot navigation in human-populated environments poses challenges due to the diversity of human behaviors and the unpredictability of human paths. However, existing Reinforcement Learning (RL)-based methods often rely on simulators that lack sufficient diversity in human behavior, resulting in navigation policies that overfit specific human behavior and perform poorly in unseen environments.

To address this, we propose a diversity-aware crowd model based on Reinforcement Learning, employing Constrained Variational Exploration (VE) with a Mutual Information (MI)-based auxiliary reward to capture fine-grained behavioral diversity.The proposed model leverages a Centralized Training Decentralized Execution (CTDE) paradigm, which ensures stable exploration under multi-agent settings.

Using the proposed diversity-aware model for training, we obtain robust

robot navigation policies capable of handling diverse unseen scenarios.

Extensive simulation and real-world experiments demonstrate the superior performance of our approach in achieving diverse crowd behaviors and enhancing robot navigation robustness. These findings highlight the potential of our method to advance safe and efficient robot operations in complex dynamic environments.

Keywords: Localization, Vision-Based Navigation, Data Sets for Robotic Vision
Abstract: Visual Place Recognition (VPR) in mobile robotics enables robots to localize themselves by recognizing previously visited locations using visual data. While the reliability of VPR methods has been extensively studied under conditions such as changes in illumination, season, weather and viewpoint, the impact of motion blur is relatively unexplored despite its relevance not only in rapid motion scenarios but also in low-light conditions where longer exposure times are necessary. Similarly, the role of image deblurring in enhancing VPR performance under motion blur has received limited attention so far. This paper bridges these gaps by introducing a new benchmark designed to evaluate VPR performance under the influence of motion blur and image deblurring. The benchmark includes three datasets that encompass a wide range of motion blur intensities, providing a comprehensive platform for analysis. Experimental results with several well-established VPR and image deblurring methods provide new insights into the effects of motion blur and the potential improvements achieved through deblurring. Building on these findings, the paper proposes adaptive deblurring strategies for VPR, designed to effectively manage motion blur in dynamic, real-world scenarios.

Keywords: Vision-Based Navigation, Field Robots
Abstract: While navigating unknown environments, robots rely primarily on proximate features for guidance in decision making, such as depth information from lidar or stereo to build a costmap, or local semantic information from images. The limited range over which these features can be used may result in poor robot behavior when assumptions about the cost of the map beyond the range of proximate features misguide the robot. Integrating far-field image features that originate beyond these proximate features into the mapping pipeline has the promise of enabling more intelligent and aware navigation through unknown terrain. To navigate with far-field features, key challenges must be overcome. As far-field features are typically too distant to localize precisely, they are difficult to place in a map. Additionally, the large distance between the robot and these features makes connecting these features to their navigation implications more challenging. We propose FITAM, an approach that learns to use far-field features to predict costs to guide navigation through unknown environments from previous experience in a self-supervised manner. Unlike previous work, our approach does not rely on flat ground plane assumptions or range sensors to localize observations. We demonstrate the benefits of our approach through simulated trials and real-world deployment on a Clearpath Robotics Warthog navigating through a forest environment.

Keywords: Bimanual Manipulation, Manipulation Planning
Abstract: In this paper, we address the problem of manipulating multi-particle aggregates using a bimanual robotic system. Our approach enables the autonomous transport of dispersed particles through a series of shaping

and pushing actions using robotically-controlled tools. Achieving this advanced manipulation capability presents two key challenges: high-level task planning and trajectory execution. For task planning, we leverage Vision Language Models (VLMs) to enable primitive actions such as tool affordance grasping and non-prehensile particle pushing. For trajectory execution, we represent the evolving particle aggregate's contour using truncated Fourier series, providing efficient

parametrization of its closed shape. We adaptively compute trajectory waypoints based on group cohesion and the geometric centroid of the aggregate, accounting for its spatial distribution and collective motion. Through real-world experiments, we demonstrate the effectiveness of our methodology in actively shaping and manipulating multi-particle aggregates while maintaining high system cohesion.

Keywords: Reinforcement Learning, Machine Learning for Robot Control, Aerial Systems: Applications
Abstract: Precise and agile flight maneuvers are essential for quadrotor applications, yet traditional control methods are limited by their reliance on flat trajectories or computationally intensive optimization. Reinforcement learning (RL)-based policies offer a promising alternative by directly mapping observations to actions, reducing dependency on system knowledge and actuation constraints. However, the sim-to-real gap remains a significant challenge, often causing instability in real-world deployments. In this work, we identify five key factors for learning robust RL-based control policies capable of zero-shot real-world deployment: (1) integrating velocity and rotation matrix into actor inputs, (2) incorporating time vector into critic inputs, (3) regularizing action differences for smoothness, (4) applying system identification with selective randomization, and (5) using large batch sizes during training. Based on these insights, we develop textit{SimpleFlight}, a PPO-based framework that integrates these techniques. Extensive experiments on the Crazyflie quadrotor demonstrate that SimpleFlight reduces trajectory tracking error by over 50% compared to state-of-the-art RL baselines. It excels in both smooth polynomial and challenging infeasible zigzag trajectories, particularly on small thrust-to-weight quadrotors, where baseline methods often fail. To enhance reproducibility and further research, we integrate SimpleFlight into the GPU-based Omnidrones simulator and provide open-source code and model checkpoints. For more details, visit our project website at url{https://sites.google.com/view/simpleflight/}.

Keywords: Perception for Grasping and Manipulation, Software-Hardware Integration for Robot Systems, Grasping
Abstract: Hyperspectral imaging is an advanced technique for precisely identifying and analyzing materials or objects. However, its integration with robotic grasping systems has so far been explored due to the deployment complexities and prohibitive costs. Within this paper, we introduce a novel hyperspectral imaging-guided robotic grasping system. The system consists of PRISM (Polyhedral Reflective Imaging Scanning Mechanism) and the SpectralGrasp framework. PRISM is designed to enable high-precision, distortion-free hyperspectral imaging while simplifying system integration and costs. SpectralGrasp generates robotic grasping strategies by effectively leveraging both the spatial and spectral information from hyperspectral images. The proposed system demonstrates substantial improvements in both textile recognition compared to human performance and sorting success rate compared to RGB-based methods. Additionally, a series of comparative experiments further validates the effectiveness of our system. The study highlights the potential benefits of integrating hyperspectral imaging with robotic grasping systems, showcasing enhanced recognition and grasping capabilities in complex and dynamic environments. The project is available at: https://zainzh.github.io/PRISM.

Keywords: Robotics and Automation in Construction, Industrial Robots, Surveillance Robotic Systems
Abstract: Acoustic inspection is crucial for infrastructure maintenance, but its effectiveness is often hampered by environmental noise. Conventional denoising methods rely on prior knowledge or training data, limiting their practicability. This paper presents Zero-Shot Denoiser, a novel approach achieving noise reduction without pre-collected target sound samples or noise knowledge. Our method synergistically combines Blind Signal Separation (BSS) for unsupervised audio decomposition and Artifact-Resilient Attention (AR-Attention) for text-guided audio reconstruction. AR-Attention leverages pre-trained audio-language models and dual normalization to mitigate BSS artifacts and identify target sounds semantically. We introduce pseudo Signal-to-Noise Ratio, derived from the audio-language model, for automatic BSS hyperparameter optimization. In experiments using public datasets, our method, operating in a true zero-shot setting, achieved performance comparable to that of state-of-the-art supervised denoising methods, and experiments targeting hammering tests confirmed the effectiveness of our approach for real-world acoustic inspections. Our approach overcomes the limitations of data-dependent techniques and offers a versatile noise reduction solution for acoustic inspection and broader acoustic tasks.

Keywords: Deep Learning for Visual Perception, Visual Learning, Sensor Fusion
Abstract: Recent advances in neural rendering have enabled the 3D reconstruction of dynamic humans from monocular videos, with applications in robotics.

However, it is still challenging to reconstruct clear humans from in-the-wild video encountering motion blur, causing shape and appearance inconsistencies, especially in blurry regions like hands and

legs. In this paper, we propose ExFMan, the first neural rendering framework that unveils the possibility of rendering high-quality humans

in rapid motion with a hybrid frame-based RGB and bio-inspired event camera. The ``out-of-the-box'' insight is to leverage the high temporal

information of event data in a complementary manner and adaptively reweight the effect of losses for both RGB frames and events in the local regions, according to the velocity of the rendered human. This significantly mitigates the inconsistency associated with motion blur in the RGB frames. Specifically, we first formulate a velocity field of

the 3D body in the canonical space and render it to image space to identify the body parts with motion blur. We then propose two novel losses, i.e., velocity-aware photometric loss and velocity-relative event loss, to optimize the neural human for both modalities under the guidance of the estimated velocity. In addition, we incorporate novel pose regularization and alpha losses to facilitate continuous pose and clear boundary. Extensive experiments on synthetic and real-world datasets demonstrate that ExFMan can reconstruct sharper and higher quality humans over the compared baselines and the state-of-the-art methods for diverse blurry subjects.

Keywords: Probabilistic Inference, Reinforcement Learning, Perception for Grasping and Manipulation
Abstract: We present an integrated (or end-to-end) framework for the Real2Sim2Real problem of manipulating deformable linear objects (DLOs) based on visual perception. Working with a parameterised set of DLOs, we use likelihood-free inference (LFI) to compute the posterior distributions for the physical parameters using which we can approximately simulate the behaviour of each specific DLO. We use these posteriors for domain randomisation while training, in simulation, object-specific visuomotor policies (i.e. assuming only visual and proprioceptive sensory) for a DLO reaching task, using model-free reinforcement learning. We demonstrate the utility of this approach by deploying sim-trained DLO manipulation policies in the real world in a zero-shot manner, i.e. without any further fine-tuning. In this context, we evaluate the capacity of a prominent LFI method to perform fine classification over the parametric set of DLOs, using only visual and proprioceptive data obtained in a dynamic manipulation trajectory. We then study the implications of the resulting domain distributions in sim-based policy learning and real-world performance.

Keywords: Mapping, Range Sensing, SLAM
Abstract: The joint optimization of sensor poses and 3D structure is fundamental for state estimation in robotics and related fields. Current LiDAR systems often prioritize pose optimization, with structure refinement either omitted or treated separately using implicit representations. This paper introduces a framework for simultaneous optimization of sensor poses and 3D map, represented as surfels. A generalized LiDAR uncertainty model is proposed to address less reliable measurements in varying scenarios. Experimental results on public datasets demonstrate improved performance over most comparable state-of-the-art methods. The system is provided as open-source software to support further research.

Keywords: Robot Safety, Safety in HRI, Compliance and Impedance Control
Abstract: In the context of safety-critical control, we propose and analyse the use of Control Barrier Functions (CBFs) to limit the kinetic energy of torque-controlled robots. The proposed scheme is able to modify a nominal control action in a minimally invasive manner to achieve the desired kinetic energy limit. We show how this safety condition is achieved by appropriately injecting damping in the underlying robot dynamics independently of the nominal controller structure. We present an extensive experimental validation of the approach on a 7-Degree of Freedom (DoF) Franka Emika Panda robot. The results demonstrate that this approach provides an effective, minimally invasive safety layer that is straightforward to implement and is robust in real experiments.

Keywords: Optimization and Optimal Control, Machine Learning for Robot Control, Motion and Path Planning
Abstract: Model Predictive Control has emerged as a popular tool for robots to generate complex motions. However, the real-time requirement has limited the use of hard constraints and large preview horizons, which are necessary to ensure safety and stability. In practice, practitioners have to carefully design cost functions that can imitate an infinite horizon formulation, which is tedious and often results in local minima. In this work, we study how to approximate the infinite horizon value function of constrained optimal control problems with neural networks using value iteration and trajectory optimization. Furthermore, we experimentally demonstrate how using this value function approximation as a terminal cost provides global stability to the model predictive controller. The approach is validated on two toy problems and a real-world scenario with online obstacle avoidance on an industrial manipulator where the value function is conditioned to the goal and obstacle.

Keywords: Aerial Systems: Mechanics and Control, Aerial Systems: Applications
Abstract: Small multirotors demonstrate significant potential due to their simple airframe and human-friendly operation. However, the reduced size results in substantially higher energy consumption, which severely limits their flight endurance and restricts their range of applications. Ornithopters, while offering better aerodynamic efficiency, experience energy losses due to the mechanical complexity required to generate reciprocating motion. In this work, inspired by the samara, we present a lightweight aircraft with an exceptionally simple design featuring a single actuator and a mono airfoil. To optimize the flight configuration for minimal power consumption, we employed a Surrogate optimization method that integrates spinning airfoil dynamics, motor-propeller efficiency, and hovering equilibrium. As a result, the proposed vehicle achieves position-controlled hovering flight for up to 26 minutes with a takeoff weight of only 32 grams. Its superior power efficiency is demonstrated by a high power loading of 9.1 grams per watt. Compared to state-of-the-art systems, the proposed design shows significant improvements in both flight endurance and power efficiency. The reliable and stable position-holding flight over an extended period further validates the effectiveness of the proposed methods and the practical applicability of the fabricated prototype.

Keywords: Constrained Motion Planning, Optimization and Optimal Control, Aerial Systems: Perception and Autonomy
Abstract: Perception algorithms are ubiquitous in modern autonomy stacks, providing necessary environmental information to operate in the real world. Many of these algorithms depend on the visibility of keypoints, which must remain within the robot’s line-of-sight (LoS), for reliable operation. This paper tackles the challenge of maintaining LoS on such keypoints during robot movement. We propose a novel method that addresses these issues by ensuring applicability to various sensor footprints, adaptability to arbitrary nonlinear system dynamics, and constant enforcement of LoS throughout the robot's path. Our experiments show that the proposed approach achieves significantly reduced LoS violation and runtime compared to existing state-of-the-art methods in several representative and challenging scenarios.

Keywords: SLAM, Localization, Field Robots

Keywords: SLAM, RGB-D Perception
Abstract: 3D Gaussian Splatting (3DGS) has recently emerged as a powerful technique for representing 3D scenes. Its superior high-fidelity rendering quality and speed have driven its rapid adoption in many applications. Among them, Visual Simultaneous Localization and Mapping (VSLAM) is the most prominent application, as it requires real-time simultaneous mapping and position tracking of navigating objects. However, from our comprehensive study, we observed a fundamental hur- dle in directly applying the current 3DGS technique to VSLAM, which we define as the scale adaptation problem. The scale adaptation problem refers to the inability of existing 3DGS- based SLAM methods to address varying scales, specifically the extent of camera pose difference from the perspective of tracking, and environmental size in terms of mapping and the addition of new 3D Gaussians. To overcome this limitation, we propose SAGA-SLAM, the first scale-adaptive RGB-D Dense SLAM framework based on 3DGS. We optimize the tracking and mapping stages robustly over various scales by utilizing the Polyak step size and momentum. Additionally, we present gaussian fission method to address the scale problem during the addition of 3D Gaussians. Experiments show that our method achieves state-of-the-art results robustly on both large and small scales, such as KITTI, Replica, and TUM-RGBD. By adapting without the need for hyperparameter tuning, our method demonstrates both superior performance and practical applicability.

Keywords: Soft Robot Materials and Design, Biologically-Inspired Robots, Mechanism Design
Abstract: Soft robots have gained widespread attention due to their lightweight nature and inherent safety. Among them, soft growing robots (SGRs) are inspired by the growth mechanism of vines, achieving movement through tip eversion. However, their load-bearing capacity remains a significant challenge due to material limitations. The stiffness modulation approach based on layer jamming is constrained in high-curvature tip regions, preventing it from fully exhibiting its potential in unstructured environments. In this letter, motivated by enhancing the load-bearing capacity of SGR and optimizing their tip motion performance, we propose a novel mixed-layer soft growing robot (MLSGR) and introduce an innovative modification to the conventional layer jamming fabrication method. Furthermore, we establish a more accurate kinematics model and, for the first time, propose a statics model to characterize tip behavior. Experimental results demonstrate that, compared to previous work, MLSGR exhibits more than twice in load capacity, a 9% reduction in energy consumption and mechanical resistance for tip growth, a 17% improvement in tip retraction capability, and a 41.2% enhancement in kinematic model prediction accuracy (MAPE).

Keywords: Object Detection, Segmentation and Categorization, Deep Learning for Visual Perception, Probability and Statistical Methods
Abstract: We propose CLEVER, an active learning system for robust semantic perception with Deep Neural Networks (DNNs). For data arriving in streams, our system seeks human support when encountering failures and adapts DNNs online based on human instructions. In this way, CLEVER can eventually accomplish the given semantic perception tasks. Our main contribution is the design of a system that meets several desiderata of realizing the aforementioned capabilities. The key enabler herein is our Bayesian formulation that encodes domain knowledge through priors. Empirically, we not only motivate CLEVER's design but further demonstrate its capabilities with a user validation study as well as experiments on humanoid and deformable objects. To our knowledge, we are the first to realize stream-based active learning on a real robot, providing evidence that the robustness of the DNN-based semantic perception can be improved in practice. The project website can be accessed at https://sites.google.com/view/thecleversystem.

Keywords: Formal Methods in Robotics and Automation, Deep Learning Methods, Software Tools for Robot Programming
Abstract: Signal Temporal Logic (STL) offers a concise yet expressive framework for specifying and reasoning about spatio-temporal behaviors of robotic systems. Attractively, STL admits the notion of robustness, the degree to which an input signal satisfies or violates an STL specification, thus providing a nuanced evaluation of system performance. Notably, the differentiability of STL robustness enables direct integration to robotics workflows that rely on gradient-based optimization, such as trajectory optimization and deep learning. However, existing approaches to evaluating and differentiating STL robustness rely on recurrent computations, which become inefficient with longer sequences, limiting their use in time- sensitive applications. In this paper, we present STLCG++, a masking-based approach that parallelizes STL robustness evaluation and backpropagation across timesteps, achieving significant speed-ups compared to a recurrent approach. We also introduce a smoothing technique to enable differentiation of time interval bounds, expanding STL’s applicability in gradient-based optimization tasks over spatial and temporal variables. Finally, we demonstrate STLCG++’s benefits through three robotics use cases and provide JAX and PyTorch libraries for seamless integration into modern robotics workflows.

Keywords: Flexible Robotics, Motion Control, Motion and Path Planning
Abstract: This article presents a motion planning and control framework for flexible robotic manipulators, integrating deep reinforcement learning (DRL) with a nonlinear partial differential equation (PDE) controller. Unlike conventional approaches that focus solely on control, we demonstrate that the desired trajectory significantly influences endpoint vibrations. To address this, a DRL motion planner, trained using the soft actor-critic (SAC) algorithm, generates optimized trajectories that inherently minimize vibrations. The PDE nonlinear controller then computes the required torques to track the planned trajectory while ensuring closed-loop stability using Lyapunov analysis. The proposed methodology is validated through both simulations and real-world experiments, demonstrating superior vibration suppression and tracking accuracy compared to traditional methods. The results underscore the potential of combining learning-based motion planning with model-based control for enhancing the precision and stability of flexible robotic manipulators.

Keywords: Collision Avoidance, Reinforcement Learning, Engineering for Robotic Systems
Abstract: Ensuring safe and efficient operation of collaborative robots in human environments is challenging, especially in dynamic settings where both obstacle motion and tasks change over time. Current robot controllers typically assume full visibility and fixed tools, which can lead to collisions or overly conservative behavior. In our work, we introduce a tool aware collision avoidance system that adjusts in real time to different tool sizes and modes of tool-environment interaction. Using a learned perception model, our system filters out robot and tool components from the point cloud, reasons about occluded area, and predicts collision under partial observability. We then use a control policy trained via constrained reinforcement learning to produce smooth avoidance maneuvers in under 10 milliseconds. In simulated and real world tests, our approach outperforms traditional approaches(APF, MPPI) in dynamic environments, while maintaining sub-millimeter accuracy. Moreover, our system operates with approximately 60 % lower computational overhead compared to a state-of-the-art GPU-based planner. Our approach provides modular, efficient, and effective collision avoidance for robots operating in dynamic environments. We integrate our method into a collaborative robot application and demonstrate its practical use for safe and responsive operation.

Keywords: Visual Tracking, Sensor Fusion, Computer Vision for Automation
Abstract: Multi-object tracking (MOT) plays a critical role in applications such as autonomous driving and surveillance. Camera-based approaches offer rich texture features for object association, while LiDAR-based methods

provide accurate geometric information for spatial reasoning. Although each modality addresses different challenges, their intrinsic discrepancies hinder effective cross-modal fusion and unified representation learning. To overcome these limitations, we propose IMH-MOT, an interactive multi-hierarchical MOT framework comprising three key modules. The Multi-modality~Alignment~Module~(MMAM) enhances spatial representations by sampling and clustering instance-level point

clouds. From different modalities are motion cues integrated by the Multi-modality~Motion~Estimation~Module~(MMEM) to build a unified motion model. To mitigate the impact of occlusion on single-frame appearance features, the Long-term~Appearance~Module~(LAM) captures temporal appearance consistency by constructing a long-term appearance embedding. Guided by modality-aware cues from MMAM, MMEM generates reliable spatial representations, while LAM encodes robust long-term appearance features. These components are jointly integrated through a Multi-hierarchical~Data~Association~(MHDA) strategy, enabling stable and accurate tracking. Extensive experiments on the KITTI MOT benchmark

demonstrate the effectiveness of our framework, achieving 80.90% HOTA, 89.73% MOTA, and 470 IDSW, outperforming state-of-the-art methods in both standard and challenging scenarios.

Keywords: SLAM, Field Robots, Robotics and Automation in Agriculture and Forestry
Abstract: Accurate state estimation is essential for an autonomous agricultural robot’s reliable operations. The effectiveness of state estimation is influenced by a number of factors, such as sensor-fusion algorithms, the environment, and sensor quality. When the robot traverses in large-scale scenarios, the distance travelled and high-speed mobility produce a drift in the estimation process and should be carefully considered. Moreover, the time-varying noise in sensors affects odometry accuracy further; this is especially noticeable in long travel. This research work is related to the multi-constraints-based state estimation in large unstructured environments with uneven terrain, with a focus on agricultural applications. Using LiDAR-IMU based fusion, our goal is to provide a reliable & accurate localization solution in complex environments like agricultural fields. Furthermore, the agricultural environments become more challenging due to the uneven terrain and lack of features. Our research proposes a hybrid framework which combines factor graph-based optimization & adaptive Kalman filtering to address these challenges in complex environments. Furthermore, performance evaluation is conducted on self-collected datasets from agricultural environments as well as on open-access datasets such as GRACO & KITTI.

Keywords: Shared Autonomy, Cognitive Human-Robot Interaction, AI-Based Methods, Motion and Path Planning
Abstract: This paper presents a novel shared autonomy and baseline policy adapting framework for human-robot interactions in high-level context-aware robotic tasks. With a unique methodology that leverages hierarchies in decision-making as well as variational analysis of human policy, we propose a mathematical model of shared autonomy policy. The framework aims at interpretable high-level decision-making for efficient robot operation with human in the loop. We modeled the decision-making process using hierarchical Markov decision processes (MDPs) in an algorithm we called policy adapting, where the autonomous system policy is adapted, and hence, shaped by incorporating design variables contextual to the robot, human, task, and pre-training. By integrating deep reinforcement learning within a multi-agent hierarchical context, we present an end-to-end algorithm to train a baseline policy designed for shared autonomy. We showcase the effectiveness of our framework, and particularly the interplay between different design elements and human's skill level, in a pilot study with a human user in a simulated sequence of high-level pick-and-place tasks. The proposed framework advances the state-of-the-art in shared autonomy for robotic tasks, but can also be applied to other domains of autonomous operation.

Keywords: Collision Avoidance, Machine Learning for Robot Control, Robot Safety
Abstract: This paper presents a novel learning-based approach for online estimation of maximal safe sets for local trajectory planning in unknown static environments. The neural representation of a set is used as the terminal set constraint for a model predictive control (MPC) local planner, resulting in improved recursive feasibility and safety. To achieve real-time performance and desired generalization properties, we employ the idea of hypernetworks. We use the Hamilton-Jacobi (HJ) reachability analysis as the source of supervision during the training process, allowing us to consider general nonlinear dynamics and arbitrary constraints. The proposed method is extensively evaluated against relevant baselines in simulations for different environments and robot dynamics. The results show a success rate increase of up to 52 % compared to the best baseline while maintaining comparable execution speed. Additionally, we deploy our proposed method, NTC-MPC, on a physical robot and demonstrate its ability to safely avoid obstacles in scenarios where the baselines fail.

Keywords: Autonomous Agents, Deep Learning Methods, Performance Evaluation and Benchmarking
Abstract: As the prediction horizon increases, predicting the future evolution of traffic scenes becomes increasingly difficult due to the multi-modal nature of agent motion. Most state-of-the-art (SotA) prediction models primarily focus on forecasting the most likely future. However, for the safe operation of autonomous vehicles, it is equally important to cover the distribution for plausible motion alternatives. To address this, we introduce EP-Diffuser, a novel parameter-efficient diffusion-based generative model designed to capture the distribution of possible traffic scene evolutions. Conditioned on road layout and agent history, our model acts as a predictor and generates diverse, plausible scene continuations. We benchmark EP-Diffuser against two SotA models in terms of plausibility, diversity, and accuracy of predictions on the Argoverse 2 dataset. Despite its significantly smaller model size, our approach achieves both highly plausible and diverse traffic scene predictions with comparable accuracy. We further evaluate model generalization in an out-of-distribution (OoD) test setting using Waymo Open dataset and show superior robustness of our approach. The code and model checkpoints will be made publicly available to ensure reproducibility.

Keywords: Field Robots, Search and Rescue Robots, Reinforcement Learning
Abstract: Mobile robot teams often require decentralised autonomous navigation through narrow gaps in limited commu- nication environments (e.g., underground search-and-rescue op- erations). Existing navigation approaches exhibit suboptimal per- formance for avoiding multi-robot collisions in such bottlenecks due to an inability to address the dynamic nature of the robots. Initial work utilising reinforcement learning has demonstrated success in navigating a single robot through narrow gaps. However, when training agents to produce give-way behaviour for navigat- ing through constrained gaps, end-to-end reinforcement learning using simple rewards suffers from slow convergence due to the increased search space of viable policies. This paper introduces a novel curriculum reinforcement learning framework, incorpo- rating a multi-robot bootstrap curriculum with preprogrammed behaviour to guide initial policy formation, subsequently refined by a gap curriculum that progressively reduces training complexity towards an optimal policy. This framework learns multi-robot in- teraction behaviours, which are impractical to program manually. Our model achieves a 99% success-rate in give-way behaviour generation without inter-agent communications in high-fidelity simulations. The success-rate reduced to 73% in simulations incor- porating noisy sensors, and 60% in field-robot tests, substantiating our model’s practical viability despite sensor noise and real-world uncertainties. The simple benchmark methods lack efficiency in basic interaction behaviours.

Keywords: SLAM, Sensor Fusion, Modeling, Control, and Learning for Soft Robots
Abstract: For robots with low rigidity, determining the robot's state based solely on kinematics is challenging. This is particularly crucial for a robot whose entire body is in contact with the environment, as accurate state estimation is essential for environmental interaction. We propose a method for simultaneous articulated robot posture estimation and environmental mapping by integrating data from proximity sensors distributed over the whole body. Our method extends the discrete-time model, typically used for state estimation, to the spatial direction of the articulated structure. The simulations demonstrate that this approach significantly reduces estimation errors.

Keywords: Localization, SLAM, Vision-Based Navigation
Abstract: In point-line Simultaneous Localization and Mapping (SLAM) systems, the utilization of line structural information and the optimization of lines are two significant problems. The former is usually addressed through structural regularities, while the latter typically involves using minimal parameter representations of lines in optimization. However, separating these two steps leads to the loss of constraint information to each other. To solve both problems, we anchor lines with similar directions to one principal axis. Precisely, our method models the line-axis probabilistic data association using the Expectation Maximization (EM) algorithm and provides the pipelines for axis creation, updating, and optimization, enhancing the system's robustness and avoiding mismatch. Our system can optimize n co-directional lines with only n+2 parameters, significantly reducing the number of line parameters to be optimized and enabling rapid mapping and tracking. Additionally, considering that most real-world scenes conform to the Atlanta World (AW) hypothesis, we provide an AW constraint by detecting structural lines based on vertical priors and vanishing points. Experimental results and ablation studies on various indoor and outdoor datasets demonstrate the effectiveness of our system.

Keywords: Calibration and Identification, Deep Learning Methods, Recognition
Abstract: This paper proposes a simple yet effective markerless hand-eye calibration method that achieves low cost, high accuracy, and strong generalization across different types of robots. The method utilizes a circular flange, a standardized structure in industrial robots, for calibration via the perspective-n-point (PnP) algorithm, achieving superior performance with a simpler pipeline. The entire system is built using mature, off-the-shelf components, avoiding complex architectures. By combining a lightweight object detection network (e.g., Faster R-CNN) with classical geometric techniques, we construct a flange detector that is both accurate and robust. The training process requires no manual annotations, and the resulting model generalizes well across various robot platforms. Experiments demonstrate that our method achieves higher calibration accuracy than more complex existing approaches. Notably, the method maintains consistent precision even when applied to previously unseen robots. Code and pre-trained models will be made available.

Keywords: Reinforcement Learning, Machine Learning for Robot Control, Mobile Manipulation
Abstract: We address the challenge of learning to manipulate deformable objects with unknown dynamics. In non-rigid objects, the dynamics parameters define how they react to interactions --how they stretch, bend, compress, and move-- and they are critical to determining the optimal actions to perform a manipulation task successfully. In other robotic domains, such as legged locomotion and in-hand rigid object manipulation, state-of-the-art approaches can handle unknown dynamics using Rapid Motor Adaptation (RMA). Through a supervised procedure in simulation that encodes each rigid object's dynamics, such as mass and position, these approaches learn a policy that conditions actions on a vector of latent dynamic parameters inferred from sequences of state-actions. However, in deformable object manipulation, the object's dynamics not only includes its mass and position, but also how the shape of the object changes. Our key insight is that the recent ground-truth particle positions of a deformable object in simulation capture changes in the object's shape, making it possible to extend RMA to deformable object manipulation. This key insight allows us to develop RAPiD, a two-phase method that learns to perform real-robot deformable object mobile manipulation by: 1) learning a visuomotor policy conditioned on the object's dynamics embedding, which is encoded from the object's privileged information in simulation, such as its mass and ground-truth particle positions, and 2) learning to infer this embedding using non-privileged information instead, such as robot visual observations and actions, so that the learned policy can transfer to the real world. On a mobile manipulator with 22 degrees of freedom, RAPiD enables over 80%+ success rates across two vision-based deformable object mobile manipulation tasks in the real world, under unseen object dynamics, categories, and instances.

Keywords: Sensor Fusion, Wheeled Robots, Probability and Statistical Methods
Abstract: This paper presents a novel calibration system for odometric sensors using an Adaptive Particle Filter (Adaptive-PF) to achieve high precision pose estimation and improve localization in wheeled mobile robots. The system compensates for intrinsic systematic errors in the odometric sensor by adjusting its parameters in realtime. Likewise, a comparative analysis of resampling methods —multinomial, stratified, systematic, and residual resampling— is conducted to evaluate their impact on calibration performance. The system validation is demonstrated by its implementation in an autonomous wheelchair, where the localization module integrates wheel encoders, an Inertial Measurement Unit (IMU), and a LIDAR sensor, providing robust navigation in dynamic environments. Experimental results demonstrate that systematic approach and resampling based on the effective number of particles, N(eff), yield the best performance. Additionally, the system dynamically adjusts prediction error based on the differences between LIDAR and odometry data. It also adapts the number of particles according to the dispersion and uncertainty, optimizing computational time without sacrificing accuracy. The proposed system outperforms another well-known method, namely the DKF (Dual Kalman Filter). Consequently, this research introduces a new Adaptive-PF for odometric parameter calibration under changing conditions.

Keywords: Visual-Inertial SLAM, SLAM, Vision-Based Navigation
Abstract: Multi-camera systems hold considerable promise for enhancing visual odometry by expanding the field of view, yet simply adding more cameras does not guarantee higher accuracy. Because increasing the number of cameras also raises the likelihood of degraded or misaligned views, appropriate handling is essential to prevent severe outliers and corrupted global pose estimates. Previous methods discard points in back-end optimization based on residuals, which has been a bottleneck for real-time performance since erroneous measurements are inevitably incorporated into the main pipeline before removal. In response, we propose a direct Multi-RGB-D Inertial Odometry framework driven by confidence-based weighting, which adaptively down-weights unreliable cameras based on photometric quality and viewpoint alignment. To manage the heavy data load typical of multi-camera setups, we also incorporate a motion-guided selection strategy, filtering out non-informative points before costly alignment. This early pruning reduces computation yet retains critical constraints for odometry. By combining these techniques, our system achieves robust, scale-consistent pose estimation in real time, even with four cameras, as validated through challenging indoor-outdoor experiments involving saturation, occlusions, low-light conditions, and severe glare. We publicly release our multi-RGB-D-inertial dataset at https://github.com/seungsang07/multi-rgbd-inertial-dataset.

Keywords: Aerial Systems: Mechanics and Control, Multi-Robot Systems, Optimization and Optimal Control
Abstract: In this paper, we address the problem of cooperative manipulation of a cable-suspended load by a team of aerial robots. Unlike classical approaches that rely on centralized controllers, we propose a Distributed Nonlinear Model Predictive Control (DNMPC) framework in which the UAVs communicate over a peer-to-peer network a reduced amount of variables. In the proposed method, each robot handles only a small subset of the global optimization problem. The optimal motion computed by the distributed DNMPC loop is then used as a reference for local nonlinear controllers that track the trajectory and compute the robot's actuation inputs. We validate the proposed scheme both through numerical simulations and real-world experiments on the Fly-Crane system: a rigid platform connected to three robots by pairs of cables.

Keywords: Mechanism Design, Tendon/Wire Mechanism
Abstract: Direct laser deposition, a specialized form of additive manufacturing, shows good potential in numerous high-value applications such as the repair of aeroengine blades. However, the traditional setup for this technique is bulky and not suited for in-situ repair, requiring the costly disassembly of the aeroengine. This letter presents a miniaturized high-repeatability tendon-driven robot that showed good potential for delivering additive manufacturing equipment for in-situ techniques like direct laser deposition. The integrated actuation and ruggedized control unit make the robot portable and suitable for a variety of aeroengines. The design of the robot actuation prevents excessive bending and damage to the fiber optic. Continuum robots have the advantage of flexible and redundant structures but present limited accuracy and repeatability. The optimized kinematics and actuation of the robot presented permitted to achieve an excellent repeatability with a standard deviation of 0.02 mm on a linear path and below 0.1 mm on a path that simulates the reconstruction of a blade. The robot showed excellent linearity on each segment of the path with a coefficient of determination to the 3D best-fit line of 0.999, while maintaining the commanded velocity magnitude of the end effector with a standard deviation along the whole path of 0.05 mm/s.

Keywords: Aerial Systems: Perception and Autonomy, Vision-Based Navigation, Software-Hardware Integration for Robot Systems
Abstract: Autonomous aerial robots are increasingly being deployed in real-world scenarios, where transparent glass obstacles present significant challenges to reliable navigation. Researchers have investigated the use of non-contact sensors and passive contact-resilient aerial vehicle designs to detect glass surfaces, which are often limited in terms of robustness and efficiency. In this work, we propose a novel approach for robust autonomous aerial navigation in unknown environments with transparent glass obstacles, combining the strengths of both sensor-based and contact-based glass detection. The proposed system begins with the incremental detection and information maintenance about potential glass surfaces using visual sensor measurements. The vehicle then actively engages in touch actions with the visually detected potential glass surfaces using a pair of lightweight contact-sensing modules to confirm or invalidate their presence. Following this, the volumetric map is efficiently updated with the glass surface information and safe trajectories are replanned on the fly to circumvent the glass obstacles. We validate the proposed system through real-world experiments in various scenarios, demonstrating its effectiveness in enabling efficient and robust autonomous aerial navigation in complex real-world environments with glass obstacles.

Keywords: Deep Learning for Visual Perception, Recognition, Computer Vision for Automation
Abstract: 6D pose estimation of textureless objects is valu- able for industrial robotic applications, yet remains challenging due to the frequent loss of depth information. Current multi-view methods either rely on depth data or insufficiently exploit multi-view geometric cues, limiting their performance. In this paper, we propose DKPMV, a pipeline that achieves dense keypoint-level fusion using only multi-view RGB images as input. We design a three-stage progressive pose optimization strategy that leverages dense multi-view keypoint geometry information. To enable effective dense keypoint fusion, we enhance the keypoint network with attentional aggregation and symmetry-aware training, improving prediction accuracy and resolving ambiguities on symmetric objects. Extensive experiments on the ROBI dataset demonstrate that DKPMV outperforms state-of-the-art multi-view RGB and RGB-D approaches. The code will be available at https://github.com/chenjiahongbq/DKPMV.

Keywords: Robotics and Automation in Agriculture and Forestry, Collision Avoidance, Field Robots
Abstract: Precision agriculture offers the opportunity to auto- mate routine or difficult tasks in orchards and vineyards, such as spraying or inspection, with Unmanned Ground Vehicles (UGV). In this context, human operators should be kept in the closed-loop control of the robot for safety and reliability. This work is motivated by the challenges of deploying effectively human-robot shared control in the field. First, an asymptotically stable controller must keep the robot on the desired trajectory between rows of trees, whose distance is on the order of the robot’s width. Second, the robot must efficiently avoid static and moving obstacles (e.g. a rock or a human) in its path. Third, the control inputs must not exceed the actuator limits, which can degrade the trajectory tracking performance, cause instability, or damage critical hardware. Finally, in real-life scenarios, user intervention is sometimes required to manage unpredictable situations. To overcome these challenges, we propose and deploy a shared controller that smoothly blends automatic trajectory tracking, collision avoidance, and human commands. At the same time, it guarantees the system is stable and control actions are within the actuator limits at all times. We extensively tested our approach in simulation and field experiments in an apple orchard.

Keywords: Computer Vision for Transportation, Sensor Fusion, Localization
Abstract: 4D radar super-resolution, which aims to reconstruct sparse and noisy point clouds into dense and geometrically consistent representations, is a foundational problem in autonomous perception. However, existing methods often suffer from high training cost or rely on complex diffusion-based sampling, resulting in high inference latency and poor generalization, making it difficult to balance accuracy and efficiency. To address these limitations, we propose MSDNet, a multi-stage distillation framework that efficiently transfers dense LiDAR priors to 4D radar features to achieve both high reconstruction quality and computational efficiency. The first stage performs reconstruction-guided feature distillation (RGFD), aligning and densifying the student’s features through feature reconstruction. In the second stage, we propose diffusion-guided feature distillation (DGFD), which treats the stage-one distilled features as a noisy version of the teacher's representations and refines them via a lightweight diffusion network. Furthermore, we introduce a noise adapter that adaptively aligns the noise level of the feature with a predefined diffusion timestep, enabling a more precise denoising. Extensive experiments on the VoD and in-house datasets demonstrate that MSDNet achieves both high-fidelity reconstruction and low-latency inference in the task of 4D radar point cloud super-resolution, and consistently improves performance on downstream tasks.

Keywords: Multi-Robot SLAM, SLAM, Localization
Abstract: Decentralized Collaborative Simultaneous Localization And Mapping (C-SLAM) techniques often struggle to identify map overlaps due to significant viewpoint variations among robots. Motivated by recent advancements in 3D foundation models, which can register images despite large viewpoint differences, we propose a robust loop closing approach that leverages these models to establish inter-robot measurements. In contrast to resource-intensive methods requiring full 3D reconstruction within a centralized map, our approach integrates foundation models into existing SLAM pipelines, yielding scalable and robust multi-robot mapping. Our contributions include: (1) integrating 3D foundation models to reliably estimate relative poses from monocular image pairs within decentralized C-SLAM; (2) introducing robust outlier mitigation techniques critical to the use of these relative poses; and (3) developing specialized pose graph optimization formulations that efficiently resolve scale ambiguities. We evaluate our method against state-of-the-art approaches, demonstrating improvements in localization and mapping accuracy, alongside significant gains in computational and memory efficiency. These results highlight the potential of our approach for deployment in large-scale multi-robot scenarios.

Keywords: Robust/Adaptive Control, Optimization and Optimal Control, Motion Control
Abstract: We propose PRED-MPPI, the first MPPI variant that seamlessly integrates real-time disturbance preview and adaptive discretization for quadrotor tracking control under significant model inaccuracies and time-varying disturbances. Unlike prior MPPI variants (e.g., mathcal{L}_1-MPPI, DA-MPPI), which assume constant or matched disturbances, PRED-MPPI leverages a high-order Generalized Extended State Observer for disturbance preview and a Variable Discretization Grid (VDG) to reduce computation and control variance. The synergy enables real-time (50 Hz) quadrotor control under time-varying and mismatched disturbances. Extensive comparative simulation and real-world Crazyflie experiments demonstrate substantial performance gains. In AirSim simulation, PRED-MPPI reduces computation time by over 30%, and mean RMSE by 10.3%, 13.5%, and 14.6% compared to baseline MPPI, and by 2.59%, 3.62%, and 5.80% compared to DA-MPPI across three representative scenarios. In real-world Crazyflie experiments, for ground-effect-disturbed hovering, PRED-MPPI reduces mean and standard deviation (Std) of X–Y plane error by 14.2%/17.9% and 6.03%/21.6% compared to MPPI and DA-MPPI; for fan-induced wind experiments, PRED-MPPI yields improvements of 23.4%/36.8% and 13.8%/25.0% in RMSE and tracking error Std. These results establish PRED-MPPI as the first disturbance-preview MPPI achieving real-world UAV robustness and efficiency, paving the way for deployment on resource-limited robotic platforms. GitHub page with videos is at https://pred-mppi.github.io/

Keywords: Modeling, Control, and Learning for Soft Robots, Deep Learning in Grasping and Manipulation, Telerobotics and Teleoperation
Abstract: Soft robotic hands promise to provide compliant and safe interaction with objects and environments. However, designing soft hands to be both compliant and functional across diverse use cases remains challenging. Although co-design of hardware and control better couples morphology to behavior, the resulting search space is high-dimensional, and even simulation-based evaluation is computationally expensive. In this paper, we propose a Cross-Entropy Method with Reward Model (CEM-RM) framework that efficiently optimizes tendon-driven soft robotic hands based on teleoperation control policy, reducing design evaluations by more than half compared to pure optimization while learning a distribution of optimized hand designs from pre-collected teleoperation data. We derive a design space for a soft robotic hand composed of flexural soft fingers and implement parallelized training in simulation. The optimized hands are then 3D-printed and deployed in the real world using both teleoperation data and real-time teleoperation. Experiments in both simulation and hardware demonstrate that our optimized design significantly outperforms baseline hands in grasping success rates across a diverse set of challenging objects.

Keywords: Deep Learning Methods, Motion and Path Planning, Learning from Demonstration

Keywords: Deep Learning in Grasping and Manipulation, Grippers and Other End-Effectors, Imitation Learning
Abstract: This paper presents GSWorld, a robust, photo-realistic simulator for robotics manipulation that combines 3D Gaussian Splatting with physics engines. Our framework advocates ‘closing the loop’ of developing manipulation policies with reproducible evaluation of policies learned from real-robot data and sim2real policy training without using real robots. To enable photo-realistic rendering of diverse scenes, we propose a new asset format, which we term GSDF (Gaussian Scene Description File), that infuses Gaussian-on-Mesh representation with robot URDF and other objects. With a streamlined reconstruction pipeline, we curate a database of GSDF that contains 3 robot embodiments for single-arm and bimanual manipulation, as well as more than 40 objects. Combining GSDF with physics engines, we demonstrate several immediate interesting applications: (1) learning zero-shot sim2real pixel-to- action manipulation policy with photo-realistic rendering, (2) automated high-quality DAgger data collection for adapting policies to deployment environments, (3) reproducible bench- marking of real-robot manipulation policies in simulation, (4) simulation data collection by virtual teleoperation, and (5) sim2real visual reinforcement learning. We plan to open-source both the simulation assets and code.

Keywords: Prosthetics and Exoskeletons, Rehabilitation Robotics, Deep Learning Methods
Abstract: Lower-limb exoskeleton robots play a significant role in both rehabilitation and assisted walking, where accurate prediction of lower-limb joint angles is crucial for achieving natural gait. However, due to inter-subject variability and differences across locomotion modes, achieving cross-task generalization in joint angle prediction remains a major challenge. This work proposes a novel framework for multi-joint angle prediction in the lower-limb, which includes a non-redundant muscle synergy feature extraction algorithm and a Generalizable Joint Angle Prediction Network (GenJAPNet) across speeds and subjects. The feature extraction algorithm employs Non-negative Matrix Factorization (NMF) to extract activation coefficient matrix from Surface Electromyography (sEMG) signals, followed by further dimensionality reduction using Uniform Manifold Approximation and Projection (UMAP) to obtain more discriminative and non-redundant features. GenJAPNet leverages pre-trained shared features and few-shot fine-tuning to rapidly adapt to new task. Through feature extraction algorithm comparison experiments, cross-speed and cross-subject experiments, and exoskeleton-assisted walking physical experiments, the effectiveness and generalizability of this method are validated, demonstrating its potential for enhancing the performance of lower-limb exoskeleton rehabilitation and assistive applications.

Keywords: Human-Aware Motion Planning, Reinforcement Learning, Human-Centered Robotics
Abstract: The demand for assistive robots for passenger transport, such as intelligent wheelchairs, is increasing rapidly due to demographic changes. To allow passengers to navigate in crowded environments, such as shopping malls and hospitals, these systems must navigate in a socially accepted manner that ensures the comfort of both passengers and surrounding pedestrians. Although deep reinforcement learning (DRL) has shown promising results for social navigation, existing planners often learn overly passive behaviors, not engaging in the mutual adaptation characteristic of human interaction. In this paper, we introduce a novel DRL-based local planner that learns navigation behaviors by integrating the Social Force Model (SFM) directly into its reward function, allowing more cooperative interactions for mobile robots and intelligent wheelchairs. This approach encourages the agent to learn more forward-looking and mutual navigation policies by rewarding actions that align with the dynamics of pedestrians. To ensure generalization and straightforward deployment, our method utilizes the standard Navigation 2 local costmap augmented with pedestrian detections as an observation. The experiments demonstrate that our agent achieves a higher success rate in crowded scenarios with fewer space intrusions, outperforming the state-of-the-art DRL planner based on velocity obstacles by up to 11%.

Keywords: Agent-Based Systems, Force and Tactile Sensing, AI-Enabled Robotics
Abstract: Effectively integrating diverse sensory modalities is crucial for robotic manipulation. However, the typical approach of feature concatenation is often suboptimal: dominant modalities such as vision can overwhelm sparse but critical signals like touch in contact-rich tasks, and monolithic architectures cannot flexibly incorporate new or missing modalities without retraining. Our method factorizes the policy into a set of diffusion models, each specialized for a single representation (e.g., vision or touch), and employs a router network that learns consensus weights to adaptively combine their contributions, enabling incremental of new representations. We evaluate our approach on simulated manipulation tasks in {RLBench}, as well as real-world tasks such as occluded object picking, in-hand spoon reorientation, and puzzle insertion, where it significantly outperforms feature-concatenation baselines on scenarios requiring multimodal reasoning. Our policy further demonstrates robustness to physical perturbations and sensor corruption. We further conduct perturbation-based importance analysis, which reveals adaptive shifts between modalities.

Keywords: Grippers and Other End-Effectors, Dexterous Manipulation, Grasping
Abstract: Handling fragile objects at high speeds presents a significant challenge. To address this challenge, researchers explored hybrid grippers and grippers with adjustable stiffness. However, grippers exhibiting anisotropic stiffness have received little attention. This paper presents an extreme case of an anisotropic stiffness gripper designed for high-speed handling of delicate foods. The proposed gripper incorporates linear motors and low-friction linear guides. Its extremely low friction in the grasping direction ensures minimal grasping force (less than 0.1 N) and high compliance, enabling the secure handling of fragile objects. Simultaneously, its rigid structure provides sufficient stiffness in the translational direction, ensuring stability during high-speed motion. Leveraging anisotropic stiffness, the gripper achieves both gentle grasp and stable high-speed translation--two requirements that typically necessitate a trade-off. Theoretical analysis was conducted to determine the maximum permissible acceleration under two conditions: with and without stiffness anisotropy. Results indicate that stiffness anisotropy enables significantly higher acceleration during translational motion, thereby reducing task time. Pick-and-place experiments on a 3D printed object, delicate foods of a potato chip, a block of tofu, and a piece of dried seaweed validated theoretical findings and demonstrated the gripper's capability to handle fragile objects at high speeds effectively.

Keywords: Intelligent Transportation Systems, Computer Vision for Transportation, Cooperating Robots
Abstract: Collaborative perception enhances the reliability and spatial coverage of autonomous vehicles by sharing complementary information across vehicles, offering a promising solution to long-tail scenarios that challenge single-vehicle perception. However, the bandwidth constraints of vehicular networks make transmitting the entire feature map infeasible. Recent methods therefore adopt a foreground-centric paradigm, transmitting only predicted foreground region features but discarding background which encodes essential context. We propose FadeLead, a foreground-centric framework that overcomes this limitation by learning to encapsulate background context into compact foreground features during training. At the core of our design is a curricular learning strategy that leverages background cues early on but progressively prunes them away, forcing the model to internalize context into foreground representations without transmitting background itself. Extensive experiments on both simulated and real-world scenarios show that FadeLead outperforms prior methods under different bandwidth settings, underscoring the effectiveness of context-enriched foreground sharing. Code, demos, and checkpoints are available at https://wyhallenwu.github.io/FadeLead/.

Keywords: Force and Tactile Sensing, Sensor Fusion, Perception for Grasping and Manipulation
Abstract: Visuotactile sensors typically employ sparse marker arrays that limit spatial resolution and lack clear analytical force-to-image relationships. To solve this problem, we present MoiréTac, a dual-mode sensor that generates dense interference patterns via overlapping micro-gratings within a transparent architecture. When two gratings overlap with misalignment, they create moiré patterns that amplify microscopic deformations. The design preserves optical clarity for vision tasks while producing continuous moiré fields for tactile sensing, enabling simultaneous 6-axis force/torque measurement, contact localization, and visual perception. We combine physics-based features (brightness, phase gradient, orientation, and period) from moiré patterns with deep spatial features. These are mapped to 6-axis force/torque measurements, enabling interpretable regression through end-to-end learning. Experimental results demonstrate three capabilities: force/torque measurement with R²>0.98 across tested axes; sensitivity tuning through geometric parameters (threefold gain adjustment); and vision functionality for object classification despite moiré overlay. Finally, we integrate the sensor into a robotic arm for cap removal with coordinated force and torque control, validating its potential for dexterous manipulation.

Keywords: SLAM, Mapping, Intelligent Transportation Systems
Abstract: This paper introduces Dr-PoGO, a method for Simultaneous Localization And Mapping (SLAM) using a 2D spinning radar. Unlike cameras or lidars that require line-of-sight, millimetre-wave radars can `see' through dust, falling snow, rain, etc. Accordingly, it is a great modality for robust perception regardless of the weather conditions. While most existing radar-based SLAM methods rely on the extraction of point clouds or features to perform ego-motion estimation, Dr-PoGO leverages direct registration techniques for odometry (DRO) and loop-closure registration. An off-the-shelf radar-focused place recognition algorithm, RaPlace, provides loop-closure candidates. As RaPlace does not provide relative transformations, Dr-PoGO introduces a coarse-to-fine registration that uses visual features and descriptors to obtain an initial guess for the direct transformation refinement. The global trajectory is optimized in a pose-graph optimization. Dr-PoGO demonstrates state-of-the-art performance over 300km of data in various real-world automotive environments. Our implementation is publicly available: https://github.com/utiasASRL/dr_pogo.

Keywords: Swarm Robotics, Aerial Systems: Applications
Abstract: Due to the limited online computational resources and the inherent probability of hardware and software failures of real-world robots, large-scale formation planning faces two common challenges: computational intractability and agent failures. Based on the theory of sparse graphs and the maximum clique, {we achieve a resilient and efficient formation planning (mathbf{RE}-mathbf{Formation}) to address these issues.} To improve the computational efficiency of trajectory planning while ensuring flexible formation maneuvers, we introduce sparse graphs to describe connection relationships and present a sparse graph construction method with closed-form solutions. The sparse graphs ensure the {}{underline{G}}lobal {}{underline{R}}igidity for uniquely corresponding to a geometric shape and {}{underline{P}}reserve the main {}{underline{F}}eatures of complete graphs, {}{denoted as the mathbf{GRPF} sparse graph}. To prevent the impact of abnormal agents, the problem of eliminating abnormal agents is transformed into an outlier rejection problem that can be solved by computing the maximum clique. {}{We approximate the maximum clique by periodically triggering the calculation of the maximum k-core to meet the real-time computational demands of large-scale swarms.} We validate the performance through real-world experiments and implement formation planning with 100 drones in simulation. Benchmark comparisons and ablation experiments demonstrate the effectiveness of

Keywords: Aerial Systems: Perception and Autonomy, Product Design, Development and Prototyping
Abstract: An aircraft's airspeed, angle of attack, and angle of side slip are crucial to its safety, especially when flying close to the stall regime. Various solutions exist, including pitot tubes, angular vanes, and multihole pressure probes. However, current sensors are either too heavy (>30 g) or require large airspeeds (>20 m/s), making them unsuitable for small uncrewed aerial vehicles. We propose a novel multihole pressure probe, integrating sensing electronics in a single-component structure, resulting in a mechanically robust and lightweight sensor (9 g), which we released to the public domain. Since there is no consensus on two critical design parameters, tip shape (conical vs spherical) and hole spacing (distance between holes), we provide a study on measurement accuracy and noise generation using wind tunnel experiments. The sensor is calibrated using a multivariate polynomial regression model over an airspeed range of 3-27 m/s and an angle of attack/sideslip range of +/-35 deg, achieving a mean absolute error of 0.44 m/s and 0.16 deg. Finally, we validated the sensor in outdoor flights near the stall regime. Our probe enabled accurate estimations of airspeed, angle of attack and sideslip during different acrobatic manoeuvres. Due to its size and weight, this sensor will enable safe flight for lightweight, uncrewed aerial vehicles flying at low speeds close to the stall regime.

Keywords: Motion and Path Planning, Collision Avoidance, Mobile Manipulation
Abstract: Differential drive mobile manipulators combine the mobility of wheeled bases with the manipulation capability of multi-joint arms, enabling versatile applications but posing considerable challenges for trajectory planning due to their high-dimensional state space and nonholonomic constraints. This paper introduces TopAY, an optimization-based planning framework designed for efficient and safe trajectory generation for differential drive mobile manipulators. The framework employs a hierarchical initial value acquisition strategy, including topological paths search for the base and parallel sampling for the manipulator. A polynomial trajectory representation with arc length–yaw parameterization is also proposed to reduce optimization complexity while preserving dynamic feasibility. Extensive simulation and real-world experiments validate that TopAY achieves higher planning efficiency and success rates than state-of-the-art method in dense and complex scenarios. The source code is released at https://github.com/TopAY-Planner/TopAY.

Keywords: Industrial Robots, SLAM, Localization
Abstract: Reliable, drift-free global localization presents significant challenges yet remains crucial for autonomous navigation in large-scale dynamic environments. In this paper, we introduce a tightly-coupled Semantic-LiDAR-Inertial-Wheel Odometry fusion framework, which is specifically designed to provide high-precision state estimation and robust localization in large-scale dynamic environments. Our framework leverages an efficient semantic-voxel map representation and employs an improved scan matching algorithm, which utilizes global semantic information to significantly reduce long-term trajectory drift. Furthermore, it seamlessly fuses data from LiDAR, IMU, and wheel odometry using a tightly-coupled multi-sensor fusion Iterative Error-State Kalman Filter (iESKF). This ensures reliable localization without experiencing abnormal drift. Moreover, to tackle the challenges posed by terrain variations and dynamic movements, we introduce a 3D adaptive scaling strategy that allows for flexible adjustments to wheel odometry measurement weights, thereby enhancing localization precision. This study presents extensive real-world experiments conducted in a one-million-square-meter automated port, encompassing 3,575 hours of operational data from 35 Intelligent Guided Vehicles (IGVs). The results consistently demonstrate that our system outperforms state-of-the-art LiDAR-based localization methods in large-scale dynamic environments, highlighting the framework's reliability and practical value.

Keywords: Multi-Robot Systems, Optimization and Optimal Control, Human-Robot Collaboration
Abstract: In this paper, we tackle real-time formation trajectory planning for collaborative object transportation in complex environments using a team of nonholonomic robots and a human. The object is transported in a deformable sheet, and robots should follow the human’s lead while autonomously avoiding obstacles. By including a human in the formation, we leverage their adaptability and decision-making to improve transportation. However, it can be difficult for a human to predict how autonomous robots will behave in complex situations, such as when the formation must cross an obstacle, i.e. where the object is transported above it. This could cause human decisions that compromise safety. To overcome these challenges, we introduce a multi-modal formation planning framework. By default the human leads the formation, and the robots plan to remain in the same homotopy class as the human to avoid collisions. If obstacle crossing is necessary the robots take the lead of the formation, where human motion is constrained to a feasible region projected visually in front of them. We demonstrate the efficacy of our framework in simulation and on hardware.

Keywords: Multi-Robot Systems, Reinforcement Learning
Abstract: Accounting for heterogeneity among robots and tasks adds additional complexity to multi-robot task allocation. While existing task allocation methods effectively handle heterogeneity among robots and tasks, they do not scale well in the number of different robots and tasks. To address this gap, we formulate the Team Formation Markov Decision Process (TF-MDP) for training Teamformer: a scalable, decentralized transformer policy for dynamically forming heterogeneous teams of robots to complete diverse tasks. Combining the TF-MDP with the autoregressive capability of transformers enables Teamformer to scale linearly in the number of robots, tasks, and combinations of different heterogeneous robots. Simulations demonstrate Teamformer generalizing to combinations of 100 different types of robots and tasks. Hardware experiments using Georgia Tech's Robotarium show Teamformer decentrally coordinating up to 20 heterogeneous robots for task completion.

Keywords: Localization, Range Sensing, SLAM
Abstract: Light Detection and Ranging (LiDAR) sensors have become a de-facto sensor for many robot state estimation tasks, spurring development of many LiDAR Odometry (LO) methods in recent years. While some smoothing-based LO methods have been proposed, most require matching against multiple scans, resulting in sub-real-time performance. Due to this, most prior works estimate a single state at a time and are ``submap''-based. This architecture propagates any error in pose estimation to the fixed submap and can cause jittery trajectories and degrade future registrations. We propose Fixed-Lag Odometry with Reparative Mapping (FORM), a LO method that performs smoothing over a densely connected factor graph while utilizing a single iterative map for matching. This allows for both real-time performance and active correction of the local map as pose estimates are further refined. We evaluate on a wide variety of datasets to show that FORM is robust, accurate, real-time, and provides smooth trajectory estimates when compared to prior state-of-the-art LO methods.

Keywords: Deep Learning in Grasping and Manipulation, Imitation Learning, Representation Learning
Abstract: End-to-end learning is emerging as a powerful paradigm for robotic manipulation, but its effectiveness is limited by data scarcity and the heterogeneity of action spaces across robot embodiments. In particular, diverse action spaces across different end-effectors create barriers for cross-embodiment learning and skill transfer. We address this challenge through diffusion policies learned in a latent action space that unifies diverse end-effector actions. We first show that we can learn a semantically aligned latent action space for anthropomorphic robotic hands, a human hand, and a parallel jaw gripper using encoders trained with a contrastive loss. Second, we show that by using our proposed latent action space for co-training on manipulation data from different end-effectors, we can utilize a single policy for multi-robot control and obtain up to 25.3% improved manipulation success rates, indicating successful skill transfer despite a significant embodiment gap. Our approach using latent cross-embodiment policies presents a new method to unify different action spaces across embodiments, enabling efficient multi-robot control and data sharing across robot setups. This unified representation significantly reduces the need for extensive data collection for each new robot morphology, accelerates generalization across embodiments, and ultimately facilitates more scalable and efficient robotic learning.

Keywords: RGB-D Perception, Computer Vision for Transportation, Deep Learning for Visual Perception
Abstract: We present a novel approach, termed ADGaussian, for generalizable street scene reconstruction. The proposed method enables high-quality rendering from merely single-view input. Unlike prior Gaussian Splatting methods that primarily focus on geometry refinement, we emphasize the importance of joint optimization of image and depth features for accurate Gaussian prediction. To this end, we first incorporate sparse LiDAR depth as an additional input modality, formulating the Gaussian prediction process as a joint learning framework of visual information and geometric clue. Furthermore, we propose a Multi-modal Feature Matching strategy coupled with a Multi-scale Gaussian Decoding model to enhance the joint refinement of multi-modal features, thereby enabling efficient multi-modal Gaussian learning. Extensive experiments on Waymo and KITTI demonstrate that our ADGaussian achieves state-of-the-art performance and exhibits superior zero-shot generalization capabilities in novel-view shifting.

Keywords: Deep Learning for Visual Perception, Sensor Fusion, Semantic Scene Understanding
Abstract: RGB-thermal (RGB-T) semantic segmentation improves the environmental perception of autonomous platforms in challenging conditions. Prevailing models employ encoders pre-trained on RGB images to extract features from both RGB and infrared inputs, and design additional modules to achieve cross-modal feature fusion. This results in limited thermal feature extraction and suboptimal cross-modal fusion, while the redundant encoders further compromises the model’s real-time efficiency. To address the above issues, we propose TUNI, with an RGB-T encoder consisting of multiple stacked blocks that simultaneously perform multi-modal feature extraction and cross-modal fusion. By leveraging large-scale pre-training with RGB and pseudo-thermal data, the RGB-T encoder learns to integrate feature extraction and fusion in a unified manner. By slimming down the thermal branch, the encoder achieves a more compact architecture. Moreover, we introduce an RGB-T local module to strengthen the encoder’s capacity for cross-modal local feature fusion. The RGB-T local module employs adaptive cosine similarity to selectively emphasize salient consistent and distinct local features across RGB-T modalities. Experimental results show that TUNI achieves competitive performance with state-of-the-art models on FMB, PST900 and CART, with fewer parameters and lower computational cost. Meanwhile, it achieves an inference speed of 27 FPS on a Jetson Orin NX, demonstrating its real-time capability in deployment. Codes are available at https://github.com/xiaodonguo/TUNI.

Keywords: Swarm Robotics, Localization, Sensor Fusion
Abstract: The autonomous cooperation of micro-Unmanned Aerial Vehicle (UAV) swarms remain key challenges. Existing swarm relative positioning and control methods demand high sensing, computing, and communication resources and rely on external equipment like GPS and ground stations. To address these issues, this paper proposes an all-onboard and external-aiding-free swarm relative measurement, positioning and control framework. The framework utilizes an onboard Vision-Optoelectronic-Ultra-Wideband (UWB) coupled measurement system to acquire inter-UAV relative distance and direction. Subsequently, the swarm's relative positions are solved via a distributed graph optimization (DGO) approach. Based on the solved relative positions, swarm cooperative control is implemented through a distributed Voronoi diagram approach. Experimental results demonstrate that the proposed method enables 150 g micro-UAVs to achieve nearly 100-meter autonomous outdoor formation flight and collaborative tracking of dynamic targets, with swarm relative localization accuracy reaching approximately 0.262 m. This work pioneers fully autonomous measurement and control for 100-gram scale UAV swarms without external infrastructure, significantly advancing autonomy and enabling swarm intelligence emergence.

Keywords: Multifingered Hands, Compliant Joints and Mechanisms
Abstract: Biological synergies have emerged as a widely adopted paradigm for dexterous hand design, enabling human-like manipulation with a small number of actuators. Nonetheless, excessive coupling tends to diminish the dexterity of hands. This paper tackles the trade-off between actuation complexity and dexterity by proposing an anthropomorphic finger topology with 4-DoFs driven by 2 actuators, and by developing an adaptive, modular dexterous hand based on this finger topology. We explore the biological basis of hand synergies and human gesture analysis, translating joint-level coordination and structural attributes into a modular finger architecture. Leveraging these biomimetic mappings, we design a five-finger modular hand and establish its kinematic model to analyze adaptive grasping and in-hand manipulation. Finally, we construct a physical prototype and conduct preliminary experiments, which validate the effectiveness of the proposed design and analysis.

Keywords: Visual Servoing, Perception for Grasping and Manipulation, Computer Vision for Automation
Abstract: Visual servoing is fundamental to robotic applications, enabling precise positioning and control. However, applying it to textureless objects remains a challenge due to the absence of reliable visual features. Moreover, adverse visual conditions, such as occlusions, often corrupt visual feedback, leading to reduced accuracy and instability in visual servoing. In this work, we build upon learning-based keypoint detection for textureless objects and propose a method that enhances robustness by tightly integrating perception and control in a closed loop. Specifically, we employ an Extended Kalman Filter (EKF) that integrates per-frame keypoint measurements to estimate 6D object pose, which drives pose-based visual servoing (PBVS) for control. The resulting camera motion, in turn, enhances the tracking of subsequent keypoints, effectively closing the perception-control loop. Additionally, unlike standard PBVS, we propose a probabilistic control law that computes both camera velocity and its associated uncertainty, enabling uncertainty-aware control for safe and reliable operation. We validate our approach on real-world robotic platforms using quantitative metrics and grasping experiments, demonstrating that our method outperforms traditional visual servoing techniques in both accuracy and practical application.

Keywords: Computer Vision for Transportation, Deep Learning for Visual Perception, Semantic Scene Understanding
Abstract: Panoramic perception holds significant potential for autonomous driving, enabling vehicles to acquire a comprehensive 360° surround view in a single shot. However, autonomous driving is a data-driven task. Complete panoramic data acquisition requires complex sampling systems and annotation pipelines, which are time-consuming and labor-intensive. Although existing street view generation models have demonstrated strong data regeneration capabilities, they can only learn from the fixed data distribution of existing datasets and cannot achieve high-quality, controllable panoramic generation. In this paper, we propose the first panoramic generation method, Percep360, for autonomous driving. Percep360 enables coherent generation of panoramic data with control signals based on the stitched panoramic data. Percep360 focuses on two key aspects: coherence and controllability. Specifically, to overcome the inherent information loss caused by the pinhole sampling process, we propose the Local Scenes Diffusion Method (LSDM). LSDM reformulates the panorama generation as a spatially continuous diffusion process, bridging the gaps between different data distributions. Additionally, to achieve the controllable generation of panoramic images, we propose a Probabilistic Prompting Method (PPM). PPM dynamically selects the most relevant control cues, enabling controllable panoramic image generation. We evaluate the effectiveness of the generated images from three perspectives: image quality assessment (i.e.,no-reference and with reference), controllability, and their utility in real-world Bird’s Eye View (BEV) segmentation. Notably, the generated data consistently outperforms the original stitched images in no-reference quality metrics and enhances downstream perception models, leading to an improvement of 2.5% in mIoU for panoramic BEV segmentation. The source code will be publicly available.

Keywords: Localization
Abstract: Visual Place Recognition (VPR) enables systems to identify previously visited locations within a map, a fundamental task for autonomous navigation. Prior works have developed VPR solutions using event cameras, which asynchronously measure per-pixel brightness changes with microsecond temporal resolution. However, these approaches rely on dense representations of the inherently sparse camera output and require tens to hundreds of milliseconds of event data to predict a place. Here, we break this paradigm with Flash, a lightweight VPR system that predicts places using sub-millisecond slices of event data. Our method is based on the observation that active pixel locations provide strong discriminative features for VPR. Flash encodes these active pixel locations using efficient binary frames and computes similarities via fast bitwise operations, which are then normalized based on the relative event activity in the query and reference frames. Flash improves Recall@1 for sub-millisecond VPR over existing baselines by 11.33x on the indoor QCR-Event-Dataset and 5.92x on the 8 km Brisbane-Event-VPR dataset. Moreover, our approach reduces the duration for which the robot must operate without awareness of its position, as evidenced by a localization latency metric we term Time to Correct Match (TCM). To the best of our knowledge, this is the first work to demonstrate sub-millisecond VPR using event cameras.

Keywords: Object Detection, Segmentation and Categorization, Computer Vision for Transportation
Abstract: Single-domain generalization is essential for object detection, particularly when training models on a single source domain and evaluating them on unseen target domains. Domain shifts, such as changes in weather, lighting, or scene conditions, pose significant challenges to the generalization ability of existing models. To address this, we propose Cross-Domain Feature Knowledge Distillation (CD-FKD), which enhances the generalization capability of the student network by leveraging both global and instance-wise feature distillation. The proposed method uses diversified data through downscaling and corruption to train the student network, whereas the teacher network receives the original source domain data. The student network mimics the features of the teacher through both global and instance-wise distillation, enabling it to extract object-centric features effectively, even for objects that are difficult to detect owing to corruption. Extensive experiments on challenging scenes demonstrate that CD-FKD outperforms state-of-the-art methods in both target domain generalization and source domain performance, validating its effectiveness in improving object detection robustness to domain shifts. This approach is valuable in real-world applications, like autonomous driving and surveillance, where robust object detection in diverse environments is crucial.

Keywords: Continual Learning, Incremental Learning, Imitation Learning
Abstract: The main challenge in lifelong imitation learning lies in the balance between mitigating catastrophic forgetting of previous skills while maintaining sufficient capacity for acquiring new ones. However, current approaches typically address these aspects in isolation, overlooking their internal correlation in lifelong skill acquisition. We address this limitation with a unified framework named Tokenized Skill Scaling (T2S). Specifically, by tokenizing the model parameters, the linear parameter mapping of the traditional transformer is transformed into cross-attention between input and learnable tokens, thereby enhancing model scalability through the easy extension of new tokens. Additionally, we introduce language-guided skill scaling to transfer knowledge across tasks efficiently and avoid linearly growing parameters. Extensive experiments across diverse tasks demonstrate that T2S: 1) effectively prevents catastrophic forgetting (achieving an average NBT of 1.0% across the three LIBERO task suites), 2) excels in new skill scaling with minimal increases in trainable parameters (needing only 8.0% trainable tokens in an average of lifelong tasks), and 3) enables efficient knowledge transfer between tasks (achieving an average FWT of 77.7% across the three LIBERO task suites), offering a promising solution for lifelong imitation learning.

Keywords: Environment Monitoring and Management, Robotics and Automation in Life Sciences
Abstract: Coral aquaculture for reef restoration requires accurate and continuous spawn counting for resource distribution and larval health monitoring, but current methods are labor-intensive and represent a critical bottleneck in the coral production pipeline. We propose the Coral Spawn and Larvae Imaging Camera System (CSLICS), which uses low cost modular cameras and object detectors trained using human-in-the-loop labeling approaches for automated spawn counting in larval rearing tanks. This paper details the system engineering, dataset collection, and computer vision techniques to detect, classify and count coral spawn. Experimental results from mass spawning events demonstrate an F1 score of 82.4% for surface spawn detection at different embryogenesis stages, 65.3% F1 score for sub-surface spawn detection, and a saving of 5,720 hours of labor per spawning event compared to manual sampling methods at the same frequency. Comparison of manual counts with CSLICS monitoring during a mass coral spawning event on the Great Barrier Reef demonstrates CSLICS' accurate measurement of fertilization success and sub-surface spawn counts. These findings enhance the coral aquaculture process and enable upscaling of coral reef restoration efforts to address climate change threats facing ecosystems like the Great Barrier Reef.

Keywords: Dynamics, Humanoid and Bipedal Locomotion, Motion Control
Abstract: A class of planar bipedal robots with unique mechanical properties has been proposed, where all links are balanced around the hip joint, preventing natural swinging motion due to gravity. A common property of their equations of motion is that the inertia matrix is a constant matrix, there are no nonlinear velocity terms, and the gravity term contains simple nonlinear terms. By performing a Taylor expansion of the gravity term and making a linear approximation, it is easy to derive a linearized model, and calculations for future states or walkability determination can be performed instantaneously without the need for numerical integration. This paper extends the method to a planar biped robot model with knees. First, we derive the equations of motion, constraint conditions, and inelastic collisions for a planar 6-DOF biped robot, design its control system, and numerically generate a stable bipedal gait on a horizontal plane. Next, we reduce the equations of motion to a 3-DOF model, and derive a linearized model by approximating the gravity term as linear around the expansion point for the thigh frame angle. Through numerical simulations, we demonstrate that calculations for future states and walkability determination can be completed in negligible time. By applying control inputs to the obtained model, performing state-space realization, and then discretizing it, instantaneous walkability determination through iterative calculation becomes possible. Through detailed gait analysis, we discuss how the knee joint flexion angle and the expansion point affect the accuracy of the linear approximation, and the issues that arise when descending a small step.

Keywords: Intelligent Transportation Systems, Autonomous Vehicle Navigation, Imitation Learning
Abstract: Accurate trajectory prediction and motion planning are crucial for autonomous driving systems to navigate safely in complex, interactive environments characterized by multimodal uncertainties. However, current generation-then-evaluation frameworks typically construct multiple plausible trajectory hypotheses but ultimately adopt a single most likely outcome, leading to overconfident decisions and a lack of fallback strategies that are vital for safety in rare but critical scenarios. Moreover, the usual decoupling of prediction and planning modules could result in socially inconsistent or unrealistic joint trajectories, especially in highly interactive traffic. To address these challenges, we propose a contingency-aware diffusion planner (CoPlanner), a unified framework that jointly models multi-agent interactive trajectory generation and contingency-aware motion planning. Specifically, the pivot-conditioned diffusion mechanism anchors trajectory sampling on a validated, shared short-term segment to preserve temporal consistency, while stochastically generating diverse long-horizon branches that capture multimodal motion evolutions. In parallel, we design a contingency-aware multi-scenario scoring strategy that evaluates candidate ego trajectories across multiple plausible long-horizon evolution scenarios, balancing safety, progress, and comfort. This integrated design preserves feasible fallback options and enhances robustness under uncertainty, leading to more realistic interaction-aware planning. Extensive closed-loop experiments on the nuPlan benchmark demonstrate that CoPlanner consistently surpasses state-of-the-art methods on both Val14 and Test14 datasets, achieving significant improvements in safety and comfort under both reactive and non-reactive settings. The source code is available on GitHub.

Keywords: Humanoid and Bipedal Locomotion, Humanoid Robot Systems, Reinforcement Learning
Abstract: This paper tackles the challenge of enabling real-world humanoid robots to perform expressive and dynamic whole-body motions while maintaining stability. We propose ExBody2, a whole-body tracking framework trained in simulation with Reinforcement Learning and then transferred to the real world. The framework decouples keypoint tracking from velocity control and leverages a privileged teacher policy to distill precise mimic skills into the student policy, enabling robust, high-fidelity reproduction of complex motions such as walking, crouching, and dancing. A significant contribution is the identification of an empirical trade-off between feasibility and diversity in motion datasets, which guides the development of an automatic dataset curation method. This principle facilitates pretraining a versatile model generalizing well across diverse motions and can be fine-tuned for specific tasks to achieve superior tracking accuracy. Extensive experiments show that Exbody2 achieves consistently better performance than strong baselines and provides insights that may inform future work on whole-body humanoid control.

Keywords: Semantic Scene Understanding, Deep Learning Methods
Abstract: Abstract— Understanding the surrounding scene geometrically and semantically is a key requirement for autonomously navigating systems. Vision-based 3D panoptic occupancy prediction aims to provide a 3D representation of the surroundingsincluding semantic meaning and identifying individual objectssuch as traffic participants in the context of urban navigation. The majority of vision-based approaches to occupancy prediction require 3D voxel labels or segmented LiDAR scan as supervision signal. While other vision-based approaches use only a few consecutive images for supervision, these approaches typically do not provide instance-level information, which is crucial for achieving a holistic understanding of the scene. In this paper, we propose a novel method for 3D panoptic occupancy prediction that relies solely on image data for both training and inference. We use bundle adjustment to align all available images in the training set to obtain depth information. We further use a pre-trained open-vocabulary image model to obtain panoptic segmentation of the RGB images and generate occupancy pseudo labels to directly optimize for the 3D panoptic occupancy prediction task. Furthermore, we use a 3D foundation model to obtain depth predictions for individual images to add dynamic objects into the pseudo labels. Without any manual or LiDAR-based annotations, our approach outputs occupancy, semantic class, and instance ID for each 3D voxel in the full voxel grid. We achieve state-of-the-art results on 3D semantic occupancy prediction among label-free methods, and we propose the first method for 3D panoptic occupancy without any LiDAR supervision.

Keywords: Imitation Learning, Learning from Demonstration
Abstract: Imitation learning (IL) offers a scalable framework for teaching robots complex manipulation skills from human demonstrations. However, conventional end-to-end visuomotor IL models often suffer from poor performance and robustness due to the significant modality mismatch between high-dimensional visual inputs and low-dimensional motor actions. The redundant information in RGB image, such as color of ambient light, leads models to depend on strong yet brittle task irrelevant priors that ultimately degrade performance across diverse visual environments. To address these limitations, we propose Motion-Aware Two-Stream Policy (MTP) -- a novel imitation learning architecture that explicitly incorporates motion priors via optical flow alongside RGB observations. MTP employs a two-stream perception module that separately encodes spatial (RGB) and temporal (optical flow) information. These spatial-temporal features are fused and fed into a conditional flow matching module to generate actions. We evaluate MTP extensively in both simulation and real-world robot tasks. Results show that MTP significantly outperforms state-of-the-art baselines in terms of success rate and robustness to visual perturbations, demonstrating its effectiveness in generalizable robotic manipulation. To benefit the community, our code will be released.

Keywords: Dual Arm Manipulation, Reinforcement Learning, Dexterous Manipulation
Abstract: Bimanual robot manipulation for long-horizon (LH) tasks is crucial for the practical use of humanoids, but it struggles with robust planning and generalization. Approaches based on Task and Motion Planning (TAMP), transformers, and Large Language Models (LLMs) suffer from critical limitations, including costly human demonstrations, task planner hallucination, and unsatisfactory generalization performance. To address these challenges, this paper introduces the Multi-modal Affordance Planner with Temporal-Context Action Policy (MAP-TCA), a novel hierarchical framework that learns and performs diverse bimanual long-horizon (LH) tasks by generating action plans from MAP. The MAP-TCA consists of a planner based on Bimanual Robot Manipulation Retrieval-Augmented Generation (Bi-RAG)-enhanced Large-Language Model (LLM) and a low-level Temporal Context Action Policy (TCA). With multimodal inputs including vision, language, and affordance for primitive action demonstration, Bi-RAG generates a Primitive Action (PA)-specific embedded space. Then, MAP generates LH plans, LH demonstrations, and reward functions within the PA-specific embedded space, thereby mitigating hallucinations and reducing training cost. The generated plan, demos, and rewards then guide TCA, which learns the LH tasks via behavior cloning (BC) and online fine-tuning. We demonstrate that the proposed MAP-TCA achieves an average success rate of 86.75%, comparable to a baseline model, TCA, which is trained extensively on direct human demonstrations and manually designed rewards. Our work presents a scalable and generalizable solution for complex bimanual LH manipulation, significantly reducing the dependency on human supervision.

Keywords: Object Detection, Segmentation and Categorization, Sensor Fusion, Computer Vision for Automation
Abstract: Multi-modal 3D object detection is pivotal for autonomous driving, integrating complementary sensors like LiDAR and cameras. However, its real-world reliability is challenged by transient data interruptions and missing, where modalities can momentarily drop due to hardware glitches, adverse weather, or occlusions. This poses a critical risk, especially during a simultaneous modality drop, where the vehicle is momentarily blind. To address this problem, we introduce ModalPatch, the first plug-and-play module designed to enable robust detection under arbitrary modality-drop scenarios. Without requiring architectural changes or retraining, ModalPatch can be seamlessly integrated into diverse detection frameworks. Technically, ModalPatch leverages the temporal nature of sensor data for perceptual continuity, using a history-based module to predict and compensate for transiently unavailable features. To improve the fidelity of the predicted features, we further introduce an uncertainty-guided cross-modality fusion strategy that dynamically estimates the reliability of compensated features, suppressing biased signals while reinforcing informative ones. Extensive experiments show that ModalPatch consistently enhances both robustness and accuracy of state-of-the-art 3D object detectors under diverse modality-drop conditions. Code will be available at https://github.com/Castiel-Lee/MM3Det_MD.

Keywords: Deep Learning for Visual Perception, Localization
Abstract: Event cameras offer high-temporal-resolution sensing that remains reliable under high-speed motion and challenging lighting, making them promising for localization from LiDAR point clouds in GPS-denied and visually degraded environments. However, aligning sparse, asynchronous events with dense LiDAR maps is fundamentally ill-posed, as direct correspondence estimation suffers from modality gaps. We propose LEAR, a dual-task learning framework that jointly estimates edge structures and dense event–depth flow fields to bridge the sensing-modality divide. Instead of treating edges as a post-hoc aid, LEAR couples them with flow estimation through a cross-modal fusion mechanism that injects modality-invariant geometric cues into the motion representation, and an iterative refinement strategy that enforces mutual consistency between the two tasks over multiple update steps. This synergy produces edge-aware, depth-aligned flow fields that enable more robust and accurate pose recovery via Perspective-n-Point (PnP) solvers. On several popular and challenging datasets, LEAR achieves superior performance over the best prior method. The source code, trained models, and demo videos are made publicly available online.

Keywords: Automation Technologies for Smart Cities
Abstract: In the vision of smart cities, technologies are being developed to enhance the efficiency of urban services and improve residents' quality of life. However, most existing research focuses on optimizing individual services in isolation, without adequately considering the reciprocal interactions among heterogeneous urban services that could yield higher efficiency and improved resource utilization. For example, human couriers could collect traffic and air quality data along their delivery routes, while sensing robots could assist with on-demand delivery during peak hours, enhancing both sensing coverage and delivery efficiency. However, the joint optimization of different urban services is challenging due to their potentially conflicting objectives and real-time coordination in dynamic environments. In this paper, we propose UrbanHuRo, a two-layer human-robot collaboration framework for joint optimization of heterogeneous urban services, demonstrated through the examples of crowdsourced delivery and urban sensing. There are two innovative designs in UrbanHuRo, i.e., (i) a scalable distributed MapReduce-based K-Submodular maximization module for efficient order dispatch and (ii) a deep submodular reward reinforcement learning algorithm for sensing route planning. Experimental evaluations on real-world datasets from a food delivery platform demonstrate that our UrbanHuRo improves sensing coverage by 29.7% and courier income by 39.2% on average in most settings, while also significantly reducing the number of overdue orders.

Keywords: Multi-Robot Systems, Swarm Robotics
Abstract: In this paper, we present a receding-horizon, sampling-based planner capable of reasoning over multimodal policy distributions. By using the cross-entropy method to optimize a multimodal policy under a common cost function, our approach increases robustness against local minima and promotes effective exploration of the solution space. We show that our approach naturally extends to multi-robot collision-free planning, enables agents to share diverse candidate policies to avoid deadlocks, and allows teams to minimize a global objective without incurring the computational complexity of centralized optimization. Numerical simulations demonstrate that employing multiple modes significantly improves success rates in trap environments and in multi-robot collision avoidance. Hardware experiments further validate the approach's real-time feasibility and practical performance.

Keywords: AI-Enabled Robotics, Deep Learning Methods, Learning from Demonstration
Abstract: While Vision–Language–Action (VLA) models map visual inputs and language instructions directly to robot actions, they often rely on costly hardware and struggle in novel or cluttered scenes. We introduce EverydayVLA, a 6-DOF manipulator that can be assembled for 300, capable of modest payloads and workspaces. A single unified model jointly outputs discrete and continuous actions, and our adaptive-horizon ensembler monitors motion uncertainty to trigger on-the-fly replanning for safe, reliable operation. On LIBERO, EverydayVLA matches state-of-the-art success rates, and in real-world tests it outperforms prior methods by 49% in-distribution and 34.9% out-of-distribution. By combining a state-of-the-art VLA with cost-effective hardware, EverydayVLA democratizes access to a robotic foundation model, and paves the way for economical use in homes and research labs alike.

Keywords: Tendon/Wire Mechanism, Flexible Robotics, Force and Tactile Sensing
Abstract: This work introduces an analytical approach for detecting and estimating external forces acting on deformable linear objects (DLOs) using only their observed shapes. In many robot-wire interaction tasks, contact occurs not at the end-effector but at other points along the robot’s body. Such scenarios arise when robots manipulate wires indirectly (e.g., by nudging) or when wires act as passive obstacles in the environment. Accurately identifying these interactions is crucial for safe and efficient trajectory planning, helping to prevent wire damage, avoid restricted robot motions, and mitigate potential hazards. Existing approaches often rely on expensive external force-torque sensor or that contacts occur at the end-effector for accurate force estimation. Using wire shape information acquired from a depth camera and under the assumption that the wire is in or near its static equilibrium, our method estimates both the location and magnitude of external forces without additional prior knowledge. This is achieved by exploiting derived consistency conditions and solving a system of linear equations based on force-torque balance along the wire. The approach was validated through simulation, where it achieved high accuracy, and through real-world experiments, where accurate estimation was demonstrated in selected interaction scenarios.

Keywords: Data Sets for Robot Learning, Deep Learning in Grasping and Manipulation, Deep Learning for Visual Perception
Abstract: Robotic grasp has been employed in various industrial, household, and medical applications. However, neglecting the final grasping state and objects' stiffness and affordance, prevailing strategies predominantly emphasize the grippers’ initial state upon reaching grasp positions and often fail due to damage or grasp slippage. Here, we propose an AdapGrasp strategy with a dataset named AdapGraspDataset and a corresponding model named AdapGraspNet. The dataset focuses on the object stiffness and grasp affordance. Specifically, for objects with different stiffness properties, the corresponding final grasp width (FGW) is annotated to ensure the object's intactness. For objects’ grasp affordance properties, higher grasp affordance weight (GAW) is typically annotated closer to the centroid, increasing grasp stability. Meanwhile, to output the set of grasping configurations (initial and final grasp states) more accurately, a denoising principle is introduced to build a corresponding transformer-based model. It enables more accurate convergence of FGW and GAW, achieving a precision of 98.04% and a mean absolute final width error of 2.71 pixels. Finally, extensive real-world experiments are conducted, where the AdapGrasp strategy ensures the intactness of fragile objects and thus enhances grasping stability without any additional sensors. It achieves a grasping accuracy of 95% and yields a 19.5% improvement compared with those without FGW and GAW. The AdapGrasp strategy is publicly available at https://embodied-soft-intelligence.github.io/AdapGrasp/.

Keywords: Soft Robot Applications, Intelligent and Flexible Manufacturing, Disassembly
Abstract: This study addresses a flexible holding tool for robotic disassembly. We propose a shell-type soft jig that securely and universally holds objects, mitigating the risk of component damage and adapting to diverse shapes while enabling soft fixation that is robust to recognition, planning, and control errors. The balloon-based holding mechanism ensures proper alignment and stable holding performance, thereby reducing the need for dedicated jig design, highly accurate perception, precise grasping, and finely tuned trajectory planning that are typically required with conventional fixtures. Our experimental results demonstrate the practical feasibility of the proposed jig through performance comparisons with a vise and a jamming-gripper-inspired soft jig. Tests on ten different objects further showed representative successes and failures, clarifying the jig's limitations and outlook.

Keywords: Surgical Robotics: Steerable Catheters/Needles, Medical Robots and Systems, Telerobotics and Teleoperation

Keywords: Computer Vision for Automation, Computer Vision for Transportation, Visual Learning
Abstract: The rapid advancement of 3D scene understanding techniques presents a significant opportunity for enhancing autonomous driving simulation systems. As these systems are increasingly required to operate in complex, large-scale, and unbounded real-world environments, efficient and high-fidelity 3D reconstruction of common outdoor scenes has become a critical prerequisite for realistic and extensible autonomous driving simulation. 3D Gaussian Splatting has achieved state-of-the-art performance in novel view synthesis, coupled with real-time rendering efficiency. However, large-scale reconstruction for autonomous driving scenarios faces several challenges as scenes grow in complexity: (1) limited views with insufficient pose diversity, (2) inadequate representation of geometric structural details, and (3) complex lighting conditions involving saturation and shadow variations. To cope with these challenges, we propose LSADS-Gaussian, a novel model for large-scale autonomous driving scene reconstruction. The model consists of a Multimodal Gaussian Network (MGN) module composed of two Gaussian sub-networks, designed to perform Gaussian aggregation and optimization from multi-sensor data, a Geometric Representation Guidance (GRG) module refines and enhances geometric consistency, and a Lighting Enhancement (LE) module introduces learnable illumination coefficients to maintain illumination consistency. Extensive experiments show that LSADS-Gaussian outperforms the state-of-the-art methods.

Keywords: Humanoid and Bipedal Locomotion, Parallel Robots, Reinforcement Learning
Abstract: Several recently released humanoid robots, in- spired by the mechanical design of Cassie, employ actuator configurations in which the motors are displaced from the joints to reduce leg inertia. While studies accounting for the full kinematic complexity have demonstrated the benefits of these designs, the associated loop-closure constraints greatly increase computational cost and limit their use in control and learning. As a result, the non-linear transmission is often approximated by a constant reduction ratio, preventing exploitation of the mechanism’s full capabilities. This paper introduces a compact analytical formulation for the two standard knee and ankle mechanisms that captures the exact non-linear transmission while remaining computationally efficient. The model is fully differentiable up to second order with a minimal formulation, enabling low-cost evaluation of dynamic derivatives for trajectory optimization and of the apparent transmission impedance for reinforcement learning. We integrate this formulation into trajectory optimization and locomotion policy learning, and compare it against simplified constant-ratio approaches. Hardware experiments demonstrate improved accuracy and robustness, showing that the proposed method provides a practical means to incorporate parallel actuation into modern control algorithms.

Keywords: Medical Robots and Systems, Vision-Based Navigation, Surgical Robotics: Planning
Abstract: Retinal Surgery Robotics is a rapidly emerging field that offers enhanced precision by overcoming human tremors. A key trend of these robotic designs is toward more compact and lightweight structures for improved positioning accuracy and precise force delivery. However, this compactness sacrifices the robot's working volume, making it difficult for ophthalmic surgeons to intuitively assess if retinal targets are accessible by the surgical tool tip. This paper proposes a methodology for visualizing the actual accessible area in the microscopic view to provide surgeons with an intuitive visual guide of the tool's reach, reducing uncertainty and streamlining extraocular robotic maneuvers. We validated this method on a commercial phantom with a surgical robot system, achieving <=1.0 deg error for 83.3% of tested points across four retinal subareas and demonstrating its clinical potential.

Keywords: Deep Learning Methods, SLAM, Localization
Abstract: Robust monocular visual Simultaneous Localization and Mapping (SLAM) serves as a cornerstone for various applications. However, its performance frequently suffers degradation in challenging scenarios including fast motion, dynamic objects, and scale ambiguity. This paper proposes CDV-SLAM, a compact deep visual SLAM framework that unifies geometric and semantic perception through a shared visual foundation model. A tight semantic-geometric fusion network is devised to predict optical flow in fast motion. Semantic features are efficiently reused to obtain segmentation and monocular depth for dynamic objects exclusion and scale acquisition. To further address scale drift, we introduce local scale correction in bundle adjustment. Experimental results demonstrate a 42% decrease in average Absolute Trajectory Error (ATE) on the KITTI dataset over the state-of-the-art. Furthermore, our flow-only visual odometry surpasses geometric-only methods on the TartanAir and EuRoC datasets, with a marginal speed reduction of 6%. Our code is publicly available at https://github.com/FrankYard/CDV-SLAM.

Keywords: Computer Vision for Automation, Autonomous Vehicle Navigation, Simulation and Animation
Abstract: Urban scene reconstruction from real-world observations has emerged as a powerful tool for self-driving development and testing. While current neural rendering approaches achieve high-fidelity rendering along the recorded trajectories, their quality degrades significantly under large viewpoint shifts, limiting the applicability for closed-loop simulation. Recent works have shown promising results in using diffusion models to enhance quality at these challenging viewpoints and distill improvements back into 3D representations. However, they often require costly per-scene optimization, and the distilled representations remain fragile and fail to generalize beyond limited synthesized views. To address these limitations, we propose GenRe, a novel diffusion-guided generalizable enhancer for urban scene reconstruction. GenRe takes as input any pretrained 3D Gaussian representation and fixes the deficiencies within a few minutes. By learning to distill generative priors across diverse scenes, GenRe produces robust and high-fidelity representation efficiently that generalizes reliably to challenging unseen viewpoints (e.g., lane change). Experiments show that GenRe outperforms existing methods in both quality and efficiency and benefits various downstream tasks, enabling robust and scalable sensor simulation for autonomous driving.

Keywords: AI-Based Methods, Agent-Based Systems, Autonomous Agents
Abstract: We present an action sequence transfer system that adaptively transfers user action sequences across different target spaces. Given an input action sequence from a source space and scene graph representations of both the source and target environments, our system predicts a corresponding action sequence in the target space by adapting to the spatial and object constraints of the new environment. To achieve this, we leverage multi-level representations of user activity to generalize actions at varying levels of abstraction. To demonstrate our system, we collect a new scene graph-based dataset derived from the Ego4D GoalStep dataset for valuation. Results indicate that our system can generate valid action sequences even between spaces with drastically different object configurations.

Keywords: Deep Learning for Visual Perception, Computer Vision for Automation, Data Sets for Robotic Vision
Abstract: Transparent object depth perception remains a major challenge in robotics and logistics due to the limitations of standard 3D sensors in capturing accurate depth on transparent and reflective surfaces. This affects applications relying on depth maps and point clouds, particularly in robotic manipulation. To address this, we propose ClearDepth, a vision transformer-based algorithm for stereo depth recovery of transparent objects, enhanced by a novel feature post-fusion module that refines depth estimation using structural visual features. To mitigate the high costs of stereo dataset collection, we introduce a physically realistic, domain-adaptive Sim2Real framework for efficient data generation. Our method outperforms state-of-the-art stereo matching approaches on transparent depth recovery. Furthermore, in transparent object grasping experiments, ClearDepth improves transparent-scene perception and achieves at least an 18% higher grasp success rate compared to the state-of-the-art methods for transparent object manipulation. Our method demonstrates strong Sim2Real generalization, enabling precise depth perception of transparent objects for robotic applications in the real world. Dataset and project details are available at https://sites.google.com/view/cleardepth-anonymous.

Keywords: Aerial Systems: Perception and Autonomy, Localization, Deep Learning for Visual Perception
Abstract: Aerial–ground localization is difficult due to large viewpoint and modality gaps between ground-level LiDAR and overhead imagery. We propose TransLocNet, a cross-modal attention framework that fuses LiDAR geometry with aerial semantic context. LiDAR scans are projected into a bird’s-eye-view representation and aligned with aerial features through bidirectional attention, followed by a likelihood map decoder that outputs spatial probability distributions over position and orientation. A contrastive learning module enforces a shared embedding space to improve cross-modal alignment. Experiments on CARLA and KITTI show that TransLocNet outperforms state-of-the-art baselines, reducing localization error by up to 63% and achieving sub-meter, sub-degree accuracy. These results demonstrate that TransLocNet provides robust and generalizable aerial–ground localization in both synthetic and real-world settings.

Keywords: Deep Learning in Grasping and Manipulation, Perception for Grasping and Manipulation, Manipulation Planning
Abstract: Robotic manipulation of deformable linear objects (DLOs) presents significant challenges due to complex dynamics and frequent self-occlusions. Existing robotic knot tying methods typically rely on precise topological state tracking with ordered keypoints and explicit edge connectivity. This reliance makes them prone to failures due to tracking drift and topology mismatch caused by repeated bending and crossings during knot formation. To address these limitations, we introduce RoboHitch, a novel framework that learns to perform hitch knot tying from human demonstrations using only disordered 3D keypoints and RGB images. This eliminates the need for explicit topological order, allowing for more flexible manipulation. Our method employs a dynamic Graph Autoencoder to extract geometric features from untracked keypoints, complemented by a Convolutional Autoencoder that captures essential visual context. A bidirectional cross-attention mechanism then fuses these modalities to jointly predict pick and place affordances, facilitating implicit reasoning about the rope's state and enabling knot tying under occlusion. Real-world experiments demonstrate the effectiveness and generalizability of our approach, successfully completing hitch knots in scenarios with self-occlusions.

Keywords: Physical Human-Robot Interaction, Learning from Demonstration, Intention Recognition
Abstract: Learning from Demonstration (LfD) for contact-rich tasks faces a fundamental challenge: arbitrating between tracking a demonstrated trajectory and reproducing an interaction force. This paper introduces a novel one-shot LfD framework that resolves this conflict by leveraging the operator's grip force as an intuitive, continuous signal for arbitration. This signal allows the controller to seamlessly transition between a trajectory-tracking impedance controller and a force-tracking admittance controller, prioritizing path accuracy when the demonstrated grip was light and interaction force fidelity when it was firm. To ensure verifiably safe interaction, the adaptive control law is integrated within a dual-layer passivity assurance framework. This mechanism intelligently distributes potentially non-passive energy between an energy tank and adaptive null-space dissipation to guarantee energetic stability. The proposed framework was experimentally validated on a 7-DOF manipulator, demonstrating that the controller autonomously reproduces interaction forces and shows significant robustness against environmental position uncertainties, a scenario where conventional impedance controllers can fail.

Keywords: Visual Tracking, Deep Learning Methods
Abstract: Point cloud single object tracking is critical in autonomous driving. However, current methods heavily rely on frame-by-frame human annotations, which do not scale well with the growing amount of unlabeled LiDAR data. In this paper, we propose the first self-supervised point cloud single object tracking framework, eliminating the need for any manual labels. Our method integrates motion, geometry, and semantic cues to generate plausible object proposals and tracks the target using a predictive filter. Specifically, we generate pseudo labels by clustering local motion patterns from scene flow, while pre-training a proposal network using point cloud forecasting as a proxy task to learn global motion patterns and geometric shape priors. Then, we train the proposal network using the initial pseudo labels and iteratively refine them by treating semantic features as evolving prototypes in each training round. Finally, a simple motion filter is employed to predict the target’s current state based on its past dynamics. Evaluated on KITTI, nuScenes, and Waymo, our self-supervised point cloud single object tracking approach is on par with—and in some cases outperforms—fully supervised trackers, demonstrating that self-supervision is a scalable path forward for 3D single object tracking.

Keywords: Vision-Based Navigation, AI-Enabled Robotics, Task and Motion Planning
Abstract: Vision-and-Language Navigation in Continuous Environments (VLN-CE), which links language instructions to perception and control in the real world, is a core capability of embodied robots. Recently, large-scale pretrained foundation models have been leveraged as shared priors for perception, reasoning, and action, enabling zero-shot VLN without task-specific training. However, existing zero-shot VLN methods depend on costly perception and passive scene understanding, collapsing control to point-level choices. As a result, they are expensive to deploy, misaligned in action semantics, and short-sighted in planning. To address these issues, we present DreamNav that focuses on the following three aspects: (1) for reducing sensory cost, our EgoView Corrector aligns viewpoints and stabilizes egocentric perception; (2) instead of point-level actions, our Trajectory Predictor favors global trajectory-level planning to better align with instruction semantics; and (3) to enable anticipatory and long-horizon planning, we propose an Imagination Predictor to endow the agent with proactive thinking capability. On VLN-CE and real-world tests, DreamNav sets a new zero-shot state-of-the-art (SOTA), outperforming the strongest egocentric baseline with extra information by up to 7.49% and 18.15% in terms of SR and SPL metrics. To our knowledge, this is the first zero-shot VLN method to unify trajectory-level planning and active imagination while using only egocentric inputs.

Keywords: Aerial Systems: Applications, Reinforcement Learning, Vision-Based Navigation
Abstract: Autonomous drone racing has attracted increasing interest as a research topic for exploring the limits of agile flight. However, existing studies primarily focus on obstacle free racetracks, while the perception and dynamic challenges introduced by obstacles remain underexplored, often resulting in low success rates and limited robustness in realworld flight. To this end, we propose a novel vision-based curriculum reinforcement learning framework for training a robust controller capable of addressing unseen obstacles in drone racing. We combine multi-stage cu rriculum learning, domain randomization, and a multi-scene updating strategy to address the conflicting challenges of obstacle avoidance and gate traversal. Our end-to-end control policy is implemented as a single network, allowing high-speed flight of quadrotors in environments with variable obstacles. Both hardware-in-the-loop and real-world experiments demonstrate that our method achieves faster lap times and higher success rates than existing approaches, effectively advancing drone racing in obstacle-rich environments. The video and code are available at: https://github.com/SJTU-ViSYS-team/CRL-Drone-Racing.

Keywords: Mapping, Object Detection, Segmentation and Categorization, Computational Geometry
Abstract: In recent years, parametric representations of point clouds have been widely applied in tasks such as memory-efficient mapping and multi-robot collaboration. Highly adaptive models, like spline surfaces or quadrics, are computationally expensive in detection or fitting. In contrast, real-time methods, such as Gaussian mixture models or planes, have low degrees of freedom, making high accuracy with few primitives difficult. To tackle this problem, a multi-model parametric representation with real-time surface detection and fitting is proposed. Specifically, the Gaussian mixture model is first employed to segment the point cloud into multiple clusters. Then, flat clusters are selected and merged into planes or curved surfaces. Planes can be easily fitted and delimited by a 2D voxel-based boundary description method. Surfaces with curvature are fitted by B-spline surfaces and the same boundary description method is employed. Through evaluations on multiple public datasets, the proposed surface detection exhibits greater robustness than the state-of-the-art approach, with 3.78 times improvement in efficiency. Meanwhile, this representation achieves a 2-fold gain in accuracy over Gaussian mixture models, operating at 36.4 fps on a low-power onboard computer.

Keywords: Intelligent Transportation Systems, Deep Learning Methods
Abstract: The inherent sequential modeling capabilities of autoregressive models make them a formidable baseline for end-to-end planning in autonomous driving. Nevertheless, their performance is constrained by a spatio-temporal misalignment, as the planner must condition future actions on past sensory data. This creates an inconsistent worldview, limiting the upper bound of performance for an otherwise powerful approach. To address this, we propose a Time-Invariant Spatial Alignment (TISA) module that learns to project initial environmental features into a consistent ego-centric frame for each future time step, effectively correcting the agent's worldview without explicit future scene prediction. In addition, we employ a kinematic action prediction head (i.e., acceleration and yaw rate) to ensure physically feasible trajectories. Finally, we introduce a multi-objective post-training stage using Direct Preference Optimization (DPO) to move beyond pure imitation. Our approach provides targeted feedback on specific driving behaviors, offering a more fine-grained learning signal than the single, overall objective used in standard DPO. Our model achieves a state-of-the-art 89.8 PDMS on the NAVSIM dataset among autoregressive models.

Keywords: Learning from Demonstration, Deep Learning in Grasping and Manipulation, Simulation and Animation
Abstract: Imitation learning is a popular paradigm to teach robots new tasks, but collecting robot demonstrations through teleoperation or kinesthetic teaching is tedious and time-consuming. In contrast, directly demonstrating a task using our human embodiment is much easier and data is available in abundance, yet transfer to the robot can be non-trivial. In this work, we propose Real2Gen to train a manipulation policy from a single human demonstration. Real2Gen extracts required information from the demonstration and transfers it to a simulation environment, where a programmable expert agent can demonstrate the task arbitrarily many times, generating an unlimited amount of data to train a flow matching policy. We evaluate Real2Gen on human demonstrations from three different real-world tasks and compare it to a recent baseline. Real2Gen shows an average increase in the success rate of 26.6% and better generalization of the trained policy due to the abundance and diversity of training data. We further deploy our purely simulation-trained policy zero-shot in the real world. We make the data, code, and trained models publicly available at real2gen.cs.uni-freiburg.de.

Keywords: Computational Geometry, Computer Vision for Automation, Collision Avoidance
Abstract: We present a novel dual-end complementary method for online depth estimation and mesh reconstruction, termed DepthMesh. Unlike most existing state-of-the-art methods that produce either only depth online or surface mesh offline, our method tightly couples online multiview depth estimation and Truncated Signed Distance Function (TSDF) reconstruction to achieve fast online mesh reconstruction. For each keyframe from 6DoF tracking, we first obtain the prior depth and normal maps via ultra-fast raycasting from TSDF, which is incrementally fused from historical keyframe depths. Then, these priors, combined with segmentation results, are used to generate local planar hypotheses that optimize both depth accuracy and computational efficiency. Finally, the optimized depth estimates further enhance the accuracy of mesh reconstruction. Through this dual-end complementary mechanism, our system achieves high accuracy and efficiency. Experiments with qualitative and quantitative evaluations on the ScanNetV2 and self-collected datasets demonstrate the effectiveness of our method. Our method can generate depth and mesh online with accuracy (< 3 cm) on mobile devices, which is useful for robotic autonomous navigation and mixed reality applications such as real-time occlusion and collision handling.

Keywords: Motion and Path Planning, Robotics in Hazardous Fields
Abstract: Unmanned aerial vehicles (UAVs) have been widely employed to achieve autonomous exploration of 3D unknown environments. However, most existing algorithms suffer from low exploration efficiency caused by inaccurate motion time cost evaluation, which typically leads to the motion inconsistency during the UAV flight. In this work, we propose a learning-based motion time prediction method for real-time evaluating the accurate motion time costs to candidate viewpoints. Specifically, the prediction method takes the current state of the UAV and its surrounding environment features as input to predict the arrival time to each viewpoint. Based on the motion time cost prediction, the UAV can minimize the time wasted by unnecessary acceleration and deceleration during exploration. To further improve the efficiency, we also develop an optimal exploration target decision algorithm that benefits from the predicted motion time costs and the adaptive upper-bound constraints. Simulation and real-world experiments demonstrate that our method can significantly improve the exploration efficiency and increase the average flight speed of the UAV.

Keywords: Deep Learning for Visual Perception, Computer Vision for Transportation, Semantic Scene Understanding
Abstract: 3D occupancy prediction is critical for comprehensive scene understanding in vision-centric autonomous driving. Recent advances have explored utilizing 3D semantic Gaussians to model occupancy while reducing computational overhead, but they remain constrained by insufficient multi-view spatial interaction and limited multi-frame temporal consistency. To overcome these issues, in this paper, we propose a novel Spatial-Temporal Gaussian Splatting (ST-GS) framework to enhance both spatial and temporal modeling in existing Gaussian-based pipelines. Specifically, we develop a guidance-informed spatial aggregation strategy within a dual-mode attention mechanism to strengthen spatial interaction in Gaussian representations. Furthermore, we introduce a geometry-aware temporal fusion scheme that effectively leverages historical context to improve temporal continuity in scene completion. Extensive experiments on the large-scale nuScenes occupancy prediction benchmark showcase that our proposed approach not only achieves state-of-the-art performance but also delivers markedly better temporal consistency compared to existing Gaussian-based methods.

Keywords: Human Factors and Human-in-the-Loop, Human-Robot Collaboration, Learning from Demonstration
Abstract: Recent evidence has shown that, contrary to expectations, it is difficult for novices to teach robots tasks through learning from demonstration (LfD). Novices often struggle with understanding the relationship between robot states and actions, leading to suboptimal demonstrations. This paper introduces a framework that leverages machine teaching algorithms to train novices in a controlled, ideal environment where optimal control parameters are predefined. The training enables participants to internalise fundamental control principles, preparing them to adapt to new skills that share similar properties. The study evaluates whether such teaching ability is (i) retained beyond the training period (including a long-term follow-up) and (ii) generalised so that novices teach robots more effectively in environments where control parameters are not predefined. It reports a series of between-subjects studies that demonstrate that trained novice teachers achieve a 75% improvement in teaching ability, with these gains retained even after guidance is removed, and exhibit a 71% enhancement in applying skills beyond the training content.

Keywords: Multi-Robot Systems, Integrated Planning and Learning, Task and Motion Planning
Abstract: The multi-robot unlabeled motion planning problem of concurrently assigning robots to goals and generating safe trajectories is central in many collaborative tasks. Recent Graph Neural Network methods offer scalable decentralized solutions but rely on simplified dynamics and simulation environments, overlooking key challenges of real-world deployment such as dynamic feasibility and communication constraints. To address these gaps, we propose a hierarchical framework that combines a Graph ATtention Planner (GATP) with a decentralized Nonlinear Model Predictive Controller (NMPC). GATP provides intermediate subgoals through multi-robot cooperation, and the NMPC enforces safety under nonlinear dynamics and actuation constraints. We evaluate our framework in both simulation and real-world quadrotor experiments. Thanks to attention mechanisms and minimal communication requirements, we demonstrate improved generalization to larger teams, robustness to communication delays up to 200 ms and practical feasibility with decentralized on-board inference.

Keywords: Field Robots, Probabilistic Inference, Vision-Based Navigation
Abstract: Accurate estimation of wheel–terrain interaction parameters is important for efficient navigation of Unmanned Ground Vehicles in unstructured outdoor environments. In this paper, we propose a hybrid data-driven and model-based method to estimate a priori emph{motion resistance}, a terrain-specific parameter representing the force opposing wheel motion, which is largely influenced by terrain class and geometry. The proposed method relies on learning motion resistance from proprioceptive feedback collected on reference terrains. This learned model is then transferred to new environments, where motion resistance is inferred from exteroceptive observation, including LiDAR and cameras, leveraging terrain geometry and class information. To capture uncertainty from terrain roughness and sensor noise, we evaluate two probabilistic models predicting motion resistance distributions: a Gaussian-MLP and a Gaussian Process Regressor. Their robustness to domain shifts is assessed by measuring performance degradation as the target diverges from the source domain. Extensive off-road field experiments validate the method’s effectiveness, demonstrating accurate prediction of motion resistance and its potential for deployment.

Keywords: View Planning for SLAM, Reinforcement Learning
Abstract: Autonomous robot exploration (ARE) is the process of a robot autonomously navigating and mapping an unknown environment. Recent Reinforcement Learning (RL)-based approaches typically formulate ARE as a sequential decision-making problem defined on a collision-free informative graph. However, these methods often demonstrate limited reasoning ability over graph-structured data. Moreover, due to the insufficient consideration of robot motion, the resulting RL policies are generally optimized to minimize travel distance, while neglecting time efficiency. To overcome these limitations, we propose GRATE, a Deep Reinforcement Learning (DRL)-based approach that leverages a Graph Transformer to effectively capture both local structure patterns and global contextual dependencies of the informative graph, thereby enhancing the model’s reasoning capability across the entire environment. In addition, we deploy a Kalman filter to smooth the waypoint outputs, ensuring that the resulting path is kinodynamically feasible for the robot to follow. Experimental results demonstrate that our method exhibits better exploration efficiency (up to 21.5% in distance and 21.3% in time to complete exploration) than state-of-the-art conventional and learning-based baselines in various simulation benchmarks. We also validate our planner in real-world scenarios.

Keywords: Intelligent Transportation Systems, Object Detection, Segmentation and Categorization, Deep Learning Methods
Abstract: 4D radars, which provide 3D point cloud data along with Doppler velocity, are attractive components of modern automated driving systems due to their low cost and robustness under adverse weather conditions. However, they provide a significantly lower point cloud density than LiDAR sensors. This makes it important to exploit not only local but also global contextual scene information. This paper proposes DRIFT, a model that effectively captures and fuses both local and global contexts through a dual-path architecture. The model incorporates a point path to aggregate fine-grained local features and a pillar path to encode coarse-grained global features. These two parallel paths are intertwined via novel feature-sharing layers at multiple stages, enabling full utilization of both representations. DRIFT is evaluated on the widely used View-of-Delft (VoD) dataset and a proprietary internal dataset. It outperforms the baselines on the tasks of object detection and/or free road estimation. For example, DRIFT achieves a mean average precision (mAP) of 52.6% (compared to, say, 45.4% of CenterPoint) on the VoD dataset.

Keywords: Representation Learning, Motion and Path Planning, Grasping
Abstract: Human-like motion generation for robots often draws inspiration from biomechanical studies, which categorize complex human motions into hierarchical taxonomies. While these taxonomies provide rich structural information about how movements relate to one another, this information is frequently overlooked in motion generation models, leading to a disconnect between the generated motions and their underlying hierarchical structure. This paper introduces the Gaussian Process Hyperbolic Dynamical Model (GPHDM), a novel approach that learns latent representations preserving both the hierarchical structure of motions and their temporal dynamics to ensure physical consistency. Our model achieves this by extending the dynamics prior of the Gaussian Process Dynamical Model (GPDM) to the hyperbolic manifold and integrating it with taxonomy-aware inductive biases. Building on this geometry- and taxonomy-aware frameworks, we propose three novel mechanisms for generating motions that are both taxonomically-structured and physically-consistent: two probabilistic recursive approaches and a method based on pullback-metric geodesics. Experiments on generating realistic motion sequences on the hand grasping taxonomy show that the proposed GPHDM faithfully encodes the underlying taxonomy and temporal dynamics, and it generates novel physically-consistent trajectories.

Keywords: Disassembly, Task and Motion Planning, Planning under Uncertainty
Abstract: To support the circular economy, robotic systems must not only assemble new products but also disassemble end-of-life (EOL) ones for reuse, recycling, or safe disposal. Existing approaches to disassembly sequence planning often assume deterministic and fully observable product models, yet real EOL products frequently deviate from their initial designs due to wear, corrosion, or undocumented repairs. We argue that disassembly should therefore be formulated as a Partially Observable Markov Decision Process (POMDP), which naturally captures uncertainty about the product's internal state. We present a mathematical formulation of disassembly as a POMDP, in which hidden variables represent uncertain structural or physical properties. Building on this formulation, we propose a task and motion planning framework that automatically derives specific POMDP models from CAD data, robot capabilities, and inspection results.

To obtain tractable policies, we approximate this formulation with a reinforcement-learning approach that operates on stochastic action outcomes informed by inspection priors, while a Bayesian filter continuously maintains beliefs over latent EOL conditions during execution. Using three products on two robotic systems, we demonstrate that this probabilistic planning framework outperforms deterministic baselines in terms of average disassembly time and variance, generalizes across different robot setups, and successfully adapts to deviations from the CAD model, such as missing or stuck parts.

Keywords: Deep Learning for Visual Perception, RGB-D Perception, Computer Vision for Automation
Abstract: Layout estimation and 3D object detection are two fundamental tasks in indoor scene understanding. When combined, they enable the creation of a compact yet semantically rich spatial representation of a scene. Existing approaches typically rely on point cloud input, which poses a major limitation since most consumer cameras lack depth sensors and visual-only data remains far more common. We address this issue with TUN3D, the first method that tackles joint layout estimation and 3D object detection in real scans, given multi-view images as input, and does not require ground-truth camera poses or depth supervision. Our approach builds on a lightweight sparse-convolutional backbone and employs two dedicated heads: one for 3D object detection and one for layout estimation, leveraging a novel and effective parametric wall representation. Extensive experiments show that TUN3D achieves state-of-the-art performance across three challenging scene understanding benchmarks: (i) using ground-truth point clouds, (ii) using posed images, and (iii) using unposed images. While performing on par with specialized 3D object detection methods, TUN3D significantly advances layout estimation, setting a new benchmark in holistic indoor scene understanding.

Keywords: Task Planning, Task and Motion Planning, Multi-Robot Systems
Abstract: Object rearrangement planning in complex, cluttered environments is a common challenge in warehouses, households, and rescue sites. Prior studies largely address monotone instances, whereas real-world tasks are often non-monotone—objects block one another and must be temporarily relocated to intermediate positions before reaching their final goals. In such settings, effective multi-agent collaboration can substantially reduce the time required to complete tasks. This paper introduces Centralized, Asynchronous, Multi-agent Monte Carlo Tree Search (CAM-MCTS), a novel framework for general-purpose makespan-efficient object rearrangement planning in challenging environments. CAM-MCTS combines centralized task assignment—where agents remain aware of each other’s intended actions to facilitate globally optimized planning—with an asynchronous task execution strategy that enables agents to take on new tasks at appropriate time steps, rather than waiting for others, guided by a one-step look-ahead cost estimate. This design minimizes idle time, prevents unnecessary synchronization delays, and enhances overall system efficiency. We evaluate CAM-MCTS across a diverse set of monotone and non-monotone tasks in cluttered environments, demonstrating consistent reductions in makespan compared to strong baselines. Finally, we validate our approach on a real-world multi-agent system under different configurations, further confirming its effectiveness and robustness. Videos can be found at https://www.youtube.com/watch?v=kNRg2kNnFxg.

Keywords: Imitation Learning, Learning from Demonstration, Grippers and Other End-Effectors
Abstract: Recent advances in robot learning have motivated integrated pipelines that combine hardware for data collection with imitation learning algorithms. Existing data collection methods like leader–follower, VR/AR, and exoskeletons rely on costly hardware and exhibit limited scalability, while imitation learning algorithms built on them remain highly sensitive to viewpoint shifts, further constraining generalizability. Handheld grippers provide a low-cost, robot-agnostic alternative, but prior systems bypass exocentric view alignment by relying solely on wrist-mounted cameras, resulting in narrowed observation and reduced policy robustness. We propose VIL, a framework pairing customized handheld gripper with zero-shot, exocentric viewpoint-robust imitation learning algorithm, bridging the handheld gripper with exocentric views. Our approach employs adapters for appearance alignment and a hybrid encoder design to extract view-consistent representations for an ACT-style policy, enabling robust execution across diverse perspectives. We further optimize the data collection pipeline and validate the system in both simulation and real-world tasks. Experiments show that VIL achieves stable performance under viewpoint shifts, challenging low-horizon scenarios, and dynamic perspectives, outperforming SOTA methods and demonstrating a scalable pipeline for manipulator-independent, viewpoint-robust policy learning. The project repository containing code and hardware is available at https://github.com/liboyan233/VIL.git.

Keywords: Energy and Environment-Aware Automation, Optimization and Optimal Control, Probability and Statistical Methods
Abstract: We propose a novel model of a virtual energy storage system (ESS) that leverages the aggregate battery capacity of parked and idling electric vehicles (EVs). Such an energy service is offered to a community of prosumers as a temporary energy buffer and managed by a parking lot manager (PLM), which absorbs the risks arising from the unreliability of the EV-based ESS due to the arrival and departure of EVs. Hence, from the prosumers' perspective, such a virtual storage service behaves deterministically. Both the PLM and the prosumers act as self-interested agents that optimize their own objectives, subject to operational constraints, leading to a non-cooperative game. To deal with the uncertainty of prosumers' renewable net generation and EVs' arrivals/departures, we use a data-driven distributionally robust approach, showing that a tractable reformulation can be obtained, where the equilibrium solutions can be computed as a variational inequality. Numerical simulations based on real data illustrate the behavior of the proposed model.

Keywords: Human-Robot Collaboration, Social HRI, Natural Dialog for HRI
Abstract: Robot failures during collaborative tasks can frustrate users and reduce trust. To address this, we developed a failure communication module that combines large language models (LLMs) with Behavior Trees (BTs) to generate interactive, context-aware explanations for task failures. The module supports three key processes: (1) initial (high/medium/low) leveled explanations, (2) interactive clarifications for user follow-up questions, and (3) explicit verification of user actions to close the recovery loop. By leveraging the BT structure and persistent interaction history, it generates responsive, multi-turn explanations and reduces redundancy for repeated failures. We implemented and evaluated this module in real-time robotic pick-and-place tasks as a user study with 33 participants across three high/medium/low explanation conditions. The user study showed that the module improved resolution rates for challenging failures and reduced resolution times for simpler failures, demonstrating the effectiveness of LLM-powered, BT-grounded explanations in human-robot collaboration (HRC).

Keywords: Whole-Body Motion Planning and Control, Imitation Learning, Reinforcement Learning
Abstract: Whole-body control (WBC) of humanoid robots has witnessed remarkable progress in skill versatility, enabling a wide range of applications such as locomotion, teleoperation, and motion tracking. Despite these achievements, existing WBC frameworks remain largely task-specific, relying heavily on labor-intensive reward engineering and demonstrating limited generalization across tasks and skills. These limitations hinder their response to arbitrary control modes and restrict their deployment in complex, real-world scenarios. To address these challenges, we revisit existing WBC systems and identify a shared objective across diverse tasks: the generation of appropriate behaviors that guide the robot toward desired goal states. Building on this insight, we propose the Behavior Foundation Model (BFM), a generative model pretrained on large-scale behavioral datasets to capture broad, reusable behavioral knowledge for humanoid robots. BFM integrates a masked online distillation framework with a Conditional Variational Autoencoder (CVAE) to model behavioral distributions, thereby enabling flexible operation across diverse control modes and efficient acquisition of novel behaviors without retraining from scratch. Extensive experiments in both simulation and on a physical humanoid platform demonstrate that BFM generalizes robustly across diverse WBC tasks while rapidly adapting to new behaviors. These results establish BFM as a promising step toward a foundation model for general-purpose humanoid control.

Keywords: Motion and Path Planning, Reinforcement Learning, Collision Avoidance
Abstract: Offline reinforcement learning (RL) holds significant potential for crowd robot navigation in human-robot coexistence applications. However, the inherent complexity of pedestrian motion renders the design of effective reward functions for promoting socially compliant robot behaviors a persistent challenge. This paper proposes a Social Preference Learning for Crowd Robot Navigation (SPLC) algorithm to eliminate the need for detailed reward design. Its core innovation lies in the introduction of a social preference feedback mechanism to automatically generate preference data through principled preference evaluation criteria. By explicitly accounting for the intricacies of pedestrian dynamics, the pipeline mitigates the reward bias and facilitates the systematic quantification of broad social norms, thereby fostering socially compliant behaviors. Extensive experiments integrating SPLC with offline RL methods demonstrate consistent improvements over state-of-the-art baselines across standard performance metrics. Furthermore, real-world experiments on the TurtleBot4 further validate the effectiveness of SPLC in practical human–robot coexistence settings. Our code and video demos are available at https://github.com/sklus949/SPLC.

Keywords: Biologically-Inspired Robots, Machine Learning for Robot Control, Compliance and Impedance Control

Keywords: Telerobotics and Teleoperation, Dual Arm Manipulation
Abstract: In dual-arm dexterous teleoperation, cross-platform generalization of motion retargeting and interactivity of grasping are crucial. However, the heterogeneity of robotic architectures and the wide variety of grasping objects pose significant challenges to achieving precise motion retargeting and compliant grasping in dual-arm dexterous teleoperation. To address these challenges, a dual-arm dexterous teleoperation system (DexTele) is proposed based on motion retargeting and adaptive force control. First, a vision-based motion retargeting module is designed to generate preliminary robot motions from human images. In this module, a motion-graph encoder and latent optimization are proposed for precise and convenient cross-platform motion retargeting. Second, an adaptive grasping module is designed to achieve compliant grasping. This module combines a vision-language model (VLM) with model predictive control (MPC), allowing the system to predict the required grasping force for a target object and perform gradient-based online optimization. Finally, extensive experiments demonstrate that the DexTele achieves precise motion retargeting and compliant grasping with generalization across multiple robot platforms. The source code will be released upon paper acceptance.

Keywords: Grasping, Dexterous Manipulation, Grippers and Other End-Effectors
Abstract: This study presents dynamic scoop-and-flick manipulation, a robotic technique that achieves desired projectile motions of target objects through rapid, non-prehensile physical interactions. The method allows a robot to scoop objects resting on a surface and quickly launch them into projectile trajectories. We formulate a theoretical model of the technique and realize it through a hybrid approach that combines model-based reasoning and data-driven learning. The advantages—-namely, rapid and accurate pick-and-place with reduced planning complexity-—are validated in experiments conducted with a particularly challenging class of objects: low-profile items with small thickness.

Keywords: Manipulation Planning, Planning under Uncertainty, Sensor-based Control
Abstract: Enabling robots to manipulate articulated objects is essential for their successful integration into human-centric environments. Such manipulation is often part of a larger multistep task, where achieving a specific joint configuration is necessary for subsequent actions-for example, in a cluttered environment, a cabinet door must be rotated to a precise angle that creates just enough clearance to retrieve an object, beyond which it would collide with surrounding obstacles. In this work, we present an approach to learning goal-directed policies for articulated object manipulation using force and proprioceptive feedback. We formulate the manipulation problem as a Partially Observable Markov Decision Process (POMDP) with a continuous state space and a set of low-level control actions. Due to the limitations of standard POMDP solvers in this setting, we introduce Planning using Belief Summaries (PuBS), which approximates the POMDP as a Markov Decision Process (MDP) over compact particle-filter belief summaries encoding estimated state and uncertainty. This approximate MDP is then solved using reinforcement learning techniques to learn goal-directed policies that enable safe exploration while efficiently guiding the object toward the goal. We evaluate our approach through simulation and real-world robotic experiments, demonstrating reliable goal-reaching performance.

Keywords: Localization, Deep Learning Methods, Deep Learning for Visual Perception
Abstract: The pooling layer plays a vital role in aggregating local descriptors into the metrizable global descriptor in the LiDAR Place Recognition (LPR). In particular, the second-order pooling is capable of capturing higher-order interactions among local descriptors. However, its existing methods in the LPR adhere to conventional implementations and post-normalization, and incur the descriptor unsuitable for Euclidean distancing. Based on the recent interpretation that associates NetVLAD with the second-order statistics, we propose to integrate second-order pooling with the inductive bias from Voronoi cells. Our novel pooling method aggregates local descriptors to form the second-order matrix and whitens the global descriptor to implicitly measure the Mahalanobis distance while conserving the cluster property from Voronoi cells, addressing its numerical instability during learning with diverse techniques. We demonstrate its performance gains through the experiments conducted on the Oxford Robotcar and Wild-Places benchmarks and analyze the numerical effect of the proposed whitening algorithm.

Keywords: Failure Detection and Recovery, Field Robots
Abstract: Autonomous inspection robots for monitoring industrial sites can reduce costs and risks associated with human-led inspection. However, accurate readings can be challenging due to occlusions, limited viewpoints, or unexpected environmental conditions. We propose a hybrid framework that combines supervised failure classification with anomaly detection, enabling classification of inspection tasks as a success, known failure, or anomaly (i.e., out-of-distribution) case. Our approach uses a world model backbone with compressed video inputs. This policy-agnostic, distribution-free framework determines classifications based on two decision functions set by conformal prediction (CP) thresholds before a human observer. We evaluate the framework on gauge inspection feeds collected from office and industrial sites and demonstrate real-time deployment on a Boston Dynamics Spot. Experiments show over 90% accuracy in distinguishing between successes, failures, and OOD cases, with classifications occurring earlier than a human observer. These results highlight the potential for robust, anticipatory failure detection in autonomous inspection tasks or as a feedback signal for model training to assess and improve the quality of training data. Project website: https://autoinspection-classification.github.io/

Keywords: Multi-Robot Systems, Cooperating Robots, Reinforcement Learning
Abstract: Multi-agent deep Reinforcement Learning (RL) has made significant progress in developing intelligent game-playing agents in recent years. However, the efficient training of collective robots using multi-agent RL and the transfer of learned policies to real-world applications remain open research questions. In this work, we first develop a comprehensive robotic system, including simulation, distributed learning framework, and physical robot components. We then propose and evaluate reinforcement learning techniques designed for efficient training of cooperative and competitive policies on this platform. To address the challenges of multi-agent sim-to-real transfer, we introduce Out of Distribution State Initialization (OODSI) to mitigate the impact of the sim-to-real gap. In the experiments, OODSI improves the Sim2Real performance by 20%. We demonstrate the effectiveness of our approach through experiments with a multi-robot car competitive game and a cooperative task in real-world settings.

Keywords: Computer Vision for Automation, Localization, Mapping
Abstract: Image-to-point-cloud registration (I2P) is a fundamental task in robotic applications such as manipulation, grasping and localization. Existing deep learning-based I2P methods seek to align image and point cloud features in a learned representation space to establish correspondences, and have achieved promising results. However, due to issues such as scale ambiguity, many geometrically inconsistent outlier correspondences persist in the feature space. To address this limitation, we propose Angle-I2P, an outlier rejection network that leverages angle-consistent geometric constraints and hierarchical attention. First, we design a scale-invariant, cross-modality geometric constraint based on angular consistency. This explicit geometric constraint guides the model in distinguishing inliers from outliers. Furthermore, we propose a global-to-local hierarchical attention mechanism that effectively filters out geometrically inconsistent matches under rigid transformation, thereby improving the Inlier Ratio (IR) and Registration Recall (RR). Experimental results demonstrate that our method achieves state-of-the-art performance on the 7Scenes, RGBD Scenes V2, and a self-collected dataset, with consistent improvements across all benchmarks.

Keywords: Soft Robot Applications, Soft Robot Materials and Design, Simulation and Animation
Abstract: Tool-based scooping is vital in robot-assisted tasks, enabling interaction with objects of varying sizes, shapes, and material states. Recent studies have shown that flexible, reconfigurable soft robotic end-effectors can adapt their shape to maintain consistent contact with container surfaces during scooping, improving efficiency compared to rigid tools. These soft tools can adjust to varying container sizes and materials without requiring complex sensing or control. However, the inherent compliance and complex deformation behavior of soft robotics introduce significant control complexity that limits practical applications. To address this challenge, this paper presents the development of a physics-based simulation model of a deformable soft conical robotic hand that captures its passive reconfiguration dynamics and enables systematic trajectory optimization for scooping tasks. We propose a novel physics-based simulation approach that accurately models the soft tool’s morphing behavior from flat sheets to adaptive conical structures, combined with an evolutionary strategy framework that automatically optimizes scooping trajectories without manual parameter tuning. We validate the optimized trajectories through both simulation and real-robot experiments. The results demonstrate strong generalization and successfully address a range of challenging tasks previously beyond the reach of existing approaches. Videos of our experiments are available online: https://sites.google.com/view/scoopsh

Keywords: Multi-Modal Perception for HRI, Hybrid Logical/Dynamical Planning and Verification, Acceptability and Trust
Abstract: Vision-Language Models (VLMs) are becoming the cornerstone of high-level reasoning for robotic automation, enabling robots to parse natural language commands and perceive their environments. However, their susceptibility to hallucinations introduces critical failures in decision-making, posing significant safety and reliability risks in physical deployments. This challenge is exacerbated by the open-ended nature of real-world tasks, where questions vary vastly in difficulty and modality, demanding robust and adaptable reasoning strategies. To tackle this, we propose the underline{P}seudocode-guided underline{St}ructured Reunderline{a}soning funderline{r}amework (PStar), which adaptively selects structured pseudocode reasoning paths to help VLMs perform flexible and step-by-step reasoning. We design a set of abstract reasoning functions and formulate a structured pseudocode library to represent modular reasoning strategies. Crucially, we design a Difficulty Feature Vector (DFV) that allows the model to assess question complexity and adaptively choose appropriate reasoning strategies—enhancing robustness and interpretability. Extensive experiments demonstrate that PStar significantly reduces hallucination rates, achieving state-of-the-art scores of 87.1% on POPE and 68.0% on MMStar, outperforming even GPT-4V. By providing a validated mechanism to reduce visual-language errors, PStar offers a critical step toward deploying more trustworthy and deterministic VLMs for real-world automated systems, where such errors can lead to catastrophic outcomes.

Keywords: Path Planning for Multiple Mobile Robots or Agents, Collision Avoidance, Distributed Robot Systems
Abstract: Multi-robot motion planning and crowd simulations are crucial in social navigation, enabling agents to avoid collisions with one another in dynamic environments. While existing methods typically use simple circular models for robot and pedestrian boundaries, superquadric models offer greater flexibility in accurately representing non-circular objects. This paper addresses the challenges of employing superquadric models to avoid dynamic obstacles and other moving robots. We tackle three primary challenges: (i) approximating the complex parametric boundary surface of Minkowski sum for easier differentiation; (ii) computing the boundary of velocity obstacles; and (iii) rapidly calculating velocity changes. The approximation of differentiable Minkowski sum boundary is formulated as a semidefinite programming problem using convex sum-of-squares polynomials. We then develop a tangency point-finding algorithm with superlinear convergence speed, and introduce a rule-based collision-avoidance approach, named SSCA (Superquadric-based Sum-of-Squares Collision Avoidance for Multi-Robot Systems) for efficient velocity change calculation. Our proposed method is evaluated through extensive experiments, demonstrating millisecond-level computational efficiency and scalability to dozens of robots. This work provides a more effective solution for collision avoidance algorithm using superquadric models, enhancing the safety and performance of robots in dynamic shared environments.

Keywords: Imitation Learning, Learning from Experience, Deep Learning in Grasping and Manipulation
Abstract: Intelligent agents progress by continually refining their capabilities through actively exploring environments. Yet robot policies often lack sufficient exploration capability due to action mode collapse. Existing methods that encourage exploration typically rely on random perturbations, which are unsafe and induce unstable, erratic behaviors, thereby limiting their effectiveness. We propose Self-Improvement via On-Manifold Exploration (SOE), a framework that enhances policy exploration and improvement in robotic manipulation. SOE learns a compact latent representation of task-relevant factors and constrains exploration to the manifold of valid actions, ensuring safety, diversity, and effectiveness. It can be seamlessly integrated with arbitrary policy models as a plug-in module, augmenting exploration without degrading the base policy performance. Moreover, the structured latent space enables human-guided exploration, further improving efficiency and controllability. Extensive experiments in both simulation and real-world tasks demonstrate that SOE consistently outperforms prior methods, achieving higher task success rates, smoother and safer exploration, and superior sample efficiency. These results establish on-manifold exploration as a principled approach to sample-efficient policy self-improvement.

Keywords: Collision Avoidance, Motion and Path Planning, Task and Motion Planning
Abstract: Real-time path planning in unknown non-convex environments is challenging, as obstacle updates can invalidate existing paths while narrow passages restrict feasible connectivity. This paper presents textbf{InBi-RRT}, an incremental bidirectional tree-based framework that grows a reverse tree from the goal and maintains a reusable forward tree from the start. When the current path becomes invalid, a cost-guided expansion selectively extends the forward tree to establish collision-free connections with the reverse tree, followed by backtracking and lightweight path optimization for efficient repair. Simulation results in unknown and non-convex scenarios demonstrate that InBi-RRT achieves significantly faster replanning than baseline methods, being up to textbf{5.5times} faster than RT-RRT and textbf{22times} faster than RRT^{text{X}}, with paths up to 19.8% shorter than RRT^{text{X}} under the same sample count. Furthermore, real-world experiments in an indoor maze-like environment verify the practicality and robustness of the proposed planner in unknown non-convex scenarios.

Keywords: Planning under Uncertainty, Optimization and Optimal Control, Robot Safety
Abstract: In this paper, we combine first-order approximations of hybrid systems (i.e., the so-called saltation matrix) with previous works on parametric sensitivity for continuous systems to propose a general framework for robust trajectory optimization of hybrid systems subject to parametric uncertainties. A method for computing parametric sensitivities of both continuous dynamics and hybrid events is presented. The obtained "hybrid parametric sensitivity" is then combined with sensitivity-based tubes that encapsulate all possible perturbed states and control trajectories given a known bounded range for the uncertain parameters. The proposed method is then applied to the problem of planning robust trajectories for legged robot systems, which allows obtaining trajectories that remain feasible w.r.t.~the contact constraints even in presence of uncertainties in the dynamics, guard conditions, and reset maps. We also illustrate one of the fundamental limitations of first-order approximations, that is, the fact that the sensitivity reset time is fixed, and propose an extension to the sensitivity analysis that can form the basis for future developments.

Keywords: Motion and Path Planning, Collision Avoidance, Simulation and Animation
Abstract: Motion planning for robotic manipulators is a fundamental problem in robotics. Classical optimization-based methods typically rely on the gradients of signed distance fields (SDF) to impose collision-avoidance constraints. However, these methods are susceptible to local minima and may fail when the SDF gradients vanish. Recently, Configuration Space Distance Fields (CDF) have been introduced, which directly model distances in the robot’s configuration space. Unlike workspace SDF, CDF are differentiable almost everywhere and thus provide reliable gradient information. On the other hand, gradient-free approaches such as Model Predictive Path Integral (MPPI) control leverage long-horizon rollouts to achieve collision avoidance. While effective, these methods are computationally expensive due to the large number of trajectory samples, repeated collision checks, and the difficulty of designing cost functions with heterogeneous physical units. In this paper, we propose a framework that integrates CDF with MPPI to enable direct navigation in the robot’s configuration space. Leveraging CDF gradients, we unify the MPPI cost in joint-space and reduce the horizon to one step, substantially cutting computation while preserving collision avoidance in practice. We demonstrate that our approach achieves nearly 100% success rates in 2D environments and consistently high success rates in challenging 7-DOF Franka manipulator simulations with complex obstacles. Furthermore, our method attains control frequencies exceeding 750 Hz, substantially outperforming both optimization-based and standard MPPI baselines. These results highlight the effectiveness and efficiency of the proposed CDF-MPPI framework for high-dimensional motion planning.

Keywords: Autonomous Agents, Motion and Path Planning, Reinforcement Learning
Abstract: As the foundation of closed-loop training and evaluation in autonomous driving, traffic simulation still faces two fundamental challenges: covariate shift introduced by open-loop imitation learning and limited capacity to reflect the multimodal behaviors observed in real-world traffic. Although recent frameworks such as RIFT have partially addressed these issues through group-relative optimization, their forward simulation procedures remain largely non-reactive, leading to unrealistic agent interactions within the virtual domain and ultimately limiting simulation fidelity. To address these issues, we propose ForSim, a stepwise closed-loop forward simulation paradigm. At each virtual timestep, the traffic agent propagates the virtual candidate trajectory that best spatiotemporally matches the reference trajectory through physically grounded motion dynamics, thereby preserving multimodal behavioral diversity while ensuring intra-modality consistency. Other agents are updated with stepwise predictions, yielding coherent and interaction-aware evolution. When incorporated into the RIFT traffic simulation framework, ForSim operates in conjunction with group-relative optimization to fine-tune traffic policy. Extensive experiments confirm that this integration consistently improves safety while maintaining efficiency, realism, and comfort. These results underscore the importance of modeling closed-loop multimodal interactions within forward simulation and enhance the fidelity and reliability of traffic simulation for autonomous driving.

Keywords: Wearable Robotics, Safety in HRI, Biologically-Inspired Robots
Abstract: Back injuries resulting from manual material handling have long constituted a prominent threat to occupational safety. While back-support exosuits offer the potential to augment human strength, their practical implementation is hindered by persistent challenges pertaining to comfort and safety. Drawing inspiration from human biomechanics and muscle behavior, we develop a lightweight assistive exosuit that synchronizes with natural load-handling rhythms. By integrating a deployable Kresling origami structure with a twostage transmission mechanism, a single motor can sequentially assist both the waist and arms, achieving motion-conforming support with minimal complexity. An energy-aware compliance control strategy allows the system to yield passively during unassisted motion, avoiding interference with voluntary human behavior. We propose an event-triggered impedance control strategy based on an energy tank framework, which adaptively intervenes only when interaction energy exceeds safety thresholds. Experimental results demonstrate substantial reductions in muscle activation during load-handling tasks, with decreases of up to 22.8%, 15.4%, and 14.8% in the biceps, triceps, and erector spinae (MVC%), respectively.

Keywords: Mapping, SLAM, Semantic Scene Understanding
Abstract: To advance 3D reconstruction from static digital replicas towards semantically interactive Living Maps responsive to an agent's queries, we propose ARTEMIS, a system for Active Real-time Textured Environment Meshing with Interactive Semantics. At its core, our Semantic Brush is a methodology comprised of tightly-coupled modules for segmentation, constraint, and refinement that operate in a two-stage, coarse-to-fine pipeline. Initially, its segmentation and constraint modules translate natural language into a semantically-aware mesh, enforcing sharp object boundaries with a unified energy function. Subsequently, its refinement module computes a unified reliability metric from color and depth consistency to guide the joint optimization of the texture map and semantic labels. This holistic process inherently filters unreliable measurements, establishing a complete interactive workflow from language input to real-time highlighting on a high-fidelity textured mesh. We evaluated ARTEMIS on public datasets and in real-world scenarios. The results demonstrate its state-of-the-art accuracy in mesh reconstruction, while simultaneously attaining high fidelity in both texture and semantics. To share our findings and make contributions to the community, our code will be made publicly available.

Keywords: Reinforcement Learning, Motion and Path Planning
Abstract: Currently, many manipulation tasks for deformable objects focus on activities like folding clothes, handling ropes, and manipulating bags. However, research on contact-rich tasks involving deformable objects remains relatively underdeveloped. When humans use cloth or sponges to wipe surfaces, they rely on both vision and tactile feedback. Yet, current algorithms still face challenges with issues like occlusion, while research on tactile perception for manipulation is still evolving and requires further development. Tasks such as covering surfaces with deformable objects demand not only perception but also precise robotic manipulation. To address this, we propose a method that leverages efficient and accessible simulators for task execution. Specifically, we train a reinforcement learning agent in a simulator to manipulate deformable objects for surface wiping tasks. We simplify the state representation of object surfaces using UV mapping, process contact feedback from the simulator on 2D feature maps, and use scaled grouped convolutions to extract features from these maps. The agent then outputs actions in a reduced-dimensional action space to generate coverage paths. Experiments demonstrate that our method outperforms previous approaches in key metrics, including total path length and coverage area. We deploy these paths on the Kinova Gen3 manipulator to perform wiping experiments on the back of a torso model, validating the feasibility of our approach.

Keywords: Reactive and Sensor-Based Planning, Grasping, Manipulation Planning
Abstract: We present a fast and reactive grasping framework that combines task-space velocity fields with joint-space Quadratic Program (QP) in a hierarchical structure. Reactive, collision-free global motion planning is particularly challenging for high-DoF systems, as simultaneous increases in state dimensionality and planning horizon trigger a combinatorial explosion of the search space, making real-time planning intractable. To address this, we plan globally in a lower-dimensional task space – such as fingertip positions – and track locally in the full joint space while enforcing all constraints. This approach is realized by constructing velocity fields in multiple task-space coordinates (or, in some cases, a subset of joint coordinates) and solving a weighted joint-space QP to compute joint velocities that track these fields with appropriately assigned priorities. Through simulation experiments and real-world tests using the recent pose-tracking algorithm FoundationPose, we verify that our method enables high-DoF arm–hand systems to perform real-time, collision-free reaching motions while adapting to dynamic environments and external disturbances.

Keywords: Deep Learning Methods, Intelligent Transportation Systems
Abstract: This work proposed a 3D autoencoder architecture, named LiLa-Net, which encodes efficient features from real traffic environments, employing only the LiDAR's point clouds. For this purpose, we have real semi-autonomous vehicle, equipped with Velodyne LiDAR. The system leverage skip connections concept to improve the performance without using extensive resources as the state-of-the-art architectures. Key changes include reducing the number of encoder layers and simplifying the skip connections, while still producing an efficient and representative latent space which allows to accurately reconstruct the original point cloud. Furthermore, an effective balance has been achieved between the information carried by the skip connections and the latent encoding, leading to improved reconstruction quality without compromising performance. Finally, the model successfully reconstruct objects unrelated to the original traffic environment.

Keywords: Autonomous Vehicle Navigation, Vision-Based Navigation, Integrated Planning and Control

Keywords: Mechanism Design, Actuation and Joint Mechanisms, Biologically-Inspired Robots
Abstract: Underwater robots have significant potential for a wide range of applications, including deep-sea exploration, hydrocarbon extraction, marine biodiversity observation, and waste retrieval. A hybrid actuation system that combines electromagnets and permanent magnets preserves the main benefits of magnetic-driven robots, addressing the issues of bulky coil systems and limited mobility. However, most electromagnetically actuated underwater robots are limited to a fixed swimming mode due to their relatively simple designs, which restrict their adaptability to unpredictable and unstructured aquatic environments. In this work, we present an untethered multi-mode swimming robot driven by four 2-degreesof-freedom (DoF) electromagnetic actuators, each with a rigid shell interconnected by flexible connectors and covered with silicone membranes. Initially, we conducted tests to determine the optimal hardness of the flexible connector by validating the module’s range of motion across different activation times. Next, we demonstrated that the robot can swim forward and backward in a water tank, exhibiting snake-inspired motion, front- and rear-undulation, and wave-shaped motion, and reaching a maximum speed of 87.8 mm/s. Finally, we showed the lateral translation and steering motions achieved with different control signals, resulting in an average turning speed of 3◦/s. This approach enables a novel robot design strategy based on compact multi-DoF electromagnetic modules, facilitating potential applications in search-and-rescue missions and environmental inspections.

Keywords: Imitation Learning, Learning from Demonstration, Deep Learning in Grasping and Manipulation
Abstract: Recent advancements in robotics have focused on developing foundation models capable of generating both actions and future states. Typically, these policies leverage world models to depict human-like imagination. However, most methods remain confined to the 2D domain, where they forecast only the final outcome state rather than the evolving interaction process, thereby offering limited guidance for step-by-step control. To address these limitations, we propose a hierarchical framework that couples 3D imagination, 3D perception, and action generation. A triplane-based world model captures future scene dynamics in a computationally efficient manner, providing predictive cues for decision-making. Based on these representations, the action expert, implemented with a flow-based policy network, converts the outputs of 3D imagination and perception into executable commands. We further introduce an adaptive Classifier-Free Guidance strategy to balance action quality with condition adherence. On Adroit, Meta-World, and real-world tasks, our method achieves a 92% voxel IoU in future state prediction and up to 8% higher success rates than state-of-the-art baselines. The performance gains highlight the effectiveness and generalizability of our method in complex robotic manipulation.

Keywords: Surgical Robotics: Planning, Surgical Robotics: Laparoscopy, Medical Robots and Systems
Abstract: We present a supervised autonomous robot-assisted ureteroscopy (ARA-URS) system for the treatment of kidney stones, integrated with a digital-twin (DT). A three degrees of freedom (3-DOF) robotic system was developed to actuate a Wolf disposable ureteroscope, enabling the ARA-URS to autonomously position the ureteroscope for laser lithotripsy procedures. The DT of the robotic system was developed to enable the generation of diverse synthetic intraoperative scenarios, which are used to train vision and control models for improved precision in ARAURS. By mapping joint motion to virtual counterparts via precise physics simulation, the system ensures realistic representation and reliable validation. This framework’s performance was assessed with particular focus on endoscopic tip positioning. Initial in-air simulation experiments demonstrated root mean square error (RMSE) values of (1.871, 1.725, 1.194) mm in the x-, y-, and z-directions, respectively, computed as the deviation between the desired laser target position and the achieved ureteroscope tip position. Corresponding real-world experiments yielded RMSE values of (5.029, 3.919, 6.681) mm. The comparison between simulated and physical experiments indicates that the DT is able to reproduce the motion behavior of the physical system with good agreement. Further benchtop and simulation experiments demonstrated the system’s capacity for stone targeting (quantified by the percentage of the image occluded by stone). In digital simulation, the ureteroscope achieved 88.77% average stone area coverage, while in the benchtop model, coverage averaged 59.04%. Together, this proof of concept highlights the potential of DT technology in robotic-assisted URS, offering a scalable and interactive platform for refining surgical techniques.

Keywords: Motion and Path Planning, Optimization and Optimal Control, Task and Motion Planning
Abstract: Autonomous navigation often requires the simultaneous optimization of multiple objectives. The most common approach scalarizes these into a single cost function using a weighted sum, but this method is unable to find all possible trade-offs and can therefore miss critical solutions. An alternative, the weighted maximum of objectives, can find all Pareto-optimal solutions, including those in non-convex regions of the trade-off space that weighted sum methods cannot find. However, the increased computational complexity of finding weighted maximum solutions in the discrete domain has limited its practical use. To address this challenge, we propose a novel search algorithm based on the Large Neighbourhood Search framework that efficiently solves the weighted maximum planning problem. Through extensive simulations, we demonstrate that our algorithm achieves comparable solution quality to existing weighted maximum planners with a runtime improvement of 1-2 orders of magnitude, making it a viable option for autonomous navigation.

Keywords: Motion and Path Planning, Machine Learning for Robot Control, Autonomous Vehicle Navigation
Abstract: Guiding Vector Fields (GVFs) are a powerful tool for robotic path following. However, classical methods assume smooth, ordered curves and fail when paths are unordered, multi-branch, or generated by probabilistic models. We propose a unified framework, termed the Score-Induced Guiding Vector Field (SGVF), which leverages score-based generative modeling to construct vector fields directly from data distributions. SGVF learns tangent fields from point clouds with unit-norm, orthogonality, and directional-consistency losses, ensuring geometric fidelity and control feasibility. This approach removes the reliance on ad-hoc path segmentation and enables guidance along complex topologies such as branching and pseudo-manifolds. The study establishes a correspondence between score vanishing in diffusion models and GVF singularities and highlights representational capacity near sharp path curvatures. Experiments on robotic navigation in planar environments demonstrate that SGVF achieves reliable path following in scenarios where classical GVFs fail, underscoring its potential as a bridge between generative modeling and geometric control. Code and experiment video are available at https://github.com/czr-gif/Guiding-Vector-Field-Generation-via-Score-based-Diffusion-Model.

Keywords: Imitation Learning, Representation Learning
Abstract: The spatial information inherent in 3D point clouds is crucial for robotic manipulation. However, existing 3D pre-training methods face a fundamental trade-off: Masked Autoencoding (MAE) excels at capturing spatial-geometric features but lacks semantics, whereas contrastive learning, while able to distill semantics from 2D foundation models, is ill-suited for the fine-grained details required for manipulation tasks. To address these challenges, we propose CLAR, a novel 3D pre-training framework that synergizes global understanding with fine-grained local alignment. Our framework unifies MAE with global cross-modal contrastive learning to integrate robust spatial awareness with rich semantic understanding. To enhance its focus on fine-grained details, at the local level, we introduce an adaptive alignment mechanism that leverages deformable attention to force precise correspondences between local 3D geometry and 2D visual features, thereby overcoming the limitations of conventional global alignment in manipulation tasks. Extensive experiments in simulation and the real world demonstrate that CLAR achieves state-of-the-art performance, significantly outperforming existing methods in visuomotor policy learning.

Keywords: Marine Robotics, Robust/Adaptive Control, Failure Detection and Recovery
Abstract: Thruster failures in unmanned surface vehicles (USVs) can critically compromise mission completion, particularly when severe degradation eliminates controllability in essential degrees of freedom. While traditional fault-tolerant control treats environmental disturbances as impediments to be rejected, this paper presents a novel approach: strategically exploiting wind and wave forces as virtual actuators for emergency harbor return. The proposed environment-assisted model predictive control (EAMPC) framework adaptively modulates environmental force utilization factors based on fault severity and the environmental force prediction confidence, transforming natural disturbances into environmental assistance. The hierarchical architecture integrates state estimation and prediction with physics-informed learning for short-term environmental forces, and reachability-based trajectory planning that exploits environmental forces to expand feasible zones. Theoretical analysis establishes practical input-to-state stability with explicit bounds quantifying degradation. Extensive validation across 320 trials demonstrates 91.25% mission success under 95% thruster degradation compared to 0% for baseline methods. This work demonstrates that strategic environmental exploitation fundamentally transforms fault recovery capabilities in marine robotics.

Keywords: Nonholonomic Motion Planning, Constrained Motion Planning, Autonomous Vehicle Navigation
Abstract: Autonomous underwater robotics faces significant challenges, particularly in the reliable recovery of Autonomous Underwater Vehicles (AUVs) after mission completion. To address this, small AUVs can dock onto a moving mothership for safe transport to recovery sites, reducing operational risks. This paper presents an explicit analytical derivation of the LSL (Left-Straight-Left) and RSR (Right-Straight-Right) Dubins' Paths for intercepting an uniformly moving target, a critical problem for robust rendezvous in dynamic marine and underwater environments. The proposed approach leverages the classical Dubins' Path model to generate time optimal, real-time, curvature-constrained paths suitable for 2D AUVs. Experimental validation on an Unmanned Surface Vehicle (USV) demonstrates the effectiveness of the developed motion planning strategy.

Keywords: SLAM, Multi-Robot SLAM, Swarm Robotics

Keywords: Mapping, Intelligent Transportation Systems, Sensor Fusion
Abstract: Crowd-sourced mapping offers a scalable alternative to creating maps using traditional survey vehicles. Yet, existing methods either rely on prior high-definition (HD) maps or neglect uncertainties in the map fusion. In this work, we present a complete pipeline for HD map generation using production vehicles equipped only with a monocular camera, consumer-grade GNSS, and IMU. Our approach includes on-cloud localization using lightweight standard-definition maps, on-vehicle mapping via an extended object trajectory (EOT) Poisson multi-Bernoulli (PMB) filter with Gibbs sampling, and on-cloud multi-drive optimization and Bayesian map fusion. We represent the lane lines using B-splines, where each B-spline is parameterized by a sequence of Gaussian distributed control points, and propose a novel Bayesian fusion framework for B-spline trajectories with differing density representation, enabling principled handling of uncertainties. We evaluate our proposed approach, B^2F-Map, on large-scale real-world datasets collected across diverse driving conditions and demonstrate that our method is able to produce geometrically consistent lane-level maps.

Keywords: Brain-Machine Interfaces, Virtual Reality and Interfaces, Cognitive Modeling
Abstract: Accurately assessing brain activity to modulate training parameters online is crucial for improving the human-robot cognitive interaction (HRCI) performance in closed-loop brain training. The major challenge for this technique lies in how to accurately model and characterize the intrinsic behavior of brain activity in HRCI process, which typically exhibits a dynamic manner across spatial and temporal scales. In this study, we propose a dynamic perspective to visualize the spatiotemporal evolution of brain activity during HRCI process, thus enabling assessment of brain states and adaptive modulation during rehabilitation. A novel framework is developed to model the spatiotemporal dynamics of brain activity by integrating deterministic learning with neural population theory. It demonstrates a remarkable capability to mine and visualize the complex nonlinear dynamics of brain activity, encompassing both temporal evolution and spatial connectivity patterns. The proposed model not only visualizes of spatiotemporal brain dynamics but also enables online assessment of brain states, which can facilitate optimal modulation of HRCI process and improve the brain training efficiency. The method is validated using a panoramic virtual reality system. Results show that our method improves the accuracy of brain activity assessment by 8.86%, effectively demonstrating that it accurately visualizes spatiotemporal brain dynamics and enhances training outcomes when integrated with HRCI.

Keywords: Aerial Systems: Applications, Aerial Systems: Perception and Autonomy, Localization

Keywords: Imitation Learning, Bimanual Manipulation, Mobile Manipulation
Abstract: Learning whole-body mobile manipulation via imitation is essential for generalizing robotic skills to diverse environments and complex tasks. However, this goal is hindered by significant challenges, particularly in effectively processing complex observation, achieving robust generalization, and generating coherent actions. To address these issues, we propose DSPv2, a novel policy architecture. DSPv2 introduces an effective encoding scheme that aligns 3D spatial features with multi-view 2D semantic features. This fusion enables the policy to achieve broad generalization while retaining the fine-grained perception necessary for precise control. Furthermore, we extend the Dense Policy paradigm to the whole-body mobile manipulation domain, demonstrating its effectiveness in generating coherent and precise actions for the whole-body robotic platform. Extensive experiments show that our method significantly outperforms existing approaches in both task performance and generalization ability. Project page is available at: https://selen-suyue.github.io/DSPv2Net/.

Keywords: Vision-Based Navigation, Autonomous Vehicle Navigation, Planning under Uncertainty

Keywords: Dual Arm Manipulation, Assembly, Motion and Path Planning
Abstract: We provide a novel end-to-end framework for the execution of an assembly operation by two robotic arms, given the digital CAD models of the parts and their desired relative placement in their assembled state. We analyze and demonstrate the advantages of using two robotic arms simultaneously in tight assembly operations, compared to single-arm systems. Our method is implemented in both simulation and using physical robots. It provides theoretical guarantees on execution time and trajectory accuracy, supported by empirical evidence. In particular, we show that coordinated movement of two arms reduces average execution time by more than 50% compared to using a single arm only, produces higher-quality trajectories, and accelerates the search for valid robot placements. Furthermore, we establish bounds on the required dimensions of the robotic cell.

Our open source software together with real-life video demonstrations are available in our project page.

Keywords: Force and Tactile Sensing, Soft Sensors and Actuators
Abstract: Recent advances in visuotactile sensors increasingly employ biomimetic curved surfaces to enhance sensorimotor capabilities. Although such curved visuotactile sensors enable more conformal object contact, their perceptual quality is often degraded by non-uniform illumination, which reduces reconstruction accuracy and typically necessitates calibration. Existing calibration methods commonly rely on customized indenters and specialized devices to collect large-scale photometric data, but these processes are expensive and labor-intensive. To overcome these calibration challenges, we present NLiPsCalib, a physics-consistent and efficient calibration framework for curved visuotactile sensors. NLiPsCalib integrates controllable near-field light sources and leverages Near-Light Photometric Stereo (NLiPs) to estimate contact geometry, simplifying calibration to just a few simple contacts with everyday objects. We further introduce NLiPsTac, a controllable-light-source tactile sensor developed to validate our framework. Experimental results demonstrate that our approach enables high-fidelity 3D reconstruction across diverse curved form factors with a simple calibration procedure. We emphasize that our approach lowers the barrier to developing customized visuotactile sensors of diverse geometries, thereby making visuotactile sensing more accessible to the broader community.

Keywords: Biologically-Inspired Robots, Soft Robot Applications
Abstract: The development of bio-inspired jellyfish robots holds significant benefits for autonomous aquatic systems due to jellyfish’s efficient water jet propulsion. However, the current design of jellyfish robots still faces challenges in balancing high biological fidelity with the demands of lightweight, compact design and high-speed locomotion. This study presents a bio-inspired jellyfish robot that emulates the shape and efficient water jet propulsion of natural jellyfish. The origami-based bell and supporting structure are designed to form an efficient water-jet cavity. As the primary components of the robot, they endow the robot with the ability to self-recover to a stable state during locomotion, thereby reducing energy consumption. Additionally, they replace traditional transmission mechanisms, thereby reducing the weight and complexity of the robot. An optimization model is established to determine the optimal parameters of the robot. Furthermore, a near-field magnetic actuation system is designed to drive the robot, enabling contactless and silent underwater driving without waterproofing requirements. The robot features a diameter of 101.6 mm, a height of 63.8 mm, and a weight of 12.5 g. Experimental results demonstrate a maximum locomotion speed of up to 96.2 mm/s.

Keywords: Robotics and Automation in Agriculture and Forestry, Agricultural Automation, AI-Enabled Robotics
Abstract: Though robotic systems are now being commercialized and deployed in various industries, many of these systems are highly specialized and often require an advanced skill set to operate and ensure they perform as instructed. To mitigate this problem, it has been proposed to use large language models (LLMs) to synthesize mission plans in precision agriculture and other domains based on mission descriptions provided in natural language (NL). While these systems demonstrate impressive performance, they also suffer from the inherent ambiguities of NL. In this paper, we address this issue by introducing a planning architecture that combines LLMs with linear temporal logic (LTL) to ensure that, through formal verification, the mission planning system meets the specifications formulated by the user while still using NL. In our proposed system, the mission plan is seen as the implementation and the LTL formalization is seen as the specification. Both are automatically extracted from mission descriptions provided in NL. To mitigate potential bias, two separate LLMs are tasked with the implementation and specification generation. Through feedback loops, the system self-corrects when syntax or verification errors are encountered, thus offering a fully hands-off solution. Through extensive experiments, we highlight the strengths and limitations of integrating mission verification into a fully autonomous pipeline, particularly regarding an LLM's ability to generate valuable LTL formulas, and show how our proposed implementation addresses and solves these challenges.

Keywords: Machine Learning for Robot Control, Industrial Robots, Assembly
Abstract: We aim to solve the problem of learning user-intended granular skills from multi-granularity demonstrations. Traditional learning-from-demonstration methods typically rely on extensive fine-grained data, interpolation techniques, or dynamics models, which are ineffective at encoding or decoding the diverse granularities inherent in skills. To overcome it, we introduce a novel diffusion-SSM based policy (DiSPo) that leverages a state-space model, Mamba, to learn from diverse coarse demonstrations and generate multi-scale actions. Our proposed step-scaling mechanism in Mamba is a key innovation, enabling memory-efficient learning, flexible granularity adjustment, and robust representation of multi-granularity data. DiSPo outperforms state-of-the-art baselines on coarse-to-fine benchmarks, achieving up to an 81% improvement in success rates while enhancing inference efficiency by generating inexpensive coarse motions where applicable. We validate DiSPo's scalability and effectiveness on real-world manipulation scenarios. Code and Videos are available at https://robo-dispo.github.io.

Keywords: Localization, Mapping, Field Robots
Abstract: Reliable localization in prior maps is essential for autonomous navigation, particularly under adverse weather, where optical sensors may fail. We present CFEAR-TR, a teach-and-repeat localization pipeline using a single spinning radar, which is designed for easily deployable, lightweight, and robust navigation in adverse conditions. Our method localizes by jointly aligning live scans to both stored scans from the teach mapping pass, and to a sliding window of recent live keyframes. This ensures accurate and robust pose estimation across different seasons and weather phenomena. Radar scans are represented using a sparse set of oriented surface points, computed from Doppler-compensated measurements. The map is stored in a pose graph that is traversed during localization. Experiments on the held-out test sequences from the Boreas dataset show that CFEAR-TR can localize with an accuracy as low as 0.117 m and 0.096°, corresponding to improvements of up to 63% over the previous state of the art, while running efficiently at 29 Hz. These results substantially narrow the gap to lidar-level localization, particularly in heading estimation. We make the C++ implementation of our work available to the community.

Keywords: Deep Learning for Visual Perception, Visual Learning
Abstract: Vision-Language Pre-training models (VLMs) have emerged as a highly promising solution to the generative problem, achieving remarkable success in the field of 2D image generation. However, extending these 2D paradigms to 3D domains is still unexplored due to the scarcity of text-3D pairs and shape ambiguity. To address this challenge, we introduce UM3D, a two-stage pre-training architecture towards unified multimodal 3D shape generation. Our approach first optimizes a Finite Scalar Quantization based Autoencoder (FSQ-AE) to learn a compact yet powerful implicit representation with improved codebook utilization. We then encode sketch features into CLIP's multimodal embedding space to incorporate additional geometric information. This unified space conditions our well-designed Instance-Normalized Glow model (Glow-IN) to model the distribution of 3D shape representations while mitigating distribution shift issues. During inference, UM3D can accept individual text, image, sketch, or combined inputs to generate corresponding 3D shapes. Quantitative and qualitative evaluations confirm our method's effectiveness in synthesizing high-fidelity, input-consistent 3D geometries.

Keywords: Deep Learning for Visual Perception, Visual Learning, Computer Vision for Transportation
Abstract: The YOLO series of models are pivotal for real-time object detection, yet their deployment on resource-constrained edge devices necessitates effective model compression. Post-Training Quantization (PTQ) offers a promising, low-cost solution, but existing methods, primarily designed for classification tasks, often lead to significant performance degradation when applied to YOLO models. In this paper, we systematically analyze the key challenges in quantizing YOLO architectures. We identify three primary obstacles: (1) the high sensitivity of detection tasks to quantization errors, exacerbated by the non-linear IoU metric; (2) the pronounced long-tail distribution of activations, particularly with the SiLU function, which complicates low-bit quantization; and (3) the structural heterogeneity of the multi-scale, multi-task detection head, which renders conventional block-wise quantization strategies ineffective. To address these challenges, we propose a novel framework, Task-Aware and Structure-Knowledge-guided Quantization (TASKQ). Our framework introduces three key components: a sparse quantization strategy to mitigate the impact of long-tailed activations, a Detection-aware Task Regularization (DTR) mechanism that incorporates IoU-based loss to guide parameter fine-tuning, and a Scale-and-Task-Aware Head-wise Quantization (STAHQ) scheme that aligns quantization granularity with the head's functional structure. Extensive experiments on various YOLO models demonstrate that TASKQ significantly outperforms existing PTQ methods, especially in low-bit scenarios, establishing a new state-of-the-art for end-to-end YOLO quantization.

Keywords: Data Sets for Robot Learning, Visual Learning, Learning from Demonstration
Abstract: Egocentric videos capture how humans manipulate objects and tools, providing diverse motion cues for learning object manipulation. Unlike the costly, expert-driven manual teleoperation commonly used in training Vision-Language-Action models (VLAs), egocentric videos offer a scalable alternative. However, prior studies that leverage such videos for training robot policies typically rely on auxiliary annotations, such as detailed hand-pose recordings. Consequently, it remains unclear whether VLAs can be trained directly from raw egocentric videos. In this work, we address this challenge by leveraging EgoScaler, a framework that extracts 6DoF object manipulation trajectories from egocentric videos without requiring auxiliary recordings. We apply EgoScaler to four large-scale egocentric video datasets and automatically refine noisy or incomplete trajectories, thereby constructing a new large-scale dataset for VLA pre-training. Our experiments with a state-of-the-art pi_0 architecture in both simulated and real-robot environments yield three key findings: (i) pre-training on our dataset improves task success rates by over 20% compared to training from scratch, (ii) the performance is competitive with that achieved using real-robot datasets, and (iii) combining our dataset with real-robot data yields further improvements. These results demonstrate that egocentric videos constitute a promising and scalable resource for advancing VLA research.

Keywords: Mapping
Abstract: Planar structures, ubiquitous in man-made indoor environments, enable compact and accurate scene abstraction for various downstream tasks. Recent methods distill planar features into learning-based MVS geometries to obtain coherent 3D plane estimation from multi-view inputs. However, the lack of explicit planar instance definitions hinders semantic–geometry alignment, leading to distorted geometry and mismatched semantics. To address this, we propose PIPS, a planar-instance 3D reconstruction method that leverages planar structural priors for both single-view planar segmentation (SGPS module) and multi-view instance association (MVPI module). The planar instance point clouds are regularized by planar distances and then converted into complete planar meshes via an instance-level planar meshing strategy. Extensive experiments on hundreds of indoor scenes demonstrate the superior performance of our method, which is less dependent on annotations and requires no feature optimization. The effectiveness of each component is further verified through comprehensive ablation studies. The project page of PIPS is available at https://pips325.github.io.

Keywords: Contact Modeling, Dexterous Manipulation, Bimanual Manipulation
Abstract: Tactile sensors have long been valued for their perceptual capabilities, offering rich insights into the otherwise hidden interface between the robot and grasped objects. Yet their inherent compliance—a key driver of force-rich interactions—remains underexplored. The central challenge is to capture the complex, nonlinear dynamics introduced by these passive compliant elements. Here, we present a computationally efficient non-holonomic hydroelastic model that accurately models path-dependent contact force distributions and dynamic surface area variations. Our insight is to extend the object’s state space, explicitly incorporating the distributed forces generated by the compliant sensor. Our differentiable formulation not only accounts for path-dependent behavior but also enables gradient-based trajectory optimization, seamlessly integrating with high-resolution tactile feedback. We demonstrate the effectiveness of our approach across a range of simulated and real-world experiments and demonstrate the importance of modeling the path dependence of sensor dynamics.

Keywords: Learning from Demonstration, Probabilistic Inference
Abstract: Learning from human video demonstrations offers a scalable alternative to teleoperation or kinesthetic teaching, but poses challenges for robot manipulators due to embodiment differences and joint feasibility constraints. We address this problem by proposing the Joint Flow Trajectory Optimization (JFTO) framework for grasp pose generation and object trajectory imitation under the video-based Learning from Demonstration (LfD) paradigm. Rather than directly imitating human hand motions, our method treats demonstrations as object-centric guides, balancing three objectives: (i) selecting a feasible grasp pose, (ii) generating object trajectories consistent with demonstrated motions, and (iii) ensuring collision-free execution within robot kinematics. To capture the multimodal nature of demonstrations, we extend flow matching to SE(3) for probabilistic modeling of object trajectories, enabling density-aware imitation that avoids mode collapse. The resulting optimization integrates grasp similarity, trajectory likelihood, and collision penalties into a unified differentiable objective. We validate our approach in both simulation and real-world experiments across diverse real-world manipulation tasks.

Keywords: Grasping, Dexterous Manipulation
Abstract: Cross-embodiment dexterous grasp synthesis refers to adaptively generating and optimizing grasps for various robotic hands with different morphologies. This capability is crucial for achieving versatile robotic manipulation in diverse environments and requires substantial amounts of reliable and diverse grasp data for effective model training and robust generalization. However, existing approaches either rely on physics-based optimization that lacks human-like kinematic understanding or require extensive manual data collection processes that are limited to anthropomorphic structures. In this paper, we propose CEDex, a novel cross-embodiment dexterous grasp synthesis method at scale that bridges human grasping kinematics and robot kinematics by aligning robot kinematic models with generated human-like contact representations. Given an object's point cloud and an arbitrary robotic hand model, CEDex first generates human-like contact representations using a Conditional Variational Auto-encoder pretrained on human contact data. It then performs kinematic human contact alignment through topological merging to consolidate multiple human hand parts into unified robot components, followed by a signed distance field-based grasp optimization with physics-aware constraints. Using CEDex, we construct the largest cross-embodiment grasp dataset to date, comprising 500K objects across four gripper types with 20M total grasps. Extensive experiments show that CEDex outperforms state-of-the-art approaches and our dataset benefits cross-embodiment grasp learning with high-quality diverse grasps.

Keywords: Prosthetics and Exoskeletons, Biologically-Inspired Robots, Hardware-Software Integration in Robotics
Abstract: Lower extremity exoskeletons designed for multi-joint assistance are increasingly explored for rehabilitation and human augmentation. However, conventional monoarticular designs often suffer from joint misalignment and actuator redundancy, limiting their efficiency and user comfort. This study presents a biarticular rigid powered lower extremity exoskeleton that simultaneously assists the knee and ankle joints through a single actuator, enabling coordinated torque generation across adjacent joints. A hierarchical control framework combining gait segmentation, impedance-based torque generation, and gravity/friction compensation is implemented to provide phase-specific assistance. Experimental results show that the proposed exoskeleton reduces gastrocnemius activation by up to 63.4% and metabolic cost by up to 11.6% during stair ascent, with corresponding reductions of 28.3% and 8.2% during level walking. These findings demonstrate the effectiveness of the biarticular and underactuated structure in enhancing locomotor efficiency, highlighting its potential as a compact and practical solution for dynamic and diverse mobility scenarios.

Keywords: Model Learning for Control, Machine Learning for Robot Control, Optimization and Optimal Control
Abstract: Learning-based model predictive control (MPC) can enhance control performance by correcting for model inaccuracies, enabling more precise state trajectory predictions than traditional MPC. A common approach is to model unknown residual dynamics as a Gaussian process (GP), which leverages data and also provides an estimate of the associated uncertainty. However, the high computational cost of online learning poses a major challenge for real-time GP-MPC applications. This work presents an efficient implementation of an approximate spatio-temporal GP model, offering online learning at constant computational complexity. It is optimized for GP-MPC, where it enables improved control performance by learning more accurate system dynamics online in real-time, even for time-varying systems. The performance of the proposed method is demonstrated by simulations and hardware experiments in the exemplary application of autonomous miniature racing.

Keywords: Localization, SLAM, Mapping
Abstract: LiDAR-Inertial Odometry (LIO) is crucial for robot navigation and autonomous driving. Most existing methods rely on the assumption of a static environment, indiscriminately using all LiDAR measurements for localization. However, LiDAR data acquired in urban scenes often contain dynamic objects such as vehicles and pedestrians, which can adversely affect localization accuracy—particularly when using solid-state LiDAR with a relatively narrow field of view. To address this issue, we propose a novel Real-time and Robust solid-state LiDAR-Inertial Odometry (R2-LIO) framework that remove the dynamic objects to improve the localization accuracy and robustness. Specifically, we design a dynamic point removal mechanism based on voxel state changes, which removes dynamic points and preserving most static points to effectively reduce interference from dynamic objects. In addition, we introduce a line search mechanism into the Error State Iterated Kalman Filter (ESIKF) to improve the localization accuracy. Experimental results on the challenging YULAN and HeLiPR datasets show that R2-LIO surpasses existing methods, verifying its effectiveness in improving the localization accuracy and robustness.

Keywords: Field Robots, Marine Robotics, Biomimetics
Abstract: MorphoBall is a bio-inspired, deformable spherical robot designed for multimodal locomotion across terrestrial and aquatic environments. By integrating a dual-mode drive system (spherical rolling and differential-drive) with a morphology-mediated propulsion mechanism, MorphoBall achieves adaptive mobility in diverse terrains, including flat ground, slopes, and water surfaces. A key innovation lies in the dual-function ciliary band, which provides both passive damping during terrestrial rolling and active propulsion during aquatic navigation. Model-based controllers are developed to regulate forward velocity, trajectory curvature, and roll tilt angle, demonstrating superior stability and responsiveness compared to baseline PID implementations. Experimental results validate MorphoBall's ability to autonomously navigate structured indoor environments and traverse unstructured outdoor terrains, achieving seamless mode transitions and completing missions 34% faster than single-morphology strategies. This work highlights the potential of morphology adaptation as a tool for enhancing environmental adaptability in mobile robotics.

Keywords: Semantic Scene Understanding, Deep Learning for Visual Perception, Object Detection, Segmentation and Categorization
Abstract: Superpoint-based pipelines provide an efficient alternative to point- or voxel-based 3D semantic segmentation, but are often bottlenecked by their CPU-bound partition step. We propose a learnable, fully GPU partitioning algorithm that generates geometrically and semantically coherent superpoints 13× faster than prior methods. Our module is compact (under 60k parameters), trains in under 20 minutes with a differentiable surrogate loss, and requires no handcrafted features. Combined with a lightweight superpoint classifier, the full pipeline fits in < 2 MB of VRAM, scales to multi-million-point scenes, and supports real-time inference. With 72× faster inference and 120× fewer parameters, EZ-SP matches the accuracy of point-based SOTA models across three domains: indoor scans (S3DIS), autonomous driving (KITTI-360), and aerial LiDAR (DALES). Code and pretrained models are accessible at github.com/drprojects/superpoint_transformer.

Keywords: Assembly, Compliant Assembly, Intelligent and Flexible Manufacturing
Abstract: Classical robotic systems typically rely on custom planners designed for constrained environments. While effective in restricted settings, these systems lack generalization capabilities, limiting the scalability of embodied AI and general‑purpose robots. Recent data‑driven Vision‑Language‑Action (VLA) approaches aim to learn policies from large‑scale simulation and real‑world data. However, the continuous action space of the physical world significantly exceeds the representational capacity of linguistic tokens, making it unclear if scaling data alone can yield general robotic intelligence. To address this gap, we propose ActionReasoning, an LLM-driven framework that performs explicit action reasoning to produce physics-consistent, prior-guided decisions for robotic manipulation. ActionReasoning leverages the physical priors and real-world knowledge already encoded in Large Language Models (LLMs) and structures them within a multi-agent architecture. We instantiate this framework on a tractable case study of brick stacking, where the environment states are assumed to be already accurately measured. The environmental states are then serialized and passed to a multi-agent LLM framework that generates physics-aware action plans. The experiments demonstrate that the proposed multi-agent LLM framework enables stable brick placement while shifting effort from low-level domain-specific coding to high-level tool invocation and prompting, highlighting its potential for broader generalization. This work introduces a promising approach to bridging perception and execution in robotic manipulation by integrating physical reasoning with LLMs.

Keywords: Deep Learning for Visual Perception
Abstract: In this work, we propose a novel Mamba block DenVisCoM, as well as a novel hybrid architecture specifically tailored for accurate and real-time estimation of optical flow and disparity estimation. Given that such multi-view geometry and motion tasks are fundamentally related, we propose a unified architecture to tackle them jointly. Specifically, the proposed hybrid architecture is based on DenVisCoM and a Transformer-based attention block that efficiently addresses real-time inference, memory footprint, and accuracy for at the same time for joint estimation of motion and 3D dense perception tasks. We extensively analyze the benchmark trade-off of accuracy and real-time processing on a large number of datasets. Our experimental results and related analysis suggest that our proposed model can accurately estimate optical flow and disparity estimation in real time. All models and associated code are available at https://github.com/vimstereo/DenVisCoM.