Keywords: Manipulation Planning and Control
Abstract: Mixed integer bilinear programs (MIBLPs) offer tools to resolve robotics motion planning problems with orthogonal rotation matrices or static moment balance, but require long solving times. Recent work utilizing data-driven methods has shown potential to overcome this issue allowing for applications on larger scale problems. To solve mixed-integer bilinear programs online with data-driven methods, several re-formulations exist including mathematical programming with complementary constraints (MPCC), and mixed-integer programming (MIP). In this work, we compare the data-driven performances of various MIBLP reformulations using a book placement problem that has discrete configuration switches and bilinear constraints. The success rate, cost, and solving time are compared along with non-data-driven methods. Our results demonstrate the advantage of using data-driven methods to accelerate the solving speed of MIBLPs, and provide references for users to choose the suitable re-formulation.

Keywords: Manipulation Planning and Control, Multisensor Data Fusion, AI Reasoning Methods for Robotics
Abstract: One of the most critical aspects of multimodal Reinforcement Learning (RL) is the effective integration of different observation modalities. Having robust and accurate representations derived from these modalities is key to enhancing the robustness and sample efficiency of RL algorithms. However, learning representations in RL settings for visuotactile data poses significant challenges, particularly due to the high dimensionality of the data and the complexity involved in correlating visual and tactile inputs with the dynamic environment and task objectives. To address these challenges, we propose Multimodal Contrastive Unsupervised Reinforcement Learning (M2CURL). Our approach employs a novel multimodal self-supervised learning technique that learns efficient representations and contributes to faster convergence of RL algorithms. Our method is agnostic to the RL algorithm, thus enabling its integration with any available RL algorithm. We evaluate M2CURL on the Tactile Gym 2 simulator and we show that it significantly enhances the learning efficiency in different manipulation tasks. This is evidenced by faster convergence rates and higher cumulative rewards per episode, compared to standard RL algorithms without our representation learning approach.

Keywords: Robotics in Hazardous Applications, Manipulation Planning and Control
Abstract: Robot skill learning and execution in uncertain and dynamic environments is a challenging task. This paper proposes an adaptive framework that combines Learning from Demonstration (LfD), environment state prediction, and high-level decision making. Proactive adaptation prevents the need for reactive adaptation, which lags behind changes in the environment rather than anticipating them. We propose a novel LfD representation, Elastic-Laplacian Trajectory Editing (ELTE), which continuously adapts the trajectory shape to predictions of future states. Then, a high-level reactive system using an Unscented Kalman Filter (UKF) and Hidden Markov Model (HMM) prevents unsafe execution in the current state of the dynamic environment based on a discrete set of decisions. We first validate our LfD representation in simulation, then experimentally assess the entire framework using a legged mobile manipulator in 36 real-world scenarios. We show the effectiveness of the proposed framework under different dynamic changes in the environment. Our results show that the proposed framework produces robust and stable adaptive behaviors.

Keywords: Manipulation Planning and Control, Dynamics and Control, Contact: Modeling, Sensing and Control 
Abstract: In robotic peg-in-hole assembly, contact dynamics provide critical information for precise control. This study introduces a novel peg-in-hole framework for a dynamic continuous contact environment using a supervised learning-based peg-hole extrinsic pose estimation model. The transformer-based model is trained on contact data sequences, generated in simulation with various shapes. With a single model, pose estimation performance approached 3.62mm and 1.68° for position XY and orientation XYZ, respectively. The proposed framework demonstrates effective peg-in-hole performance across 12 shapes with 5 degrees of freedom, outperforming conventional search methodologies in accuracy and efficiency, over 1.8 and 5.6 times better respectively. The study highlights the potential in dynamic continuous contact situations, contributing to precise assembly in dynamic environments.

Keywords: Manipulation Planning and Control
Abstract: Ongoing research is being conducted on the autonomous operation of robots for manufacturing automation. However, it faces challenges due to uncertainty in unstructured environments. Particularly, tasks with frequent contact, such as peg-in-hole assembly, pose a risk of hazardous forces on the robot system due to uncertainty-induced collisions. To address this, we propose a regression learning model to infer the orientation error angles of the peg and hole based on contact data. We demonstrate its effectiveness in overcoming jamming caused by misalignment in peg-in-hole insertion by integrating it with a traditional parallel force/position controller and a learned misalignment error estimation network (MEN). Experimental results show that the system integrated with MEN achieves a 100% insertion success rate and a faster average duration compared to the single system, demonstrating stable peg-in-hole task without jamming.

Keywords: Social and Socially Assistive Robotics, Learning From Humans, Motion Planning and Obstacle Avoidance
Abstract: Robots are becoming popular in assisting humans. The mobile robot navigation in human crowd environments has become more important. We propose a deep reinforcement learning-based mobile robot navigation method that takes the observation from the robot's onboard Lidar sensor as input and outputs the velocity control to the robot. A customized deep deterministic policy gradient (DDPG) method is developed that incorporates guiding points to guide the robot toward the global goal. We built a 3D simulation environment using an open dataset of real-world pedestrian trajectories that were collected in a large business center. The neural network models are trained and tested in such environments. We compare the performance of our proposed method with existing algorithms that include a classic motion planner, an existing DDPG method, and a generative adversarial imitation learning (GAIL) method. Using the measurement metrics of success rate, freezing times, and normalized path length, extensive simulation results show that our method outperforms other state-of-the-art approaches in both trained and untrained environments. Our method has also better generalizability compared with the GAIL method.

Keywords: Social and Socially Assistive Robotics, Physical and Cognitive Human-Robot Interaction, Biomimetic and Bioinspired Robots
Abstract: Reading books is a fundamental cornerstone in the formation of young minds, and it influences various facets of children's lives. Recent technological advances have led to extensive research into the potential role of robots as companions in children's reading experiences. This study explores how to engage children in reading through human-robot interaction, deriving three design specifications: a robot with an animal-like appearance; a robot that responds as a real organism; and a robot that reacts based on reading status. An interactive reading companion robot named Book-Toki was developed, which has the appearance of a rabbit. The robot's ears, made of deformable silicone, and the robot's head move in response to the child's reading. As the child reads the book aloud, the ears change shape from folded to unfolded, and the head slowly peeks out of the burrow, symbolizing active concentration. This visual response sparks children's imagination and gives them a sense of 'I am listening to you,' which enriches the reading experience. In this way, robots can play a significant role in making reading a more enjoyable experience for children.

Keywords: Roboethics, Physical and Cognitive Human-Robot Interaction, Social and Socially Assistive Robotics
Abstract: Science fiction has a great influence on how we view and develop technology, including robots. However, the science fiction we often cite in these discussions are visual: films and TV shows. While those stories are crucial to the discussion, we seek to expand and shift the conversation around science fiction to include more literature, especially recent literature. We highlight some stories, principally books released in the last decade, and the ways they might be relevant to roboticists today in how they portray the themes of empathy, embodiment, and the place of robots in a human world.

Keywords: Physical and Cognitive Human-Robot Interaction, Social and Socially Assistive Robotics, Robotic Systems Architectures and Programming
Abstract: In Human-Robot Interaction (HRI), the ability of robots to understand social atmosphere is essential for the quality improvement of the interaction. However, robots have limitations in recognizing and performing appropriate behaviors for social atmosphere that are not externally revealed. In this study, based on Large Language Models (LLMs), we aimed to investigate whether adding atmosphere elements to robots' social behavior generation could enable robots to generate and perform more appropriate behaviors in social contexts. A total of 50 participants participated in the experiment. As a result of the experiment, a robot scenario that incorporates atmospheric elements showed significantly higher differences in terms of the robot's social behavior, sociability, and human-robot interaction, compared to a robot scenario without atmospheric elements. This study provides new insights into the importance of considering social context when setting prompting elements for LLM-based social robots. By adding social atmospheric elements, it is expected that robots will be able to improve human’s understanding of social context and perform more natural and effective interactions.

Keywords: Multisensor Data Fusion, Range, Sonar, GPS and Inertial Sensing
Abstract: Deployment of multiple robots in real-world scenarios requires simultaneous information exchange from all platforms to ensure effective task performance. The robot’s relative position to its peers is an important data needed to predict collision between robots or task distribution. This paper introduces a robust and simple method for achieving cooperative localization among multiple robots, utilizing a single ultra-wideband (UWB) sensor for each platform. Each robot uses a visual-inertial odometry (VIO) system to track its own trajectory. Given the inherent drift associated with VIO systems, we leverage UWB data to estimate and correct this drift, enhancing each robot’s localization accuracy. Our approach substantially improves the result compared to other cooperative localization methods and can even correct the VIO ego-motion.

Keywords: Simultaneous Localization and Mapping (SLAM), Multisensor Data Fusion, Range, Sonar, GPS and Inertial Sensing
Abstract: Scientific and engineering applications of solid-state light detection and ranging (LiDAR) sensors with no rotating mechanisms and a limited field of view have attracted research attention in recent years because of their cost-effectiveness and durability. However, it is challenging to perform place recognition, which is one of the most important problems in simultaneous localization and mapping (SLAM), using limited field-of-view measurements. Considering a terrestrial SLAM framework for quadruped robots with limited field-of-view LiDAR sensors, this study proposes a terrain-based place recognition algorithm that uses a set of foot contact information for quadruped robots. The practical feasibility of the proposed approach is demonstrated through experimental results using a quadruped robot system with a limited field-of-view LiDAR sensor.

Keywords: World Modelling
Abstract: Perceptual anchoring traditionally relies on data from sensors mounted on a mobile robot. This allows the sensors to be close to objects in the environment, making it possible to acquire details with high accuracy. IoT sensors are becoming more and more ubiquitous, and are found in both private and public buildings. IoT cameras are often mounted on ceilings or walls, allowing them to observe a larger part of the environment than robot-mounted sensors, but making them unsuitable for acquiring detailed visual information. They often have a lower sampling rate, keeping them cost-effective. We hypothesize that IoT and robot sensors can be combined in a way that exploits the details of the robot sensors and the immediately high overview of the IoT sensors by embracing ubiquitous sensing. In this work, we evaluate and compare different methods for associating IoT and robot sensing data, including a novel context-based similarity measure and a simple geometric baseline. The results support our hypothesis and we find that all methods outperform the baseline method in most scenarios. Using context-similarity is most beneficial for the affinity propagation clustering algorithm for setups with 16 and 12 objects. These results can serve as a guideline for designing anchoring or world modeling systems, using IoT and robot sensing data.

Keywords: Simultaneous Localization and Mapping (SLAM), Multisensor Data Fusion, Range, Sonar, GPS and Inertial Sensing
Abstract: This study presents MSCKF-DVIO (Multi-State Constraint Kalman Filter - Depth-aided Visual Inertial Odometry) as an innovative approach to address the limitations of existing methods. MSCKF-DVIO leverages RGB-D images and low-cost IMU measurements to enhance the accuracy and efficiency of visual-inertial odometry systems. The proposed framework jointly optimizes RGB-D images and low-cost IMU measurements, enabling robust and precise state estimation.

By reducing the number of state variables compared to the existing MSCKF-VIO, the efficiency of state augmentation and covariance computations is significantly enhanced. Notably, the prediction of the future state is achieved through the interpolation of past keyframe viewpoint poses from the image and short-time IMU integration. The proposed MSCKF-DVIO improves the accuracy of estimation by applying nonholonomic constraints (NHCs) of a differential-wheeled mobile robot and utilizing the zero velocity update (ZUPT) based on stationary state determination. The efficacy of MSCKF-DVIO is evaluated through the Absolute Pose Error (APE) using 3D LiDAR-based localization as the ground truth. The evaluation includes pose estimation outcomes and performance evaluations in comparison to other algorithms. The results demonstrate significant promise of MSCKF-DVIO for improving the performance and reliability of visual-inertial odometry systems.

Keywords: Multisensor Data Fusion, Robotics in Hazardous Applications, Object Recognition
Abstract: In recent years, we have witnessed significant progress in the field of autonomous mobility. However, these advancements have been highly limited to urban environments. Autonomous mobility in off-road environments is more challenging because of diverse environments, illumination conditions, lack of distinct features, among others. This paper delves into the issue of semantic segmentation for off-road conditions, focusing on the fusion of RGB (Red-Green-Blue) & NIR (Near Infrared) under intense lighting conditions. Given the huge variability associated with off-road environments, the existing datasets fail to capture strong illumination variations in the environment and therefore, to address this, we present "RPTU-Forest dataset" with 285 RGB images along with their respective multi-spectral images including (RGB, NIR, GREEN, RED and Red Edge (REG)) channel images. The paper explores the different fusion approaches for multispectral semantic segmentation documented in literature such as concatenation, Variational autoencoders, dual branch and DenseFuse fusion and provides a thorough analysis of each approach for the task of semantic segmentation. Two state-of-the-art semantic segmentation networks (UNet & DeepLab V3) are additionally compared for our use-case and the best among them is selected. The paper concludes with a qualitative and quantitative analysis. This work represents a significant contribution to the ongoing research in off-road autonomous mobility. The code is publicly available at (https://github.com/ShubhamAbhayDeshpande/RobustSemanticSegmentationWithSensorFusion)

Keywords: Telerobotics, Simultaneous Localization and Mapping (SLAM)
Abstract: Space Exploration is a continuously flourishing field of research, as NASA has a plethora of ongoing missions to be achieved over the next few years. With the advent of many robotic platforms dedicated for space exploration such as NASA's Dragonfly, their Mars Perserverance Rover, and many more, it is evident that these types of robots will continue to play a key role. Despite their success, the limited man power for such specialized operators, reliability concerns with Unmanned Aerial Vehicles (UAVs or drones) in such harsh environments, and the limited battery life justify the consideration of different approaches. This paper presents work towards a suspended Cable-Driven Parallel Robot (CDPR), paired with an intuitive Virtual Reality interface designed for space exploration. Real-time 3D Point Cloud visualization can potentially grant the operator a greater sense of immersion, and can allow any operator to view the environment around the CDPR. Along with the benefits of a CDPR, an immersive VR interface gives operators intuitive control through rigorous tasks.

Keywords: Object Recognition, Computer Vision and Visual Servoing
Abstract: Recent years have seen notable advancements in wildfire smoke detection, particularly in Uncrewed Aircraft Systems (UAS)-based detection employing diverse deep learning (DL) approaches. Despite the promise exhibited by these approaches, the task of detecting smoke from UAS imagery remains challenging due to difficulties in differentiating smoke from similar phenomena such as clouds and water. This work introduces a novel DL-based method for smoke detection from UAS visual observations. The core idea involves segregating forest areas from non-forest regions, such as sky and lake, and exclusively applying smoke detection to forested areas, thus eliminating the chance of misidentifying clouds and water as smoke. Specifically, we utilized a Mask Region-Based Convolutional Neural Network (Mask R-CNN) for semantic segmentation to remove non-forest regions (e.g., sky and lake): Subsequently, a customized You Only Look Once-version 7 (YOLOv7) model was trained to detect smoke within the forest areas. The proposed method was validated on an image dataset collected from our previous prescribed burn experiment, where we extracted 246 images to train both MASK R-CNN and YOLOv7 models. Additionally, we extract another 128 images to validate and confirm the efficacy of our enhanced wildfire smoke detection approach. The test results demonstrate that our proposed approach, employing MASK R-CNN and YOLOv7 models, outperforms the YOLOv7-only model by 25.3% in precision, 18.7% in recall, and 45% in mean Average Precision (mAP). The datasets are available at: https://github. com/khanRmahmud/wildfire-smoke-detection.

Keywords: Object Recognition, Roboethics, World Modelling
Abstract: Semantic mapping is computationally expensive, requiring either large GPUs on the robot, or significant numbers of uploaded images to the cloud. Neither solution is appropriate for home robots, where the hardware must be inexpensive, and privacy is a real concern. Instead of resorting fully to hand-labeled maps to address privacy concerns, where label noise can be a big problem depending on the quality of the input interface, we propose an interactive solution integrating hand drawn boxes with robot exploration data. Specifically, nonlinear optimization is conducted on each user-submitted proposal based on the bounding boxes and detection information collected by the robot, generating higher quality estimates quickly for human review as part of an interaction. In this manner, images are processed once on the robot with cost effective algorithms, and then discarded, minimizing the risk of exposing sensitive information. This privacy-aware approach leads to an improvement in map and object quality compared to using hand-labeled maps directly, even when working with user proposals that have up to 50% label noise.

Keywords: Object Recognition, Computer Vision and Visual Servoing, Intelligent Robotic Vehicles
Abstract: Human-machine interaction can be supported by the detection of humans through the simultaneous localization and distinction from non-human objects. This paper compares modern object detection algorithms (Damo-YOLO, YOLOv6, YOLOv7 and YOLOv8) in combination with Transfer Learning and Super Resolution in different scenarios to achieve human detection on low resolution infrared images. The data set created for this purpose includes images of an empty room, images of warm coffee cups, and images of people in various scenarios and at distances ranging from two to six meters. The Average Precision AP@50 and AP@50:95 values achieved across all scenarios reach up to 98.02 % and 66.99% respectively.

Keywords: Object Recognition, Physical and Cognitive Human-Robot Interaction, Robot Surveillance and Security
Abstract: Security operations are all about detecting change. Looking for out of place or suspicious things or people is the job, so the first step is to learn what is normal and then recognize what is not. Change detection in robotics, however, has focused on the big picture – extracting mask images of new buildings or construction to support autonomous cars, or correcting semantic maps. If we want robots to help patrol a facility, a different type of change detection is required that can be quickly adapted for working with humans to address new security concerns. To this end, we propose a highly dynamic change detection system based on Contrastive Language-Image Pre-Training (CLIP) and Neural Radiance Fields (NeRF). NeRF is used to generate images from the viewpoint that are high quality indoor reconstructions, while CLIP-based segmentation allows a security guard to search for a variety of potential threats using natural language queries. The resulting robotic system is demonstrated to be effective on an office environment with no additional manual annotation.

Keywords: Object Recognition, Computer Vision and Visual Servoing
Abstract: This paper introduces a deep learning-based semantic segmentation framework tailored for robotic apple harvesting, leveraging synthetic data generated within a 3D simulated apple orchard. The proposed simulation environment replicates real-world scenarios, encompassing challenges such as occlusion, variety in apple types, and changes in lighting conditions. This approach eliminates the extensive costs and complexities associated with collecting real-world datasets, particularly in unpredictable agricultural settings. The synthetic dataset, rendered from perspectives consistent with a robotic harvester's camera in the Gazebo physics engine, provides a comprehensive range of scenarios for robust model training. For validation, we deploy U-Net, a fully convolutional neural network, emphasizing its adaptability to domain shifts between synthetic and real-world data. By integrating strategies such as domain adaptation, data augmentation, and the inclusion of pre-trained ResNet-50 encoders in the U-Net framework, we demonstrate superior performance in detecting and segmenting apples in diverse real-world conditions compared to standard U-Net models and traditional computer vision techniques. The results highlight the potential of synthetic data in deep learning-based semantic segmentation, offering a cost-effective and scalable solution when real-world data is limited or hard to collect.

Keywords: Object Recognition, Computer Vision and Visual Servoing, Learning From Humans
Abstract: Lots of well-labelled data are needed to utilize and understand new types of sensors such as event camera systems. To reduce the effort for labelling dataset, we utilized a Swin transformer as a backbone and a transformer decoder with mask attention for segmentation tasks. More and more labelled data has been collected by just selecting well-labelled data and fine-tuning from the collected ones iteratively. Stereo event datasets are being built by non-experts by labelling them via fine-tuning on Swin transformer backbone and a pre-trained transformer decoder. So far one-fifth of the images collected by a traditional camera aligned with the stereo event camera system have been accepted as properly labelled data.

Keywords: Manipulation Planning and Control, Performance Evaluation and Optimization, Humanoids
Abstract: This paper focuses on implementing Spinova, a drum stroke often discovered when drummers hit a drum over a large distance in a limited amount of time, by minimizing wrist displacement. In order to execute the minimization at the right time, a triggering condition is introduced by appropriately quantifying the drum distance and stroke time length. The method is implemented on MOFFETT, a drumming humanoid platform capable of autonomous drumming based on a given drum score. Finally, the impact of Spinova on energy consumption and reaction torque on shoulder is analyzed.

Keywords: Manipulation Planning and Control, Computer Vision and Visual Servoing, Humanoids
Abstract: This paper presents the development and evaluation of specialized perception and manipulation methodologies for a dual-arm robot system. The goal of this research is to enhance the adaptability and performance of this system for intricate real-world bimanual tasks. The proposed approach is validated through a key real-world task - autonomous door-opening. This particular task provides an excellent test bed, as it requires robust perception, coordinated manipulation, and context-aware decision-making in a human-centric environment. To tackle these challenges, we employ methods such as camera-based object recognition and localization, task-specific motion planning algorithms, and integrated force feedback mechanisms. The proposed strategies for door-opening have been successfully implemented, leading to improvements in the overall performance, robustness, and adaptability of the dual-arm robot system under varied conditions.

Keywords: Telerobotics, Mechanism and Design, Humanoids
Abstract: This research addresses the need for advanced teleoperation in challenging environments where human access is limited, introducing MUAT (Modularized Ultralight Arm Tracker). Leveraging advancements in humanoid robotics and motion retargeting, MUAT creatively overcomes the limitations of previous teleoperation methods, including IMU-based and optical sensor-based systems. These methods often struggled with issues related to measurement accuracy, space requirements, and operational constraints. MUAT employs a position sensor-based exoskeleton approach, offering a novel solution. The design of MUAT is characterized by its lightweight and modular construction, allowing it to be easily attached to an operator's forearm. This design minimizes interference with movement and grants unrestricted use of the hands. To assess the feasibility and effectiveness of MUAT, a preliminary study version, referred to as p-MUAT (preliminary MUAT), was developed in this study. Through rigorous testing with p-MUAT, we successfully validated the device's precision in tracking the operator's wrist position, utilizing a motion capture system for validation. Additionally, this study demonstrated the device's capability to effectively teleoperate a humanoid robot.

Keywords: Humanoids, Dynamics and Control, Telerobotics
Abstract: The complexity of the control and operation is one of the roadblocks of widespread utilization of humanoid robots. In this study, we introduce a novel approach to humanoid robot control by leveraging a mixed reality (MR) interface for whole-body balancing and manipulation. This interface system uses an MR headset to track the operator’s movement and provide the operator with useful visual information for the control. The robot mimics the operator’s movement through a motion retargeting method based on linear scaling and inverse kinematics. The operator obtains visual access to the robot’s perspective view augmented with fiducial detection and perceives the current stability of the robot by evaluating the robot’s center-of-mass state in real-time against the precomputed balanced state basin. In experimental demonstrations, the operator successfully controlled the robot to grasp and lift an object without falling. The common issues in teleoperation with virtual reality headsets, motion sickness and unawareness of their surroundings, are reduced to a low level by using the MR headset with transparent glasses. This study demonstrates the potential of MR in teleoperation with a motion retargeting and stability monitoring method.

Keywords: Actuation and Actuators, Dynamics and Control, Humanoids
Abstract: In this paper, we will discuss the development of linear actuators for agile humanoid robots. Most linear actuators have greater rigidity than rotary actuators and are configured to be used in high-load environments using large deceleration ratios or hydraulic systems. However, to apply the linear actuator to a very dynamic humanoid robot, in addition to the large rigidity capable of withstanding impact, it must satisfy requirements such as accurate speed control, low impedance force control, and high power density. In particular, for high-bandwidth force control, mechanical impedance in the joint stage must be minimized. To compensate for these shortcomings, this paper proposes a linear actuator that minimizes mechanical impedance and ultimately has excellent back-drive performance by directly fastening only ball screws with large screw gaps to high power density BLDC motors. Taking into account these characteristics, a linear actuator controller was designed and the performance and function of the controller were verified on the actual actuator. Finally, we demonstrate that it is optimized for humanoid robots with agile motor performance by applying linear actuators to the leg of humanoid robots for actual bipedal walking.

Keywords: Social and Socially Assistive Robotics
Abstract: The primary goal of our project is to generate artificial humanoid avatars, specifically talking faces from a single image and a text for enhanced human-robotic interaction. We put a specific emphasis on avatars that exhibit precise lip motion, head movement, and dynamic facial expressions. We believe that these attributes are essential components, making avatars significantly more engaging to human users. Contrary to traditional 3D modelling techniques that are commonly used in many modern state-of-the-art systems, our project aims to build avatars from machine-learned image augmentations. While numerous studies have been conducted on talking face generation systems, most have explored lip-motion in isolation from emotional facial shifts. Additionally, many methods depend heavily on audio or video inputs. In this paper, we propose an emotional talking face generation, called EmoFaceGen, which generates realistic talking face videos with emotions. Our system is unique in that it creates emotional talking face videos taking a single facial image and a text as inputs, then producing a talking face video with emotions as an output. The text input is converted to an audio source using the Text-To-Speech method for our project. Based on our findings, EmoFaceGen provides a more realistic talking face representation compared to other open-source models, highlighting a positive direction in overcoming present challenges in this area especially when considering the memory and hardware limitations associated with conventional 3D graphics methods.

Keywords: Social and Socially Assistive Robotics, Physical and Cognitive Human-Robot Interaction
Abstract: This research investigates the use of compact deep neural network designs to teach the humanoid robot Pepper to comprehend American Sign Language (ASL), enhancing non-verbal interactions between humans and robots. Initially, we developed a streamlined and powerful model specifically for ASL interpretation, optimized for embedded systems. This model ensures swift sign language recognition while minimizing computational demands. Furthermore, we incorporate large language models (LLMs) to enhance the robot's interactive abilities. By carefully crafting prompts, Pepper can produce Co-Speech Gesture responses, fostering more fluid and realistic human-robot conversations. We also introduce a comprehensive software framework that encapsulates these advancements in a socially conscious AI interaction model. Utilizing Pepper's capabilities, we showcase the practicality and impact of our method in actual scenarios. Our findings underscore the significant possibilities for improving human-robot interactions through non-verbal means, bridging communication barriers, and making technology more accessible and understandable.

Keywords: Social and Socially Assistive Robotics, AI Reasoning Methods for Robotics
Abstract: With the ongoing advancements in AI technology, human-robot interactions have become increasingly prevalent, extending across diverse domains such as AI speakers and service robots. Despite the progress, users often perceive interactions with robots as lacking naturalness. One factor contributing to this perception is the improper involvement of robots in specific situations. To address these issues, this paper proposes a method for defining and recognizing social conversation contexts. Furthermore, the paper outlines plan for constructing a database to assess the performance of the defined problem. By enabling robots to recognize social conversational situations based on the speaker and addressee and generate context-aware actions, we envision achieving more natural interactions. Through the newly proposed situation definition and problem-solving approach, we anticipate alleviating some of the unnatural interaction elements in Human-Robot Interaction (HRI) scenarios.

Keywords: Social and Socially Assistive Robotics, Physical and Cognitive Human-Robot Interaction, Foundations of Sensing and Estimation
Abstract: In this paper, we propose a new real-time recognition system that predicts the Big Five personality traits - extroversion, agreeableness, conscientiousness, neuroticism, and openness. This system continuously evaluates these traits over time and across various context. By treating each moment individually to predict personality scores, we have implemented and compared various multimodal approaches to enhance the accuracy of these predictions. Our framework has shown the capability to obtain robust personality predictions extrapolated from complex information. Additionally, we have successfully implemented this framework in a real robot, confirming its potential applicability in the realm of social robotics. Based on these research findings, our personality prediction model is expected to operate stably in a wide range of environments, contributing to social interactions and applications.

Keywords: Manipulation Planning and Control, Performance Evaluation and Optimization
Abstract: Our research presents an improvement to task planning using Large Language Models (LLMs) by incorporat- ing a simple approach to consider uncertainty in planning. This strategy, which differs from standard LLM-based planners, emphasizes quantifying uncertainty and exploring alternative paths for task execution. By establishing a method to measure uncertainty by setting appropriate thresholds on probabilities in skill selection, our planner is more capable at selecting a better path for carrying out tasks. Through our experiments in high- level planning within the ALFRED task domain, we observed an improvement in plan execution success rates by 0.96–2.41 percent points over conventional LLM-based task planners. These results demonstrate that uncertainty-aware strategies can lead to more precise and effective task planning.

Keywords: Underwater Robotics, Telerobotics, Manipulation Planning and Control
Abstract: In an effort to lower the barrier to entry in underwater manipulation, this paper presents an open-source, user-friendly framework for bimanual teleoperation of a light- duty underwater vehicle-manipulator system (UVMS). This framework allows for the control of the vehicle along with two manipulators and their end-effectors using two low-cost haptic devices.

The UVMS kinematics are derived in order to create an independent resolved motion rate controller for each manipulator, which optimally controls the joint positions to achieve a desired end-effector pose. This desired pose is computed in real-time using a teleoperation controller developed to process the dual haptic device input from the user. A physics-based simulation environment is used to implement this framework for two example tasks as well as provide data for error analysis of user commands. The first task illustrates the functionality of the framework through motion control of the vehicle and manipulators using only the haptic devices. The second task is to grasp an object using both manipulators simultaneously, demonstrating precision and coordination using the framework. The framework code is available at https://github.com/stevens-armlab/uvms_bimanual_sim.

Keywords: Underwater Robotics, Multi-Robot Systems, Range, Sonar, GPS and Inertial Sensing
Abstract: Multiple autonomous marine vehicles can be used for complementary cooperative operation to perform given marine tasks. This paper presents a method of cooperative positioning system for ASV-AUV. ASVs can perform accurate localization based on GNSS. On the other hand, AUVs suffer from lack of absolute localization and the accumulation of inertial sensor based localization error. The proposed method is developed to perform reliable localization of both ASV and AUV by cooperative localization system. The proposed cooperative positioning system utilizes relative geometric information between vehicles acquired by acoustic sensors. AUV can correct its own location by using the relative geometric information. The proposed method is implemented by Kalman filter based estimation for the cooperation of single ASV and single AUV case. Simulation results verify the performance of the proposed cooperative location method.

Keywords: Underwater Robotics, AI Reasoning Methods for Robotics
Abstract: In underwater diving monitoring, camera sensors capture respiratory bubbles, which are useful for tracking breathing cycles, assessing breath volume, and identifying anomalies. However, segmenting these dynamic and irregular bubbles is challenging. Supervised deep learning offers high accuracy but demands significant training data. To overcome this, we used unsupervised K-means clustering, combining RGB color space with HSV color space images with relative coordinates to extract bubbles from diver images. Relative coordinates address spatial issues in clustering. We applied Contrast Limited Adaptive Histogram Equalization to balance color channels affected by the underwater environment. This validated our ability to segment respiratory bubbles by suppressing seabed reflections using cluster-based ensembles. We evaluate the result of the algorithm with the mean IoU score. And we also compared the impact of pre-processing on the result.

Keywords: Motion Planning and Obstacle Avoidance, Manipulation Planning and Control, Dynamics and Control
Abstract: We design a method to control the motion of a manipulator robot while strictly enforcing collision avoidance in a dynamic obstacle field. We rely on model predictive control while formulating collision avoidance as a hard constraint. We express the constraint as the requirement for a signed distance function to be positive between pairs of strictly convex objects. Among various formulations, we provide a suitable definition for this signed distance and for the analytical derivatives needed by the numerical solver to enforce the constraint. The method is completely implemented on a manipulator "Panda" robot, and the efficient open-source implementation is provided along with the paper. We experimentally demonstrate the efficiency of our approach by performing dynamic tasks in an obstacle field while reacting to non-modeled perturbations.

Keywords: Motion Planning and Obstacle Avoidance, Wheeled Mobile Robots, Range, Sonar, GPS and Inertial Sensing
Abstract: This paper introduces a mobile ground robot designed for autonomous navigation and data collection in agricultural fields, utilizing precise localization through an extended Kalman filter (EKF) that integrates data from GPS, an inertial measurement unit (IMU), and wheel encoders. We propose a novel method based on an artificial electric potential field (AEPF) for reliable and autonomous navigation in these robots. Implemented on a four-wheeled robot interfaced, our experiments showed that AEPF-based navigation processed data more quickly than the traditional Nav2 local path planner. Additionally, the robot reliably collected RGB and depth images while navigating crop rows, highlighting the method's effectiveness and its potential for extensive applications in autonomous crop monitoring. Additionally, a graphical user interface was developed to enable users to define target areas, assign tasks, and monitor the robot's performance in real time.

Keywords: Motion Planning and Obstacle Avoidance, Intelligent Robotic Vehicles, Wheeled Mobile Robots
Abstract: In winter, snowfall has a risk of impairing the autonomous navigation capabilities of mobile robots by obscuring road lane markings and causing sensor noise. This problem complicates the development of safe and efficient snow removal robots. In our research, we propose a novel supervised machine learning method for autonomous robot navigation in snowy environments based on Dual-Type Discriminator Adversarial Reservoir Computing (DDARC) which integrates Reservoir Computing (RC) with Generative Adversarial Networks (GANs). Utilizing depth and thermal imagery as inputs, our method can generate reliable control values for the robot's movement. Experiments in simulated environments have demonstrated that our method significantly improves the autonomous navigation capabilities of mobile robots, even in substantial environmental noise from snowfall.

Keywords: Motion Planning and Obstacle Avoidance, Dynamics and Control, Behavior-Based Systems
Abstract: Successful robotic deployment in challenging environments requires diverse reasoning and reactive control techniques to deal with uncertainty and risk. In this work, we propose a novel behavioral control framework to navigate in such environments with static and dynamic sources of risks. Different agent behaviors can create distinct environment assessments, leading to a variety of reactions while dealing with uncertain and risky situations. We construct a class of perceived risk functions to capture these different behaviors by taking inspiration from behavioral decision making models from Cumulative Prospect Theory (CPT). We then incorporate these perceived risks via local costmaps into a Model Predictive Controller (MPC) framework. Specifically, we use Model Predictive Path Integral (MPPI) Control framework that is capable of handling more general cost functions like our proposed perceived risks. Using this framework, we generate reactive control policies for any given behavioral profile, resulting in a diverse AI for reactive controls. We then illustrate the proposed algorithm in virtual experiments conducted in a high fidelity indoor ROS-Unity environment embedded with static and dynamic sources of risk. We show that our proposed framework is capable of producing a larger range of reactive behaviors leading to a more successful robot deployment.

Keywords: Motion Planning and Obstacle Avoidance, Multi-Robot Systems
Abstract: This paper presents a unified approach to safety-critical, multi-agent motion planning for connected autonomous robotic systems, seamlessly integrating the kinematic, dynamic, and safety constraints of individual agents, while reducing computational expense to ensure real-time applicability. By integrating Voronoi Cells with an adaptive blend of A* and Hybrid A* algorithms, the proposed combinational planner ensures the generation of feasible and executable trajectories, guaranteeing efficient and collision-free navigation of multiple agents in dynamically complex environments. An additional deadlock avoidance strategy is proposed to further enhance the safety layer. We demonstrate the effectiveness and robustness of our approach in terms of efficiency, collision avoidance, and deadlock resolution through simulations in diverse, randomly generated environments. The results show that the proposed method outperforms existing methods in terms of dynamic considerations and obstacle avoidance, making it a practical real-time motion planning approach for connected non-holonomic agents in complex environments.

Keywords: Motion Planning and Obstacle Avoidance, Intelligent Robotic Vehicles, Wheeled Mobile Robots
Abstract: In situations following catastrophic events such as fires and earthquakes, the deployment of autonomous robotic technology plays a crucial role in ensuring the success of exploration and assessment tasks. Despite significant progress in robot navigation, there remains a critical need for autonomous navigation systems adept at executing adaptive motion strategies, particularly in complex environments with obstacles of varying material properties. This study proposes an autonomous navigation system for disaster response scenarios, utilizing advanced deep reinforcement learning techniques. We develop a novel route opening policy that enhances the robot's ability to interact with and navigate around obstacles, thereby improving its route opening capabilities. Our method distinguishes between general navigation and collision-pushing scenarios to identify optimal routes. Experiments demonstrate the system's effectiveness in navigating and opening routes, as well as in locating victims, utilizing both the range and intensity data provided by Light Detection and Ranging (LiDAR) sensors.

Keywords: Actuation and Actuators, Legged Robots, Dynamics and Control
Abstract: The aim of this paper is to apply the water-based Electro-Hydraulic Actuator (EHA) to a robotic leg device. To assess its applicability, kinematic and dynamic analyses have been performed. The EHA is a linear actuator where the power cylinder is driven by hydraulic pressure from a pump. In this study, one robotic leg was considered, and its motion was simulated by exoskeletal parallel actuation, mimicking a human configuration. Water was used as the working fluid for the EHA in this study, as conventional oil hydraulics have drawbacks such as contamination, cost, and fire risk. Therefore, a new type of actuator was tested and verified for a robotic leg. The Euler-Lagrange equation was employed to derive the dynamic equation, using an inverted pendulum as a simplified model for the robotic leg. An experimental set-up was constructed, and pressure measurements of the EHA were compared with calculations from the dynamics, incorporating measured angle and angular acceleration values. The experimental results indicated that the pressure percentage difference averaged ~22%. The graph of the calculated pressure appeared smoother than the measured curve, as the dynamics calculation utilized measurements from the mechanical system. In the future, more rigorous dynamics will be considered, and a feedback controller will be designed for improved robot operation.

Keywords: Simultaneous Localization and Mapping (SLAM), Multisensor Data Fusion, Legged Robots
Abstract: In recent years, various studies on simultaneous localization and mapping~(SLAM) have achieved outstanding performance in terms of accuracy. Accordingly, various SLAM methods can generate a precise 3D map of the surroundings in usual environments and estimate the pose accurately on a pre-built map. However, the localization should be not only accurate but also robust in various situations to achieve a fully autonomous navigation system. Unfortunately, existing localization algorithms are not robust in some cases. For example, the aggressive walking motion of quadruped robots frequently causes a divergence of the odometry algorithm, leading to a catastrophic failure of a fully autonomous system. In this study, we propose a robust localization system leveraging a pose divergence manager, which is applicable to various odometry algorithms. The localization system integrates a pose divergence manager with a 3D LiDAR map-based global localizer that estimates the global pose of the robot on the pre-built 3D LiDAR map. We conducted real-world experiments using a quadruped robot and verified that our proposed method is accurate and robust in indoor and outdoor environments.

Keywords: Legged Robots, AI Reasoning Methods for Robotics, Motion Planning and Obstacle Avoidance
Abstract: The field of legged robots has seen tremendous progress in the last few years. Locomotion trajectories are commonly generated by optimization algorithms in a Model Predictive Control (MPC) loop. To achieve online trajectory optimization, the locomotion community generally makes use of heuristic-based contact planners due to their low computation times and high replanning frequencies. In this work, we propose ContactNet, a fast acyclic contact planner based on a multi-output regression neural network. ContactNet ranks discretized stepping locations, allowing to quickly choose the best feasible solution, even in complex environments. The low computation time, in the order of 1 ms, enables the execution of the contact planner concurrently with a trajectory optimizer in a MPC fashion. In addition, the computational time does not scale up with the configuration of the terrain. We demonstrate the effectiveness of the approach in simulation in different scenarios with the quadruped robot Solo12. To the best knowledge of the authors, this is the first time a contact planner is presented that does not exhibit an increasing computational time on irregular terrains with an increasing number of gaps.

Keywords: Legged Robots, Dynamics and Control
Abstract: Replicating the remarkable athleticism seen in animals has long been a challenge in robotics control. Although Reinforcement Learning (RL) has demonstrated significant progress in dynamic legged locomotion control, the substantial sim-to-real gap often hinders the real-world demonstration of truly dynamic movements. We propose a new framework to mitigate this gap through frequency-domain analysis-based impedance matching between simulated and real robots. Our framework offers a structured guideline for parameter selection and the range for dynamics randomization in simulation, thus facilitating a safe sim-to-real transfer. The learned policy using our framework enabled jumps across distances of 55 cm and heights of 38 cm. The results are, to the best of our knowledge, one of the highest and longest running jumps demonstrated by an RL-based control policy in a real quadruped robot. Note that the achieved jumping height is approximately 85% of that obtained from a state-of-the-art trajectory optimization method, which can be seen as the physical limit for the given robot hardware. In addition, our control policy accomplished stable walking at speeds up to 2 m/s in the forward and backward directions, and 1 m/s in the sideway direction.

Keywords: Simultaneous Localization and Mapping (SLAM), Legged Robots, Computer Vision and Visual Servoing
Abstract: The majority of visual SLAM systems are not robust in dynamic scenarios. The ones that deal with dynamic objects in the scenes usually rely on deep-learning-based methods to detect and filter these objects. However, these methods cannot deal with unknown moving objects. This work presents Panoptic-SLAM, a visual SLAM system robust to dynamic environments, even in the presence of unknown objects. It uses panoptic segmentation to filter dynamic objects from the scene during the state estimation process. Panoptic-SLAM is based on ORB-SLAM3, a state-of-the-art SLAM system for static environments. The implementation was tested using real-world datasets and compared with several state-of-the-art systems from the literature, including DynaSLAM, DS-SLAM, SaD-SLAM, PVO and FusingPanoptic. For example, Panoptic-SLAM is on average four times more accurate than PVO, the most recent panoptic-based approach for visual SLAM. Also, experiments were performed using a quadruped robot with an RGB-D camera to test the applicability of our method in real-world scenarios. The tests were validated by a ground-truth created with a motion capture system.

Keywords: Mechanism and Design, Legged Robots
Abstract: Steel structures can pose potential risks, including damage and collapse due to aging. Safety inspections can help you determine the condition of a structure, identify problems, and develop a maintenance and reinforcement plan. This paper proposes the design of a permanent magnet-based attachment mechanism for Non-Destructive Testing of steel structures and a walking linkage structure for overcoming various obstacles. The applicability of the designed robotic system was verified by selecting the drive part specification and motion verification through dynamics and motion analysis.

Keywords: Haptics, Physical and Cognitive Human-Robot Interaction, Mechanism and Design
Abstract: Designing car doors to provide optimal haptic sensations is essential for enhancing user experience. Consequently, research has focused on simulating these haptic sensations using haptic devices during the design phase. However, the virtual implementation of car door mechanisms presents unique challenges, including significant resistive torque, self-opening and closing behavior, and variable torque profiles during the opening and closing phases. Therefore, analyzing the characteristics of the torque components that constitute the car door torque profile and developing suitable devices for their rendering is necessary. This paper presents the measurement of the car door torque profile and introduces a method that classifies the torque into active and passive components, based on whether they aid or impede rotation. By employing each torque component as an active actuator (motor) and a passive actuator (brake), we will design a suitable hybrid haptic device and can realistically implement the haptic feeling of a car door.

Keywords: Legged Robots, Dynamics and Control
Abstract: In this study, we focused on developing a controller of quadruped robots using a Proximal Policy Optimization (PPO) algorithm to achieve stable and effective locomotion of the robot in a moving ship. In the initial stage, the basic performance of the PPO algorithm was checked by performing simulation-based training in OpenAI Gym, targeting the Unitree Go1 platform. Then, the Gazebo simulator was used to verify the performance of the algorithm in a situation mimicking the inside of the moving ship hull in marine environments. The data generated through parallel learning reduced computational efforts in the learning process. Through the realistic gazebo simulations, we developed a quadruped control algorithm that can be applied to the various ship operation scenarios.

Keywords: Underwater Robotics, Range, Sonar, GPS and Inertial Sensing, Multisensor Data Fusion
Abstract: This paper present a preliminary research on designing and applying a maritime visualization framework for CPOS technology in ocean environments. We developed a virtual underwater environment by mapping the seafloor and then created a virtual reality setting to make this environment visible to people. To assess its effectiveness, we built a test-bed for testing, verification, and analysis.

Keywords: Learning From Humans, Robotic Systems Architectures and Programming, Telerobotics
Abstract: Learning from Demonstration has garnered considerable attention for its ability to teach human-performed skills to robots. The method of providing demonstrations can limit the tasks that can be performed and significantly impact the learning outcomes. Therefore, we propose a demonstration method that is unrestricted by the task, which is the demonstration through the virtual environment. The demonstrations performed in virtual settings allow the operator to fully concentrate on the demonstration itself, extracting only the pure task skills of the operator. The proposed method has demonstrated its efficacy experimentally in the case of tight peg-in-hole tasks through CHAI3D simulator.

Keywords: Wheeled Mobile Robots, Social and Socially Assistive Robotics, Intelligent Robotic Vehicles
Abstract: In this study, we introduce an electric mobility scooter with autonomous driving algorithms, with particular emphasis on caregiver tracking capabilities to address mobility challenges faced by older adults in Korea. Utilizing a LiDAR-based person tracking algorithm, the scooter successfully recognizes and follows the user's guardian. Integration of various sensors ensures functional autonomy and safety. The experimental results demonstrated the effectiveness of the follow-the-guardian algorithm in a controlled environment and highlighted that further validation in crowded spaces is needed for successful commercialization. This solution has the potential to provide a safe and convenient transportation option for our aging population.

Keywords: Simultaneous Localization and Mapping (SLAM), Computer Vision and Visual Servoing
Abstract: Robust spatial perception is one fundamental requirement for a safe and reliable autonomous driving system against adverse weather and lighting conditions, such as rain, fog, haze, snow, and low-light environments. However, the RGB sensor, the most widely used sensor, has a critical vulnerability to lighting and weather conditions. On the other hand, thermal sensors provide clear visibility and robustness under harsh rain, snow, fog, smoke, and light conditions. Therefore, this paper investigates the feasibility of a visual odometry method from a monocular thermal image as a potential rescue for a safe and reliable autonomous driving system. Due to the difficulty of ground-truth data collection in harsh environments, we propose a self-supervised learning method that trains a dense correspondence matching network only from sequential thermal images. After that, the odometry is estimated by RANSAC and the 5-point algorithm from the dense correspondence. Lastly, we discuss the current challenges of thermal visual odometry by comparing it with RGB image results.

Keywords: Aerial and Flying Robots, Mechanism and Design, Biomimetic and Bioinspired Robots
Abstract: This paper underscores the importance of environmental monitoring, and specifically of freshwater ecosystems, which play a critical role in sustaining life and global economy. Despite their importance, insufficient data availability prevents a comprehensive understanding of these ecosystems, thereby impeding informed decision-making concerning their preservation. Aerial-aquatic robots are identified as effective tools for freshwater sensing, offering rapid deployment and avoiding the need of using ships and manned teams. To advance the field of aerial aquatic robots, this paper conducts a comprehensive review of air-water transitions focusing on the water entry strategy of existing prototypes. This analysis also highlights the safety risks associated with each transition and proposes a set of design requirements relating to robots' tasks, mission objectives, and safety measures. To further explore the proposed design requirements, we present a novel robot with VTOL capability, enabling seamless air water transitions.

Keywords: Telerobotics, Learning From Humans, Physical and Cognitive Human-Robot Interaction
Abstract: In this paper, we present a novel motion constraint-based segmentation method to extract the interaction skill primitives in robotic interaction tasks. The proposed approach effectively leverages motion constraints to segment complex tasks into distinct sub-motions. Our experiments, conducted with a robot possessing 3 degrees of freedom in translation, demonstrated successful segmentation performance during interaction tasks. Demonstrations were provided by a remote operator relying solely on force feedback, simulating conditions where the robot operates without visual information. The segmented sub-motions accurately captured the exploration, confirmation, and insertion phases of the task, showcasing the method's efficacy.

Keywords: Modular Robots, Dynamics and Control, Actuation and Actuators
Abstract: This paper presents a preliminary study for applying force control to the Variable Topology Truss (VTT) system. The elements of the VTT system have so far only used position control as the main prismatic actuator (spiral zipper) is very difficult to model including friction, cogging, and other non-linear characteristics. An end-effector load cell provides a means for closed-loop control.

Keywords: Manipulation Planning and Control
Abstract: In this paper, we present our work in progress towards creating a library of motion primitives. This library facilitates easier and more intuitive learning and reusing of robotic skills. Users can teach robots complex skills through Learning from Demonstration, which is automatically segmented into primitives and stored in clusters of similar skills. We propose a novel multimodal segmentation method as well as a novel clustering method. Then, when needed for reuse we transform primitives into new environments using trajectory editing. We present simulated results for our framework with demonstrations taken on real-world robots.

Keywords: Object Recognition, Simultaneous Localization and Mapping (SLAM), Range, Sonar, GPS and Inertial Sensing
Abstract: In recent years, filter-based visual-inertial odometry (VIO) systems have demonstrated remarkable accuracy and robustness in static environments. However, the failure of these methods to account for dynamic objects renders them impractical in real-world scenarios, particularly in indoor environments with high pedestrian presence. To address this limitation, this paper proposes a filter-based state estimation method tailored for dynamic scenes, termed Dynamic MSCKF-3D VIO. The Dynamic MSCKF-3D VIO system comprises three primary components. Firstly, it involves the detection and segmentation of potential dynamic objects within the scene. Subsequently, feature tracking is conducted, excluding features associated with moving objects. Finally, the tracked features are utilized to execute the MSCKF-3D algorithm. To evaluate the efficacy of the proposed system, real-world data collected by the Twinny Deohago 60 is employed. The results demonstrate a notable improvement compared to the performance of the MSCKF-3D VIO system without dynamic object feature handling.

Keywords: Force and Tactile Sensing, Performance Evaluation and Optimization, Modeling, Identification, Calibration
Abstract: In recent years, research focus on force sensing for delicate tasks in robotic applications has grown. This has led to increased demands for force sensors with improved sensitivity, expanded measurement ranges, and more compact designs. The study introduces a compact three-axis force sensor with an enhanced sensitivity amplification mechanism for high sensitivity and capacity measurements. The mechanism, shaped like Wi-Fi, utilizes eccentric structures placed 120^{circ} apart between electrodes for three-axis force detection. The sensor comprises three main components: a top plate, sensing part, and bottom part. The Wi-Fi-shaped electrode structures are situated in the sensing part, where the capacitance information of the electrodes is measured. With high sensitivity (2.72 mN), a wide measurement range (100 N), and compactness (thickness: 6.6 mm), the proposed sensor offers the advantages of low cost and easy manufacture. The sensor's performance was validated through various experiments.

Keywords: Computer Vision and Visual Servoing, Modeling, Identification, Calibration, Wheeled Mobile Robots
Abstract: The 3D structure of indoor environments is essential for comprehensive robot perception and scene analysis. Current floor plan generation methods predominantly rely on 360-degree cameras or point cloud-based techniques, which are unsuitable for devices with limited sensor capabilities, such as robot vacuums that only have monocular cameras and wheel odometry sensors. Addressing this gap, our paper introduces a new method for floor plan creation compatible with these simpler devices. Our approach utilizes partially observed monocular images combined with wheel odometry data to detect and assemble mask images. The process involves segmenting the ceiling area from these images, overlaying the segmented masks, and applying homography transformations guided by the wheel odometry data. The methodology includes the development of two deep learning models: one for segmenting the ceiling from partially visible images, and another for refining the mask for enhanced processing. The segmentation model is designed to improve performance through the use of continuous image sequences, while the refinement model focuses on the hierarchical prediction of vertices, edges, and surfaces of the ceiling. This approach offers a practical solution for floor plan generation in settings where advanced sensory equipment is not available.

Keywords: Foundations of Sensing and Estimation, Range, Sonar, GPS and Inertial Sensing
Abstract: This study proposes a learning-based approach for estimating the orientation of a 6D rigid body object. The framework estimates the orientation in SO(3) by utilizing only the sensor values from Inertial Measurement Units. We validate the performance of the proposed orientation estimator by comparing it with the Invariant extended Kalman filter in the simulation of a 6D rigid body object moving in various poses.

Keywords: Mechanism and Design, Legged Robots, Search and Rescue Robotics
Abstract: This paper proposes design of a magnetic legged robot foot designed to adhere to various ferromagnetic terrains, including uneven, stepped, and curved surfaces. For the robot to navigate these challenging environments, we introduce a novel magnetic foot equipped with multiple miniaturized electropermanent magnet toes (MME) and a differential ankle mechanism (DAM). The MME comprises eight coin-sized electropermanent magnets (EPMs) that can adaptively adhere to surfaces characterized by unevenness, steps, and curves. DAM seamlessly integrates the MME and evenly distributes the force among the eight EPMs using a pulley differential mechanism. Consequently, our study demonstrates that multiple EPMs can be adaptively attached to curved and uneven surfaces.

Keywords: Object Recognition, Computer Vision and Visual Servoing, Robotic Hands
Abstract: Capturing the relationship and position of individual joints of a real human hand is a critical task in the study of AI-Powered Prosthetic Hands. Computer Vision technology aids in acquiring training datasets for this research and understanding the relationship between joints according to actions. In this paper, we research Computer Vision for AI-Powered Prosthetic Hands and conduct experiments applying CV algorithms to various scenarios. Based on the experimental results, we identify current limitations and problems, and propose directions for future research.

Keywords: Haptics, Actuation and Actuators, Soft Robotics
Abstract: In response to the escalating demand for immersive engagements in Virtual Reality (VR) and Augmented Reality (AR), the imperative for advanced haptic feedback systems, particularly for finger interactions, has grown significantly. This study introduces a novel Shape Memory Alloy (SMA)-driven finger haptic device tailored to deliver skin stretch feedback. The SMA actuator configuration, comprising four serpentine structures, enables individual or simultaneous control with diverse actuation patterns. Integrated seamlessly into a 3D printed flexible guide structure and an elastic cover, the proposed device effectively mitigates the challenges of bulkiness and weight associated with traditional designs. Emphasizing its streamlined and lightweight nature, the study delves into the dynamic performances of the SMA-driven finger haptic device on a fingertip. This haptic device with SMA actuators showcased remarkable achievements, realizing a total of eight distinct motions in both tangential and diagonal directions. Furthermore, this system excels in user-finger compatibility, seamlessly integrating into daily activities. Experimental results not only underscore its thin and compact system but also affirm its potential applications in VR and AR environments, marking a significant stride in the evolution of haptic technology.

Keywords: Underwater Robotics, Dynamics and Control, Actuation and Actuators
Abstract: This paper presents a method that detects the faults in the operation of the thruster of an unmanned surface vehicle (USV). The method uses the dynamic model of the USV, and detects the fault in real-time. The dynamic model describes the USV velocity as a function of the thruster rotation speed. The proposed method compares the measured speed of the USV and the speed estimated using the dynamic model. The proposed method is verified by offshore experiments using a USV.

Keywords: Range, Sonar, GPS and Inertial Sensing, Simultaneous Localization and Mapping (SLAM), Multisensor Data Fusion
Abstract: This study is about positioning an AGV in the open air through LiDAR and IMU fusion. In this study, we estimate the position of an AGV through a proposed two-stage matching using LiDAR and IMU on the AGV

Keywords: Multi-Robot Systems, Simultaneous Localization and Mapping (SLAM), AI Reasoning Methods for Robotics
Abstract: We propose a Deep Reinforcement Learning (DRL) based autonomous exploration algorithm designed for distributed multi-robot teams, which takes into account map and localization uncertainties of range-sensing mobile robots. An exploration graph, incorporating current SLAM pose estimation and potential future actions, is introduced to characterize the robot state at each iteration. A Graph Neural Network (GNN) is integrated into DRL agents to enhance their understanding of the topology within the exploration graph. The results of our experiments demonstrate the algorithm’s capacity to strike a balance between ensuring map uncertainty and achieving efficient exploration with a multi-robot team.

Keywords: Simultaneous Localization and Mapping (SLAM), Manipulation Planning and Control
Abstract: Mobile manipulators are widely used and are expected to see even broader applications due to the increasing demand for automation. These mobile manipulators perceive their surroundings through Simultaneous Localization and Mapping (SLAM). However, the integration of mobile robots and manipulators for unified SLAM has not been extensively studied. The visual SLAM of a mobile robot with a statically linked camera required rotation to change the camera orientation unrelated to the robot's position movement due to its limited field of view (FOV), which resulted in an inefficient driving path decision. "In this work, we address the previously mentioned problem by tracking unexplored areas with a sensor-equipped manipulator. The manipulator autonomously adjusts the camera orientation without rotating the base and scans unexplored areas identified through point cloud density. Areas where scanning is impossible due to obstacles always maintain a low point cloud density, which can cause the manipulator to become trapped. These shadowed areas are identified through vertical point cloud density and excluded from the target angle. The experiment was conducted through gazebo simulation. The experiment showed that manipulator assistance SLAM is possible with a more efficient driving path than the statically linked camera only method.

Keywords: Underwater Robotics, Object Recognition
Abstract: Various methods have been proposed to develop smart aquaculture, but fish detection models designed for automated fish monitoring suffer from problems such as illumination changes, low contrast, high noise, fish deformation, frequent occlusion, and dynamic background. In this study, a dataset containing images of fish backs was developed to solve the problem that models trained with existing fish dataset models cannot detect the backs of fish. These limitations arise when monitoring fish in cage farms using ROVs. By evaluating the results of training the fish detection model using the proposed dataset, an mAP of 96.7% was achieved for the test dataset. Additionally, the model successfully detected fish even when the fish were not visible due to lack of contrast with the net.

Keywords: Aerial and Flying Robots, Dynamics and Control
Abstract: Energy expenditure for quadrotor control has a likelihood of being costly given parameter-dependent controllers that are less than optimal. The cost can grow proportionally when applied to multiple quadrotors for tracking and collaborative navigation tasks. This research aims to establish a basic approach to tuning PID (Proportional-Integral-Derivative) parameters for a simulated quadrotor drone. Implementing a PID controller for autonomy provides a straightforward method for correcting robotic movement based on its current state. However, applying a PID system to a flight controller poses challenges with an inherently under-actuated system, which includes the likelihood of large overshoots and lengthy adjustment times. To address this, we propose utilizing Particle Swarm Optimization (PSO) for tuning PID parameters in a simulated quadrotor using Webots. The PSO algorithm is employed to find optimal PID values for thrust, yaw, and translational PIDs for x- and y-positions by identifying converging values across randomly created particles. The results demonstrate converging properties for particles that achieve minimal fitness scores, particularly in reducing overshoot. The results indicate that the optimized PID controller outperforms the default PID controller without optimization. We also present a proposed application for transferring our PSO implementation to find optimal gains for a physical quadrotor, including carrying found parameters into multiple physical robots for collaborative navigation and tracking.

Keywords: Rehabilitation and Healthcare Robotics
Abstract: In this study, we quantitatively compared muscle activity during manual care and robot-assisted care when a caregiver assisted a caregiver in transferring from a bed to a wheelchair. The muscle activity measurement and comparison system can be used to quantitatively evaluate workload reduction by applying various manual care systems and robot-based care systems to elderly care tasks.

Keywords: Rehabilitation and Healthcare Robotics
Abstract: Most of care works are known to be major harmful risk factors for musculoskeletal disorders. Transferring care involves a high proportion of physical labor and causes physical strain to caregivers. To evaluate transfer care using ergonomic tool, a work analysis model based on Job Strain Index was developed.

Keywords: Multisensor Data Fusion, Multi-Robot Systems, Object Recognition
Abstract: This paper presents an innovative parking robot system designed to alleviate urban parking congestion. The system incorporates advanced recognition and localization methodologies crucial to its parking function. Empirical operations conducted in a real parking lot environment validate the effectiveness of our approach, particularly in ensuring the smooth operation of parking robot systems. Using the proposed method can be offer a promising solution to the persistent issue of urban parking congestion.

Keywords: Multisensor Data Fusion, Legged Robots, Robotics in Hazardous Applications
Abstract: In this study, we proposed a method to remove fog and generate an elevation map to ensure effective terrain recognition even in dense fog or smoky environments, and verified the proposed method through experiments.

Keywords: Foundations of Sensing and Estimation, Contact: Modeling, Sensing and Control , Actuation and Actuators
Abstract: Recently, linear motion (LM) guides have been used in high-tech facilities or robots that require high precision or high-speed operations. In the event of a sudden failure of an LM guide, an automated factory can experience production delays. The LM guide includes high-speed motion and complex motion, which can reduce the diagnostic accuracy of the existing fault diagnosis methodology based on single input of torque ripple. To overcome this difficulty, this paper proposes a method to increase diagnostic accuracy by weighting motion information to concatenate failure information. The proposed method concatenates three images: the noise-removed speed signal containing motion information, the speed spectrogram, and the torque spectrogram, which contain fault information. To address this issue, the Multi-Modal 1D-CNN methodology is utilized to integrate and extract features from heterogeneous signals for fault detection. Validation was conducted with data concerning LM guide failures in actual industrial settings. The proposed method was confirmed to enhance diagnostic accuracy.

Keywords: Wheeled Mobile Robots, Robotics in Hazardous Applications, Intelligent Robotic Vehicles
Abstract: This paper introduces a robot that enhances worker safety, increases efficiency, and improves the quality of painting work in high-height painting operations. Sophisticated navigation and control technology has been developed to automatically complete these painting tasks. Previous mural robots had limited movement capabilities and were mainly suitable for low working heights. The key to the efficiency of high wall painting robots lies in the implementation of map-based task organization and control technology. This paper aims to address specific challenges associated with high mural operations by developing innovative map-based task organization methods and control techniques, with the goals of improving safety, increasing efficiency, and improving overall performance.

Keywords: Manipulation Planning and Control, Dynamics and Control
Abstract: The recent promises of Model Predictive Control in robotics have motivated the development of tailored second-order methods to solve optimal control problems efficiently. While those methods benefit from strong convergence properties, tailored efficient implementations are challenging to derive. In this work, we study the potential effectiveness of first-order methods and show on a torque controlled manipulator that they can equal the performances of second-order methods.

Keywords: Behavior-Based Systems, Robotic Systems Architectures and Programming, Motion Planning and Obstacle Avoidance
Abstract: Behavior Trees (BTs) are an effective method for organizing autonomous agent tasks in robotics and AI. However, conventional BTs' rigid design limits flexibility and exhibits limitations in designing intelligent agents with adaptable behaviors. This paper proposes a Reinforcement Learning-based Modification Structure of Behavior Tree (RLMS-BT), which is a modified design of a conventional BT structure aimed at applying RL algorithms. In this design, Sub-nodes are transformed into RL nodes through hierarchical restructuring, which can enhance overall performance by optimizing task sequence. Furthermore, a task planning framework is presented, which enables the optimization of the modified BT structure using various RL algorithms. In the simulation of task planning within an obstacle environment, the RLMS-BT structure exhibits superior adaptability, surpassing conventional BT from a time-cost perspective. This demonstrates its potential for implementing complex behavioral systems in autonomous agents.

Keywords: Robotic Systems Architectures and Programming
Abstract: In order to achieve intelligent manufacturing system capable of high-mix low-volume production, it is essential to integrate multiple systems including sensors, robot manipulators, mobile robots and other systems. Open Platform Communications Unified Architecture (OPC UA) is one of the most promising methods for such integration. In this study, an OPC UA-based control architecture for robot manipulators is proposed. The OPC UA client acts as the main process controller while each device, such as robot manipulators or sensors, contains OPC UA servers. This configuration allows the main controller to effectively control multiple devices by sending requests. Furthermore, this configuration enables easy modularization of each device, which is particularly helpful for high-mix low-volume manufacturing. To illustrate the feasibility of the proposed control architecture, the proposed architecture is applied to a system consists of a robot manipulator and a camera.

Keywords: Aerial and Flying Robots
Abstract: Unmanned Aerial Vehicles (UAVs), commonly called drones, are a promising and revolutionary technology with the potential to impact industry and the living environment. We present an integrated and intuitive human-centered collaborative management system for multiple drones based on oneM2M IoT platform. The system consists of CGCS (Central Ground Control System), FGCS (Field Ground Control System), and XR (eXtended Reality) platform to support the management and operation of multiple drones in the field. By adapting user-centered factors such as voice controlling and hand gesture interfaces, we expect to adapt human-centered factors in the UAV management environment to achieve intuitive aerial operations in the future.

Keywords: Aerial and Flying Robots, Multi-Robot Systems
Abstract: This paper is about the development of a ground control system to support flight path setting in terms of communication connection stability for unmanned aerial vehicles. The proposed ground control system aims to build an environment that allows to select the flight path by utilizing communication quality data obtained through actual measurements. We describe the structure of an global IoT standard-based server platform for storing and managing measured communication quality data. As a result, we propose a ground control system that links actual measurement data with the system and allows the operator to set the route in terms of communication when setting the flight path.

Keywords: AI Reasoning Methods for Robotics, Object Recognition
Abstract: This study proposes a novel method for detecting Out-Of-Distribution (OOD) data in image classification tasks, aimed at improving the trustworthiness of robot vision tasks e.g. surveillance or safety inspection. The proposed approach utilizes class-wise confidence thresholds, determined analytically through a grid search, to effectively identify data that falls outside the model's training distribution. Experimental results demonstrate that the proposed method achieves competitive performance across various OOD detection scenarios, with significant improvements in Area Under the Receiver Operating Characteristic (AUROC) curve and False Positive Rate at 95% True Positive Rate (FPR95) compared to existing research. By accurately detecting model uncertainty, this study contributes to expanding the scope of indoor safety check robots, enhancing system reliability, safety, and efficiency. The proposed method's feasibility in improving the trustworthiness of robot vision intelligence highlights its potential for real-world applications.

Keywords: Multi-Robot Systems
Abstract: Robot swarms exhibit optimal efficiency when operating with a small number of robots. However, as the size of a robot swarm increases, the collective performance on tasks such as foraging tends to decline due to heightened competition rather than cooperation among individual robots. A primary obstacle encountered during swarm navigation within search arenas is the occurrence of collisions among robots. Our objective is to enhance the scalability of robot swarms, enabling them to efficiently execute foraging tasks even with a large number of robots. To achieve this goal, we introduce a novel trajectory analysis method designed to predict potential collisions for each robot. By collecting and analyzing 2-dimensional trajectory data from a simulated environment featuring 16 robots engaged in foraging tasks, we categorized the data into congested and normal states, subsequently evaluating the accuracy of this classification. Utilizing the predictions generated by our method, robots can adjust their behaviors proactively to mitigate the likelihood of collisions, thereby enhancing foraging performance and overall swarm scalability. Furthermore, our preliminary results underscore the potential of trajectory analysis to identify anomalies within robot swarms, offering promising prospects for future research in anomaly detection methodologies.

Keywords: Force and Tactile Sensing
Abstract: Tactile sensors for wearable applications, e.g., electronic skins, robotic manipulators, etc., often require multi-modal sensing ability that enables combinations of pressure, shear, and bending stress/strain measurements. Carbon nanomaterials (CNMs) and CNM-derived polymer composites offer a high degree of freedom for sensor design and performance tunability. This paper presents recent studies on CNM-based, multi-modal tactile sensors, namely: (1) pressure-shear multi-modal sensor based on carbon nanocomposite pillar arrays; (2) multi-layered, micro-patterned nanocomposite pressure sensor with modulus gradient for wide-range sensitivity; and (3) bio-inspired, carbon nanotube(CNT)-coated hexagonal micro-columnar arrays with interlocking structure for pressure, shear and bending sensing.

Keywords: Physical and Cognitive Human-Robot Interaction, Behavior-Based Systems
Abstract: In the Industry 5.0 framework, due to the close collaboration between humans and robots, providing a safe environment and balance workload becomes an essential requirement. In this context, evaluating the trustworthiness of robots from a human-centric perspective is essential as trust impacts the interaction in human-robot collaborations. Numerous researchers in the literature have delved into physiological responses as indicators of user trust in robots. In this research endeavor, multiple machine learning models were employed, leveraging skin conductance response (SCR) to classify the trust level of the human operator. A chemical industry scenario was developed, where a collaborative robot supported a human operator by handing over a beaker used for the pouring of chemicals. The machine learning models achieved a moderate accuracy rate of 68.99% and AUC of 0.73 for the handover task. Nonetheless, this study underscores the importance of sensor fusion techniques to improve the accuracy of trust assessment within the context of human-robot collaborations.