Keywords: Rescue robotics, Safety standards for robots and systems, Performance evaluation and benchmarking
Abstract: While rescue robots are becoming more established as part of disaster response, they are typically teleoperated in actual disasters. (Autonomous) assistance functions can improve performance, extend functionality and reduce operator overload.

It is necessary to understand relevant requirements to ensure that developed capabilities apply to real-world needs. Previous analyses focused on general aspects of rescue robots, leaving a gap in understanding requirements for (autonomous) assistance functions. We address this gap and provide a detailed, evidence-driven analysis of application requirements and research challenges for (autonomous) assistance functions for rescue robots in reconnaissance missions. We base our analysis on a comprehensive model for technology acceptance and consider reports of past deployments, related analyses, our own experience from deploying robots, and insights from workshops with first responders. We define relevant aspects of an integrated function capability and analyze general and specific requirements for assistance functions and autonomy. We relate our results with current assistance functions and identify several research challenges. A key insight is the need for an increased research focus on novel approaches combining the complementary capabilities of human operators and robotic assistance functions.

Keywords: Human-robot interaction, Mobile Robotics, Rescue robotics
Abstract: The remote human operator's user interface (UI) is an important link to make the robot an efficient extension of the operator's perception and action. In rescue applications, several studies have investigated the design of operator interfaces based on observations during major robotics competitions or field deployments. Based on this research, guidelines for good interface design were empirically identified. The investigations on the UIs of teams participating in competitions are often based on external observations during UI application, which may miss some relevant requirements for UI flexibility. In this work, we present an open-source and flexibly configurable user interface based on established guidelines and its exemplary use for wheeled, tracked, and walking robots. We explain the design decisions and cover the insights we have gained during its highly successful applications in multiple robotics competitions and evaluations. The presented UI can also be adapted for other robots with little effort and is available as open source.

Keywords: Rescue robotics, Localization, Mapping, and Navigation, Robotics and automation for safety and security
Abstract: Disasters in tunnels and underground infrastructures pose significant challenges to responders due to the confined and hazardous nature of these environments. Robots are seen as valuable tools in assisting first responders in such scenarios by providing situational awareness and allowing to maintain a safe distance to disaster sites in the initial reconnaissance. However, using robots with automated skills in confined environments like tunnels comes with its own set of challenges, mainly regarding perception, localization, and navigation. This study presents a comprehensive support system for challenging underground operations, utilizing a robot equipped with situation awareness sensors and automated navigation skills addressing the GNSS-denied and featureless environment.The developed system also includes a communication framework as well as a visualization tool that is able to provide a detailed 3D environment map, to monitor the temperature, and to detect gas and fire sources, which enhances safety and efficiency during subterranean activities. The system was evaluated through simulated rescue scenarios in a highway tunnel where responder used the system to find victims and to localize a gas leak after an accident in the tunnel.

Keywords: Aerial robotics, Human-robot interaction, Rescue robotics
Abstract: This paper offers a brief overview of the history of small-scale aerial platform teleoperation, its current state, and potential alternatives to traditional teleoperation techniques. We evaluate the pros and cons of augmented reality, virtual reality, and mixed reality technologies, based on our own and related research, as well as observations from a live search and rescue exercise. Additionally, we emphasize the potential benefits and challenges that new teleoperation systems could bring to search and rescue operations.

Keywords: Human-robot interaction, Rescue robotics
Abstract: There is a need for an interface for the teleoperation of rescue robots that can be used with less training and reduces the mental burden. In recent years, numerous teleoperation interfaces for robots have been developed using commercially available VR headsets and VR controllers. In this study, to investigate the usefulness of VR interfaces in tasks involving both movement and manipulation, a VR interface capable of both movement and manipulation was proposed and validated by users in a door-opening task. The validation results confirm that operations using the VR interface contribute to a shorter working time with the manipulator compared to operations using the gamepad. Additionally, mental demand, temporal demand, and effort tended to be reduced in manipulator operation using the VR interface in the NASA-TLX mental workload assessment. However, the gamepad was also identified as a suitable input device for driving operations, and combining both operations or using them differently depending on the situation may reduce the operator's workload.

Keywords: Rescue robotics, Multi-robot systems, Human-robot interaction
Abstract: This research explores evacuee responses to robot-guided evacuation. During the conduct of a human-robot experiment we trigger a smoke alarm on unsuspecting subjects. The robot offers to led the subjects to a distant, less familiar exit. We capture the subjects decision to follow, evacuation characteristics, and impressions of the robot over individual and group conditions and two different approaches to designing evacuation robots. We ran a total of 112 subjects across various conditions and find that, overall, 95.28% of participants followed the robot's guidance. We further show that, although the amount of time evacuees wait prior to evacuating does not differ between the conditions, the total evacuation time is less when a multi-robot approach is taken. We also present a series of preliminary results on related exploratory studies.

Keywords: Rescue robotics, Aerial robotics, Perception for navigation, hazard detection, and victim identification
Abstract: This paper presents a convolutional neural network (CNN) model for event detection from Unmanned Aerial Vehicles (UAV) in disaster environments. The model leverages the YOLOv5 network, specifically adapted for aerial images and optimized for detecting Search and Rescue (SAR) related classes for disaster area recognition. These SAR-related classes are person, vehicle, debris, fire, smoke, and flooded areas. Among these, the latter four classes lead to unique challenges due to their lack of discernible edges and/or shapes in aerial imagery, making their accurate detection and performance evaluation metrics particularly intricate. The methodology for the model training involves the adaptation of the pre-trained model for aerial images and its subsequent optimization for SAR scenarios. These stages have been conducted using public datasets, with the required image labeling in the case of SAR-related classes. An analysis of the obtained results demonstrates the model's performance while discussing the intricacies related to complex-shape classes. The model and the SAR datasets are publicly available.

Keywords: Sensing and sensor fusion, Perception for navigation, hazard detection, and victim identification, Field robotics
Abstract: Obstacle detection is crucial for ensuring the safety of autonomous robots and their surroundings in unstructured outdoor environments. Objects with minimal lateral dimensions can pose risks to the robot or serve as important elements in the infrastructure it operates in. Detecting these structures becomes particularly challenging when tall vegetation is present. Distinguishing between soft, traversable objects, such as tufts of grass, and potentially lethal solid obstacles is paramount to a robot's ability to operate.

This paper presents a novel approach that focuses on point cloud generation and vegetation identification to facilitate the safe navigation of autonomous outdoor robots. Our approach uses a single multispectral stereo camera system that employs a novel stereo matching strategy based on binary descriptors for spectrally non-identical image pairs.

Keywords: Rescue robotics, Artificial Intelligence, Autonomous search and rescue
Abstract: This paper evaluates four semantic segmentation models in Search-and-Rescue (SAR) scenarios obtained from ground vehicles. Two base models are used (U-Net and PSPNet) to compare different approaches to semantic segmentation, such as skip connections between encoder and decoder stages and using a pooling pyramid module. The best base model is modified by including two attention mechanisms to analyze their performance and computational cost. We conduct a quantitative and qualitative evaluation using our SAR dataset defining eleven classes in disaster scenarios. The results demonstrate that the attention mechanisms increase model performance while minimally affecting the computation time.

Keywords: Sensing and sensor fusion, Performance evaluation and benchmarking, Artificial Intelligence
Abstract: The advancement of computer vision and machine learning has made datasets crucial for further research and applications. However, the creation and development of robots with advanced recognition capabilities are hindered by the lack of appropriate datasets. Existing image or video processing datasets are unable to depict observations from a moving robot accurately, and they do not contain the kinematics information necessary for robotic tasks. Synthetic data, on the other hand, are cost-effective to create and offer greater flexibility for adapting to various applications. Hence, they are widely utilized in both research and industry. In this paper, we propose the dataset HabitatDyn, which contains both synthetic RGB videos, semantic labels, depth information, as well as kinetics information. HabitatDyn was created from the perspective of a mobile robot with a moving camera and contains 30 scenes featuring six different types of moving objects with varying velocities. To demonstrate the usability of our dataset, two existing algorithms are used for evaluation and an approach to estimate the distance between the object and camera is implemented based on these segmentation methods and evaluated through the dataset. With the availability of this dataset, we aspire to foster further advancements in the field of mobile robotics, leading to more capable and intelligent robots that can navigate and interact with their environments more effectively. The code is publicly available at https://github.com/ignc-research/HabitatDyn.

Keywords: Rescue robotics, Autonomous search and rescue, Robotics simulation
Abstract: Robots that perform rescue and search operations at disaster sites are called rescue robots. These robots should be able to be navigated operate autonomously because remote control is difficult. The objective of this research is to enable rescue robots to be navigated autonomously on international standard rough terrain. To achieve this objective, we built a learning environment in a simulator using Unity, a physics engine, and conducted deep reinforcement learning using machine learning framework of Unity, ML-Agents. Comparative verification with remote control showed that autonomous navigation was superior to remote control in both time and success rate. The reason for this result was found to be the difference in the motion of the robot.

Keywords: Aerial robotics, Autonomous search and rescue, Perception for navigation, hazard detection, and victim identification
Abstract: This paper describes gaps in acquisition of drone imagery that impair the use with computer vision/machine learning (CV/ML) models and makes five recommendations to maximize image suitability for CV/ML post-processing. It describes a notional work process for the use of drones in wilderness search and rescue incidents. The large volume of data from the wide area search phase offers the greatest opportunity for CV/ML techniques because of the large number of images that would otherwise have to be manually inspected. The 2023 Wu-Murad search in Japan, one of the largest missing person searches conducted in that area, serves as a case study. Although drone teams conducting wide area searches may not know in advance if the data they collect is going to be used for CV/ML post-processing, there are data collection procedures that can improve the search in general with automated collection software. If the drone teams do expect to use CV/ML, then they can exploit knowledge about the model to further optimize flights.