Keywords: Software, Middleware and Programming Environments, Integration Platform, Software Platform
Abstract: In recent years, the problem of the aging population and declining birth rate in Japan inspired research and development of service robots. Although we can provide a variety of services through coordination between the robots, it is difficult to synchronize the robots running on different software. In this paper, we propose a system concept and the coordination of different robot middlewares and demonstrate the system concept by implementing an actual robot system. More specifically, we develop a guidance system for a mobile robot using two middlewares OpenRTM and ROS with coordinating system eSEAT to convert the data types used in both systems. Both OpenRTM and ROS are robot technology middleware that supports asynchronous communication for robots, such as sensor data communication and equipment control. Each middleware has data that has the same meaning, such as robot velocity information and odometry information. By converting the message type, it is possible to exchange only the data that we want to communicate or share between different systems efficiently. We evaluated evaluate (1) the delay which is a concern when sending the position on the map from ROS to OpenRTM, and (2) whether the robot was able to reach the specified position in executing the task experimentally. Additionally, we will verify the efficiency of the system by combining it with Node-RED (superordinate system for scenario generation of robot) developed in our previous research in the future.

Keywords: Entertainment Systems, Human-Robot/System Interaction, Formal Methods in System Integration
Abstract: Mapping spoken text to gestures is an important research topic for robots with conversation capabilities. According to studies on human co-speech gestures, a reasonable solution for mapping is using a concept-based approach in which a text is first mapped to a semantic cluster (i.e., a concept) containing texts with similar meanings. Subsequently, each concept is mapped to a predefined gesture. By using a concept-based approach, this paper discusses the practical issue of obtaining concepts for a unique vocabulary personalized for a conversational agent. Using Microsoft Rinna as an agent, we qualitatively compare concepts obtained automatically through a natural language processing (NLP) approach to those obtained manually through a sociological approach. We then identify three limitations of the NLP approach: at the semantic level with emojis and symbols; at the semantic level with slang, new words, and buzzwords; and at the pragmatic level. We attribute these limitations to the personalized vocabulary of Rinna. A follow-up experiment demonstrates that robot gestures selected using a concept-based approach leave a better impression than randomly selected gestures for the Rinna vocabulary, suggesting the usefulness of a concept-based gesture generation system for personalized vocabularies. This study provides insights into the development of gesture generation systems for conversational agents with personalized vocabularies.

Keywords: Surveillance Systems, Welfare systems
Abstract: Handwashing and hand disinfection are essential for the prevention of infections. Both caregivers and people requiring long-term care must wash their hands appropriately. However, caregivers may make mistakes or forget handwashing procedures owing to unexpected events. Hence, it is necessary to implement regular efforts to ensure thorough hand hygiene. In addition, it is assumed that people requiring long-term care may not wash their hands properly because of the cognitive function decline. Therefore, a system that can monitor handwashing daily, check if it is done correctly, and promote improvements in handwashing behavior, when necessary; would help prevent and reduce the risk of diseases. The purpose of this study is to propose and develop a non-contact handwashing monitoring system and to confirm the feasibility of the proposed system through basic experiments. We developed a non-contact whole-body 3D skeletal model measurement system using an RGB-D camera. Moreover, based on the acquired 3D skeletal information, we developed an algorithm to recognize handwashing actions with respect to those recommended by the Japanese Ministry of Health, Labor, and Welfare. The basic experiments as a proof of concept were conducted with a single participant. We confirmed that the handwashing action recognition accuracy for the test data was 91.1%. We confirmed that the recognition result of a series of handwashing by the proposed system shifted in a stepwise sequence. In addition, the proposed system detected missing actions from a series of handwashing, including intentional errors. In conclusion, by setting a time threshold for each action, the proposed system was expected to detect the recommended handwashing actions.

Keywords: Surveillance Systems, Machine Learning, Vision Systems
Abstract: In this paper, we propose a scene-independent robust unsupervised video anomaly detection method based on future frame prediction as a breakthrough and better video anomaly detection technique. Most conventional methods evaluate and develop a static camera and dashcam approach as independent tasks, and no method has been proposed that is independent of the capture conditions. The proposed method introduces a frame-wide future prediction–based spatio-temporal adversarial networks that can handle arbitrary series lengths to cope with various scenes. The noise in the prediction error caused by constant background changes is improved by weighting the regions of interest for the discriminator of the generative adversarial networks (GANs). This framework can be applied to all cases regardless of the scene environment. Experiments on public datasets of general traffic scenes and crowded scenes confirm the superiority of the proposed method over current state-of-the-art methods.

Keywords: System Simulation, Micro/Nano Systems
Abstract: ALE-1 is a microsatellite that needs to change its initial launch orbital altitude before its main mission can start. A passive de-orbiter system is used for this maneuver. This paper summarizes the history of this orbital maneuver sub-mission and the results of ALE-1’s de-orbiting over the months since its launch. Furthermore, these results are compared to a simulation and the end date of this orbital maneuver is estimated.

Keywords: Micro/Nano Systems, Vision Systems, Systems for Field Applications
Abstract: Gossamer structures have been playing important roles in space application, such as solar sail for deep space explorations, and also as drag-sail for de-orbiting of satellites into Earth atmosphere catching the atmospheric drag to prevent space debris. There are several methods suggested to securely deploy such gossamer structures in space, but those technologies are still under development especially for the realization of large scale structures in space. One of the challenge in this development roadmap is the measurement method of the 3-D distributed shape of those gossamer structures in space. This paper summarizes orbital demonstration results of shape estimation method based on a Time-of-Flight camera system mounted on a micro-satellite. The result of space demonstration provided valuable lessons learned in structural design of such a camera system as well as noise elimination signal processing. The suggested system provides a new type of shape estimation method for a wide range of space system applications.

Keywords: Modeling and Simulating Humans, Medical Systems, Systems for Service/Assistive Applications
Abstract: This paper presents analysis of sleep electroencephalogram (EEG) based on a scale mixture model. In the scale mixture model, the EEG signal is assumed to be a random variable that follows a infinite mixture of Gaussian distributions with the same mean and different covariance matrices, thereby allowing the representation of the stochastic fluctuation of the EEG amplitude. First, a sleep EEG analysis method was proposed by combining the scale mixture model with band-pass filters and a sliding window, thereby allowing the time-series estimation of the model parameters in a specific frequency band. Then, in experiments, we analyzed the EEG signals during rapid eye movement (REM) sleep and sleep stage II using the proposed analysis method. The results showed that the proposed method captures the characteristic changes in the amplitude distribution of the EEG depending on the sleep stage. Furthermore, we focused on sleep spindles in sleep stage II, which are distinctive waves in sleep EEG, and verified their detectability by machine learning using the features defined by the proposed method as input.

Keywords: Modeling and Simulating Humans, Rehabilitation Systems, Welfare systems
Abstract: The purpose of this study is to verify the effectiveness of the balance map analysis in assessing the risk of fall while walking by analyzing the steady-state gait, the perturbed walking and the subsequent steps. The balance map visualizes three states as the region of stable touch down, the region of the risk of fall forward, and the region of the risk of fall backward, respectively. These regions are determined by ballistic trajectories of a linear compass gait model. The whole body movement during walking was measured by a motion capture. The state of contact between the foot and the ground was measured by foot sensors. The rope which equips with an auto-reel mechanism was attached on the ankle joints. The disturbance to prevent the swing leg from moving forward was induced by manual lock of released rope. The tension and the timing of disturbance were measured by a load cell. The results showed that the measured trajectories during the steady-state walking were within the region of stable touch down. When the disturbance was applied, the trajectory approached and went into the region of fall risk. The perturbed trajectory gradually converged to the steady-state gait by recovery steps. We also confirmed that the margin from the region of fall forward risk was after perturbation onset, and gradually increased by recovery steps. We conclude that the balance map analysis is effective for assessment of balance during walking.

Keywords: Modeling and Simulating Humans, Intelligent Transportation Systems, Human Factors and Human-in-the-Loop
Abstract: With the development of autonomous driving technology, verifying the system by a simulator is an essential process, and pedestrians' behavior is a particularly important factor in the process. To express the pedestrians' behavior, this paper proposes the model of pedestrians' crossing decisions for a non-signalized intersection, where pedestrian fatalities are the most frequent environment in Japan. This crossing decision, whether to go through a crosswalk or to wait for a vehicle, is an interactive factor that is influenced by other traffic participants. For this reason, we developed a multi-player interactive simulator, in which vehicles and pedestrians can be controlled by the human. This simulator makes it easy to measure the crossing decision data that takes into account the differences in individual characteristics. Besides, the decision model was constructed for this simulation data, and multi-class logistic regression was used as the model to express human decision ambiguity. The individual decision characteristics were analyzed by the logistic regression model in two- and three-person interactive situations. Finally, to evaluate the performance and decision characteristics of the model, simulation experiments were conducted for 15 groups.

Keywords: Virtual Reality and Interfaces, Human Interface, Entertainment Systems
Abstract: Electrical muscle stimulation, which has been often used in the medical field, such as rehabilitation, has also come to be used as a haptic feedback technique. However, electrical muscle stimulation has a unique sense of electricity and pain. We aim to suppress this undesired sensation by changing the electrical muscle stimulation waveform. This study investigates the change in sensation due to the electrical muscle stimulation waveform change and describes the results in detail.

Keywords: Multi-Modal Perception, Modeling and Simulating Humans, Environment Monitoring and Management
Abstract: We propose a system using a multimodal probabilistic approach to solve the human action recognition challenge. This is achieved by extracting the human pose from an ongoing activity from a single camera. This pose is used to capture additional body information using generalized features such as location, time, distances, and angles. A probabilistic model, multimodal latent Dirichlet allocation (MLDA), which uses this multimodal information, is then used to recognize actions through topic modeling. We also investigate the influence of each modality and their combinations to recognize human actions from multimodal observations. The experiments show that the proposed generalized features captured significant information that enabled the classification of various daily activities without requiring prior labeled data.

Keywords: Rehabilitation Systems, Systems for Service/Assistive Applications, Virtual Reality and Interfaces
Abstract: Stroke causes physical dysfunction due to brain damage, leading to aftereffects, such as hemiplegia. Until now, a lower limb rehabilitation system was developed by us to help patients to restore leg function owing to brain plasticity in neurorehabilitation. Visual Feedback (FB) in the system, together with a mechanical assistant for lower limb, linked to the motor execution is presented to form a FB learning model matching the motor execution and its imagination. This feasibility study on the healthy subjects aims to apply the knowledge to rehabilitation program in the future. Meanwhile, the feasibility study aims to investigate the effect of linked visual FB on the imagination of motor execution and furthermore, to clarify an effective design of visual FB conditions focusing on the similarity between visual FB and motor execution. The experiments under three different conditions were conducted using the system: (i) no visual FB, (ii) with visual FB and motion in the FB different from motor execution, (iii) with visual FB and motion in the FB closer to motor execution. Electroencephalogram (EEG) was performed as an indicator to evaluate the imagination of motor execution, being the reduction of EEG values during motion recall - Event Related Desynchronization (ERD) mainly used. According to the results, by comparing (i) and (ii), significantly greater ERD in (ii) verified that visual FB helps acquire the imagination of motor execution (p = 0.028). Moreover, between (ii) and (iii), ERD in β-band were significantly greater in (iii) (p = 0.028), but no significant difference showed for ERD in α-band (p = 0.600) that further confirmation might be needed. In conclusion, the linkage between visual FB and motor execution was verified in this study, which helps to acquire the imagination and may be related with brain recovery.

Keywords: Multi-Robot Systems, Building Automation
Abstract: In this paper, we describe a mutual position estimation method for multiple omni-directional mobile robots using only internal field sensors. Today's material transportation requires multiple manpower and is a repetitive and frequent task. In order to solve this problem, a plurality of small robots are used for transportation. By moving each robot restrained by materials appropriately by hand, mutual position estimation between the robots can be performed, and it is possible to move by operating the controller. In this paper, we experiment with two approaches to estimate the mutual position and center of rotation between robots based on a single equation.

Keywords: Multi-Robot Systems, Path Planning for Multiple Mobile Robots or Agents
Abstract: In this paper, we present a coordinated omnidirectional transporting system of a large object by multiple mobile robots. Conventional differential drive wheeled mobile robots are used for the system, but the connection point with the transported object is installed at a position separated from the midpoint of the two drive wheels in the traveling direction. The mobile robot is driven to control the movement of that point. The control method for the differential drive wheeled mobile robot is a kind of active caster control. From this, it is possible to perform independent running of the mobile robot, docking with the transport object, and omnidirectional motion control of the transport object by multiple mobile robots. In this paper, to verify the effectiveness of the proposed system, we present some simulation studies together with the system design and mechanical design of a prototype of mobile robot prototype.

Keywords: Network Systems, Hardware Platform
Abstract: In this paper, we propose and experimentally demonstrate a network switch to improve scalability and fault tolerance for a ring network applied to articulated robots. A ring network is suitable as a network for a robot with slim joints because it can reduce the number of wires. However, it is difficult to implement when the connection state changes during operation, such as a cable break due to unexpected trouble or addition of nodes. Our network switch can dynamically reconfigure the ring network based on the connection status. By combining this network switch with a previously proposed network protocol which performs in low latency and high efficiency, we realize a more practical design that solves the issues of scalability and fault tolerance in current ring networks.

Keywords: Network Systems, Automation Systems
Abstract: Deployment of the Internet of Things (IoT) in industries have enabled remote control of various mechanical system over the Internet. It is known that such networked control systems (NCSs) might suffer large time-varying communication delays. Not only performance degradation of the controlled system but also stability issues may occur as a consequence. The present paper proposes a two degree-of-freedom (DOF) digital servo system that stabilizes a feedback control system under the presence of large data transmission delay. We formulated a round trip time (RTT) predictor using a recurrent neural network (RNN). We then designed a dynamic state prediction controller (DSPC) to stabilize the feedback control system under the presence of large delay without causing control performance degradation. The controller gain is determined by solving LMI-based delay-dependent optimization problem. The designed controller is applied to a virtual inverted pendulum attached to the tip of an industrial robot manipulator controlled through a network that includes a wifi router. Results of experiments revealed that the proposed controller exhibited good performance under large fluctuation of RTT.

Keywords: Network Systems, Integration Platform, Multi-Robot Systems
Abstract: We propose a safe operation architecture for a semi-autonomous remote control system for mobile robots using mobile communications. The currently used architectures are unable to cope with several issues: adapting to various types of mobile robots, maintaining a sufficient level of safety in an area of unstable communications, flexibly interpreting input from the controller side, and flexibly changing the control mode according to the robot’s surrounding environment and the situation. To overcome these issues, we propose a safe operation architecture. Each functional unit is designed such that it is suitable for different types of mobile robots, and a range of control inputs from the controller side. Since the communication status between a base station and a mobile terminal is monitored by heartbeats, operation of the robot can be suspended safely when communication is lost. An initial system based on the proposed architecture has been developed using ROS (Robot Operating System). We conducted experiments with the system using delivery robots that served as the mobile robots. The experiments were performed in an outdoor field and used a 60 GHz mobile communication system.

Keywords: Multi-Robot Systems, Sensor Networks, Path Planning for Multiple Mobile Robots or Agents
Abstract: Power domain Non-orthogonal multiple access (NOMA) is a candidate multiple access technique for future wireless networks, which can improve spectrum efficiency and user connectivity. NOMA requires superposition coding (SC) at the transmitter side and successive interference cancellation (SIC) at the receiver side that can be achieved with reasonable system complexity. A heterogeneous network (Het-net) assisted with unmanned aerial vehicles (UAVs) is considered an indispensable part of the future wireless networks. Hence, weighted max-min fairness (MMF) data rate-based power allocation optimization for UAV-assisted networks in a heterogeneous cellular system is studied. Since power allocation for weighted MMF-based optimization is a non-convex problem, we characterized Perron-Frobenius (PF) theory-based optimization algorithm that converges very fast. The theoretical base and simulation results with imperfect SIC conditions and different weights for user data rates are presented.

Keywords: Control Theory and Technology, Multi-Robot Systems, Mechatronics Systems
Abstract: Methods for designing a multitask controller for redundant robots are often implemented by a null space projector. Although null-space projection-based controllers can achieve the task decoupling performance among prioritized tasks, it is vulnerable to uncertainty due to modeling errors in practice. Accordingly, this paper provides an analytical proof of the task coupling effect caused by the uncertainty and verifies it through a simulation.

Keywords: Modeling and Simulating Humans, Multi-Modal Perception, Virtual Reality and Interfaces
Abstract: A variety of Autism Spectrum Disorder (ASD) Perception Simulators have been developed in order to model these perceptions and analyze them in a quantitative way. We propose to further enhance that progress by producing a new unified simulator, building on and extending previous versions, which will run on mobile devices. Using this app version, we intend to broaden the experimental data pool of previous studies and develop a better model for environmental triggers and core functionality of applications for proactive support to those who wish to understand potential triggers in their living and working environments.

Keywords: Systems for Service/Assistive Applications, Software Platform, Human Interface
Abstract: Online communication is rapidly replacing face-to-face group communication in real space due to COVID-19 pandemic. However, in online group communication, when the number of participants increases and the entire members are arbitrarily divided into groups, some participants find it difficult to decide which group to join; it becomes difficult to see the topics discussed in other groups. To overcome the problem, we are developing a system called EVEA to assist the online group communication for text chatting. In EVEA, the discussed topics and the activeness of discussions based on our metrics in other groups are provided to participants as a hint for deciding which group to join. In this paper, we describe an overview of EVEA and explain how the system is designed.

Keywords: Micro/Nano Systems, Biomimetics, Medical Systems
Abstract: In this study, we designed a microfluidic device that utilizes the chemotaxis of Caenorhabditis elegans (C.elegans). In previous studies, the device showed an increase in the locomotion speed of C.elegans, but the reverse locomotion of C.elegans in the devices was frequently observed. Therefore, in this study, we aim to improve the movement speed of C.elegans by designing the device to match the wavelength of the meandering motion of the C.elegans and fabricate a channel with a check valve to prevent the reverse locomotion and quickly check the number of individuals that showed an attraction response. In this paper, we also investigated the chemotaxis of C.elegans mutants to urine obtained from a healthy person and cancer patient using the device. As a result, the chemotaxis was observed when the urine of cancer patients was used, indicating the developed device could be used for cancer tests.

Keywords: Modeling and Simulating Humans, Medical Systems, Human-Robot/System Interaction
Abstract: Currently, a new type of coronavirus infection (COVID-19) is rampant in the world. In this process, the medical doctor who performs the PCR test is also at risk of being infected with COVID-19. Therefore, we are developing a swab manipulation robot to mechanize this process. This robot can prevent infection of laboratory doctors and technicians by remotely controlling the dangerous virus collection process and increase the number of tests performed per day to detect COVID-19 at an early stage and perform mass testing. In this study, we created a model of the nasopharynx, which is the area to be wiped in the PCR test for the medical staff to improve their skills of sampling operation.

Keywords: Human-Robot/System Interaction, Human-Robot Cooperation/Collaboration, Sensor Networks
Abstract: This paper describes sonic communication using an electrostatic driver. The electrostatic driver is capable of transmitting and receiving audible and inaudible sounds. This allows sonic communication between robots and humans or between robots using a single sensor. We designed an electrostatic driver, and measured and evaluated its directivity, frequency characteristics, and transmission/reception characteristics between the same device.

Keywords: Vision Systems, Virtual Reality and Interfaces, Human-Robot/System Interaction
Abstract: High-accuracy marker is a fiducial marker developed by the author, and its six degrees of freedom (6DoF) pose estimation error is less than 10% of that of conventional fiducial markers. This paper proposes a method to turn any object into an input device using only one camera and one high-accuracy marker. As an example, a prototype of passive stylus was achieved by simply attaching a high-accuracy marker to a pen. The results of the experiment showed that the position error of the pen tip was less than 1 mm without using any filtering technique.

Keywords: Autonomous Vehicle Navigation, Automation Systems, Systems for Field Applications
Abstract: We have previously proposed the mapping method with graph-based SLAM using architectural floor plans. The previous method estimated the robot poses on the architectural floor plan and added them to the pose graph as the floor plan constraint. However, in an environment where the walls are covered by obstacles, it was difficult to estimate the robot poses on the architectural floor plan. In this paper, we introduce wall detection based on Hough transform to deal with environments where the walls are occluded. We use the wall detection to extract the point cloud of the wall surface and align it with the architectural floor plan to estimate the robot poses on the architectural floor plan.

Keywords: Formal Methods in System Integration, Autonomous Vehicle Navigation, Factory Automation
Abstract: Automatic AGVs (Auto Guided Vehicles) used in factories are automatically recharged using wireless power transfer to reduce labor cost and improve productivity. However, the current system requires AGVs to be stopped for charging, and there is still room for improvement in productivity. Therefore, the authors proposed an automatic charging system for AGVs using an autonomous mobile robot with wireless power supply function and a chase function, called a "wireless power transfer robot(WPT-Robot)", as an automatic charging system for AGVs. In this paper, we implemented functions about wireless charging and tracking, then evaluated them and combination function.

Keywords: Motion and Path Planning, Machine Learning, Vision Systems
Abstract: For autonomous mobile robots, obstacle avoidance is a necessary capability for navigation. We have thus far proposed a discrete motion planner based on a fully-connected feedforward deep neural network. The motion planner resulted in a sufficient capability with discrete motions, such as straight, right turn, and left turn. Once a robot stops, however, it is impossible to restart navigation toward a goal destination. This problem occurs when obstacles form a dead end in an environment. In order for the robot to escape from the dead end, further motions are required. Moreover, the robot is required to take into account a series of motions for the dead end. In this paper, therefore, we propose a discrete motion planner based on a fully-connected deep recurrent neural network. For network training, we use imitation learning from motion instructions. In the experiment, two of the motion planners are applied to the robot. Finally, we show that the robot based on the proposed motion planner successfully moves toward the goal destination while avoiding obstacles even in a dead-end environment.

Keywords: Systems for Field Applications, Vision Systems
Abstract: Crop and weed detection is an essential technique for the automation of spot spraying and mechanical weeding. Previous studies developed crop and weed detection methods by using crop rows. However, those methods cannot perform with high accuracy when weeds are heavily present. The reason is that the crop row detection is adversely affected by the presence of large amounts of weed. And even if crop rows can be detected accurately, the methods wrongly label the weeds within crop rows as crop. Therefore, we propose a crop and weed detection method which can be used in presence of large amounts of weeds by combining crop row detection using depth data and crop/weed classification by k-means clustering. The experiment showed the effectiveness of the method by using images taken in unweeded cabbage field.

Keywords: Automation Systems, Systems for Field Applications, Sensor Fusion
Abstract: Inspecting concrete structures such as tunnels is very important to keep them safe and durable. Due to the shortage of human inspectors, automated system for inspection is highly required. Hammering test is one of the popular inspection methods, and previous studies proposed automated systems for hammering test. Most works based on machine learning models to train a classifier to recognize hammering sounds suffer when the training data is not adequate for the considered data during deployment. This problem is also known as domain gap problem. In this paper, a methodology for concrete defect detection even when the available training data was collected from a tunnel that differs from the actually inspected tunnel is proposed. The proposed method selects part of the data from the inspection target tunnel, for which labels are not available, to use along traditional labeled training data in the training of a classifier within the semi-supervised support vector machine framework. This selection is conducted using the integration of visual information from an ordinary camera and acoustic information obtained using the hammering test. Experimental results showed that the proposed method yielded satisfying results in the laboratory conditions.

Keywords: Sensor Fusion, Systems for Field Applications, Machine Learning
Abstract: Most data fusion methods assume all data sources to be conditionally independent. In most cases, this is a practical choice, rather than an assumption derived from the observed phenomena. Wifi-based localization uses wireless signal strength information from all access points in an environment to estimate a robot's locations, which has been used both in indoor and outdoor (urban) environments. In such areas, the number of access points often ranges from several tens to a few hundred access points, several of which are strongly correlated to each other. In such a case, assuming all data sources are independent results in extremely peaked likelihood functions (overconfident) that are not suitable for robot localization under a Bayesian approach. Previous work has shown experimentally that a rather conservative, complete dependence assumption holds better in practice, as it is more robust against sensor and mapping errors. However, such an assumption generates less informative posteriors than desired, lowering localization accuracy. In this work, rather than taking either of these extremes, we propose a data-based method that learns more balanced data fusion rules, which generate informative yet robust likelihood functions. Testing performed on both indoor and outdoor datasets show the feasibility of our method.

Keywords: Machine Learning, Vision Systems, Motion and Path Planning
Abstract: One of the fundamental problems in mobile robot navigation in challenging environment is to improve autonomy in the sensor information processes. This paper presents an automatic parameter selection method for path recognition of outdoor semi-unstructured environment, based on human knowledge on the paths. As a preliminary process for path recognition, line segment detection integrating 2D image and 3D point cloud is adopted. Human knowledge on the path, line segments are mostly horizontal and around the ground level, is reflected to parameter selection through Pareto optimality in the multiple objective optimization. The proposed selection was evaluated by depth images taken in outdoor environments where boundaries of paths are obscure. It was shown in the experiment that resolution and edge parameters are influential to boundary detection performance. It was verified that the proposed method could select appropriate set of processing parameters based on the knowledge. The proposed idea is expected to contribute automatic selection of sensor modalities for more various environmental recognition scenarios.

Keywords: Multi-Robot Systems, Control Theory and Technology
Abstract: When multiple robots perform a cooperative task that a single robot cannot solve with its capabilities, such as transportation or monitoring, the robot system achieves efficient operation by controlling a robot swarm with the density that is suitable for the task. For instance, the robot swarm requires high- and low-density structures for transporting heavy objects and monitoring a large area, respectively. Thus far, researchers have studied lattice-structure-based collective movement with regular organizations and amorphous-structure-based flocking behavior with irregular organizations. However, both control methods use a high-density swarm or aim for that state. In this study, we propose distance and angular potentials to control the filling density of robots based on regular tessellations and a collective movement method to maintain the latticed structure constructed by these potentials. This study demonstrates the collective movement of the robot swarm with lattice structures with different densities and quantitatively evaluates the structural changes caused by the movement.

Keywords: Modeling and Simulating Humans, Machine Learning, Vision Systems
Abstract: In this study, we proposed a method to evaluate the viewpoint of a robot arm in a reaching movement using reinforcement learning. The optimal viewpoint for operators in teleoperation was studied by conducting a subject experiment. However, in some special situations, such as inside the pedestal of a nuclear plant crushed in a disaster, the lack of environmental information makes it challenging to prepare the subject experiment in advance. In addition, individual differences cannot be eliminated by conducting the subject experiment. In this study, we used reinforcement learning to select viewpoints and found that the world model inspired by the prediction function of the brain exhibited similar performance to that of humans in the reaching motion of a robot arm. This study demonstrated that the world model can evaluate viewpoints using reinforcement learning in the reaching task.

Keywords: Motion and Path Planning, Machine Learning, Autonomous Vehicle Navigation
Abstract: In this paper, we propose a novel approach to allow exploration robots to solve navigation problems in complex environments including rough terrain. Previous studies show that it is difficult for robots to autonomously navigate without prior information such as an environmental map. Recently, advances in deep reinforcement learning (DRL) have made it possible to complete the autonomous motion control without the map. In this respect, we apply DRL to realize fully autonomous navigation on rough terrain for the exploration robot. The exploration robot can generate suitable control motion converted from observed depth information through a neural network. The simulation results show that our DRL-based approach can successfully solve the collision-free navigation of robots on rough terrain.

Keywords: Environment Monitoring and Management
Abstract: For accurate identification of radiation sources using 4π gamma imaging, the selection rule of measurement points around target sources was investigated. In a simple computational model of the 4π gamma imaging, the detector transfer algorithm was optimized by decision tree analysis. According to the proposed algorithm, a 137Cs point source was identified using a multi-pixel CdTe detector mounted on an unmanned vehicle as a prototype 4π gamma imager.

Keywords: Systems for Search and Rescue Applications, Energy and Environment-Aware Automation
Abstract: This study proposes a highly accurate method for estimating the distribution of radiation sources, using a particle filter of the Bayesian estimation framework, which incorporates the structural information of the reactor at the Fukushima Daiichi Nuclear Power Plant as the prior information. From the experimental result, unlike the maximum likelihood estimation method, our method performed well in an extreme measurement environment where the detector could not reach, and showed high accuracy in the estimation of the point-like and distributed radiation sources.

Keywords: Vision Systems, Machine Learning, Building Automation
Abstract: This paper describes a method for discriminating structures in a nuclear facility based on deep learning using three-dimensional (3D) point cloud data. To promote safe and secure decommissioning, estimating and assuming the conditions of a nuclear facility based on measured sensor data are important. Especially, data on dose rate in a workspace are useful to plan a decommissioning task, and the shape and material of structures in the workspace are required for radiation dose simulation. Shape data can be obtained using equipment such as a 3D laser scanner. However, obtaining the material data of objects is difficult. Therefore, we consider that major material can be estimated from the category of structures in a nuclear facility. In this study, we propose a structure discrimination method based on 3D semantic segmentation using 3D point cloud data comprising labeled data points by referring to the structural category labels of 3D computer-aided design data of an existing nuclear facility. We evaluated the discrimination performance of the proposed method through hold-out validation.

Keywords: Vision Systems, Systems for Search and Rescue Applications, Plant Engineering
Abstract: This paper describes the image selection method for the efficient 3D reconstruction computation from the image sequence. We confirmed that the proposed method can select the adequate images for 3D reconstruction computation by calculating the optical flow from the image sequence obtained from camera mounted on the remotely operated robot. This suggests that the computational cost could be reduced by eliminating the duplicate and high similarity images for performing the efficient 3D reconstruction.

Keywords: Vision Systems, Systems for Service/Assistive Applications
Abstract: The socio-economic need for service robots has become more evident during the ongoing pandemic. To boost their deployment, robots need to improve their manipulation capabilities, which includes solving one of the biggest challenges: determine the position and orientation of the target objects. While conventional approaches use markers which require constant maintenance, deep-learning-based approaches require a host computer with high specifications. In this paper, we propose a 6D-pose estimation system whose segmentation algorithm is embedded into OAK-D, a camera capable of running neural networks on-board, which reduces the host requirements. Furthermore, we propose a point cloud selection method to increase the accuracy of the 6D-pose estimation. We test our solution in a convenience store setup where we mount the OAK-D camera on a mobile robot developed for straightening and disposing of items, and whose manipulation success depends on 6D-pose estimation. We evaluate the accuracy of our solution by comparing the estimated 6D-pose of eight items to the ground truth. Finally, we discuss technical challenges faced during the integration of the proposed solution into a fully autonomous robot.

Keywords: Virtual Reality and Interfaces, Human Interface, Modeling and Simulating Humans
Abstract: The authors have proposed Motion-Less VR, an immersive VR system without actual body motion. Motion-Less VR is a system that enables virtual body operation by measuring the joint torques output under body immobilization and simulating the motion that would typically occur. Our previous study developed a system to realize a 2- DOF (shoulder and elbow) operation and showed its validity. However, it has not yet achieved the same level of operational performance as that of normal, natural body motion. One of the reasons for this is the difference in posture between the real and virtual bodies. This paper investigated the effects of postural differences caused by body immobilization on muscle output, which is the source of joint torque. Specifically, we measured surface EMG (electromyography) during a motion intention output task where the participant attempted to track a target under non-visualization of the body and compared the results between the body immobilization and normal conditions. As a result of the experiment, we confirmed that the EMGs under body immobilization differed from those under normal conditions. This result suggests that the difference in posture between the physical body and the virtual body may affect the operating performance of the virtual body.

Keywords: Motion and Path Planning, Machine Learning, Decision Making Systems
Abstract: In this paper, we address the task of rearranging items with a robot. A rearrangement task is challenging because it requires us to solve the following issues: determine how to pick the items and plan how and where to place the items. In our previous work, we proposed to solve a rearrangement task by combining the symbolic and motion planners using a Motion Feasibility Checker (MFC) and a Monte Carlo Tree Search (MCTS). The MCTS searches for the goal while it collaborates with the MFC to accept or reject instructions. We could solve the rearrangement task, but one drawback is the time it takes to find a solution. In this study, we focus on quickly accepting or rejecting tentative instructions obtained from an MCTS. We propose using a Neural Network-based Motion Feasibility Checker (NN-MFC), a Fully Connected Neural Network trained with data obtained from the MFC. This NN-MFC quickly decides if the instruction is valid or not, reducing the time the MCTS uses to find a solution to the task. The NN-MFC determines the validity of the instruction based on the initial and target poses of the item. Before the final execution of the instructions, we re-validate the instructions with the MFC as a confirmation before execution. We tested the proposed method in a simulation environment by doing an item rearrangement task in a convenience store setup.

Keywords: Systems for Service/Assistive Applications, Human-Robot/System Interaction, Human Factors and Human-in-the-Loop
Abstract: In the Future Convenience Store Challenge (FCSC) of the World Robot Summit (WRS), the Customer Interaction Task has been conducted from 2017 to 2020 to propose new services and store designs that can be achieved using robot technology to serve customers in convenience stores in the near future. In the Customer Interaction Task of WRS 2020 FCSC, held in Aichi, Japan, we present a convenience store introducing intelligent space, with new experiences, safety, and flexibility as key concepts. We propose a product recommendation that reflects the owner's intentions, an assistive robot that provides customers desired assistance, and a monitoring service with safety and relief.

Keywords: Human-Robot/System Interaction, Systems for Service/Assistive Applications, Software, Middleware and Programming Environments
Abstract: Service robot in convenience stores is an emerging domain of a human-robot interaction system (HRI). To utilize the full efficiency of the robot and meet users' needs in HRI, the robot has to recognize human action to decide suitable robot reactions. However, it is difficult for non-expert users (e.g., store staff) who do not have experience with robot programming to create robot behaviors and make sure that those behaviors work properly in the real robot. To solve these difficulties, we provide (i) a graphical interface to facilitate the robot behaviors creation by using Behavior Trees and (ii) a robot simulation environment to test and debug the created robot behaviors before deploying them to the real robot. By using our system, non-expert users can easily create the robot behaviors by making pairs of the detected human action to their desired robot reaction. Non-expert users also can make sure that the created robot behaviors are working without any issues with the surrounding environment via the robot simulation. We evaluate our proposed system by using the System Usability Scale (SUS) score and found that the system has good usability but still has points for improvement as reported during our user study.

Keywords: Virtual Reality and Interfaces, Human Interface, Modeling and Simulating Humans
Abstract: In this paper, to investigate whether there is a difference in muscle contraction when using Motion-Less VR and during natural motion, we measured and compared surface EMG during tasks under both Motion-Less VR and natural motion conditions. The results showed that the biceps muscle showed the same tendency as during natural motion, but the triceps muscle showed a different tendency.

Keywords: Human Factors and Human-in-the-Loop, Human Interface, Machine Learning
Abstract: Experimental subjects in science and engineering courses at universities and other higher education institutions are one of the most important subjects for acquiring knowledge and skills. However, due to the lack of skills among young people these days, an increasing number of students are not interested in experimental subjects because they cannot perform the experiments well. This paper proposes design guidelines for digital teaching materials for practical training in manipulating tools using the body such that casting and forging. In this practical training, motivation of the students and awareness of learning may be reduced, when low skill levels worsen work costs and product quality. And it is assumed that students who will use the digital materials are beginners and are not familiar with the tools used in practical training. Under these conditions, this paper discusses the specifications for the design of digital teaching materials that can achieve the contents of each stage and proposes a method for realizing these specifications. The proposed method introduces spatial perception and 3D models of tools, taking into account not only the student's knowledge and skills, but also the mentality of beginners. The effectiveness of the developed digital teaching materials will be examined through interviews with students.