Keywords: Multi-Modal Perception, Machine Learning, Human-Robot/System Interaction
Abstract: We propose a novel method that integrates probabilistic logic and multimodal spatial concepts to enable a robot to acquire the relationships between places and objects in a new environment with a few learning times. Using predicate logic with probability values (i.e., probabilistic logic) to represent commonsense knowledge of place-object relationships, we combine logical inference using probabilistic logic with the cross-modal inference that can calculate the conditional probabilities of other modalities given one modality. This allows the robot to infer the place of the object to find even when it does not know the likely place of the object in the home environment. We conducted experiments in which a robot searched for daily objects, including objects with undefined places, in a simulated home environment using four approaches: 1) multimodal spatial concepts only, 2) commonsense knowledge only, 3) commonsense knowledge and multimodal spatial concepts, and 4) probabilistic logic and multimodal spatial concepts (proposed). We confirmed the effectiveness of the proposed method by comparing the number of place visits it took for the robot to find all the objects. We also observed that our proposed approach reduces the on-site learning cost by a factor of 1.6 over the three baseline methods when the robot performs the task of finding objects with undefined places in a new home environment.

Keywords: Human-Robot/System Interaction, Human Factors and Human-in-the-Loop, Multi-Robot Systems
Abstract: Some people have difficulty participating in social activities due to COVID-19 or other factors such as age, physical or cognitive state, appearance, and gender hindrance. We propose a concept of active social participation with cybernetic avatars (CAs) that are tele-operated robots and CG agents. We also propose the CA platform, which is a software platform that connects CAs and tele-operators, to make it easier for people in various locations to participate in society using CAs. In this paper, we propose three types of CAs and connection methods for CAs and environment sensors to the CA platform for assisting tele-operators to achieve active social participation in Avatar Symbiotic Society.

Keywords: Multi-Modal Perception, Vision Systems
Abstract: Before grasping an object, robots should determine where is best to grasp it to increase the picking success rate. To search for feasible grasping points, placing proximity sensors based on the reflection of light at the tip of the gripper's fingers can be used to obtain the distance to objects (pre-grasping distance). However, the proximity information output by the sensor is sensitive to the object's reflectance, which leads to an error in the pre-grasping distance. In this study, we use neural network regression to estimate the reflectance of objects based on RGB images so that the pre-grasping distance error is reduced, assuming that the reflectance statistically depends on object category regarding daily objects found in the service robot environments. We created a dataset with images of 40 objects for which we measured the reflectance. Then, we trained two regressors based on well-known architectures. In the experiments, we compare the accuracy in the distance estimation for our proposed method and the conventional fixed reflectance method with both known and unseen objects. The proposed method significantly reduces the pre-grasping distance error for objects with various reflectances compared to the conventional method. The results show that our proposed method can achieve a distance estimation error under 2 [mm] at distances of 20 [mm] or less.

Keywords: Human-Robot/System Interaction, Systems for Service/Assistive Applications, Software Platform
Abstract: This paper describes the role of activity monitoring and management in a software platform (cybernetic avatar (CA) platform) to support avatar-based interactions. Avatar-based interaction differs in various aspects from conventional face-to-face interaction. For instance, it offers anonymity and parallel operability (i.e., a single operator can control multiple interactions simultaneously). On the other hand, it imposes a large burden on the operator due to the required avatar operations. Reducing the operator's burden is a crucial issue to consider in developing such a system. From the viewpoint of data analysis, it is easy to capture and record the activities of participants in avatar-based interaction because those interactions are carried out by explicit avatar operations via a network. In addition, the activities of human participants in such interaction are observed by sensor networks. Therefore, the data from avatar monitoring should contain many kinds of information on the human operators and interaction partners interacting with each other via the avatars. The activity monitoring and management functions enable various services to support human activities using avatars, including reduction of the operational burden. In this paper, we also address an operation recommendation system, as an example application of activity monitoring, which offers situation-dependent recommendations for the operator's speech and operations to reduce the operator's burden. Experimental results support the effectiveness of the proposed system.

Keywords: Software, Middleware and Programming Environments, Integration Platform, Multi-Modal Perception
Abstract: The rapid deployment of intelligent robots to perform service tasks has become an increasingly complex challenge for researchers due to the number of disciplines and skills involved. Therefore, this paper introduces Serket-SDE, a containerized Software Development Environment (SDE) for the Symbol Emergence in Robotics Toolkit (Serket) that relies on open-source technologies to build cognitive robotic systems from multimodal sensor observations. The main contribution of Serket-SDE is an integrated framework that allows users to rapidly compose, scale, and deploy probabilistic generative models with robots. The description of Serket-SDE is accompanied by demonstrations of unsupervised multimodal categorizations using a mobile robot in various simulation environments. Further extensions of the Serket-SDE framework are discussed in conclusion based on the demonstrated results.

Keywords: Systems for Service/Assistive Applications, Vision Systems, Modeling and Simulating Humans
Abstract: Academics, researchers, and industrial experts have made significant efforts in human activity recognition by considering different perspectives, such as benchmark datasets, utilization of smart sensors, and development of recognition algorithms. In addition, the emerging trend of avatar technologies has drawn the attention of researchers, which assemble cognitive abilities, including activity recognition, to realize social activities and overcome boundaries of human energy, time, and environment. These could widely assist older adults, independent living care, and medical care, by the data obtained from human activities. Since human behavior in real-life is extremely complex, assembling meaningful recognition is paramount. The performance of existing recognition models depends heavily on the datasets. However, it has constraints to acquiring datasets with rich information due to the limited budgets, the specific areas that can be used for human activity, the limited number of actors, and other ethical reasons. This paper considers the critical criteria to fill the requirements for human daily activity recognition and presents a survey of the existing datasets of activities of daily living in both real-life settings and virtual spaces. It also presents new challenges and potential advances in behavior recognition technology.

Keywords: Multi-Modal Perception, Machine Learning, Sensor Fusion
Abstract: In the field of robotics, scene understanding has been studied mainly by vision-based processing using RGB sensors such as cameras and LiDAR, depth sensors, or both. However, these sensors are difficult to use in human living spaces due to issues such as privacy and impact on the human body. In this paper, we propose a new framework for echolocation-based scene reconstruction with mixed inputs called Mixed ELSR. The problem of audible sound, which is easily affected by environmental noise, is solved by considering the characteristics of the input sound and integrating it with deep learning techniques. We have conducted experiments of recording data in real space and have shown the effectiveness of the proposed framework.

Keywords: Human-Robot/System Interaction, Human Interface, Sensor Networks
Abstract: In order to overcome the problems of current hearing aid devices, we proposed an attention-based sound selective hearing support system, where individual target and anti-target sound sources in the environment can be selected, and the target sources in the facing direction are emphasized. New functions were implemented by accounting for system’s practicability and usability. The performance of the proposed system was evaluated under different noise conditions, by elderly subjects with different levels of hearing loss. Intelligibility tests and subjective impressions in three-party dialogue interactions indicated clear improvements by using the proposed hearing support system under noisy conditions.

Keywords: Sensor Networks, Systems for Search and Rescue Applications
Abstract: This paper verify the performance of the method of calibrating their positions, orientations, and time offsets using observations with multiple microphone arrays and estimating the sound source positions in the real environment. The conventional method calculates the orientation and time offset independently, and it is difficult to correct the solution when one of the optimizations cannot be performed well. Therefore, a method was proposed that can simultaneously estimate the position, orientation, and time offset by combining two types of objective functions. We evaluate the effectiveness of this simultaneous optimization through numerical simulations and experiments with recorded acoustic signals. As a result, it was shown that the proposed method is effective under the condition that the input error is small.

Keywords: Sensor Networks
Abstract: Multiple microphone arrays have many applications in robot audition, including sound source localization, audio scene perception and analysis, etc. However, accurate calibration of multiple microphone arrays remains a challenge because there are many unknown parameters to be identified, including the Euler angles, geometry, asynchronous factors between the microphone arrays. This paper is concerned with joint calibration of multiple microphone arrays and sound source localization using graph simultaneous localization and mapping (SLAM). By using a Fisher information matrix (FIM) approach, we focus on the observability analysis of the graph SLAM framework for the above-mentioned calibration problem. We thoroughly investigate the identifiability of the unknown parameters, including the Euler angles, geometry, asynchronous effects between the microphone arrays, and the sound source locations. We establish necessary/sufficient conditions under which the FIM and the Jacobian matrix have full column rank, which implies the identifiability of the unknown parameters. These conditions are closely related to the variation in the motion of the sound source and the configuration of microphone arrays, and have intuitive and physical interpretations. We also discover several scenarios where the unknown parameters are not uniquely identifiable. All theoretical findings are demonstrated using simulation data.

Keywords: Sensor Fusion
Abstract: Human and/or asset tracking with attached sensor units is a fundamental task to understand their activities. Most common indoor localization methods for human tracking require infrastructures in the environment with costly deployment and maintenance. To overcome the problem, environmental sounds have been used for infrastructure-free localization. While they achieve room-level classification, they suffer two problems: low signal-to-noise-ratio(SNR) condition and non-uniqueness of sound over coverage area. We propose a microphone localization method based on supervised spectral decomposition and spatial likelihood to solve these problems. The proposed method was evaluated with actual recordings in our experimental room with a size of 12x30 m. The results demonstrated that the proposed method with supervised NMF was robust under low-SNR condition compared to a simple feature (mel frequency cepstrum coefficient: MFCC). Also, the proposed method could be easily integrated with prior distribution, which is available from other Bayesian localization. Our method thus would be useful to estimate spatial likelihood from environmental sound and further integration with others.

Keywords: Environment Monitoring and Management, Decision Making Systems, Machine Learning
Abstract: Scene analysis relies on sensing and understanding the events and objects in a dynamic environment. Lifelong robot learning for scene analysis is a continuous process to learn distinct events, actions, and noises using different sensory modalities in a lifelong manner. In real environments, the spatio-temporal nature of the data captured by sensors may not be stationary, therefore novel events or unseen instances of the known events may exist which affects the performance of scene analysis. In this work, a robot audition framework for Auditory Scene Analysis (ASA) is proposed which enables a real robot to acoustically detect and incrementally learn novel acoustic events in a real domestic environment. To achieve the source-specific analysis, a lifelong learning approach in ASA for robot audition is developed, which includes the following steps: (1) Sound Source Localization (SSL), (2) audio feature extraction, (3) Acoustic Event Recognition (AER), (4) Acoustic Novelty Detection (AND), and (5) adaptation of new event classes into the AER and AND models. The steps are performed on streaming raw audio signals captured in a domestic environment by a robot equipped with a microphone array. The self-learning process on acoustic signals stemming from different events occurs without human supervision. Thus, the proposed system allows the robot to have the capability for lifelong learning of novel acoustic events. The effectiveness of the proposed robot audition framework for lifelong ASA is evaluated in terms of the accuracy of acoustic event recognition and computational time to meet the demands of lifelong learning in real-time.

Keywords: Human-Robot Cooperation/Collaboration, Energy and Environment-Aware Automation, Surveillance Systems
Abstract: In this paper, we propose a reduced data representation for skeleton-based action recognition on GCNs and examine the method's influence on action recognition performance with respect to computational resources. With the straightforward strategy of omitting skeleton joints that are just mildly relevant, we want to point out that not only general reduction methods are applicable for decreasing a model's complexity but problem-specific options can likewise have a significant influence by also keeping a good accuracy. The trade-off between system accuracy and required computational resources becomes particularly relevant with an increasing focus on green computing. Evaluated on three recent GCN approaches on the NTU RGB+D dataset, our reduced skeleton representation decreases the training time by 27.4% and memory consumption by 28.6%, while decreasing the accuracy by only 1.3 ppt on average. This demonstrates the importance of descriptive joints for skeleton-based action recognition and the ability to optimize the trade-off between system performance and required resources at a representational level.

Keywords: Human-Robot/System Interaction, Systems for Service/Assistive Applications, Welfare systems
Abstract: Simultaneous localization and mapping (SLAM) have been an essential requirement for the autonomous operation of mobile robots for over a decade. However, in the wake of recent developments and successes of deep neural networks and machine learning, the conventional task of SLAM is gradually being replaced by Semantic SLAM. Extracting semantic information (such as object information) from sensory data can enable the robot to distinguish different environmental regions beyond the conventional grid assignments of free and occupied. This level of scene awareness is essential for performing higher-level navigation and manipulation tasks and enhancing human-robot interactions. This paper presents an integrated framework that not only builds such maps of indoor environments but also facilitates the execution of `textit{Go to object}' tasks with high-level user input. We also present a method to extract meaningful endpoints of navigation based on object class. Our modular stack leverages well-known object detectors (YOLOv3), RGB-D SLAM techniques (RTAB-Mapping) and local navigation planners (TEB) to perform ObjectGoal navigation tasks. We also validate the results of experiments in real environments.

Keywords: Welfare systems, Rehabilitation Systems, Systems for Service/Assistive Applications
Abstract: People depend on medical equipment to support their movements when their motor function declines. Our previous study developed a method to estimate motor function from the force applied to a vertical handrail while standing. However, the effect of the handrail direction on movement remains unclear. Additionally, the force applied to the handrail and floor reaction forces on the buttocks and feet may alsochange with a decline in motor function. Here, this study constructed a system with force plates and handles in both the horizontal and vertical directions to measure the forces applied to the handrails, buttocks, and feet. Furthermore, the change in accuracy of the estimation of motor function, depending on the direction of the handrails and input information, was investigated. In the experiment, healthy participants stood up using a handrail with unrestricted movement and while wearing elderly experience kits that artificially impaired their motor function. The results showed that people exert more downward force on horizontal handrails than on vertical handrails. However, people rely on the vertical handrail for a longer period of time to stabilize anterior-posterior movement. These results indicate that different directions of handrails cause different strategies of the standing-up motion. Additionally, the accuracy of the estimation of motor function improved when the horizontal handrail was used rather than the vertical handrail. This suggests that the classification accuracy could be improved by using different handrail directions, depending on the subject's condition and standing-up motion.

Keywords: Human Interface, Human-Robot/System Interaction, Human-Robot Cooperation/Collaboration
Abstract: In recent years, research and development of human assist robots have been actively carried out in order to support the independent lives of persons with impaired motor function. Although assisting the user's activity according to the user's motion intention is important for the human assist robots, it is difficult to objectively determine in real time whether the assisted motion by the robot is exactly same as the intended motion by the user. In this paper, a method to objectively detect a user's sense of assist mismatch using EEG signals is investigated. Experiments were conducted to analyze the user’s brain activities during the ankle joint movement with the assist of intended and unintended motion. The experimental results show that there are differences in the theta wave activities around the central area of the head and alpha wave activities around the central sulcus when assist mismatch occurs. These results suggest that a user's sense of mismatch can be detected with the proposed method.

Keywords: Mechanism Design, Systems for Field Applications, Systems for Service/Assistive Applications
Abstract: Omnidirectional driving gear is a gear mechanism having two degrees of freedom on one surface. We have developed an omnidirectional driving gear by combining flat and convex arc surfaces (free-form omnidirectional driving gear mechanism). Weight compensation and limit sensors are incorporated in the mechanism for smooth running. In this design, the motion range is small and limited by the shape of the curvature and requirement. To improve the adaptability of the mechanism to any given environment, a flexible omnidirectional driving gear whose curvature can be changed arbitrarily was developed and studied.

Keywords: Autonomous Vehicle Navigation, Systems for Service/Assistive Applications, Motion and Path Planning
Abstract: With the arrival of an ultra-smart society, there is a need for mobility support robots that can act as flexibly as humans without trial-and-error learning and re-design of controllers in public spaces where people freely come and go. One of the solutions to this problem is to judge spaces that require suitable attention and make decisions based on only time-series observation information of the surrounding environment. Therefore, we propose a spatial attention-based behavior coordination rule to achieve safer avoidance by increasing the weight of avoidance behavior in advance when attention is paid to a moving obstacle, and a scaling rule of action outputs to adapt more quickly even to obstacles with faster moving speeds. We show the effectiveness of the proposed methods by comparing them with conventional methods through experiments using a real mobile robot.

Keywords: Human Interface, Human-Robot/System Interaction, Multi-Modal Perception
Abstract: It is assumed that the work is performed in the future by switching between different machine classes sequentially using teleoperated hydraulic excavators. In this case, the operation time is reported to increase in the process of excavation and loading when switching to a different machine class, compared with the case in which no switching is performed. It is considered a larger excavation reaction force was applied when switching the machine, -compared with the case without switching. Therefore, this study validates whether two feedback methods on the joint loads of attachments to operators can effectively suppress the increase in operation time. The first method is force feedback on a parameter named “attachment joint load ratio”. The second method is force feedback on the absolute load of the attachment joint. Subsequently, we designed a simulator that simulated a moving excavator and conducted a subject experiment. Consequently, it was validated that the operation time significantly increased when feedback was not provided. Conversely, the effect of suppressing the increase in the operation time by switching between classes was validated in the two feedback methods on the joint load of attachments. Moreover, it was indicated that the method that provides feedback on the attachment joint load ratio was effective when switching to a different machine class.

Keywords: Control Theory and Technology, Mechatronics Systems
Abstract: This study addresses the approximation of the nonlinear controller for a pneumatic artificial muscle (PAM) actuator using polynomial functions. The controller approximation is described by the least absolute shrinkage and selection operator (LASSO), which is the least-squares method used to obtain a sparse solution. The nonlinear controller includes the PAM model; thus, model uncertainties affect the performance of the controller. Therefore, it is important to find a reasonable regularization parameter for approximating the controller in both numerical and experimental settings. The numerical and experimental results confirm that the resulting polynomial controller helps reduce the computational cost and maintain the control performance comparable with the original nonlinear controller.

Keywords: Multi-Modal Perception
Abstract: The same external force can alter a perceived force by applying vibrations. The objective of this study is to develop a vibration application method that can manipulate human-perceived force by maximizing this characteristic. In this study, we asked subjects to perform isometric wrist extension movements with and without an assistive force using pneumatic gel artificial muscles. We then measured perceptual characteristics by combining the conditions of vibration, presence of active exertion, and magnitude of assistive force. The results indicated that the perception-assist force increased significantly with the application of vibration, regardless of the presence or absence of active force exertion.

Keywords: Machine Learning, Human Interface, Welfare systems
Abstract: With the research on mental health, rehabilitation training and other fields, obtaining people’s real emotion feelings is frequently required in many fields. Emotion recognition method based on physiological signals can directly obtain people’s emotion states and avoid pretending expression and emotional expression disorder. In physiological signals, Electroencephalogram (EEG) signal is commonly used in the emotion valuation, and Heart Rate Variability (HRV) signal is related to people’s excited feeling. This paper proposed an emotion recognition method based on EEG and HRV to do the emotion recognition work. This method aims to solve the accuracy problem of instant emotion recognition, and achieve a higher accuracy. According to Russell’s model of emotion, the system in this paper use two dimensions, “valence” and “arousal”, to describe people’s emotion. The emotion recognition system we proposed combines more advanced neural network models and eigenvalues closely related to emotional states. This system uses DenseNet as the neural network model for machine learning process, which is more accurate than the general deep neural network. Using differential entropy as the main eigenvalue makes the system’s ability to analyze emotions based on EEG more efficient.

Keywords: Modeling and Simulating Humans, Rehabilitation Systems, Virtual Reality and Interfaces
Abstract: In recent years, neurorehabilitation is expected to apply the brain’s plasticity. This study develops a rehabilitation support system using the simultaneous presentation of visual and force sensations. In the visual presentation part, hand movements are detected by hand tracking of a head-mounted display, and real-time virtual hand movements are presented. In the force presentation part, pneumatic gel muscle supports the bending motion by using the hand’s bending angle as a trigger. In the verification, we investigated the effect on the addition of a motor for assisting in hand motion (when the bending angle of the virtual hand motion is larger than that of the actual hand motion) and showed that the potential of this system for improving motor functions.

Keywords: Modeling and Simulating Humans
Abstract: In this study, we developed a lumbar mechanical model that can reproduce the changes in joint stiffness during lumbar extension motion in humans. Earlier reports suggest that during lumbar extension motion joint stiffness varies from person to person. The individual differences in the way joint stiffness changes during lumbar extension motion are caused by individual differences in motion strategy, developed with reference to the equilibrium-point hypothesis, which is one of the clues to the elucidation of the human motor control law. We herein describe the development process of the model and examine the possibility of verifying individual differences in movement strategies by reproducing joint stiffness changes.

Keywords: Network Systems, Software, Middleware and Programming Environments, Control Theory and Technology
Abstract: In this study, we proposed a false data injection (FDI) attack model targeting passivity of an encrypted bilateral control system. The implementation of encrypted control keeps important parameters in the control system secret. In addition, the encrypted control is sensitive for streaming data falsification and useful for detecting FDI attacks. In particular, there is a lot of research aimed at achieving secure teleoperation by encrypted bilateral control systems. However, there will be some types of attacks that are difficult to detect due to the structure of the calculation process of the encrypted control. As an example of that attack, we found one effective FDI attack model that is imperceptible to the encrypted wave variable based bilateral control system. The proposed FDI attack is applied to the input and output side of the master or follower and multiplies the streaming signal more than 1. We implemented the encrypted bilateral control system into an experimental setup using the same type of pneumatic cylinders as the master and follower devices and verified the performance of the proposed attack model. As a result of the experiment, we confirmed that the proposed attack model was imperceptible by the encrypted control, and the passivity of the whole system was broken. We consider the proposed attack model will be able to detect by monitoring the total estimated energy using the passivity observer.

Keywords: Control Theory and Technology
Abstract: This study developed a commercial field-programmable gate array board for building an encrypted flight control system and a quadcopter using competition parts. It also examined the performance of the developed quadcopter through implementation and thrust tests. The experimental test results verified the performance of the encrypted feedback and feedforward controllers of the quadcopter for the thrust and roll actions. Additionally, the quantization error in the control commands was confirmed to be within 0.2%.

Keywords: Network Systems, Systems for Field Applications, Automation Systems
Abstract: Because reaction force estimation used in teleoperated robots is exposed to threats of cyberattacks, ensuring cyber security of reaction force estimation is important. The purpose of this study is to develop an encrypted reaction force observer. The state-space representation of the reaction force observer is organized and represented as a product of a matrix and a vector. ElGamal encryption, which can compute the product of ciphertexts, is applied to the calculation of the reaction force observer. The results of the reaction force estimation experiment show that the reaction force observer with encrypted control can estimate the reaction force.

Keywords: System Simulation, Control Theory and Technology, Mechatronics Systems
Abstract: The goal of the research presented in this paper is to establish functional mockup units (FMU) co-simulation methods to simulate and evaluate encrypted dynamic systems using somewhat homomorphic encryption (SHE). The proposed approach encrypts the entire dynamic system expressions including: model parameters, state variables, feedback gains, and sensor signals, and perform computation in the ciphertext space to simulate dynamic behaviors or generate motion commands to servo systems. The developed FMU co-simulation helps analyze the relationship between security parameters and performance. Two illustrative examples are presented and analyzed: 1) encrypted Duffing oscillator and 2) encrypted teleoperation. How the time delay due to FMU co-simulation affects the refresh rate is also reported.

Keywords: Control Theory and Technology
Abstract: A botnet defense system (BDS) is a cyber-security system that defends an IoT system against malicious botnets. In a BDS, malicious botnets are detected by white-hat botnets, and are deleted from an IoT system. In this paper, we propose a modeling method of the spread of malicious botnets in a BDS, where white-hat botnets patrol according to the computation result of surveillance by multiple agents. Using the graph structure of an IoT system and the paths of white-hat botnets, the spread of malicious botnets is modeled by a discrete-time Markov chain. A numerical example is presented to demonstrate the proposed method.

Keywords: Control Theory and Technology, Network Systems
Abstract: This paper considers a control problem to prevent dynamic stealth attacks using interception data.　The attacker can design a stealth attack signal from the trajectory of the system if the input data is persistently exciting (PE).　Thus, this paper considers model predictive control using the input without PE.　Since a design problem for the input without PE is a combinatorial optimization problem, nuclear and l2/l1 norm optimization problems are proposed as alternative problems to obtain approximate solutions.

Keywords: Rehabilitation Systems, Mechanism Design
Abstract: This paper deals with an above-knee prosthesis that allows stair ascending/descending with no external energy source. It has a key mechanism that hydraulically interlocks knee joint and ankle joint, of which interlocking strength is adjustable by a flow control valve (FCV) equipped in the hydraulic system. Our previous study certified that the prototype AKP allows step-over-step gait in stair ascending and a slight knee flexion at the initial stage of the stance phase in level ground walking observed in non-amputees’ walking. This paper reports a new design of the AKP aiming for a practical usage and cosmetically desirable.

Keywords: Vision Systems, Human Interface
Abstract: Corneal imaging has been studied as an eye-tracking method, and the focused object and point-of-gaze are estimated by extracting scene information reflected on the corneal surface. However, when an eye tracker is regarded as a daily-use device, it is an essential function to estimate its self-position. In our previous research, we attempted self-position estimation using interior illumination patterns reflected on the cornea. However, the accuracy of localization was not sufficient; thus, we focused on convolutional neural network developed remarkably and the inertial measurement unit (IMU) as additional improvements. In this study, we propose to improve the accuracy of localization by integrating SE-ResNet and IMU with Monte Carlo localization. Data collection and annotation become a problem when deep neural network is employed. To address this problem, three-dimensional (3D) environmental map was reconstructed, and SE-ResNet was trained using virtual images generated using the 3D environmental map. We conducted comparative experiments and confirmed the feasibility of our proposed method.

Keywords: Mechanism Design, Systems for Service/Assistive Applications, Human-Robot/System Interaction
Abstract: Fruit farmers often need to keep their arms raised continuously for cultivating or harvesting fruits, and Japanese fruit farmers are rapidly aging. In this paper, we propose a new agricultural arm support suit. This arm support suit easily locks the user's shoulder joints by pushing the support suit's levers and continues to support the user's arms. To unlock is also easy. This support suit is developed by integrating only passive mechanical elements without batteries, actuators, or controllers. Therefore, this support suit is essentially safe. Since batteries are not required, there are no battery problems such as the limitation of the time to use and the maintenance. First, we describe the features of this arm support suit. Second, we explain the structure and mechanism of the support suit. Third, we introduce the developed support suit. Finally, we provide experimental results to verify the effectiveness of the developed arm support suit.

Keywords: Medical Systems, Human-Robot/System Interaction, Multi-Modal Perception
Abstract: In developing software to analyze the emotions of patients with schizophrenia using multimodal data, a pilot case study was conducted in order to examine its accuracy in a healthy subject. This study shows a low agreement and reliability between the MTCNN and the subjective evaluation of the three examiners based on the result of the ICC and Cronbach’s alpha coefficient. However, it can be revealed that the Multi-Task Cascaded Convolutional Networks (MTCNN) facial expression recognition and the Heart Rate Variability (HRV) analysis showed consistent results. Subject who experienced anticipated feelings of happiness when the conversation was focused in the subject’s favorite food, which showed an increased in HFnu, indicating increased parasympathetic activity. It was considered that the subject felt that the conversation with the robot was lively and empathetic. Findings suggest that MTCNN can be used in combination with HRV analysis in determining the facial expression of an individual. However, further research should be done involving additional subjects in order to ascertain the validity and reliability of the MTCNN.

Keywords: Human-Robot/System Interaction
Abstract: This study aimed to determine differences in healthcare professionals' perceptions of robots’ usefulness, challenges, cautions, ethical issues, and intermediary roles by the experience robot use. The questionnaire was distributed to 180 participants, and 151 responses (84% valid response rate) were analyzed, excluding participants who did not give consent or data with missing values. The participants were divided into two groups according to whether they had had the following experiences: 1) using a robot in their daily life, 2) seeing a robot being used while at work, or 3) using a robot while at work. Group comparisons were performed using Welch's t-test or Mann–Whitney U test, and p < 0.05 was used as the significance level in all calculations. Factor 2 (Usefulness of robots) scores were significantly higher for participants who reported having used robots in their daily lives. Factor 4 (Inconveniences of using robots) was significantly higher for participants who reported having used robots while at work. However, no significant differences were found for Factor 1 (The role of the intermediaries); Factor 3 (Ethical issues of concern in robot use); and Factor 5 (Dangers of robots). Findings from this study that robots are useful but currently used robots need intermediary assistance and improvement.

Keywords: Systems for Service/Assistive Applications, Human-Robot Cooperation/Collaboration, Integration Platform
Abstract: This study describes a long-sleeved shirt dressing support system, which mainly comprises two robotic manipulators, for people with half-body paralysis. To design a feasible support system, we obtain the opinions of occupational therapists and experimental psychologists. Based on their advices, we propose herein an appropriate dressing procedure that can be supported by robots. We also construct a robotic system to achieve the required assistance and implement some essential functions, including human pose recognition, end-effector trajectory generation for shirt sleeve manipulation, and arm motion support while dressing. Accordingly, we introduce admittance control for the arm motion support. This avoids the trouble of a person getting a large load while putting his/her arm through a shirt sleeve. We present the experimental results of an actual person wearing a long-sleeved shirt to show the usefulness of the proposed system.