Keywords: Cooperation and Collaboration in Human-Robot Teams, HRI and Collaboration in Manufacturing Environments, Computational Architectures
Abstract: Robots acting in real-world environments may interact with humans at different levels of abstraction (e.g., process, task, physical), entailing different control and coordination challenges. When acting in social situations, robots should be able to pursue (joint) goals by behaving according to the context as well as the skills/features of involved humans. Although reliable and effective, standard control techniques may limit the adaptability of robots. Novel control technologies based on Artificial Intelligence can endow robots with the cognitive capabilities needed to achieve a higher level of autonomy in terms of flexibility, reliability, and awareness. In this context, this paper introduces a goal-oriented acting framework based on timeline-based planning and execution. The framework is evaluated on a realistic Human-Robot Collaboration manufacturing scenario. Results show the capability of dealing with the uncontrollable dynamics of humans achieving effective and reliable collaborations.

Keywords: Social Intelligence for Robots, Creating Human-Robot Relationships, Applications of Social Robots
Abstract: Deploying service robots in our daily life, whether in restaurants, warehouses or hospitals, calls for the need to reason on the interactions happening in dense and dynamic scenes. In this paper, we present and benchmark three new approaches to model and predict multi-agent interactions in dense scenes, including the use of an intuitive qualitative representation. The proposed solutions take into account static and dynamic context to predict individual interactions. They exploit an input- and a temporal-attention mechanism, and are tested on medium and long-term time horizons. The first two approaches integrate different relations from the so-called Qualitative Trajectory Calculus (QTC) within a state-of-the-art deep neural network to create a symbol-driven neural architecture for predicting spatial interactions. The third approach implements a purely data-driven network for motion prediction, the output of which is post-processed to predict QTC spatial interactions. Experimental results on a popular robot dataset of challenging crowded scenarios show that the purely data-driven prediction approach generally outperforms the other two. The three approaches were further evaluated on a different but related human scenarios to assess their generalisation capability.

Keywords: Novel Interfaces and Interaction Modalities, HRI and Collaboration in Manufacturing Environments, Non-verbal Cues and Expressiveness
Abstract: This article presents an open-source architecture for conveying robots' intentions to human teammates using Mixed Reality and Head-Mounted Displays. The architecture has been developed focusing on its modularity and re-usability aspects. Both binaries and source code are available, enabling researchers and companies to adopt the proposed architecture as a standalone solution or to integrate it in more comprehensive implementations. Due to its scalability, the proposed architecture can be easily employed to develop shared Mixed Reality experiences involving multiple robots and human teammates in complex collaborative scenarios.

Keywords: Motion Planning and Navigation in Human-Centered Environments, Creating Human-Robot Relationships, Social Learning and Skill Acquisition Via Teaching and Imitation
Abstract: Autonomous service robots are increasingly necessary to move without impeding the movement of pedestrians. Previous studies have determined optimal input for robots by minimizing a multi-objective function that includes the cost of reaching the destination and avoiding surrounding pedestrians. However, it is challenging to adjust the weights of each term in the cost function since they depend on the users and environment. In this study, we used the Social Force Model (SFM) as the base cost function and proposed a method to estimate SFM weights preferred by general user based on population density from human feedback. To achieve this, first we use Bayesian Optimization and derive each user’s evaluation map of SFM in a virtual reality environment that provides a realistic and immersive experience for subjects to provide feedback on the robot's movement. Second, we aggregated each user's evaluation map to estimate a general user's evaluation map. Finally, we have derived a functional relationship between the preferred SFM weights of general users and population density by Gaussian process regression. This relationship empowers the robot to navigate in a manner preferred by the general public, contingent on population density, even in the absence of human feedback obtained through virtual reality experimentation.

Keywords: Detecting and Understanding Human Activity, Machine Learning and Adaptation, HRI and Collaboration in Manufacturing Environments
Abstract: This paper presents a new method to describe spatio-temporal relations between objects and hands, to recognize both interactions and activities within video demonstrations of manual tasks. The approach exploits Scene Graphs to extract key interaction features from image sequences while simultaneously encoding motion patterns and context. Additionally, the method introduces event-based automatic video segmentation and clustering, which allow for the grouping of similar events and detect if a monitored activity is executed correctly. The effectiveness of the approach was demonstrated in two multi-subject experiments, showing the ability to recognize and cluster hand-object and object-object interactions without prior knowledge of the activity, as well as matching the same activity performed by different subjects.

Keywords: Programming by Demonstration, Motion Planning and Navigation in Human-Centered Environments, User-centered Design of Robots
Abstract: Collaborative delivery robots in hospitals are required to move safely and efficiently in a short time, without colliding with people. Hence, they must consider the risk of people rushing out from blind spots or rooms, including field context such as the role and usage of the location. However, these factors are difficult to extract solely from geometric information. Therefore, we propose a method for generating a rush-out risk map considering the field context from the hospital staff’s operation data of an electric wheelchair. We convert the wheelchair’s speed operated by staff into rush-out risk, and then place rush-out risk potentials at positions where rush-outs may occur. Subsequently, we optimize the mapping position of rush-out risk and parameters of each potential to minimize the error to obtain a rush-out risk map. We collected actual staff operation data in the hospital and confirmed that we could generate a rush-out risk map with small errors.

Keywords: Child-Robot Interaction, Personalities for Robotic or Virtual Characters, Creating Human-Robot Relationships
Abstract: Social robots can support children in their socio-emotional development. To improve the cooperation between a child and a social robot, a good relationship is vital. Self-disclosure is an essential element for building personal relationships. Yet, knowledge about the effects of self-disclosure in child-robot interactions is still lacking. To investigate effects of robot persona, child personality, and self-disclosure category on self-disclosure in child-robot interaction, we have conducted a field study at a science festival in which children had a conversation with a robot that either behaved human-like or robot-like. The results show a significant difference in the amount of self-disclosure (in conversation duration) between the two robot personas. Significant relationships were found between conscientiousness and extravesion and amount of self-disclosure (in word count) as well. The participant disclosed significantly more about the category ‘attitudes and opinions’ than about ‘school’. Finally, a thematic analysis shows that the content of the conversations can be categorised in five plus one themes. Between robot personas, the content of the conversations did not differ in terms of conversation themes. However, in both conditions, we found that children generally immediately feel comfortable sharing unpleasant experiences about current themes (such as COVID) in a first encounter with a robot.

Keywords: Child-Robot Interaction, Robots in Education, Therapy and Rehabilitation, Applications of Social Robots
Abstract: Traffic safety education is the key to safe road behavior and can prevent severe and fatal accidents, and it is suggested starting from an early age. Technologies such as Augmented Reality (AR) and Socially Assistive Robots (SAR) have been successfully used in the last years in the educational field. Therefore, it is essential to identify how SAR and AR educational tools can be optimally used to support traffic safety education. In this paper, we introduce the idea of an educational approach with the combination of SAR with AR technologies to teach traffic safety in kindergarten. We have been inspired by evidence from human-robot interaction studies, suggesting both technologies can enhance gained knowledge using the storytelling method at an early age and trigger positive attitudes in the students after interacting with them. Our educational approach was benchmarked against a single AR application. Our study initially proved, through real-world experiments, that the combination of SAR and AR was successfully used to teach kindergarten students about traffic safety. It managed to sustain their attention for statistically significantly more time than the case of the single AR application. Students taught by the single AR application also managed to gain basic knowledge regarding traffic safety. Moreover, another important outcome of our study is that students improved their attitudes and beliefs regarding using SAR in education after interacting with it, as demonstrated by individual interviews. Hence, our results suggest that both AR applications and SAR are suitable for learning purposes in kindergarten, while the employment of SAR in AR educational applications sustains the students’ attention.

Keywords: Child-Robot Interaction, Embodiment, Empathy and Intersubjectivity, Robots in Education, Therapy and Rehabilitation
Abstract: Children naturally engage in pretend play with others in their daily life. The children share each other's expectations toward objects and people around them. Such expectation-sharing can be considered the fundamental process of social communication, where we exchange beliefs, desires, and intentions. Firstly, the present paper examines some cases of such expectation-sharing in pretend play observed in our longitudinal robot-mediated interaction with 27 preschoolers (3-to-4-year-olds) at a kindergarten. The robot, which the researchers in another room remotely controlled, has a simple appearance and motion to accept various expectations from the children. The robot functioned as a pivot for the children to exchange each other's expectations towards the robot, toys, and people. The children might bring expectations incongruent with each other, but such expectations gradually converged into a consistent one, on which they understood each other's verbal and non-verbal actions. Secondly, this paper interprets and models these cases using the predictive coding theory. According to the theory, the children project their predictions onto the shared environment and update the predictions by minimizing the error against reality. This act of projection, which was performed as labeling (e.g., a block as a rice ball) or pretend action (e.g., feeding it to the robot) by the children, is a form of active inference to modify the environment to match the prediction. The children exchanged and coordinated their predictions through visible action collectively accumulated in the playing environment around the robot. We conclude that joint prediction is the fundamental process of social communication, where we coordinate each other's behavior for collaboration.

Keywords: Child-Robot Interaction, Creating Human-Robot Relationships, Storytelling in HRI
Abstract: Many people buy social robots out of curiosity, but they don’t use them continuously due to technical limitations. Rapport, an emotional relationship between humans and social robots, can be thought as a way to overcome the limitations for continuous use. The purpose of this study is to examine how the identity of a social robot affects the rapport formation between children and a social robot. Two studies were conducted to verify rapport formation. One was an experiment in which a conversation was conducted with 8 children, and the other was an online survey with 45 children to verify their personal connection to a social robot. Two studies were conducted in an experimental group interacting with a social robot with an identity, and a control group interacting with a social robot without an identity. In the first experiment, there was no significant difference between the two groups. However, in the second online survey, significant differences were found between them. The experimental group felt stronger personal connection to the social robot than the control group, and this tendency was more conspicuous among girls than boys. This study is significant because it confirmed that the identity of a social robot can have a positive effect on children's rapport with social robots.

Keywords: Child-Robot Interaction, Robot Companions and Social Robots, Robots in Education, Therapy and Rehabilitation
Abstract: The present study applies a novel Reinforcement Learning-based (RL) alphabet learning system named QWriter for the acquisition of the Kazakh Latin alphabet. We conducted a between-subject design experiment with 108 Kazakh children aged 6-8 years old in a public school and compared their learning rates across the two conditions: an RL-based QWriter robot and a human tutor (HT) as a baseline. The results show that children learned significantly more letters with the HT compared to the QWriter robot, showing that the RL-based robot is not effective for learning in the short term. Yet, we observe some interesting results by children's age and gender. The results need further investigation comparing the QWriter with other robot baselines with different roles and across various learning tasks.

Keywords: Child-Robot Interaction, Assistive Robotics, Applications of Social Robots
Abstract: The research presented in this paper investigates the feasibility of using humanoid robots like Kaspar as assistive tools in Speech, Language and Communication (SLC) therapy for children with learning disabilities. The study aims to answer two research questions: RQ1. Can a social robot be used to improve SLC skills of children with learning disabilities? RQ2. What is the measurable impact of interacting with a humanoid robot on children with learning disability and SLC needs? A co-creation approach was followed, three therapeutic educational games were developed and implemented on the Kaspar robot in collaboration with experienced SLC experts. Twenty children from two different special educational needs schools participated in the games in 9 sessions over a period of 3 weeks. Results showed significant improvement in participants’ SLC skills – i.e. language comprehension and production skills – over the intervention. Findings of this research affirms feasibility, suggesting that this type of robotic interaction is the right path to follow to help the children improve their SLC skills.

Keywords: Human Factors and Ergonomics, Virtual and Augmented Tele-presence Environments, Novel Interfaces and Interaction Modalities
Abstract: Extending the human field of view could enhance our ability to perceive our surroundings thereby improving user safety and enabling us to perform complex manipulation tasks in industrial assembly processes. We investigated the augmentation of environment perception through a study that systematically measured the performance and perception arising from the use of a third eye placed on the back of the user’s head in virtual reality. 28 participants were asked to conduct two goal-oriented tasks: One requiring the identification of targets that discretely appeared in a set of predefined locations; the other the catching of a continuously moving target. The results show that participants were able to incorporate the additional visual information in real time resulting in changes of their motion behaviour. These changes led to localised improvements in task performance for the discrete target task and more efficient motion for both tasks. Participants also showed a strong preference for performing the tasks with the third eye and perceived no accompanying increase in cognitive load.

Keywords: Human Factors and Ergonomics, User-centered Design of Robots, Cognitive Skills and Mental Models
Abstract: Trust is an essential component in HRI, yet it has been impossible to agree on a definition within the HRI context. Moreover, the volume of definitions with different conceptualizations has led trust research to be viewed by some as a quagmire. Instead of attempting to define trust, this paper takes a bottom-up approach, starting with the body of current literature and breaking it into conceptual components within a trust system. Applying concepts such as abstraction and encapsulation from computer science, this paper attempts to synthesize the current body of literature into a metamodel where the components can either represent models themselves or natural groupings of elements. By viewing trust as a system (represented by a metamodel), it is possible to hide some of the details and functionality in each component without losing semantic value. This helps to clarify the trust workspace and terminology, highlights areas of trust research that are actively being researched, shows how different areas of research are connected, and captures the current state of trust research in a single framework.

From a practical perspective, using this model also provides trust researchers with a single reference point, simplifies scoping one's research, and enhances homogeneity by making cross-referencing and comparing study methods easier. Finally, viewing trust as a system allows capturing nuances definitions cannot, and may help identify gaps in current research.

Keywords: Human Factors and Ergonomics, Novel Interfaces and Interaction Modalities
Abstract: Augmented Reality (AR) enables the transmission of intent using the physical area in which humans and robots interact as a shared canvas. Studies exploring AR for human-robot collaboration have reported mixed findings on the relationship between cognitive workload and task performance. In this study, we developed an AR user interface (UI) that guides the user to perform a pick-and-place task while collaborating with a robot. A repeated measures mixed-methods study with sixteen participants demonstrated that AR UI significantly impacted task performance, where the users had longer travel distances to pick and place objects than the control group. Additionally, UI significantly impacted the cognitive load, where participants showed significantly higher pupil diameter and reported higher NASA-TLX scores while using AR UI. Finally, users reported significantly lower situational awareness and low usability scores while using AR UI. Our findings suggest that the AR UI negatively impacts human-robot collaboration, calling for further investigation.

Keywords: Motion Planning and Navigation in Human-Centered Environments, Human Factors and Ergonomics, Monitoring of Behaviour and Internal States of Humans
Abstract: Driver support systems that include human states in the support process is an active research field. Many recent approaches allow, for example, to sense the driver's drowsiness or awareness of the driving situation. However, so far, this rich information has not been utilized much for improving the effectiveness of support systems. In this paper, we therefore propose a warning system that uses human states in the form of driver errors and can warn users in some cases of upcoming risks several seconds earlier than the state of the art systems not considering human factors. The system consists of a behavior planner Risk Maps which directly changes its prediction of the surrounding driving situation based on the sensed driver errors. By checking if this driver's behavior plan is objectively safe, a more robust and foresighted driver warning is achieved. In different simulations of a dynamic lane change and intersection scenarios, we show how the driver's behavior plan can become unsafe, given the estimate of driver errors, and experimentally validate the advantages of considering human factors.

Keywords: Cooperation and Collaboration in Human-Robot Teams, Motion Planning and Navigation in Human-Centered Environments, HRI and Collaboration in Manufacturing Environments
Abstract: Close-proximity human-robot collaboration (HRC) requires an appropriate level of trust from the operator to the robot to maintain safety and efficiency. Maintaining an appropriate trust level during robot-aided production requires non-obstructive real-time human-robot trust assessment. To this end we performed an experiment with 20 participants performing two types of HRC tasks in close proximity to a Kuka KR 300 R2500 ultra robot. The two tasks involved collaborative transport of textiles and collaborative draping, respectively. During the experiment we performed full body motion tracking and administered human-robot trust questionnaires in order investigate the correlation between trust and operator movement patterns. From the initial per-session analyses we see the effects of task types on movement patterns, but the correlations with trust are weak overall. Further analysis at higher temporal resolution and with correction for participants' base movement patterns are required.

Keywords: Cooperation and Collaboration in Human-Robot Teams, Human Factors and Ergonomics
Abstract: Human-robot teams can leverage a human's expertise and a robot's computational power to meaningfully improve mission outcomes. In command and control domains, the robot teammate can also act as a decision-support system to advise human users. However, decision-support systems are susceptible to human factors issues including miscalibrated trust and degraded team performance. Recent work has mitigated these issues by using cognitive forcing functions to structure shared decision-making systems and place users as proactive on-the-loop actors. We bring this approach to a human-robot teaming domain, and investigate how Type I and Type II errors in the robot's recommendation affects team performance and user rational trust. We present the architecture of our decision-making process and a mars rover landing experiment domain. Results from a comprehensive user study demonstrate that the error type of the robot's recommendation forms a trade-off between team performance and rational trust.

Keywords: Cooperation and Collaboration in Human-Robot Teams, Social Intelligence for Robots
Abstract: It is essential for a collaborative robot to consider the Theory of Mind (ToM) when interacting with humans. Indeed, performing an action in the absence of another agent may create false beliefs like in the well-known Sally & Anne Task. The robot should be able to detect, react to, and even anticipate false beliefs of other agents with a detrimental impact on the task to achieve. Currently, ToM is mainly used to control the task execution and resolve in a reactive way the detrimental false beliefs. Some works introduce ToM at the planning level by considering distinct beliefs, and we are in this context. This work proposes an extension of an existing human-aware task planner and effectively allows the robot to anticipate a false human belief ensuring a smooth collaboration through an implicitly coordinated plan. First, we propose to capture the observability properties of the environment in the state description using two observability types and the notion of co-presence. They allow us to maintain distinct agent beliefs by reasoning directly on what agents can observe through specifically modeled Situation Assessment processes, instead of reasoning of action effects. Then, thanks to the better estimated human beliefs, we can predict if a false belief with adverse impact will occur. If that is the case then, first, the robot's plan can be to communicate minimally and proactively. Second, if this false belief is due to a non-observed robot action, the robot's plan can be to postpone this action until it can be observed by the human, avoiding the creation of the false belief. We implemented our new conceptual approach, discuss its effectiveness qualitatively, and show experimental results on three novel domains.

Keywords: Robotic Etiquette, Social Intelligence for Robots, Robot Companions and Social Robots
Abstract: This study aimed to investigate the effects of norm-violating behaviour on human perception and attitudes towards robots. Specifically, we examined the impact of a robot performing social norm violations in the context of a competitive scavenger hunt game. During the game, the robot was programmed to engage in predefined behaviours considered as social norm violations, including both injunctive and descriptive norm violations (e.g., cheating, and making loud noises). The study used an experimental and control group, with participants either exposed to norm-violating behaviour or not, respectively. The results indicated that participants in the experimental group had a strong awareness of the norm-violating behaviour according to self-reported assessments. Additionally, post-questionnaire results revealed a significant difference in trust, overall enjoyment, and discomfort between the two groups. These findings show that in our study, participants expected robots to abide by both types of social norms (i.e., injunctive, and descriptive) and that violations of them negatively impacted participants' perceptions and attitudes towards robots. This further emphasizes the importance of considering social norms in the design and programming of robots for human-robot interactions.

Keywords: Multi-modal Situation Awareness and Spatial Cognition, Applications of Social Robots, Creating Human-Robot Relationships
Abstract: Autonomous mobile robots are now being perceived as natural objects in real environments such as warehouses or restaurants. Naturally, These robots must consider the indeterminate behaviors of people, especially those unaware of their surroundings, such as those texting while walking (awareness). Although the risk of nonawareness has been studied extensively, much less work has been done on developing the pedestrian awareness model for an autonomous mobile robot. This study uses three factors: viewing angle, distance, and updating cycle to replicate the pedestrian awareness model mathematically. We collect data from five participants with high and low–workload texting tasks to identify their parameters. In this data collection, one pedestrian is walking freely in a 6–㎡ room with three walking experimenters. Meanwhile, they carry out a task that requires repeating a given sentence, and gazing information and head position are measured. In consequence, the distance probability of pedestrian awareness can be approximated by the sigmoid. Other parameters are estimated by the time ratio of awareness time to the overall time from the gazing information. Due to the difficulty of producing the generic parameters, we verified the proposed method using the calculated parameters of each participant. The simulation evaluation result indicates that our model leads to fewer errors than the simple social force model by comparing it with the collected walking trajectories. The result indicates that the proposed model could reproduce nonawareness pedestrians with higher accuracy, and our proposed model contributes to the mobile robot and the more realistic pedestrian simulation.

Keywords: Multi-modal Situation Awareness and Spatial Cognition, Machine Learning and Adaptation
Abstract: Socially Assistive Robots are foreseen as having the potential to improve the quality of life of older adults and individuals with mental disabilities. Natural human-robot interaction in everyday settings may require robots that are capable of understanding what is happening in their operating environments so that they can respond appropriately to the experienced situations and engage people in meaningful ways. This paper presents an approach using perceptual constructs to represent what is being observed by the robot. Perceptual constructs are derived from deep neural networks used to process visual and audio data. The objective is to derive a condensed representation of the interactions observed by the robot in real-life settings. Results are provided from observations made by a robot of a room with human activity over a two-week period, outlining what works and remaining challenges in video and audio processing.

Keywords: Social Learning and Skill Acquisition Via Teaching and Imitation, Human Factors and Ergonomics, Social Intelligence for Robots
Abstract: Robots deployed in social communities must act according to the communities' social and moral norms. To acquire the large number of nuanced norms, robots can rely on human teaching. While humans tend to naturally use more than one teaching method when training a novice, current human-in-the-loop teaching frameworks have typically relied on single teaching methods (e.g., instruction or reward). To gain insight into how humans would teach robots to master social and moral norms, we present a novel paradigm in which participants interactively teach a simulated robot to behave appropriately in a healthcare setting, choosing to either instruct the robot or evaluate its proposed actions. We demonstrate that 89.5% of human teachers naturally use both teaching methods. Importantly, they dynamically change their teaching method as they observe the robot's task performance, reacting both to the task's difficulty, the robot's most recent action, and the accumulated evidence of the robot's learning progress.

Keywords: User-centered Design of Robots, Human Factors and Ergonomics, Degrees of Autonomy and Teleoperation
Abstract: Wilderness search and rescue teams face challenges in hazardous environments. While robots show promise for these teams, their success depends on their ability to account for sociotechnical considerations, including human factors, as well as the organizational, economic, and emotional realities of search missions. We investigate these considerations through interviews with wilderness search team members who handle search dogs. These interviews reveal underexplored perspectives on awareness and uncertainty, the value of training experiences, team dynamics, and financial feasibility. Our findings motivate design recommendations for semiautonomous robots in the wilderness, yet also raise key questions regarding the role that robots can and should play in this domain.

Keywords: Virtual and Augmented Tele-presence Environments, HRI and Collaboration in Manufacturing Environments, Computational Architectures
Abstract: This study introduces a novel proxemic-aware augmented reality (AR) system to mitigate information overload in AR-enabled human-robot interaction (HRI). The system leverages human-robot proxemics to automatically adjust what and how much visual content needs to be presented. Therefore, the operator can perceive the relevant data through AR interfaces without being overwhelmed by excessive information exposure. We propose a task-specific model for evaluating human-robot proxemic (HRP), where the system can identify HRP levels based on raw features, such as distance and orientation. Based on HRP levels, we design a set of visual elements for presenting robots' information at various levels of detail. To demonstrate the functionality of the system, we present a series of proof-of-concept applications showing that our system can assist the operator in a wide range of HRI tasks. The user study proves that the proxemic-aware AR system can reduce mental loading, increase visual clarity, and improve interaction efficiency in HRI.

Keywords: Virtual and Augmented Tele-presence Environments, Human Factors and Ergonomics, Anthropomorphic Robots and Virtual Humans
Abstract: Telexistence technology enables the manipulation of a remote robot using sensory feedback and leader-follower control, giving the operator the impression of controlling their own body. However, the robot's operability declines significantly due to increasing communication time delays. In order to address this issue, the "Predictive Display" technique has been proposed and developed. This technique displays both the real robot's movements with time delay and the computer-generated (CG) robot's movements without time delay. In the Telexistence robot operation system, a predictive display system can be constructed using a virtual reality (VR) system. In this study, we have developed a VR system for Telexistence operation with THK's SEED-Noid humanoid robot. Unity is utilized, which enables the predictive display function for the stereo camera mounted on the SEED-Noid. We developed and evaluated a display mode without neck joint angle restrictions to reduce VR sickness and workload using the developed VR system. Additionally, we developed a display mode with a limited neck joint angle for comparison purposes.

Keywords: Social Presence for Robots and Virtual Humans, Robot Companions and Social Robots, Cognitive Skills and Mental Models
Abstract: Human interactions with robot groups are more complex than interactions with individual robots. This is especially true for groups of robots that do not have humanlike 1-1 associations between bodies and identities, such as when multiple robots share a single identity. This is further complicated by the lack of direct observability of the relationship between body and identity, which may be inferred by users on the basis of various robot group identity performance strategies. Previous research on textit{Deconstructed Trustee Theory} has argued that this complexity is critical, as different perceived body-identity configurations may lead users to build and develop trust in distinct ways. In this paper, we thus investigate (n=94) the ways that different robot group identity performance strategies might influence the distribution of trust amongst robot group members, as well as the impact of these strategies on perceptions of robot group entitativity.

Keywords: Virtual and Augmented Tele-presence Environments
Abstract: Body ownership, which is the feeling that one's body part belongs to oneself, and agency, which is the sense of being able to control one’s own body part, can be felt towards fake body parts and those depicted by computer graphics. As part of an attempt to transfer self-body awareness to fake body segments, we investigated whether body ownership and agency are felt towards two visible left hands in an immersive virtual reality environment. One of the two hands shown through virtual reality goggles spatially matched the unseen actual hand. The other hand was fake and displayed at either the 10-cm lateral or medial side of the position of the actual hand. These two left hands moved synchronously with the actual left hand. Participants completed a behavioral test and questionnaire after adapting to the two left hands. In the behavioral test, participants accessed randomly emerging spheres using the seen hands as fast as possible. They used the fake hand to touch a sphere when it appeared near the fake hand 41% of the time when the fake hand was displayed at the medial position of the actual hand. The results of the questionnaire suggest that agency was experienced for the two visible left hands for this condition. By contrast, body ownership was felt mostly against the displayed hand that was spatially consistent with the actual hand. These findings indicate that although agency can be simultaneously felt for two seen left hands, body ownership is felt only for either of the two visible left hands.

Keywords: Virtual and Augmented Tele-presence Environments, HRI and Collaboration in Manufacturing Environments
Abstract: The concept of a Digital Twin has proved its worth over the past two decades, establishing itself as a cornerstone of contemporary industry. Augmented Reality, an emerging technology, enhances the interaction between humans and machines, including computers and robots. Today, numerous examples exist of the union of these two technologies to create real-augmented digital-twin models of collaborative robots. However, these models often lack data on motor currents and power consumption. In this study, we propose a real-augmented digital-twin model that accurately estimates energy consumption. This additional energy information equips the tool for various applications such as robot optimization, commissioning, and troubleshooting. We employ our real-augmented digital-twin model to test methods for reducing Cobots' energy consumption, using the tool to demonstrate and train Cobot practitioners on these techniques' applications. The model is also useful for anomaly detection (troubleshooting) when the robot's consumption statistically deviates from the ideal model. Moreover, the model can anticipate the robot's power consumption during the commissioning phase, prior to its installation. Through a series of experiments and a practical demonstration at a robot fair for practitioners, we illustrate the benefits and training capabilities of our approach.

Keywords: Degrees of Autonomy and Teleoperation
Abstract: The utilization of teleoperation is a crucial aspect of the construction industry, as it enables operators to control machines safely from a distance. However, remote operation of these machines at a joint level using individual joysticks necessitates extensive training for operators to achieve proficiency due to their multiple degrees of freedom. Additionally, verifying the machine's resulting motion is only possible after execution, making optimal control challenging. In addressing this issue, this study proposes a reinforcement learning-based approach to optimize task performance. The control policy acquired through learning is used to provide instructions on efficiently controlling and coordinating multiple joints. To evaluate the effectiveness of the proposed framework, a user study is conducted with a Brokk 170 construction machine by assessing its performance in a typical construction task involving inserting a chisel into a borehole. The effectiveness of the proposed framework is evaluated by comparing the performance of participants in the presence and absence of virtual fixtures. This study's results demonstrate the proposed framework's potential in enhancing the teleoperation process in the construction industry.

Keywords: Personalities for Robotic or Virtual Characters, Non-verbal Cues and Expressiveness, Multimodal Interaction and Conversational Skills
Abstract: The ability to impact the attitudes and behaviours of others is a key aspect of human-human interaction. The same capability is a desideratum in human-robot interaction, when it can have an impact on healthy behaviours. The robot's interaction style plays a significant role in achieving effective communication, leading to better outcomes, improved user experience, and overall enhanced robot performance. Nonetheless, little is known about how different robots' communication styles impact users' performance and decision-making. In this article, we build upon previous work, in which a robot was endowed with two personality behavioural patterns: one more antagonist and other-comparative and the other one more agreeable and self-comparative. We conducted a user study where N=66 participants played a game with a robot displaying the two multimodal communication styles. Our results indicated that i) participants' decision-making was not influenced by the designed robot's communication styles, ii) participants who interacted with the agreeable robot performed better in the game, and iii) the more participants are knowledgeable about robots, the lower they performed in the game.

Keywords: Cognitive Skills and Mental Models, Social Intelligence for Robots, Applications of Social Robots
Abstract: In the past few years, human-robot deception has been receiving growing attention in several fields (e.g., human-robot interaction, laws, philosophy, and psychology). While deception both in human-human and human-robot interactions may have positive consequences, it still presents philosophical and psychological controversy. In particular, verbal deceptions (i.e., in the form of lies or misleading information) may be judged as intentional behaviour at times. While intentionality has been recognised as fundamental in the development of trust, it is not yet fully clear which mechanisms can be designed to foster trust and the potential issues connected to deception. To this extent, in this study, we investigate whether the ability of mentalizing may be one of such mechanisms. We conducted a user study during a public fair, where participants played an assistive game with a robot endowed with Theory of Mind (ToM). We collected the responses from 37 participants to evaluate their perception of trust in the robot. During the game, the robot may occasionally have deceptive behaviours suggesting the wrong move to the human players. Our results showed that a deceptive robot was less trusted compared to a non-deceiving one. We also found that people's perception of the robot was positively affected by the frequency of exposure to deception (i.e., wrong suggestions).

Keywords: Ethical Issues in Human-robot Interaction Research, Philosophical Issues in Human-Robot Coexistence, Robotic Etiquette
Abstract: Failure as a concept can refer both to a lack of objective success and a perceived lack of success in living up to others’ expectations, requirements, or standards. Both kinds of failure tend to be seen as undesirable, and when an entity fails in some way, this has effects on how the entity is evaluated by those it interacts with. But failure is not all bad. Since it is human to err, erring can also potentially foster the perception of human-like qualities in non-humans. This allows for a discussion of strategic robot failure, which entails intentionally designing robots that are perceived as failing (by the human), while they are actually successful in achieving the (hidden) objectives of their designer. Such design strategies involve the use of deception to shape, for example, humans’ trust in robots, to engender effective human-robot interaction (HRI). This article begins with a brief description of research on failure in HRI, with an emphasis on understanding the implications of robot failure for human trust and reliance in the robots. I then turn to the concept of failure and distinguish between an objective component (lack of success) and the subjective component (failure as not meeting the others’ requirements or standards). This makes failure a relational concept that can only be fully understood through context and knowledge of the preferences, values, and expectations of the human in HRI. Through these considerations, I conclude by discussing the potential positive and negative implications of strategic robot failure, with a closing discussing of potential ethical objections to strategic robot failure.

Keywords: Creating Human-Robot Relationships, Non-verbal Cues and Expressiveness, Multimodal Interaction and Conversational Skills
Abstract: Trust repair strategies are an important part of human-robot interaction. In this study, we investigate how repeated failures impact users’ trust and how we might mitigate them. Specifically, we look at different repair strategies in the form of apologies, with additional features to them such as warnings and promises. Through an online study, we explore these repair strategies for repeated failures in the form of robot incongruence, where there is a mismatch of verbal and non-verbal information given by the robot. Our results show that such incongruent robot behaviour has a significant overall negative impact on participants’ trust. We found that the robot making a promise, and then breaking it, results in a significant decrease in participants’ trust, when compared to a general apology as a repair strategy. These findings contribute to the research on trust repair strategies and, additionally, shed light on how robot failures, in the form of incongruences, impact participants’ trust.

Keywords: Linguistic Communication and Dialogue, Creating Human-Robot Relationships, Robot Companions and Social Robots
Abstract: The paper discusses two current approaches to conversational AI, using large language models and knowledge graphs, and compares types of errors that occur in human-robot interactions based on these approaches. It provides example dialogues and describes solutions to several error types including false implications, ontological errors, theory of mind errors, and handling of speech recognition errors. The paper addresses issues of particular concern for earning user trust.

Keywords: Child-Robot Interaction
Abstract: Robots are being developed to help in various settings with young children. However, research suggests that children may overtrust robots, which can have a negative impact when such trust is unwanted or unsafe. Based on recent results from the community we suggest designing robots to use intentional errors to potentially reduce children’s trust in robots and to mitigate overtrust. We present a breakdown of robot errors that might affect children’s trust towards robots, and which could be used intentionally to mitigate overtrust. This includes accuracy errors, responsiveness errors, and error recovery strategies. We highlight how they could be used to decrease trust. We lastly provide an agenda for researchers to further investigate how the intentional use of robot errors could help to mitigate children’s overtrust in robots.

Keywords: Child-Robot Interaction, Assistive Robotics, Applications of Social Robots
Abstract: This paper presents an investigation into the effectiveness of introducing explicit causal explanations in a child-robot interaction setting to help children with autism improve their Visual Perspective Taking (VPT) skills. A sample of ten children participated in three sessions with a social robot on different days, during which they played several games consisting of VPT tasks. In some of the sessions, the robot provided constructive feedback to the children by giving causal explanations related to VPT; other sessions were control sessions without explanations. An analysis of the children’s learning progress revealed that they improved their VPT abilities faster when the robot provided causal explanations. However, both groups ultimately reach a similar ratio of correct answers in later sessions. These findings suggest that providing causal explanations using a social robot can be effective to teach VPT to children with autism. This study paves the way for further exploring a robot’s ability to provide causal explanations in other educational scenarios.

Keywords: Child-Robot Interaction, Robots in Education, Therapy and Rehabilitation, Evaluation Methods
Abstract: Social robots are expected to become more and more frequently used in the education field. However, in the interaction between children and social robots, it is still under investigation how robots are perceived in social contexts such as education. In this exploratory study, we aimed to investigate how the expectations and demographical characteristics of children (textit{N}= 53, 9-14 years old) influence their perception of robot NAO during an education training program in schools. MANCOVA analysis conducted over questionnaire data indicates a positive correlation between the acceptance of the robot and the entertainment in interacting with it. We found evidence that the more students accepted the robot the more they perceived the group environment positively. Through a Correspondence Analysis, we investigate which are the preferred features of a robot according to the age of participants. The study suggests that a better opinion of robotics is a factor that can improve the learning environment in this specific context. Our exploratory study encourages the chance of conducting studies in-the-wild using self-reported measures to better understand the implication of Child-Robot Interaction.

Keywords: Child-Robot Interaction, Robots in Education, Therapy and Rehabilitation, Monitoring of Behaviour and Internal States of Humans
Abstract: Stress-related health problems in children have increased in recent years, resulting in a significant negative physical and mental impact on children's daily lives. This systematic review explores the potential of embodied technologies, such as robots, smart wearables, and the Internet of Things (IoT), as tools for managing stress in children. The goal of this systematic review is to identify the design opportunities of embodied technologies in stress management, by looking for answers in terms of different technologies, users, issues and challenges addressed in selected 91 papers. Through the frequency and thematic analysis we identified six main challenges and eight design opportunities for embodied technologies. Where there are gaps and opportunities in research, we propose to focus on connectivity and active sensing through connected objects, through exploring the potential of the Internet of Robotic Things (IoRT) as an M-health solution for providing real-time and personalized stress detection and interventions for children in various daily life settings.

Keywords: Child-Robot Interaction, Detecting and Understanding Human Activity, Monitoring of Behaviour and Internal States of Humans
Abstract: Haru4Kids (H4K) is a system that emulates the physical, social, family-oriented robot Haru, designed with the goal to cohabitate with children in their home for extended periods of time. Seven families kept H4K for two net weeks in their homes. Throughout this period of cohabitation, we collected user logs comprised of the children users' head angles, the rotation angles of the platform, and the actions taken by H4K as well as captured images which were afterwards hand-annotated to estimate user engagement. We report the trends of these external metrics that we collected during every session of interaction. We also developed an annotation tool and report the Engagement Level Metric we chose to estimate child engagement throughout interactions "in-the-wild." Overall, our platform offers a feasible system that can engage with children while also allowing us to monitor their engagement and behaviour throughout each interaction.

Keywords: Child-Robot Interaction, Linguistic Communication and Dialogue, Multimodal Interaction and Conversational Skills
Abstract: Child-robot interaction (CRI) has been mostly studied in labs and classroom settings. In this work, we share a CRI language processing study carried out in children’s homes. Any automated system deployed “in-the-wild” faces practical problems, but when the target users are children, these problems get even more sensitive and challenging. In this work we analyse how each language processing layer performs with children at home with no researcher present. We carried out an experiment with 7 families [N=14 children, 6-13 years old] cohabiting with a simulated robot for 2 weeks in their own homes. Our goal in this study is to evaluate the performance of voice recognition, language understanding and dialogue management when children interact with a robot at home. Our results indicate that dialogue management capabilities are becoming the key element in the language processing pipeline; they also denote that the dialogue engine should include mixed-initiative capabilities and show the relative usage of different common built-in intents

Keywords: Child-Robot Interaction, Applications of Social Robots, Creating Human-Robot Relationships
Abstract: With the introduction of socially assistive robots in many avenues of children’s lives, it is becoming increasingly vital to understand how children’s perceptions of the robot affect their evaluation and interaction. The main objective of this work is to investigate how children’s anxiety towards robots has influenced their perceptions of their interaction with a Nao robot. We collected data from 37 children (8 - 13 years old) who interacted, for about 30-45 minutes, with the robot which delivered initial pleasantries and four different tasks to help assess their mental wellbeing in a lab setting. We collected audio-visual recordings of the interaction. At the end of the session, we asked children to answer three self-report questionnaires to evaluate: the robot’s role as a confidante, the anxiety towards the robot, and the children’s perception of the robot’s behaviour and capabilities. Based on their responses to the robot’s anxiety questionnaire, children were divided into two categories: “low anxiety” (anxiety score <= median anxiety score) and “high anxiety” (anxiety score> median anxiety score). Our results show that i) most children (89.2%) irrespective of their wellbeing, experience some degree of anxiety towards the robot, ii) children’s anxiety has influenced their willingness to participate in the initial pleasantries conducted by the robot, and iii) children’s anxiety has also affected their evaluations of the robot as a confidante and their perceptions of the robot’s behaviour and capabilities. Findings from this work have significant implications for designing effective and successful robot-led initiatives for assessing mental wellbeing in children, by taking into account their mindsets and dispositions.

Keywords: Child-Robot Interaction, User-centered Design of Robots, Applications of Social Robots
Abstract: As child-robot interactions become more and more common in daily life environment, it's important to examine how robot's errors may influence children's behavior. We explored how a robot's unexpected behaviors influence child-robot interactions during two workshops on active reading in a modern art museum and in a school. We observed the behavior and attitudes of 42 children from three age groups: 6-7 years, 8-10 years, and 10-12 years. Through our observations, we identified six different types of surprising robot behaviors: personality, movement malfunctions, inconsistent behavior, mispronunciation, delays, and freezing. Using a qualitative analysis, we examined how children responded to each type of behavior, and we observed similarities and differences between the age groups. Based on our findings, we propose guidelines for designing age-appropriate learning interactions with social robots.

Keywords: Child-Robot Interaction, Robots in Education, Therapy and Rehabilitation, Robot Companions and Social Robots
Abstract: Could robotic furnishings become reading companions for children? This paper investigates how a socially interactive robotic bookshelf may influence children’s enthusiasm, persistence, and enjoyment in reading when they are absorbed in other, usually more addictive activities such as playing iPad games or toys. We designed and developed a low-fidelity, elephant-like robotic bookshelf, and used WoZ technique to conduct an experiment in a public library in Singapore with 7 child-parent pairs to assess the effectiveness of this socially interactive, mobile robotic bookshelf in attracting children to read physical books while iPad games and toys were present. The results indicate that the robotic bookshelf was successful in attracting 5 out of 7 children to read books, with variations observed in the timing of switching to book reading from playing iPad games or toys. Currently, most robot reading companions are tabletop robots passively waiting for children to come to the robots and read with them. This study, however, highlights a unique approach in which a mobile, robotic bookshelf reading companion proactively engages children in reading by bringing physical books to them. In a modern world where screen and other addictions become increasingly real for children, such mobile robotic bookshelves have great potential to benefit children’s literacy and overall development.

Keywords: Cooperation and Collaboration in Human-Robot Teams, Human Factors and Ergonomics, Machine Learning and Adaptation
Abstract: This paper presents development and validation of a robot adaptation model to support operators in Human-Robot Collaborative tasks when they are cognitively fatigued. A human-centered robot adaptation method for providing appropriate assistance to the operator is developed with a dual objective of task performance optimization and aiding human cognitive fatigue recovery. The problem is formulated as a Markov Decision Process (MDP) and solved using Q-learning. The implementation issues resulting from modeling the MDP and performing Q-learning for cognitive fatigue recovery are discussed, together with methods to mitigate those issues and implications on the resulting optimal policies. The validity of the proposed approach is evaluated through a user study of sixteen participants performing a robotic surface polishing task under cognitive fatigue conditions. The MDP model is validated using subjective metrics, i.e., fatigue perception surveys, and objective metrics, i.e., Heart-Rate Variability (HRV), accuracy in trajectory tracking, and time efficiency of the task. Fatigue perceptions, accuracy, and time efficiency improved during the user-specific optimal adaptation policies. HRV analysis of time-domain features shows an overall improvement in fatigue conditions during the optimal adaptation policies. The results from this approach indicate that such human-centered robot adaptation can lead to efficient human-robot collaborations with robust interactions between robots and humans.

Keywords: Human Factors and Ergonomics, Child-Robot Interaction, Robots in Education, Therapy and Rehabilitation
Abstract: Handwriting practising, as any other repetitive task, often leads the practiser to an overconcentration state where their performance might be affected by postural and mental fatigue. Short breaks to perform unrelated activities, especially relaxation exercises, have shown to be a simple alternative to soften or postpone this phenomenon. Therefore, in this paper we are investigating the immediate effects of different types of short-duration relaxation exercises in the handwriting and posture qualities of children aged from 8 to 10 in handwriting training. We divided 40 children in two groups performing the sessions, guided by a social robot, with small exercises of mindfulness or stretching in the middle of their training. Additionally, we analysed participants' perceptions towards the robot leading these interactions. Results showed improvements in participants' handwriting quality and posture maintenance regardless of the condition. Additionally, more positive feedback about the pause was reported from individuals in the mindfulness condition.

Keywords: Human Factors and Ergonomics, Ethical Issues in Human-robot Interaction Research, User-centered Design of Robots
Abstract: For the aim at exploring factors determining negative attitudes toward robots, the study conducted an online questionnaire survey for a total of five hundreds of persons varying from 20’s to 60’s. The measurements consisted of the Japanese version of the Negative Attitudes toward Robots Scale, Critical Thinking Attitude Scale, and questionnaire items related to conservative attitudes and tendencies of preserving the status quo. The analysis results suggested that critical thinking attitudes, conservative attitudes, and tendencies of preserving the status quo affected negative attitudes toward robots, and this affection differed dependent on gender.

Keywords: Detecting and Understanding Human Activity, Evaluation Methods, Social Intelligence for Robots
Abstract: Appearance-based methods are a promising solution for gaze estimation, as they eliminate the need for additional devices and calibration. This makes them particularly well-suited for human-robot interaction (HRI) research. However, until recently their performance was under par compared to traditional eye-trackers. Recent breakthroughs have been made with the release of two large-scale datasets with a wide range of gaze directions (Gaze360 and ETH-XGaze) and the accompanying state-of-the-art deep neural networks (L2CS and ETH). In this paper, we systematically evaluate the performance of these two appearance-based models on a social robot. In our setup, we vary the distance from the robot (1-3 m) and camera resolution (640*480 and 3840*2160) and analyze the performance in terms of accuracy and precision. We find that the L2CS model trained on the Gaze360 dataset combined with a 4K camera achieves the best performance on the 2 m and 3 m distances. We show that a simple offset correction on pitch and yaw can further increase the accuracy and precision by 18.6% and 9.6% respectively. We conclude that for a range up to 3 m appearance-based gaze estimation models provide a promising approach for application in HRI research.

Keywords: Cooperation and Collaboration in Human-Robot Teams
Abstract: Despite the advantages of having robot swarms, human supervision is required for real-world applications. The performance of the human-swarm system depends on several factors including the data availability for the human operators. In this paper, we study the human factors aspect of the human-swarm interaction and investigate how having access to high-quality data can affect the performance of the human-swarm system--- the number of tasks completed and the human trust level in operation. We designed an experiment where a human operator is tasked to operate a swarm to identify casualties in an area within a given time period. One group of operators had the option to request high-quality pictures while the other group had to base their decision on the available low-quality images. We performed a user study with 120 participants and recorded their success rate (directly logged via the simulation platform) as well as their workload and trust level (measured through a questionnaire after completing a human-swarm scenario). The findings from our study indicated that the group granted access to high-quality data exhibited an increased workload and placed greater trust in the swarm, thus confirming our initial hypothesis. However, we also found that the number of accurately identified casualties did not significantly vary between the two groups, suggesting that data quality had no impact on the successful completion of tasks.

Keywords: Programming by Demonstration, HRI and Collaboration in Manufacturing Environments, Multimodal Interaction and Conversational Skills
Abstract: The programming of industrial robots necessitates specialized expertise and significant time and effort, particularly for small batch sizes. However, with the increasing demand for agility in production, the solutions used for robot programming have evolved significantly. Intuitive robot programming systems based on diverse concepts have been introduced to facilitate rapid deployment of robot systems. One such approved concept is no-code robot programming with finger-based gesture. In this concept, non-expert users draw a robot path via finger movement, which is subsequently translated into robot programming language to facilitate the corresponding movement. A significant challenge associated with this method is the valid replication of the corresponding robot Tool Center Point (TCP) orientation. Reachability issues, non-compliant hand contortions, and sensor occlusions make it difficult to directly derive the robot's TCP orientation from the finger's orientation.

This work presents two novel approaches for estimating robot orientation using numerical analysis and point cloud information for finger-based robot programming without requiring prior knowledge. The first approach utilizes numerical analysis to estimate the relative robot orientation based on the geometry of the trajectory. In contrast, the second approach uses point-cloud to derive the robot orientation based on the object contour. Input shaping algorithms are employed and evaluated to reduce the divergence in the orientation estimations. Experiments demonstrate the effectiveness of the proposed approach utilizing a low-cost camera as a cost-efficient alternative to existing no-code programming strategies, potentially accelerating the real-world deployment of robotic applications in industrial environments.

Keywords: Human Factors and Ergonomics, Degrees of Autonomy and Teleoperation, User-centered Design of Robots
Abstract: In this position paper, results from a video-based study on the influence of prior information given to users and explanations situationally given by the vehicle itself on trust and perceived intelligence are presented using a simulated autonomous vehicle in an ambiguous driving situation. A 2x2 between-subjects design is chosen with two independent variables ‘prior information’ (extended/short) and ‘explanations’ (yes/no) with users pseudo-randomly assigned to one of the four conditions. Significant results from 189 test persons reveal, that trust de-pends on how the capabilities of the intelligent vehicle are explained a priori and not on situational explanations, while perceived intelligence is influenced by both variables. Additional interactions of prior information and user gender is noted with respect to perceived intelligence. As one side effect, it is found, that male users felt significantly more safe than female users with also higher ratings of intention to use the vehicle independently of given information and explanations. Another side effect is that situational explanations lead to better ratings of subjective performance, while also here a significant interaction of gender and prior information is noted. Thus, contrary to expectations, a dominant role of continuous situational explanations (Explainable AI) of the intelligent vehicle for increasing trust is not confirmed and the extent of given prior information seems the deciding factor for initial trust building, which is an important aspect for the introduction of new intelligent technology into society. This is remarkable as at the same time perceived intelligence seems to be dependent on both variables. So, a vehicle able to explain itself may appear more intelligent, but not necessarily trustworthy.

Keywords: Detecting and Understanding Human Activity, Assistive Robotics, Machine Learning and Adaptation
Abstract: The article proposes a method for classifying human postures using an improved probabilistic neural network (PNN) with different distance measures and different probabilistic distribution functions (PDF). We found that the PNN with angular distance provides better accuracy, precision, and recall for the postures classification tasks than the PNN with conventional Euclidean distance. The k Nearest Neighbors (kNN) method gives slightly better prediction results than PNN, but our PNN is much faster. Such good computational performance is beneficial for posture recognition tasks that require real-time functions. An example is the needs of co-bots or service robots. The article also proposes a method for selecting the distribution smoothing parameter (σ) using the sub-optimization process based on the improved Gray Wolf Optimization (I-GWO) algorithm. It was found that the impact of PDF differences on the quality of the results can be reduced by choosing the best possible σ. In order to evaluate the developed method, selected human activities were recorded. The datasets were created using two different RGB-D systems located in two different laboratories.

Keywords: Machine Learning and Adaptation, Computational Architectures, Social Intelligence for Robots
Abstract: From learning assistance to companionship, socially aware and socially assistive robotics is promising for enhancing many aspects of daily life. However, socially aware robots face many challenges preventing their widespread public adoption. Two such major challenges are (1) lack in behavior adaptation to new environments, contexts and users, and (2) insufficient capability for privacy protection. The commonly employed emph{centralized learning} paradigm, whereby training data is gathered and centralized in a single location (i.e., machine / server) and the centralized entity trains and hosts the model, contributes to these limitations by preventing online learning of new experiences and requiring storage of privacy-sensitive data. In this work, we propose a emph{decentralized learning} paradigm that aims to improve the personalization capability of social robots while also paving the way towards privacy preservation. First, we present a new framework by capitalising on two machine learning approaches, Federated Learning and Continual Learning, to capture interaction dynamics distributed physically across robots and temporally across repeated robot encounters. Second, we introduce four criteria (adaptation quality, adaptation time, knowledge sharing, and model overhead) that should be balanced within our decentralized robot learning framework. Third, we develop a new algorithm -- Elastic Transfer -- that leverages importance-based regularization to preserve relevant parameters across robots and interactions with multiple humans (users). We show that decentralized learning is a viable alternative to centralized learning in a proof-of-concept Socially-Aware Navigation domain, and demonstrate the efficacy of Elastic Transfer across our proposed evaluation criteria.

Keywords: Machine Learning and Adaptation
Abstract: To control the lower-limb exoskeleton robot effectively, it is essential to accurately recognize user status and environmental conditions. Previous studies have typically addressed these recognition challenges through independent models for each task, resulting in an inefficient model development process. In this study, we propose a Multitask learning approach that can address multiple recognition challenges simultaneously. This approach can enhance data efficiency by enabling knowledge sharing between each recognition model. We demonstrate the effectiveness of this approach using Gait phase recognition (GPR) and Terrain classification (TC) as examples, the most conventional recognition tasks in lower-limb exoskeleton robots. We first created a high-performing GPR model that achieved a Root mean square error (RMSE) value of 2.345 ± 0.08% and then utilized its knowledge-sharing backbone feature network to learn a TC model with an extremely limited dataset. Using a limited dataset for the TC model allows us to validate the data efficiency of our proposed Multitask learning approach. We compared the accuracy of the proposed TC model against other TC baseline models. The proposed model achieved 99.5 ± 0.044% accuracy with a limited dataset, outperforming other baseline models, demonstrating its effectiveness in terms of data efficiency. Future research will focus on extending the Multitask learning framework to encompass additional recognition tasks.

Keywords: Cooperation and Collaboration in Human-Robot Teams, Machine Learning and Adaptation, Detecting and Understanding Human Activity
Abstract: Technologies in machine learning and artificial intelligence have come a long way in decision making and system automation, but still faces difficult challenges in semi-automation and human-in-the-loop frameworks. This work presents a probabilistic policy blending approach for shared control between a human operator and an intelligent agent. The proposed approach assumes that the agent can control a system and the human operator needs to communicate the system's intended goal. A comparative study is presented between different arbitration functions that are used to blend the human and agent's actions. The proposed approach can achieve a variable level of assistance to the human operator successfully within discrete action space using the Lunar Lander game environment developed by OpenAI. Furthermore, human physiological data have been analyzed while the human interacts with the system and the agent using different arbitration functions. A correlation between the physiological data, arbitration level, and task performance was observed.

Keywords: Machine Learning and Adaptation, Interaction Kinesics, Evaluation Methods
Abstract: When deploying a robot to real-world environments, it is crucial to execute tasks amidst constantly changing surroundings. The conventional kinematics control of robot arms is primarily reliant on the inverse kinematics model. Unfortunately, due to the lack of adaptability, high-precision control models often falter when the robot utilizes tools of varying lengths or when the robot arm is worn out. This work aims to address this issue by proposing a meta-learning-based control framework. We achieve rapid and seamless online adaptation by updating control models when the robot arm's configuration changes. The control framework comprises an Adaptive Global Inverse Model (Adaptive GIM) and an Adaptive Local Inverse Model (Adaptive LIM). The Adaptive GIM employs configuration-independent meta-learning, which allows the control model to swiftly adapt to different arm configurations. The Adaptive LIM adopts a meta-learning approach for location-independent training, enabling the robot to adapt to diverse local positions. As the Adaptive GIM suffers from the adverse effects stemming from the multiple solutions of inverse kinematics, utilizing the Adaptive LIM with relative position as input can alleviate this issue and enable more precise reaching towards the target. Extensive validation conducted on PKU-HR6.0 demonstrates that the proposed approach significantly enhances online adaptation speed and precision compared to existing methods.

Keywords: Cognitive and Sensorimotor Development, Cognitive Skills and Mental Models, Computational Architectures
Abstract: The number of contexts in which a word is encountered, or contextual diversity, has been shown to be a relevant predictor of word-naming and lexical decision times. In this work we present an end-to-end scenario in which we collect data with a humanoid robot in three different contextual diversity levels, use the data to train a cognitive architecture with the objective of mirroring the same phenomenon observed in the literature, and ultimately we test the model by collecting test data with the robot and matching them with the learned word-object mappings. Results show that the approach manages to capture and describe successfully a computational representation of the impact of contextual diversity on word-object mapping, showing how with greater contextual diversity the mapping is more precise compared to the cases with lower diversity.

Keywords: Motion Planning and Navigation in Human-Centered Environments
Abstract: We study the task of locating a user in a mapped indoor environment using natural language queries and images from the environment. Building on recent pretrained vision-language models, we learn a similarity score between text descriptions and images of locations in the environment. This score allows us to identify locations that best match the language query, estimating the user's location. Our approach is capable of localizing on environments, text, and images that were not seen during training. One model, finetuned CLIP, outperformed humans in our evaluation.

Keywords: Affective Computing, Machine Learning and Adaptation, Social Intelligence for Robots
Abstract: Social Robotics and Human-Robot Interaction (HRI) research relies on different Affective Computing (AC) solutions for sensing, perceiving and understanding human affective behaviour during interactions. This may include utilising off-the-shelf affect perception models that are pre-trained on popular affect recognition benchmarks and directly applied to situated interactions. However, the conditions in situated human-robot interactions differ significantly from the training data and settings of these models. Thus, there is a need to deepen our understanding of how AC solutions can be best leveraged, customised and applied for situated HRI. This paper, while critiquing the existing practices, presents four critical lessons to be noted by the hitchhiker when applying AC for HRI research. These lessons conclude that: (i) The six basic emotions categories are not always relevant in situated interactions, (ii) Affect recognition accuracy (%) improvement as the sole goal is inappropriate for situated interactions, (iii) Affect recognition may not generalise across contexts, and (iv) Affect recognition alone is insufficient for adaptation and personalisation. By describing the background and the context for each lesson, and demonstrating how these lessons have been compiled from the various studies of the authors, this paper aims to enable the hitchhiker to successfully leverage AC solutions for advancing HRI research.

Keywords: Virtual and Augmented Tele-presence Environments, Novel Interfaces and Interaction Modalities, Degrees of Autonomy and Teleoperation
Abstract: While we can see robots in more areas of our lives, they still make errors. One common cause of failure stems from the robot perception module when detecting objects. Allowing users to correct such errors can help improve the interaction and prevent the same errors in the future. Consequently, we investigate the effectiveness of a virtual reality (VR) framework for correcting perception errors of a Franka Panda robot. We conducted a user study with 56 participants who interacted with the robot in both the VR and screen interfaces. Participants learned to collaborate with the robot faster in the VR interface compared to the screen interface. Additionally, participants found the VR interface more immersive, enjoyable, and ex- pressed a preference for using it again. These findings suggest that VR interfaces may offer advantages over screen interfaces for human-robot interaction in erroneous environments.

Keywords: Cognitive and Sensorimotor Development, Virtual and Augmented Tele-presence Environments, Assistive Robotics
Abstract: Learning the advanced functionalities of modern myoelectric prostheses can be challenging and highly cognitive demanding for naive users. While virtual reality (VR) has recently emerged as a promising tool for neurorehabilitation, it is important to consider also the cognitive load aspect of the training process, for a more realistic assessment of users' capabilities. This study aims to investigate the correlation between functional performance and cognitive demand when learning a myoelectric control method in an immersive virtual reality environment for training and assessment. The developed virtual training environment simulated activities of daily living, while the assessments included standard tests, as well as a motor-cognitive dual-task methodology that combined both aspects. The study was conducted with 10 able-bodied participants, who controlled a virtual multi-grip prosthesis using a conventional myoelectric control strategy that requires muscle co-activation to switch between power and precision grasp. Performance in terms of functionalities, cognitive load, and user perception was assessed before and after training. Results show that VR training led to an immediate improvement in functionalities, enabling fast object manipulation, and highlighting the importance of including cognitive evaluation in the training progress.

Keywords: Virtual and Augmented Tele-presence Environments, Motivations and Emotions in Robotics, Machine Learning and Adaptation
Abstract: Abstract—Deep learning models require large amounts of data to be trained to fulfill their potential. To solve this problem,we propose a novel method for creating high-quality photorealistic synthetic training data and compare its performance to real data for object detection. We introduce the RealStreet and SynthStreet datasets, which were designed to enhance a safety analysis for road object detection. The objective of the project is to provide a useful synthetic environment for learning and building a road traffic experience by imitating the real environment as nearly as possible. This will improve the safety of road users and enable testing and planning before actual, dangerous events arise. The RealStreet data-set was collected in real-world scenarios of an urban city, while the SynthStreet closely recreates the RealStreet scenes: field of view, road objects, such as pedestrians, cyclists, vehicles, and background information buildings are matched. We study the performance and behavior of a network model trained on real and synthetic data-sets and both with various ratio mixes. Our approach is to evaluate data-set performances using a state-of-the-art method for object detection tasks in learning from synthetic data. We also compare the performance of each dataset, which is evaluated on real-world data, to determine the possibility of synthetic data-set benefits and analyze the effect of limited real-world data. However, it is important to note that synthetic data-sets may not always accurately represent the variability and complexity of real-world environments and may not generalize well to real-world scenarios. Hence, it is important to carefully evaluate the performance of deep learning models trained on synthetic data and validate their results using real data.

Keywords: User-centered Design of Robots, Virtual and Augmented Tele-presence Environments, Human Factors and Ergonomics
Abstract: Avatars as digital portrayals of humans play a pivotal role in fostering embodiment and immersion in virtual reality (VR) environments by providing users with a visual representation of their virtual presence. In the context of human-robot interaction (HRI), understanding the dynamics of trust formation between humans and avatars are essential for successful collaboration and communication. This study methodically investigates the effect of the similarity between players and their avatars on the establishment of trust between human users and robots in a VR environment. Our research implies that enhancing the resemblance of self-avatars boosts trust dynamics in human-robot interactions, yet it doesn't necessarily foster cooperation. Moreover, this research highlights the importance of designing virtual avatars and robots that can effectively communicate and adapt to user preferences, thereby fostering a more engaging and productive HRI experience in VR environments.

Keywords: Virtual and Augmented Tele-presence Environments, Medical and Surgical Applications, Assistive Robotics
Abstract: Maternal health remains a pervasive challenge in developing and underdeveloped countries. Inadequate access to basic antenatal Ultrasound (US) examinations, limited resources such as primary health services and infrastructure, and lack of skilled healthcare professionals are the major concerns. To improve the quality of maternal care, robot-assisted antenatal US systems with teleoperable and autonomous capabilities were introduced. However, the existing teleoperation systems rely on standard video stream-based approaches that are constrained by limited immersion and scene awareness. Also, there is no prior work on autonomous antenatal robotic US systems that automate standardized scanning protocols. To that end, this paper introduces a novel Virtual Reality (VR) platform for robotic antenatal ultrasound, which enables sonologists to control a robotic arm over a wired network. The effectiveness of the system is enhanced by providing a reconstructed 3D view of the environment and immersing the user in a VR space. Also, the system facilitates a better understanding of the anatomical surfaces to perform pragmatic scans using 3D models. Further, the proposed robotic system also has autonomous capabilities; under the supervision of the sonologist, it can perform the standard six-step approach for obstetric US scanning recommended by the ISUOG. Using a 23-week fetal phantom, the proposed system was demonstrated to technology and academia experts at MEDICA 2022 as a part of the KUKA Innovation Award. The positive feedback from them supports the feasibility of the system. It also gave an insight into the improvisations to be carried out to make it a clinically viable system.

Keywords: Virtual and Augmented Tele-presence Environments, Cooperation and Collaboration in Human-Robot Teams, User-centered Design of Robots
Abstract: This paper addresses the problem of multi-robot task scheduling by estimating the demand for the tasks in a real-world scenario. Scheduling tasks for multiple robots becomes complex when a human is involved in allocating limited resources. We propose a stochastic multi-agent multi-armed bandit based task scheduler which prioritizes the tasks based on the estimated demand for the tasks. To gain insight into the varying priorities of a human task allocator in a multi-armed bandit scenario, we conducted a user study in a Mixed Reality environment which can be used to customize the resource allocation process. We observe that the users consistently made sub-optimal choices due to their preference to minimize other parameters (such as cumulative distance travelled) of the real world scenario rather than strictly adhering to the optimal strategy. Our proposed method uses the Thompson sampling bandit algorithm with ϵ-greedy approach to solve the multi-agent multi-armed bandit problem. The approach outperformed other methods such as first-come-first-serve, rate monotonic scheduling, and heuristic-based Min-interference approaches in terms of the demand aware performance index.

Keywords: Degrees of Autonomy and Teleoperation
Abstract: We present a shared control framework for teleoperation that combines the human and autonomous robot agents operating in different dimension spaces. The shared control problem is an optimization problem to maximize the human’s internal action-value function while guaranteeing that the shared control policy is close to the autonomous robot policy. This results in a state update rule that augments the human controls using the Riemannian metric that emerges from computing the curvature of the robot’s value function to account for any cost terms or constraints that the human operator may neglect when operating a redundant manipulator. In our experiments, we apply Linear Quadratic Regulators to locally approximate the robot policy using a single optimized robot trajectory, thereby preventing the need for an optimization step at each time step to determine the optimal policy. We show preliminary results of reach-and-grasp teleoperation tasks with a simulated human policy and a pilot user study using the VR headset and controllers. However, the mixed user preference ratings and quantitative results show that more investigation is required to prove the efficacy of the proposed paradigm.

Keywords: Multimodal Interaction and Conversational Skills, Motivations and Emotions in Robotics, Social Presence for Robots and Virtual Humans
Abstract: The COVID-19 pandemic has popularized telework using video chat tools. Although video chat tools allow people to communicate each other in remote locations, psychological distance is longer than face-to-face communication, which may lead to a lack of intimacy. Previous studies have shown that interfaces combining video chat, VR, and robots with handshakes can improve the sense of closeness. However, few studies have focused on the sense of closeness in interfaces combining VR and robots. In this study, we proposed a multimodal interface combining VR and a robotic arm and verified whether shaking hands with a remote partner in VR could shorten the psychological distance compared to video chat. The results showed that the psychological distance was predominantly shorter in VR communication than in video chat communication; however, a handshake using robotic arm did not shorten the psychological distance compared to a handshake without a robotic arm. These results indicate that spatial distance is shorter in VR than in video chat and that robotic arms must be made more human-like.

Keywords: Robot Companions and Social Robots, Machine Learning and Adaptation, Assistive Robotics
Abstract: Socially-assistive robots (SARs) hold significant potential to transform the management of chronic healthcare conditions (e.g. diabetes, Alzheimer’s, dementia) outside the clinic walls. However doing so entails embedding such autonomous robots into people’s daily lives and home living environments, which are deeply shaped by the cultural and geographic locations within which they are situated. That begs the question whether we can design autonomous interactive behaviors between SARs and humans based on universal machine learning (ML) and deep learning (DL) models of robotic sensor data that would work across such diverse environments? To investigate this, we conducted a long-term user study with 26 participants across two diverse locations (United States and South Korea) with SARs deployed in each user’s home for several weeks. We collected robotic sensor data every second of every day, combined with sophisticated ecological momentary assessment (EMA) sampling techniques, to generate a large-scale dataset of over 270 million data points representing 173 hours of randomly-sampled naturalistic interaction data between the human and SAR. Models built on that data were capable of achieving nearly 84% accuracy for detecting specific interaction modalities (AUC 0.885) when trained/tested on the same location, though suffered significant performance drops when applied to a different location. Further analysis and participant interviews showed that was likely due to differences in home living environments in the US and Korea. The results suggest that our ability to create adaptable behaviors for robotic pets may be dependent on the human-robot interaction (HRI) data available for modeling.

Keywords: Assistive Robotics, Degrees of Autonomy and Teleoperation, Novel Interfaces and Interaction Modalities
Abstract: Mobile manipulator platforms, like the Stretch RE1 robot, make the promise of in-home robotic assistance feasible. For people with severe physical limitations, like those with quadriplegia, the ability to tele-operate these robots themselves means that they can perform physical tasks they cannot otherwise do themselves, thereby increasing their level of independence. In order for users with physical limitations to operate these robots, their interfaces must be accessible and cater to the specific needs of all users. As physical limitations vary amongst users, it is difficult to make a single interface that will accommodate all users. Instead, such interfaces should be customizable to each individual user. In this paper we explore the value of customization of a browser-based interface for tele-operating the Stretch RE1 robot. More specifically, we evaluate the usability and effectiveness of a customized interface in comparison to the default interface configurations from prior work. We present a user study involving participants with motor impairments (N=10) and without motor impairments, who could serve as a caregiver, (N=13) that use the robot to perform mobile manipulation tasks in a real kitchen environment. Our study demonstrates that no single interface configuration satisfies all users' needs and preferences. Users perform better when using the customized interface for navigation, but not for manipulation due to higher complexity of learning to manipulate through the robot. All participants are able to use the robot to complete all tasks and participants with motor impairments believe that having the robot in their home would make them more independent.

Keywords: Assistive Robotics, User-centered Design of Robots, Human Factors and Ergonomics
Abstract: The aged care industry is under pressure from stressors including increasing resident numbers and difficulty meeting staffing requirements. Robots may be able to support the industry by filling many vital roles, however it is currently unclear how successful implementation of robots in aged care can occur, and detailed in situ assessment and mapping of robotic deployment in these settings is lacking. The current case study examines early-stage implementation of robots at an aged care home in Australia, assessing logistical, technical and person factors. Key facilitators and barriers to successful deployment are identified, including identifying needs and roles, health and safety issues and technical support. The findings illustrate the potential for robots in aged care and provide a blueprint for the steps needed for long-term effectiveness and commercial viability.

Keywords: Linguistic Communication and Dialogue, Multi-modal Situation Awareness and Spatial Cognition, Motion Planning and Navigation in Human-Centered Environments
Abstract: The spoken language serves as an accessible and efficient interface, enabling non-experts and disabled users to interact with complex assistant robots. However, accurately grounding language utterances gives a significant challenge due to the acoustic variability in speakers' voices and environmental noise. In this work, we propose a novel speech-scene graph grounding network (SGGNet^2) that robustly grounds spoken utterances by leveraging the acoustic similarity between correctly recognized and misrecognized words obtained from automatic speech recognition (ASR) systems. To incorporate the acoustic similarity, we extend our previous grounding model, the scene-graph-based grounding network (SGGNet), with the ASR model from NVIDIA NeMo. We accomplish this by feeding the latent vector of speech pronunciations into the BERT-based grounding network within SGGNet. We evaluate the effectiveness of using latent vectors of speech commands in grounding through qualitative and quantitative studies. We also demonstrate the capability of SGGNet^2 in a speech-based navigation task using a real quadruped robot, RBQ-3, from Rainbow Robotics.

Keywords: Detecting and Understanding Human Activity, Machine Learning and Adaptation, Cooperation and Collaboration in Human-Robot Teams
Abstract: In emergency rescue scenarios, rapid identification of human casualties is a critical first step in enhancing emergency medical response. This task can be limited by the physical and cognitive capacity of rescue personnel, who are exposed to significant risk. The use of small unmanned aerial systems (sUAS) equipped with autonomous casualty assessment abilities can reduce these limitations and risks by enabling remote casualty detection, identification, and vitals assessment, providing standoff protection, and eliminating the need for human personnel to access the potentially hazardous scene. This paper presents a vision-based casualty assessment framework and specifically discusses our casualty identification software, which is designed to recognize the faces of casualties and identify their nametapes in images captured by sUAS under realistic conditions. Our approach addresses the limitations of the sUAS-captured long-distance images to enable accurate identification in challenging casualty monitoring situations. The face and nametape recognition algorithms will be integrated into the larger casualty perception framework and embedded into sUAS platforms to assist with emergency rescue operations. The total casualty perception system will detect, identify, and evaluate the condition of casualties from a remote location, providing standoff protection to first responders and rapid information to inform a suitable medical treatment plan.

Keywords: Machine Learning and Adaptation, Medical and Surgical Applications, Computational Architectures
Abstract: The quality of medical images is critical for Computer-aided diagnosis (CAD) and Image-guided robotic interventions because accurate and high-quality images are required to perform each task. High resolution and SNR images are required to analyze and navigate the robotic instruments to the accurate localization inside the body. However, medical images are often of substantially lower quality than clean photographic images due to various factors. In this study we focus on a post-processing based strategy for reducing the amount of noise in MRI images. We propose a method based on Denoising Diffusion Probablistic Models (DDPM), also known as diffusion models the reduce the amount of noise in the image. Specifically, a two-stage DDPM method is proposed -- estimating the amount of noise and designating to the correct stage in the Marchov Chain in the reverse diffusion operation, and iteratively and gradually reducing noise by reversing the process. Our experiment was performed on an actually scanned images on a clinical MR scanner, with the reference image that were averaged to match the SNR. Our quantitative and qualitative comparison shows that our method outperforms previous methods including supervised training based on two different metrics (SSIM, PSNR). It demonstrates the effectiveness of the DDPM-based method in reducing noise in the image. Moreover, the resulting image quality achieved with the proposed approach shows that tissue sub-structures are clearer. The noise reduction performance of the proposed method for multiple adjacent slices and various contrasts was tested to show the model's ability to reduce noise across a diverse set of imaging conditions, which is essential in real-world scenarios.

Keywords: Affective Computing, Non-verbal Cues and Expressiveness
Abstract: Prevention of abuse of autonomous robots is one of the major challenges in getting robots out "in the wild". The involvement of observing pedestrians of an abusive situation might be a key factor to stop abuse. We investigated how a delivery robot's different reactions to abuse affect an observer's situational empathy and affect towards the robot. In two studies, a total of 364 participants watched short videos of a delivery robot being kicked by an "abuser" and observed one of four different pre-tested robot reactions towards abuse: sad face, neutral face, white screen, and black screen. Concerning negative affect and situational empathy towards the robot and the abuser, significant differences were found between screen on conditions (sad, neutral, white) and the screen off condition (black screen). No differences were found between the different screen on conditions. Especially, negative affect, ascribed warmth to the robot, and empathetic concern significantly predicted situational empathy. Humans seem to empathize most with a delivery robot indicating "being on" (white screen or facial expression) while the emotional facial reaction had no effects. Thus, autonomous robots should always show signs of being turned on when operating autonomously on the streets in order to raise empathy and prevent abuse.

Keywords: Personalities for Robotic or Virtual Characters, Social Intelligence for Robots, Cognitive Skills and Mental Models
Abstract: The objective of this work is to develop computational cognitive models embedded in a humanoid robot. We focus on Dark Triad constructs and the so-called ``Nice and Nasty" Theory of Mind that have never been investigated through a robotic approach. To this end, DT and ToM conceptual models in psychology have been taken as a reference for developing a framework based on the popular PDDL planning language. Next, a cognitive architecture has been implemented on a humanoid robot, with the final objective of making adverse personalities emerge. The motivations of the present work are both theoretical and practical. On the one side, we aim to provide researchers with new insights into DT constructs through simulated and robotic setups. On the other side, we aim to provide a tool to train psychologists to deal with social and antisocial behaviour in a controlled setup. The article includes all the details about the model and the experiments performed.

Keywords: Storytelling in HRI, Linguistic Communication and Dialogue
Abstract: In this study, we examined scene selection methods and emotion-based descriptions for a robot's daily diary. We proposed a scene selection method and an emotion description method that take into account semantic and affective information, and created several types of diaries. Experiments were conducted to examine the change in sentiment values and preference of each diary, and it was found that the robot's feelings and impressions changed more from date to date when scenes were selected using the affective captions. Furthermore, we found that the robot's emotion generally improves the preference of the robot's diary regardless of the scene it describes. However, presenting negative or mixed emotions at once may decrease the preference of the diary or reduce the robot's robot-likeness, and thus the method of presenting emotions still needs further investigation.

Keywords: Motivations and Emotions in Robotics, Creating Human-Robot Relationships, Cooperation and Collaboration in Human-Robot Teams
Abstract: Increasing anthropomorphic robot behavioral design could affect trust and cooperation positively. However, studies have shown contradicting results and suggest a task-dependent relationship between robots that display emotions and trust.

Therefore, this study analyzes the effect of robots that display human-like emotions on trust, cooperation, and participants' emotions. In the between-group study, participants play the coin entrustment game with an emotional and a non-emotional robot.

The results show that the robot that displays emotions induces more anxiety than the neutral robot. Accordingly, the participants trust the emotional robot less and are less likely to cooperate. Furthermore, the perceived intelligence of a robot increases trust, while a desire to out-compete the robot can reduce trust and cooperation.

Thus, the design of robots expressing emotions should be task dependent to avoid adverse effects that reduce trust and cooperation.

Keywords: Motivations and Emotions in Robotics, Embodiment, Empathy and Intersubjectivity, Creating Human-Robot Relationships
Abstract: Self-disclosing to others can benefit emotional well-being, but socio-emotional barriers can limit people's ability to do so. Self-disclosing towards social robots can help overcome these obstacles as robots lack judgment and can establish rapport. To further understand the influence of affective factors on people's self-disclosure to social robots, this study examined the relationship between self-disclosure behaviour towards a social robot and people's emotional states and their perception of the robot's responses as comforting (i.e., being emphatic). The study included 1160 units of observation collected from 39 participants who conversed with the social robot Pepper (SoftBank Robotics) twice a week for 5 weeks (10 sessions in total), answering three personal questions in each session. Results show that perceiving the robot's responses as more comforting was positively related to self-disclosure behaviour (in terms of disclosure duration in seconds, and disclosure length in number of words), and negative emotional states, such as lower mood, and higher feelings of loneliness and stress, were associated with higher rates of self-disclosure towards the robot. Additionally, higher rates of introversion significantly predicted higher rates of self-disclosure towards the robot. The study reveals the meaningful influence of affective states on how people behave when talking to social robots, especially when experiencing negative emotions. These findings may have implications for designing and developing social robots in therapeutic contexts.

Keywords: Affective Computing, Motivations and Emotions in Robotics, Child-Robot Interaction
Abstract: Autism Spectrum Disorder (ASD) children struggle with social interaction and communication, prompting the development of facial emotion recognition systems to aid communication with caretakers. Researchers have shown interest in using non-invasive thermal imaging to recognize emotions. In this study, a wavelet-based technique was developed to detect changes in the thermal intensity values (TIV) of three regions of interest from frontal facial thermal image. The study analyzed thermal images of ASD children responding to audio-visual stimuli of five basic emotions and derived features from them. Wavelet coefficients were combined with thermal intensity values of each region of interest to form feature set, which were fed into a CNN model. The result showed the efficacy of the method with accuracy and precision attained at 91.81%% and 94.54% respectively. The study is useful in the development of robot assist training for the ASD children as part of early intervention program.

Keywords: Personalities for Robotic or Virtual Characters, Interaction with Believable Characters, Non-verbal Cues and Expressiveness
Abstract: Robotic personalities broaden the social dimension of an agent creating feelings of comfort in humans. In this work, we propose a taxonomy model to generate synthetic personalities based on the Big Five model. In particular, this paper describes a generalized framework for artificial personalities whose core is a Bidirectional Encoder Representations from Transformers (BERT) capable of associating different behaviors to each personality trait. The generator is fully integrated within a modular software architecture capable of performing social interaction tasks, being at the same time task- and platform-independent. The proposed framework has been tested in a pilot experiment where human subjects were asked to interact with a humanoid robot displaying different personality traits. Results obtained by the statistical analysis of validated questionnaires show interesting insights about the capability of the framework of generating personalities that are clearly perceived by users, and whose personality dimensions are strongly distinguishable.

Keywords: Ethical Issues in Human-robot Interaction Research, Social Intelligence for Robots, Philosophical Issues in Human-Robot Coexistence
Abstract: Telling lies and faking emotions is quite common in human-human interactions: though there are risks, in many situations such behaviours provide social benefits. In recent years, there have been many social robots and chatbots that fake emotions or behave deceptively with their users. In this paper, I present a few examples of such robots and chatbots, and analyze their ethical aspects. Three scenarios are presented where some kind of lying or deceptive behaviour might be justified. Then five approaches to deceptive behaviours --- no deception, blatant deception, tactful deception, nudging, and self deception -- are discussed and their implications are analyzed. I conclude by arguing that we need to develop localized and culture-specific solutions to incorporating deception in social robots and chatbots.

Keywords: Multimodal Interaction and Conversational Skills, Novel Interfaces and Interaction Modalities, Narrative and Story-telling in Interaction
Abstract: Time-Offset Interaction Applications (TOIAs) are narrative-sharing systems that use databases of previously recorded videos of real people to mimic conversations with them. These video databases comprise large (the larger, the better) collections of videos of answers paired with specific questions. This paper focuses on a solution to the challenge of creating such databases without exhausting their creators' creativity, energy, and interest. We describe the design and development process of Question Suggester (QS) - an intelligent GPT-3-based service that generates suggested questions following up a conversation based on the history of recorded questions and answers. We conduct a user study to empirically evaluate the value of QS for reducing the effort to create a video database while creating an interaction that is enjoyable. The users' average experience rating for QS is 4.6 compared to 4.0 when QS is not used (on a 1-5 scale, p-value<0.05). The experience with interactions so created is more enjoyable, too (3.7 vs. 3.3, p-value<0.05). The usage metrics and qualitative feedback confirm that QS is essential for interactive video-recording systems and for increasing their adoption.

Keywords: Machine Learning and Adaptation, Cognitive Skills and Mental Models, Creating Human-Robot Relationships
Abstract: Causality is relatively unexplored in robotics even if it is highly relevant, in several respects. In this paper, we study how a robot's causal understanding can be improved by allowing the robot to ask humans causal questions. We propose a general algorithm for selecting direct causal effects to ask about, given a partial causal representation (using partially directed acyclic graphs, PDAGs) obtained from observational data. We propose three versions of the algorithm inspired by different causal discovery techniques, such as constraint-based, score-based, and interventions. We evaluate the versions in a simulation study and our results show that asking causal questions improves the causal representation over all simulated scenarios. Further, the results show that asking causal questions based on PDAGs discovered from data provides a significant improvement compared to asking questions at random, and the version inspired by score-based techniques performs particularly well over all simulated experiments.

Keywords: Programming by Demonstration, Social Learning and Skill Acquisition Via Teaching and Imitation
Abstract: Learning from Demonstrations (LfD) transfers skills from human teachers to robots. However, data imbalance in demonstrations can bias policies towards majority situations. Previous work attempted to solve this problem after data collection, but few efforts were made to maintain a balanced distribution from the phase of data acquisition. Our method accounts for the influence of robots on human teachers and enables robots to actively guide interaction to approximate demonstration distributions to target distributions. Simulated and real-world experiments validated the method's efficacy in shaping demonstration distribution into various target distributions and robustness to various levels of uncertainties. Also, our method significantly improved the generalization ability of robot learning when LfD policies were trained with data collected by our method compared to natural data collection.

Keywords: Programming by Demonstration, Social Learning and Skill Acquisition Via Teaching and Imitation, Social Intelligence for Robots
Abstract: Human-centered interactive robot tasks (e.g., social greetings and cooperative dressing) are a type of task where humans are involved in task dynamics and performance evaluation. Such tasks require spatial and temporal coordination between agents in real-time, tackling physical limitations from constrained robot bodies, and connecting human user experience with concrete learning objectives to inform algorithm design. To solve these challenges, imitation learning has become a popular approach whereby a robot learns to perform a task by imitating how human experts do it (i.e., expert policies). However, previous works tend to isolate the algorithm design from the design of the whole learning pipeline, neglecting its connection with other modules inside the process (like data collection and user-centered subjective evaluation) from the view as a system. Going beyond traditional imitation learning, this work reexamines robot imitation learning in human-centered interactive tasks from the perspective of the whole learning pipeline, ranging from data collection to subjective evaluation. We present a process-oriented framework that consists of a guideline to collect diverse yet representative demonstrations and an interpreter to explain subjective user-centered performance with objective robot-related parameters. We illustrate the steps covered by the framework in a fist-bump greeting task as demonstrative deployment. Results show that our framework is able to identify representative human-centered features to instruct demonstration collection and validate influential robot-centered factors to interpret the gap in subjective performance between the expert policy and the imitator policy.

Keywords: Machine Learning and Adaptation, Cognitive Skills and Mental Models, Affective Computing
Abstract: Current reinforcement learning algorithms train an agent using forward-generated trajectories, which provide little guidance so that the agent can explore as much as possible. While realizing the value of reinforcement learning results from sufficient exploration, this approach leads to a trade-off in losing sample efficiency, an essential factor impacting algorithm performance. Previous tasks use reward-shaping techniques and network structure modification to increase sample efficiency. However, these methods require many steps to implement. In this work, we propose novel backward curriculum reinforcement learning that begins training the agent using the backward trajectory of the episode instead of the original forward trajectory. This approach provides the agent with a strong reward signal, enabling more sample-efficient learning. Moreover, our method only requires a minor change in the algorithm of reversing the order of the trajectory before agent training, allowing a straightforward application to any state-of-the-art algorithm.

Keywords: Machine Learning and Adaptation, Medical and Surgical Applications
Abstract: To enable the operator to control the surgical assistant robot without a separate joystick, we aim to develop an interface using artificial intelligence that responds to the movement of surgical instruments in the laparoscopic screen. In this study, we propose a method for detecting and tracking surgical tools using the YOLOv5 (YOU ONLY LOOK ONCE v5) and DeepSORT (Deep Learning based Simple Online and Realtime Tracking) algorithms. The proposed approach employs the YOLOv5 algorithm to detect surgical tools in real-time, and the DeepSORT algorithm to track the detected tools across multiple frames. The YOLOv5 model requires minimal computation, which enables the rapid detection of surgical instruments. Furthermore, the DeepSORT algorithm can precisely track object movements in complex environments. The proposed method's tracking stability was assessed using the average pixel error performance metric test. To evaluate the method's performance, we used a 2-degree-of-freedom remote center motion experimental setup and employed the PID control algorithm to control the laparoscopic camera's movements.

Keywords: Motion Planning and Navigation in Human-Centered Environments, Machine Learning and Adaptation, Cooperation and Collaboration in Human-Robot Teams
Abstract: Over the past decades, countless autonomous nav- igation and dynamic obstacle avoidance approaches have been proposed by various research works. However, to bridge the gap between research and industries, these approaches are required to be extensively evaluated and benchmarked within various different setting, scenarios, and maps. However, conducting these test runs is tedious and time-consuming. Furthermore, simulation runs and test on real robots can not always cover all potentially occurring scenarios or are inaccurate in certain settings and circumstances especially when a high number of pedestrians or other dynamic entities are involved. In this paper, we propose an approach to predict the navigational performance of navigation approaches for new and unknown maps, scenarios, and robots without the necessity to conduct the actual test runs. Therefore, we acquire a large dataset consisting of thousands of evaluation runs within crowded environments from both simulation and real-world runs, which were conducted using the arena-bench platform of our previous works [1] and trained several neural network architectures to predict relevant navigational performance metrics such as collision rates or path efficiency. We demonstrate the feasibility of our neural networks by predicting the most relevant metrics with up to 95 percent accuracy compared to the groundtruth data acquired by an actual simulation run. Using this approach could prove beneficial for a number of applications and save valuable time and costs in that the performance of new navigation algorithms for crowded environments can be estimated and predicted on new maps, scenarios, and on new robots. We made the code publicly available at https://github.com/ignc-research/nav- prediction.

Keywords: Detecting and Understanding Human Activity, Multi-modal Situation Awareness and Spatial Cognition, Applications of Social Robots
Abstract: Social robots are increasingly used in public spaces, but they still struggle to handle complex social dynamics and interactions with multiple users. This study assessed the interaction between visitors and a Furhat robot connected to an artificial intelligence (AI)-based dialogue system in an oceanographic museum. Our findings from a video analysis of 176 interactions highlight the importance of understanding social dynamics in social robotics research. At the same time, it suggests that social robots are still limited in their ability to handle multi-party interactions (MPI) and understand complex social dynamics. Those limitations are due, in part, to technical constraints, as well as the need for further research on the social and cultural factors that shape human-robot interactions.

Keywords: Human Factors and Ergonomics, Machine Learning and Adaptation, HRI and Collaboration in Manufacturing Environments
Abstract: During the process of reaching a target, a robotic arm may generate a large number of redundant movements due to the stochastic nature of planning algorithms. We think that each joint of the robotic arm has a unique functionality towards the arm's end effector. Therefore, during motion planning, we optimized the weights of each joint based on its individual functionality. We introduce the human arm operation mechanism - When a person is reaching for a distant target, the shoulder-near joints take on more actions. We proposed an advanced auxiliary loss function. By utilizing this function, we can filter out multiple solutions from the inverse model, thus reducing the amount of joint change and redundant movements. We conducted comprehensive comparative experiments on the UR3 robotic arm, and the results showed that our approach achieved better efficiency and outcomes.

Keywords: Machine Learning and Adaptation, Cooperation and Collaboration in Human-Robot Teams, User-centered Design of Robots
Abstract: In this work, we performed a user study in which participants had to solve a human-robot teaming decision-making task (the Connect 4 game) with an explainable vs non-explainable robot. During the task, the robot provided suggestions and, depending on the experimental condition, explanations to justify those suggestions. We compared participants' behaviours in interacting with both types of robots. In particular, we investigated how participants' personality dimensions and previous experiences with the iCub robot impacted participants' decision-making. We also studied how participants aligned with iCub's playing style as the interaction continued. Our results show that participants' negative agency and agreeableness substantially impacted how they accepted the robot's suggestions when it provided example-based counterfactual explanations. We also observed a learning effect: participants tended to align with the robot's playing style during the interaction. However, the participants' learning depended not only on the presence of the explanations, but also on the time spent with the robot. Moreover, the human-robot team's victories were mainly attributable to the robot's persuasiveness rather than the participants' skills in the game.

Keywords: Linguistic Communication and Dialogue, Multimodal Interaction and Conversational Skills, Machine Learning and Adaptation
Abstract: Natural language encodes rich sequential and contextual information. A task plan for robots can be extracted from natural language instruction through semantic understanding. This information includes sequential actions, target objects and descriptions of working environment. Current systems focus on single-domain understanding such as household or industrial assembly settings, and many rule-based approach have been developed in this context. Thanks to the development of deep learning, data-driven contextual language understanding shows promising results. In this work, an information extraction system is proposed for domain-independent understanding of robotic task plans. The developed approach is based on a pre-trained BERT-model and a syntactic dependency parser. To evaluate the performance, experiments are conducted on three different datasets.

Keywords: Personalities for Robotic or Virtual Characters, Linguistic Communication and Dialogue, Machine Learning and Adaptation
Abstract: Previous works in Human-Robot Interaction have demonstrated the positive potential benefit of designing social robots which express specific personalities. In this work, we focus specifically on the adaptation of language (as the choice of words, their order, etc.) following the extraversion trait. We look to investigate whether current language models could support more autonomous generations of such personality-expressive robot output. We examine the performance of two models with user studies evaluating (i) raw text output and (ii) text output when used within multi-modal speech from the Furhat robot. We find that the ability to successfully manipulate perceived extraversion sometimes varies across different dialogue topics. We were able to achieve correct manipulation of robot personality via our language adaptation, but our results suggest further work is necessary to improve the automation and generalisation abilities of these models.

Keywords: User-centered Design of Robots, Linguistic Communication and Dialogue, Cognitive Skills and Mental Models
Abstract: Politeness is a social and linguistic phenomenon that humans use in communication to build and maintain relationships and spare others' feelings. Research on whether humans also apply politeness strategies when interacting with robots – artifacts that lack feelings – yields contradictory findings. This paper presents a human–robot interaction study (N=40) and compares participants' use of face-saving politeness strategies in their responses to disconfirmation eliciting and face-threatening questions asked either by a robot or a human. An analysis of the linguistic properties of participants' answers (response type, use of politeness markers) shows a higher use of indirect politeness in disconfirming answers directed at humans than at robots. This contradicts previous theories on the automatic and mindless application of social strategies towards artificial agents. Alternative explanations for the differences in politeness behavior are discussed.

Keywords: Sound design for robots, User-centered Design of Robots, Linguistic Communication and Dialogue
Abstract: The ability to interact verbally with humans is a key requirement of many social robots. It is common however for robot speech to lack contextual human-like prosody, making it intelligible but seeming inexpressive and cold. We investigated the effect that applying human-like prosody to synthetic speech had on aural comprehension during human-robot interaction. A text-to-speech system was used to generate synthetic sentences in two conditions: "default", and "human" (informed by voice actor). A speech-in-noise experiment was then performed that required participants to transcribe perceived sentences spoken by a robot in both test conditions. Overall, we found no significant difference in comprehension between sentences spoken using the synthetic voice with prosody and the unaltered synthetic voice, however significant differences in comprehension were detected for shorter sentences (n=50), and among participants that learned English in a different country to the native dialect of the voice actor (n=26). In both of these cases, participants found the voice with human-like prosody harder to comprehend. These findings suggest that introducing human-like prosody to synthetic speech in human-robot interaction, under certain circumstances, may lead to the voice becoming less intelligible. This motivates further research and adds to the growing body of literature on the multifaceted role that voice plays in human-robot interaction.

Keywords: Linguistic Communication and Dialogue, Personalities for Robotic or Virtual Characters, Affective Computing
Abstract: A task-oriented dialogue system capable of expressing personality can improve user engagement and satisfaction. To realize such a system, this paper presents a method of building a personality-aware natural language generation (NLG) module in task-oriented dialogue using reinforcement learning (RL). This method handles both the expression of personality and system intent. During the RL process, a positive reward is given when the generated utterance correctly expresses the assigned personality and conveys its system intent simultaneously. In our experiments on the MultiWOZ dataset, we fine-tuned a personality-aware NLG module for two personality traits (extraversion and neural perception sensitivity). We experimented with data from MultiWOZ and a user simulator to confirm its effectiveness in terms of the ability to express personality and task performance.

Keywords: Cooperation and Collaboration in Human-Robot Teams, User-centered Design of Robots, Creating Human-Robot Relationships
Abstract: Despite significant improvements in robot capabilities, they are likely to fail in human-robot collaborative tasks due to high unpredictability in human environments and varying human expectations. In this work, we explore the role of explanation of failures by a robot in a human-robot collaborative task. We present a user study incorporating common failures in collaborative tasks with human assistance to resolve the failure. In the study, a robot and a human work together to fill a shelf with objects. Upon encountering a failure, the robot explains the failure and the resolution to overcome the failure, either through handovers or humans completing the task. The study is conducted using different levels of robotic explanation based on the failure action, failure cause, and action history, and different strategies in providing the explanation over the course of repeated interaction. Our results show that the success in resolving the failures is not only a function of the level of explanation but also the type of failures. Furthermore, while novice users rate the robot higher overall in terms of their satisfaction with the explanation, their satisfaction is not only a function of the robot's explanation level at a certain round but also the prior information they received from the robot.

Keywords: Multimodal Interaction and Conversational Skills, Androids, Non-verbal Cues and Expressiveness
Abstract: We present two approaches to animate an android robot head according to audio speech input, both are adopted from recent machine learning based works in computer graphics animation. More concrete we implemented a viseme-based and a mesh-based approach on our robot. After a subjective comparison we conduct a speech-reading study to evaluate our preferred, the mesh-based, approach. The results show that on average the intelligibility is not increased by the visual cues provided through the robot head in comparison to noisy audio alone. This underlines the importance of carefully designing and controlling the facial co-speech movements of talking android heads.

Keywords: Detecting and Understanding Human Activity, User-centered Design of Robots, Interaction Kinesics
Abstract: Handovers are basic yet sophisticated motor tasks performed seamlessly by humans. They are among the most common activities in our daily lives and social environments. This makes mastering the art of handovers critical for a social and collaborative robot. In this work, we present an experimental study that involved human-human handovers by 13 pairs, i.e., 26 participants. We record and explore multiple features of handovers amongst humans aimed at inspiring handovers amongst humans and robots. With this work, we further create and publish a novel data set of 8672 handovers, which includes human motion tracking and the handover-forces. We further analyze the effect of object weight and the role of visual sensory input in human-human handovers, as well as possible design implications for robots. As a proof of concept, the data set was used for creating a human-inspired data-driven strategy for robotic grip release in handovers, which was demonstrated to result in better robot to human handovers.

Keywords: Cooperation and Collaboration in Human-Robot Teams, Machine Learning and Adaptation, Detecting and Understanding Human Activity
Abstract: With the introduction of collaborative robots, humans and robots can now work together in close proximity and share the same workspace. However, this collaboration presents various challenges that need to be addressed to ensure seamless cooperation between the agents. This paper focuses on task planning for human-robot collaboration, taking into account the human's performance and their preference for following or leading. Unlike conventional task allocation methods, the proposed system allows both the robot and human to select and assign tasks to each other. Our previous studies evaluated the proposed framework in a computer simulation environment. This paper extends the research by implementing the algorithm in a real scenario where a human collaborates with a Fetch mobile manipulator robot. We briefly describe the experimental setup, procedure and implementation of the planned user study. As a first step, in this paper, we report on a system evaluation study where the experimenter enacted different possible behaviours in terms of leader/follower preferences that can occur in a user study. Results show that the robot can adapt and respond appropriately to different human agent behaviours, enacted by the experimenter. A future user study will evaluate the system with human participants.

Keywords: Multimodal Interaction and Conversational Skills, Cooperation and Collaboration in Human-Robot Teams, Linguistic Communication and Dialogue
Abstract: Human-guided robotic exploration is a useful approach to gathering information at remote locations, especially those that might be too risky, inhospitable, or inaccessible for humans. Maintaining common ground between the remotely-located partners is a challenge, one that can be facilitated by multi-modal communication. In this paper, we explore how participants utilized multiple modalities to investigate a remote location with the help of a robotic partner. Participants issued spoken natural language instructions and received from the robot: text-based feedback, continuous 2D LIDAR mapping, and upon-request static photographs. We noticed that different strategies were adopted in terms of use of the modalities, and hypothesize that these differences may be correlated with success at several exploration sub-tasks. We found that requesting photos may have improved the identification and counting of some key entities (doorways in particular) and that this strategy did not hinder the amount of overall area exploration. Future work with larger samples may reveal the effects of more nuanced photo and dialogue strategies, which can inform the training of robotic agents. Additionally, we announce the release of our unique multi-modal corpus of human-robot communication in an exploration context: SCOUT, the Situated Corpus on Understanding Transactions.

Keywords: Cooperation and Collaboration in Human-Robot Teams
Abstract: When robots interact with humans, limitations in their internal models arise due to the uncertainty and even randomness of human behavior. This has led to attempts to predict human future actions and infer their intent. However, some authors argue for combining inference engines with communication systems that explicitly elicit human intention. This work builds on our Perception-Intention-Action (PIA) cycle, a framework that considers human intention at the same level as perception of the environment. The PIA cycle is used in a collaborative task to compare the effect on different human-robot interaction aspects of using a force predictor that infers human implicit intention versus a communication system that explicitly elicits human intention. A study with 18 volunteers shows that allowing humans to directly express themselves can achieve the same improvement as an intention predictor.

Keywords: Computational Architectures, HRI and Collaboration in Manufacturing Environments, Cooperation and Collaboration in Human-Robot Teams
Abstract: In this study, force sensorless plastic deformation control is designed and implemented. This control strategy is derived based on the concept that the shift in position and posture attributable to an external force as the deformation of the robot. The Maxwell model that describes plastic deformation is derived as the simple compressed expression using the convolution. Based on the expression, a control law that does not require force information is proposed by not performing the inertia shaping. Physical simulation with a robotic arm are executed to validate the proposed control law.

Keywords: Motion Planning and Navigation in Human-Centered Environments, Machine Learning and Adaptation, Computational Architectures
Abstract: To enable a smooth co-existence between robots and human workers in an industrial setting, we implemented two robot working memory configurations onto a mobile manipulator RB-KAIROS+ robot (Robotnik): A GRU-based one and a bio-inspired alternative called WorkMATe which enabled the robot to adapt its navigation strategy depending on the presence of human workers. To evaluate the two working memory configurations against a non adaptive behavior, we tested a possible co-working scenario between two ostensible workers and the RB-KAIROS+ robot navigating in two mocked industrial set-ups. The application of behavioral adaptation through a working memory component was highly beneficial as it led to reduced energy consumption and, more importantly, to fewer acceleration anomalies in robot navigation than the non adaptive one. This suggests that a robot's adaptive navigation through working memory can increase workers' safety and improve the efficiency of the human-robot system as a whole in industrial applications.

Keywords: HRI and Collaboration in Manufacturing Environments, Creating Human-Robot Relationships, Cooperation and Collaboration in Human-Robot Teams
Abstract: We present and evaluate a pointing-gesture-based, bilateral HRI with the humanoid robot Digit. Recently, humanoid robots have become more powerful and available, but they are still not helpful and accessible to non-technical users without programming experience. Therefore, we propose a pointing gesture-based interaction modality based on monocular video input. The pointing gestures are extracted using a pre-trained pose estimation model and, together with a verbal dialogue, serve as an exemplar interaction behavior for the task of moving heavy objects using the Digit. The interaction is evaluated quantitatively and qualitatively in a user study. The results of the study show promising results in terms of usability and implementation challenges for further research in this area.

Keywords: Assistive Robotics, Applications of Social Robots, Cognitive Skills and Mental Models
Abstract: In order to achieve a widespread adoption of social robots in the near future, we need to design intelligent systems that are able to autonomously understand our beliefs and preferences. This will pave the foundation for a new generation of robots able to navigate the complexities of human societies. To reach this goal, we look into Theory of Mind (ToM): the cognitive ability to understand other agents' mental states. In this paper, we rely on a probabilistic ToM model to detect when a human has false beliefs with the purpose of driving the decision-making process of a collaborative robot. In particular, we recreate an established psychology experiment involving the search for a toy that can be secretly displaced by a malicious individual. The results that we have obtained in simulated experiments show that the agent is able to predict human mental states and detect when false beliefs have arisen. We then explored the set-up in a real-world human interaction to assess the feasibility of such an experiment with a humanoid social robot.

Keywords: Cooperation and Collaboration in Human-Robot Teams, Machine Learning and Adaptation
Abstract: One of the main challenges for successful human-robot collaborative applications lies in adapting the plan to the human agent's changing state and preferences. A promising solution is to bridge the gap between agent modelling and AI task planning, which can be done by integrating the agent state as action costs in the task planning domain. This allows for the plan to be adapted to different partners, by influencing the action allocation. The difficulty then lies in setting appropriate action costs. This paper presents a novel framework to learn a set of planning action costs considering the preferred actions for an agent based on their state. An evolutionary optimisation algorithm is used for this purpose, and an action outcome simulator is developed to act as the black-box function, based on both an agent model and an action type model. This addresses the challenge of collecting data in HRC real-world scenarios, accelerating the learning for posterior fine-tuning in real applications. The coherence of the models and the simulator is proven through a conducted survey, and the learning algorithm is shown to learn appropriate action costs, producing plans that satisfy both the agents' preferences and the prioritised plan requisites. The resulting system is a generic learning framework integrating components that can be easily extended to a wide range of applications, models and planning formalisms.

Keywords: User-centered Design of Robots, Creating Human-Robot Relationships, Human Factors and Ergonomics
Abstract: Involving users in the design of novel technologies, for instance, by adapting them to personal preferences, can improve attitudes and trust towards them. Research on the customization of robot functionalities is still scarce, with existing work predominantly focusing on the impact of perceived, rather than actual user involvement in the design process. Furthermore, it is unclear whether the effects of customization depend on merely choosing options, or on actively implementing these options. Thus, in the present online experiment, we investigated whether actual time invested in choosing options (i.e., words that a robot would learn) and implementing them (i.e., defining the words to a robot in the individual’s own terms) influenced psychological ownership and trust towards the robot. Contrary to our hypotheses, time invested in choosing options slightly decreased perceived control and psychological ownership over the robot. This might be due to task difficulty, which hints at the role of perceived self-efficacy as a determinant of psychological ownership. No effect of time spent in implementing the chosen words by defining them has been found.

Keywords: Social Touch in Human–Robot Interaction, Non-verbal Cues and Expressiveness, Motivations and Emotions in Robotics
Abstract: Robots are increasingly being developed as assistants for household, education, therapy, and care settings. Such robots can use adaptive emotional behavior to communicate warmly and effectively with their users and to encourage interest in extended interactions. However, autonomous physical robots often lack a dynamic internal emotional state, instead displaying brief, fixed emotion routines to promote specific user interactions. Furthermore, despite the importance of social touch in human communication, most commercially available robots have limited touch sensing, if any at all. We propose that users' perceptions of a social robotic system will improve when the robot provides emotional responses on both shorter and longer time scales (reactions and moods), based on touch inputs from the user. We evaluated this proposal through an online study in which 51 diverse participants watched nine randomly ordered videos (a three-by-three full-factorial design) of the koala-like robot HERA being touched by a human. Users provided the highest ratings in terms of agency, ambient activity, enjoyability, and touch perceptivity for scenarios in which HERA showed emotional reactions and either neutral or emotional moods in response to social touch gestures. Furthermore, we summarize key qualitative findings about users’ preferences for reaction timing, the ability of robot mood to show persisting memory, and perception of neutral behaviors as a curious or self-aware robot.

Keywords: Non-verbal Cues and Expressiveness, Novel Interfaces and Interaction Modalities, Social Learning and Skill Acquisition Via Teaching and Imitation
Abstract: The lack of transparency in robotic learning processes poses a significant challenge to effective human-robot collaboration. This is particularly relevant in non-industrial settings because it prevents humans from adequately comprehending a robot's intentions, progress, and decision-making rationale, which is essential for seamless interaction. To address this issue, this work presents a study where users observe a robot endowed with three distinct emotional/behavioural mechanisms for conveying transparent information about its learning process. The proposed mechanisms use inner speech, emotions, and a combination of the two communication styles (hybrid). To assess and evaluate the transparency of these behavioural models, a between-subject study was conducted with 108 participants. Results indicate that the people's perception of the robot's warmth dimension increased when it utilized a hybrid model to explain its learning state. Additionally, increased transparency was observed when the robot used inner speech during the learning process.

Keywords: Applications of Social Robots, Robot Companions and Social Robots, Personalities for Robotic or Virtual Characters
Abstract: This work studies the impact of a robot's appearance on how people judge robots' trustworthiness in a public space scenario. An online experimental study was conducted to investigate the effect of the robot's appearance on the perception of its role and on participants' willingness to comply with the robot's request to share sensitive information. The context of the interaction and the robot's role was presented to the participants using a pre-recorded video filmed from a first-person perspective, encountering and interacting with a Pepper robot at a foreign University. We recruited 54 participants of different ages and nationalities. Each participant was tested with one of the three conditions in which the robot played the same role (which was not explicitly conveyed to the participants) but with a different appearance. Qualitative and quantitative measures were used to collect participants’ responses to evaluate their trust perception in showing their ID documents to the robot and letting the robot take a picture of them. Results showed that the context of interaction played a big part in helping the participants infer the robot's role and the judgment of sensitivity of the information. Our findings provide insights and a better understanding of which are the factors affecting the perception of trustworthiness of robot for a privacy-sensitive human-robot interaction (HRI).

Keywords: Cooperation and Collaboration in Human-Robot Teams, Assistive Robotics, Degrees of Autonomy and Teleoperation
Abstract: We propose a novel interactive semantic shared control framework that exploits an active high-level communication loop between the human operator and the robot for time-efficient teleoperation. In shared control approaches, accurate prediction of the operator’s intention is crucial to enable the robot to provide meaningful assistance. Incorrect intention prediction (e.g., target objects to be interacted with) increases the task duration due to conflicts between human behaviors and robot guidance. Unlike existing methods, our approach not only passively observes and predicts the human operator’s input in the haptic control loop, but also actively communicates with the human operator in an additional semantic loop in the form of a speech user interface to optimize the effectiveness of assistance. We evaluate our ISSC framework for a peg-in-hole teleoperation task. The experimental results show that the proposed framework significantly outperforms teleoperation without assistance and conventional shared control paradigms regarding task execution efficiency and user control quality, and reduces task completion time by up to 26.68% and 39.00%, respectively.

Keywords: Medical and Surgical Applications, Degrees of Autonomy and Teleoperation, Cooperation and Collaboration in Human-Robot Teams
Abstract: In teleoperated robots, such as surgical robots, there is a desire to infer the intent of the operator and provide assistance as needed. This lofty goal is especially challenging when it comes to long-horizon inference. In this paper, we propose leveraging a Transformer-based model to predict the long-horizon trajectory of the master-side manipulators of the da Vinci surgical robot, while also investigating the role of trajectory-based haptic guidance cues as potentially assistive cues. Using the JIGSAW dataset, our model achieved an RMSE Cartesian error of 26.14mm when using the provided gesture labels and 32.13mm without gesture labels for master-side manipulators 1-second-ahead trajectory prediction. We then created resistive and assistive haptic guidance cues with a virtual spring between the current manipulator position and prior or future predicted positions, respectively. Each condition consisted of two levels, defined by 0.5s and 1s time horizons. We conducted a preliminary human subject study with 10 subjects to investigate the role of these guidance forces on completion time for a running suturing task. While there are no statistically significant time differences based on the type of haptic cue and time horizon, we observed that the long-horizon resistive guidance had weak significance to improve the mean task performance in a washout trial that immediately followed the haptic condition. We also observed a large decrease in user difficulty ratings for this trial. These results indicate that haptic guidance cues could be leveraged in surgical training, potentially resulting in lasting after-effects on performance once the guidance has been removed.

Keywords: Cognitive and Sensorimotor Development, Assistive Robotics
Abstract: This paper introduces SmartBelt, a wearable microphone array on a belt that performs sound source localization and returns the direction of arrival with respect to the user waist. One of the haptic motors on the belt then vibrates in the corresponding direction to provide useful feedback to the user. We also introduce a simple calibration step to adapt the belt to different waist sizes. Experiments are performed to confirm the accuracy of this wearable sound source localization system, and results show a Mean Average Error (MAE) of 2.90 degrees, and a correct haptic motor selection with a rate of 92.3%. Results suggest the device can provide useful haptic feedback, and will be evaluated in a study with people having hearing impairments.

Keywords: Novel Interfaces and Interaction Modalities, Robots in Education, Therapy and Rehabilitation, Assistive Robotics
Abstract: In this paper, the issue of integrating haptic technology of a robotic system and virtual environment is discussed. A robot-assisted virtual rehabilitation system is developed for bilateral training and telerehabilitation. In the system, we designed a training scene in the Unity game engine for these two rehabilitation modes, the proposed framework between two robots and virtual environment can allow the end-effectors to perform tasks as hand avatars in Unity and provide force feedback from virtual environment, the haptic rendering is generated on robot's end-effector using a task-space impedance controller. Besides providing the feeling of interaction with virtual objects, we proposed a robot-assisted strategy to provide assistance force when the patient is unable to finish the task in virtual training scene, the assistance force can guide the patient to training path. We provide experiment results to demonstrate the performance of proposed system.

Keywords: Novel Interfaces and Interaction Modalities, Monitoring of Behaviour and Internal States of Humans, Assistive Robotics
Abstract: In human physical activity, motion and sensation affect each other. By utilizing this interaction and appropriately controlling and presenting sensory input during exercise with an external device, it is expected that training will become more efficient and exercise performance will improve. This study proposes a novel visual and force sensory stimulus presentation system using a pneumatically driven gait assistive suit and a head-mounted display (HMD). We aim to develop a gait simulator that realizes the same training effect as walking in various environments by presenting designed sensory stimuli with external devices to solve the spatial limitation of gait training. As an application example of the proposed system, the gait assistive suit applies loads to the legs of a subject walking on flat ground, and an HMD is used to show images of the slope climbing, allowing the subject to virtual experience climbing a slope. In the experimental validation, four subjects measured the average muscle load and leg motion trajectory during virtual slope climbing using the proposed system. We compared the above results with other experimental conditions: (i) normal flat ground walking, (ii) normal slope walking, and (iii) walking with force presentation only. As a result, we confirmed that the root mean square of the myoelectric potential at the front thigh for all four subjects approached the result of normal climbing when the proposed system was used.

Keywords: Human Factors and Ergonomics, Degrees of Autonomy and Teleoperation, Virtual and Augmented Tele-presence Environments
Abstract: This paper proposes a Quality-of-Task-Perception (QoTP) based performance optimization approach for bilateral haptic teleoperation. For time-delayed teleoperation, stabilizing control schemes are combined with communication and data reduction algorithms to ensure stability, transparency, and Quality of Experience (QoE). An adaptive control scheme switching strategy to improve the QoE of teleoperation considering network quality of service (QoS) and quality of control (QoC) is proposed in our previous work. In this paper, we introduce a novel concept named quality of task perception (QoTP) to optimize teleoperation from another dimension in addition to QoS and QoC. QoTP represents the pre-cognition of the task and the accuracy of the environment restoration. The proposed optimization approach is applied to a haptic teleoperation system with switchable control schemes (prediction-based or passivity-based). An environment restoration model is set on the leader side using the least squares method (LSM) to fit different environment models and provide force feedback without the influence of round-trip delay. We also evaluate the system performance with different delays, control schemes, and model complexities both objectively and subjectively. Our experiments validate the proposed approach and show that the QoE performance increases when selecting the more accurate environment restoration model in the QoTP dimension considering the system's computing power.

Keywords: Long-term Experience and Longitudinal HRI Studies, Machine Learning and Adaptation
Abstract: Continual learning (CL) has emerged as an important avenue of research in recent years, at the intersection of Machine Learning (ML) and Human-Robot Interaction (HRI), to allow robots to continually learn in their environments over long-term interactions with humans. Most research in continual learning, however, has been robot-centered to develop continual learning algorithms that can quickly learn new information on static datasets. In this paper, we take a human-centered approach to continual learning, to understand how humans teach continual learning robots over the long term and if there are variations in their teaching styles. We conducted an in-person study with 40 participants that interacted with a continual learning robot in 200 sessions. In this between-participant study, we used two different CL models deployed on a Fetch mobile manipulator robot. An extensive qualitative and quantitative analysis of the data collected in the study shows that there is significant variation among the teaching styles of individual users indicating the need for personalized adaptation to their distinct teaching styles. The results also show that although there is a difference in the teaching styles between expert and non-expert users, the style does not have an effect on the performance of the continual learning robot. Finally, our analysis shows that the constrained experimental setups that have been widely used to test most continual learning techniques are not adequate, as real users interact with and teach continual learning robots in a variety of ways.

Keywords: Long-term Experience and Longitudinal HRI Studies, Linguistic Communication and Dialogue, Applications of Social Robots
Abstract: Studying Human-Robot Interaction over time can provide insights into what really happens when a robot becomes part of people’s everyday lives. “In the Wild” studies inform the design of social robots, such as for the service industry, to enable them to remain engaging and useful beyond the novelty effect and initial adoption. This paper presents an “In the Wild” experiment where we explored the evolution of interaction between users and a Robo-Barista. We show that perceived trust and prior attitudes are both important factors associated with the usefulness, adaptability and likeability of the Robo-Barista. A combination of interaction features and user attributes are used to predict user satisfaction. Qualitative insights illuminated users’ Robo-Barista experience and contribute to a number of lessons learned for future long-term studies.

Keywords: Art pieces supported by robotics, Long-term Experience and Longitudinal HRI Studies, Innovative Robot Designs
Abstract: The last few years have witnessed a sudden increase interest in religion within robotics. Among the first robots of this domain to be developed, SanTO, stands as the first Catholic robot. Its redevelopment, called, SanTO-PL is also an animated statue resembling a Catholic saint. It has been placed first into a church for trial, then on permanent exhibition in a museum. This paper describes the robot and the reactions it provoked, which are far from what a supposedly sacred object would be expected to receive.

Keywords: Art pieces supported by robotics, Interaction Kinesics, Robots in art and entertainment
Abstract: A robot can affect its social environment beyond the person who is interacting with it. Within this context, we believe it is important to explore Human-Robot Interactions (HRI) in complex social settings. We examine the effect of different robot shapes in a multi-person context during dance routines and observe how the design of the robot enhances the artistic process. We identify key factors through which human preferences are being shaped, within a novel third party setting human-robot-human interaction (HRHI).

Keywords: Cooperation and Collaboration in Human-Robot Teams, Applications of Social Robots, Detecting and Understanding Human Activity
Abstract: Human-Robot Interaction (HRI) becomes more and more important in a world where robots integrate fast in all aspects of our lives but HRI applications depend massively on the utilized robotic system as well as the deployment environment and cultural differences. Because of these variable dependencies it is often not feasible to use a data-driven approach to train a model for human intent recognition. Expert systems have been proven to close this gap very efficiently. Furthermore, it is important to support understandability in HRI systems to establish trust in the system. To address the above-mentioned challenges in HRI we present an adaptable python library in which current state-of-the-art Models for context recognition can be integrated. For Context-Based Intention Recognition a two-layer Bayesian Network (BN) is used. The bayesian approach offers explainability and clarity in the creation of scenarios and is easily extendable with more modalities. Additionally, it can be used as an expert system if no data is available but can as well be fine-tuned when data becomes available.

Keywords: Motion Planning and Navigation in Human-Centered Environments, Social Intelligence for Robots, Machine Learning and Adaptation
Abstract: It is essential for autonomous robots to be socially compliant while navigating in human-populated environments. Machine Learning (ML) and, especially, Deep Reinforcement Learning (DRL) have recently gained considerable traction in social navigation (SN). This can be partially attributed to the resulting policies not being bound by human limitations in terms of code complexity or the number of variables that are handled. Unfortunately, the lack of safety guarantees and the large data requirements by DRL algorithms make learning in the real world unfeasible. To bridge this gap, we propose SocNavGym, a social navigation simulation environment that can generate a wide variety of social navigation scenarios, that would facilitate the development of intelligent social agents that can successfully navigate around humans. SocNavGym is light-weight, fast, easy-to-use, and can be effortlessly configured to generate different types of environments. It can also be configured to work with different hand-crafted and data-driven social reward signals and to yield a variety of evaluation metrics to benchmark the performance of the agent. Further, we also provide a case study that trains a Dueling-DQN agent to learn social-navigation policies. The results provides evidence that SocNavGym can be used to train a DRL agent from scratch to navigate in simple as well as complex social scenarios. Our experiments also show that the agents trained using the data-driven reward function are socially more compliant in comparison to the heuristic based reward function.

Keywords: Multimodal Interaction and Conversational Skills, Affective Computing, Non-verbal Cues and Expressiveness
Abstract: In many cases of human-human communication, humans interact with others while assuming their emotions and impressions based on not only verbal information but also non-verbal information. Similarly, during the human-robot interaction, predicting the impressions that a person has of the robot is important for the robot to change the behavior and realize good interaction. In this work, we try to use gaze and gesture annotation data in human-robot interaction from AICO-Corpus and show LSTM approach has the potential for the prediction about impressions of interaction with a robot. We also analyzed the types of nonverbal information that influence the impressions towards the robot in English and Japanese respectively.

Keywords: Social Touch in Human–Robot Interaction, Applications of Social Robots, Social Intelligence for Robots
Abstract: Socially aware robotic applications such as companion and therapeutic robots usually require human emotions or intent to be conveyed. As the scope of these applications increases, the need for recognizing affective touch gestures which are often used to convey these emotions or intent becomes eminent. However, existing touch gesture recognition modalities either have low recognition accuracy or depend heavily on carefully hand-crafted features, therefore limiting their deployment in real-life applications. Motivated by the need for learning models with superior accuracy which do not rely on manually selected hand-crafted features, this paper proposes an optimized class-weighted CNN-LSTM for social touch gesture recognition evaluated on the CoST and HAART datasets. Specifically, contrary to vanilla training schemes where equal importance is given to each class in the dataset, different class weights are introduced to give priority to classes that are difficult for the network to distinguish during training. Furthermore, the weights associated with each of the classes are obtained through optimization using Genetic Algorithm. The proposed model demonstrates superior performance compared with other existing models in the literature.

Keywords: Social Touch in Human–Robot Interaction, Applications of Social Robots, Non-verbal Cues and Expressiveness
Abstract: Holding hands can give people a sense of security. In this study, we developed a five-fingered robotic hand that can hold hands with a person and a guidance system that uses the developed hand to hold hands with them. In this system, a robot remembers the location where people have taught it and guides people to that point. We conducted an experiment to evaluate the sense of security in guidance with hand-holding compared with guidance without hand-holding. Participants watched the videos on these two conditions and answered a questionnaire. The results confirmed that holding hands can lead to a sense of security.

Keywords: Multimodal Interaction and Conversational Skills, Applications of Social Robots, Multi-modal Situation Awareness and Spatial Cognition
Abstract: Predicting when it is an artificial agent's turn to speak/act during human-agent interaction (HAI) poses a significant challenge due to the necessity of real-time processing, context sensitivity, capturing complex human behavior, effectively integrating multiple modalities, and addressing class imbalance. In this paper, we present a novel deep learning network-based approach for predicting turn-taking events in HAI that leverages information from multiple modalities, including text, audio, vision, and context data. Our study demonstrates that incorporating additional modalities, including in-game context data, enables a more comprehensive understanding of interaction dynamics leading to enhanced prediction accuracy for the artificial agent. The efficiency of the model also permits potential real-time applications. We evaluated our proposed model on an imbalanced dataset of both successful and failed turn-taking attempts during an HAI cooperative gameplay scenario, comprising over 125,000 instances, and employed a focal loss function to address class imbalance. Our model outperformed baseline models, such as Early Fusion LSTM (EF-LSTM), Late Fusion LSTM (LF-LSTM), and the state-of-the-art Multimodal Transformer (Mult). Additionally, we conducted an ablation study to investigate the contributions of individual modality components within our model, revealing the significant role of speech content cues. In conclusion, our proposed approach demonstrates considerable potential in predicting turn-taking events within HAI, providing a foundation for future research with physical robots during human-robot interaction (HRI).

Keywords: Embodiment, Empathy and Intersubjectivity, Creating Human-Robot Relationships, Anthropomorphic Robots and Virtual Humans
Abstract: Trust is essential for social interactions, including those between humans and social artificial agents, such as robots. Several robot-related factors can contribute to the formation of trust. However, previous work has often treated trust as an absolute concept, whereas it is highly context- dependent, and it is possible that some robot-related features will influence trust in some contexts, but not in others. In this paper, we present the results of two video-based online studies aimed at investigating the role of robot voice and empathic behaviour on trust formation in a general context as well as in a task-specific context. We found that voice influences trust in the specific context, with no effect of voice or empathic behaviour in the general context. Thus, context mediated whether robot-related features play a role in people's trust formation towards robots.

Keywords: Monitoring of Behaviour and Internal States of Humans, Multimodal Interaction and Conversational Skills, Motivations and Emotions in Robotics
Abstract: In recent years, robots that estimate emotions in real time have been proposed and are expected to be used in nursing care and at home. Especially in the nursing care field, there is a demand for both reduction of care burden and maintenance, and improvement of quality. Among emotion estimation technologies, those that incorporate methods using physiological signals can constantly acquire real-time human emotional reactions using physiological signals such as EEG and HRV from wearable sensors. We believe that if this technology can be used for mental care of the elderly, it will be possible to reduce the burden and improve the quality of nursing care. For this purpose, we prototyped a robot that responds to emotion of humans in real time, and making facial expressions, body movements, and speeches to improve or maintain the emotional state of user, and we conducted experiments to apply this robot to three elderlies. As the result of experiments, it was confirmed that the emotional states of them improved during the interaction with the robot and their ratios of voluntary utterance, when they could hear the robot, ranged between 58% to 81%.

Keywords: Detecting and Understanding Human Activity
Abstract: This paper proposes an online adaptation method for Fourier series based acoustic transfer function (TF) models for robot audition systems based on microphone array signal processing. The TF represents the signal propagation characteristics between a microphone and a sound source, which is essential for real-world scene analysis, including sound source localization and separation for robots. The real-world applications of TF-based array signal processing requires two characteristics: 1) adaptability to changes in the acoustic environment (changes in the signal propagation characteristics between the sound source and the microphone), and 2) a lightweight TF set for use in embedded systems such as robots with limited memory and computational resources. This paper proposes an online adaptation method for lightweight TF models using the Fourier series expansion. This method has both of the above two characteristics. Experimental results showed that the use of TF set adapted online using the proposed method performs better sound source localization and separation performance than existing online TF adaptation methods.

Keywords: Sound design for robots, User-centered Design of Robots, Non-verbal Cues and Expressiveness
Abstract: Sound can benefit human-robot interaction, but little work has explored questions on the design of nonverbal sound for robots. The unique confluence of sound design and robotics expertise complicates these questions, as most roboticists do not have sound design expertise, necessitating collaborations with sound designers. We sought to understand how roboticists and sound designers approach the problem of robot sound design through two qualitative studies. The first study followed discussions by robotics researchers in focus groups, where these experts described motivations to add robot sound for various purposes. The second study guided music technology students through a generative activity for robot sound design; these sound designers in-training demonstrated high variability in design intent, processes, and inspiration. To unify the two perspectives, we structured recommendations through the design thinking framework, a popular design process. The insights provided in this work may aid roboticists in implementing helpful sounds in their robots, encourage sound designers to enter into collaborations on robot sound, and give key tips and warnings to both.

Keywords: Sound design for robots, Applications of Social Robots, Robot Companions and Social Robots
Abstract: Robots may be able to significantly assist older adults through making activity recommendations. Prior research suggests that gender and age of a robot's voice may affect how people respond to such recommendations, but few studies have explored how a robot's voice is perceived by older adults, and whether their perceptions differ across cultures. We conducted a survey study with older adult participants (aged 65+) in the U.S. (N=225) and Japan (N=466), asking them to evaluate a humanoid robot speaking with three different voices (male, female, child). After seeing a video of a robot making recommendations, participants rated the fit of the voice to the robot, its sociality (via the Robotic Social Attributes Scale - RoSAS), and their willingness to use the robot in various contexts. We discovered that robot's social attributes and participants' culture impacted willingness to use the robot in both countries. Having positive social attributes and lower negative attributes increases willingness to use the robot. The U.S. older adults preferred the adult robot voices, had more positive social attributes, less negative social attributes, and were more likely to accept lifestyle recommendations than Japanese older adults. This study contributes to our understanding of older adults' perceptions of robot voice and provides design implications for robots that make recommendations to older adults.

Keywords: Sound design for robots, Androids, Anthropomorphic Robots and Virtual Humans
Abstract: This study examines the impact of gender neutralization on the anthropomorphism of speech signals and is motivated by the need to find a computer-generated voice for our android robot Andrea. A filter was used to gender-neutralize recordings from natural and synthetic voices, which were then pre-tested for gender neutrality. The results of the main experiment showed that gender-neutral voices were less anthropomorphic than gender-specific voices, with the naturalness of the voice having a greater impact than gender neutralization itself. Synthetic voices were rated less anthropomorphic than natural voices, and the additional effect of gender neutralization was stronger for natural voices. Overall, anthropomorphism ranked highest for natural gender-specific voices, followed by natural gender-neutralized voices, synthetic gender-specific, and, finally, synthetic gender-neutralized voices. Gender of the participants had no significant impact. These findings have implications for the development of humanoid and social as well as android robots.

Keywords: Detecting and Understanding Human Activity, Social Intelligence for Robots, Sound design for robots
Abstract: It is useful for a social robot to detect and locate users based on their speech. Notable challenges hampering the effective localization of a speaker are background noise and overlapping speech. Convolutional Neural Networks (CNNs) have yielded good performance on locating single speakers on a curated dataset, but to a lesser extent in scenarios with two speakers. In addition, their computational cost is still too high for a timely reaction in real-world settings. We build on the current state-of-the-art CNN approach, and propose several improvements for distinguishing multiple speakers by time-alignment in the input representation and reducing computational costs by considerably shortening the input audio blocks. We evaluate this approach on an existing dataset with blocks of noisy and overlapping speech recorded in rooms of different sizes, predicting the number of active speech events and their azimuth locations. The results show that our approach outperforms other approaches in locating two speakers and is considerably faster than the best-performing alternative approach. The time-domain information in the input representation was found essential for predicting the location of the signal source.

Keywords: Applications of Social Robots, Cognitive Skills and Mental Models, Detecting and Understanding Human Activity
Abstract: Behavioral economics has revealed that human decision making is not rational and logical as previously thought. It is strongly dependent on the environment and context, and is influenced by various biases. In this paper, we focus on the type of information technology and findings that can be used to construct such environments and contexts to improve decision making. An additional objective is to help humans change their behavior to adapt to the context. For this purpose, the design of ambient intelligent systems is proposed in this study. Specifically, by integrating the concepts of “nudge” and “boost” in behavioral economics, we propose a method to design “nudge & boost agents” that encourage users to change their behavior to adapt to a situation (nudge function) and foster individual decision-making skills (boost function). The nudge and boost agents are implemented on the basis of ambient intelligent systems based on previous human-agent interaction (HAI) research. In this paper, we describe the concept of these agents and the implementation of the system. Furthermore, based on the evaluation results, the effectiveness of the system is clarified.

Keywords: Social Learning and Skill Acquisition Via Teaching and Imitation, Virtual and Augmented Tele-presence Environments, Anthropomorphic Robots and Virtual Humans
Abstract: Perspective-taking, which enables individuals to consider the thoughts and objectives of another, is well established to be a successful strategy for encouraging pro-social behavior in human-computer interactions. Nowadays, perspective-taking is no longer limited to text; it is now more frequently used in virtual reality (VR). However, most previous research has focused on simulating human-human interactions in the real world in VR by providing participants with experiences connected to different moral tasks. In this study, we investigated whether participants’ prosocial behaviors toward robots would change if they experienced an altruistic VR task involving robots from the perspective of different robots. Our findings show that participants who had the help-receiver-view exhibited more altruistic behaviors toward a robot than those who had the help-provider-view one in a dictator game. We believe that this work is the first attempt to investigate the relationship between perspective-taking in a VR environment and changes in prosocial behavior in human-robot interaction.

Keywords: Monitoring of Behaviour and Internal States of Humans, Degrees of Autonomy and Teleoperation, Curiosity, Intentionality and Initiative in Interaction
Abstract: Various corporate and commercial responses to the COVID-19 pandemic have fueled opportunities for remote communication, including online meetings, and such technological developments have increased.This situation has created a focus on telepresence avatar robots, a technology through which people feel as if they are sharing the same place, even between remote locations.Approaches based on telepresence avatar robots support communication by generating a realistic sense of conversation by incorporating such technologies as camera motion and eye contact with the speaker.However, such approaches involve conscious manipulation by the robot's operator of actions that are unconsciously performed by human operators.Compared to local communication, it is difficult to achieve a sense of presence for a robot operator.In contrast, we implemented autonomous behavior that resembles the joint attention unconsciously performed by humans and addressed whether a robot operator feels that he or she is performing joint attention and gains a sense of presence through such autonomous behaviors.Joint attention is an unconscious interaction in which both parties share an object of observation in a local conversation.We speculate that when a robot autonomously performs this behavior, its operator's awareness of the person confronting its robot and the remote environment increases, creating a sense of presence as the operator gains a sense of immersion in its robot.Our experimental results suggest that autonomous behavior generates joint attention and gaze-intention estimation, and this approach may be a useful method with which robot operators can obtain a sense of presence in environments and predict the behavior of others.

Keywords: Computational Architectures, Non-verbal Cues and Expressiveness, Social Learning and Skill Acquisition Via Teaching and Imitation
Abstract: Trust is an essential concept for human-human and human-robot interactions. Yet only a few studies have addressed this concept from a robot perspective -- that is, forming robot trust in interaction partners. Our previous robot trust model relies on assessing the trustworthiness of the interaction partners based on the computational cognitive load incurred during the interactive task [1]. However, this model does not take into account the social markers indicative of trustworthiness, such as the gestures displayed by a human partner. In this study, we make a step toward this point by extending the model by integrating a social cue processing module to achieve social human-robot interaction. This new model serves as a novel social cognitive trust framework to enable the Pepper robot to evaluate the trustworthiness of its interaction partners based on both cognitive load (i.e., the cost of perceptual processing) and social cues (i.e., their gestures). For evaluating the efficacy of the framework, the Pepper robot with the developed model is put to interact with human partners who may take the roles of a reliable, unreliable, deceptive, or random suggestion providing partner. Overall, the results indicate that the proposed framework allows the Pepper robot to differentiate the guiding strategies of the partners by detecting deceptive partners and thus select a trustworthy partner in case of a free choice to perform the next task.

Keywords: Philosophical Issues in Human-Robot Coexistence, Cooperation and Collaboration in Human-Robot Teams, Ethical Issues in Human-robot Interaction Research
Abstract: Future robots are expected to be autonomous actors, even capable of moral reasoning. Yet how they can provide transparent explanations while being socially intelligent during morally relevant interactions deserves a close examination. Our mixed-methods lab study on a human-robot moral debate on the footbridge dilemma showed that quantitatively, a robot's perceived competence was significantly higher with transparency cues (additional information presented on a screen). The robot's perceived warmth and mind were not influenced by transparency cues, but they did significantly change over time (pre- vs. post-debate). The change in the robot's perceived mind and social attributes after the debate correlated with people's trust in the robot; transparency cues did not correlate with trust. Qualitatively, the robot was described to be logical, unemotional, and intentional in making moral decisions; participants focused on its gaze and speech. While transparency may help in theory, if people do not observe relevant cues while attributing intentionality to the robot and its gaze, transparency cues may not be useful during critical decision-making though the robot appears to be competent. We discuss the implications for moral HRI research and call for broadening the notion of transparency to investigate how robots can be transparent communicators by appealing to both cognition and affect in morally sensitive interactions.

Keywords: Motivations and Emotions in Robotics, Detecting and Understanding Human Activity, Curiosity, Intentionality and Initiative in Interaction
Abstract: Feeling that a non-human object has emotions (hereinafter referred to as “emotional state attribution”) is generally known as animacy perception. Previous studies have considered appearance and motion separately as factors that evoke emotional state attribution. Thus, if both the degree of human likeness in an object’s shape and the presence or absence of motion are considered simultaneously, we must consider whether there a possibility of strong emotional state attribution occurring even for objects that are not human-like in shape. In this study, we experimentally investigated the influence of human likeness in shape and movements evoking social relations on emotional state attribution, including their interaction, using three types of objects (human figure, shimeji mushroom, and match). In Experiments 1 and 2, although the human figure was rated as more human-like than the shimeji mushroom in terms of shape, emotions were attributed more strongly to the shimeji mushroom than to the human figure when accompanied by movements that evoked social relationships. The results suggest that people may attribute emotions more strongly to objects that resemble humans only to a certain extent in terms of shape when they show movements that evoke social relations.

Keywords: Assistive Robotics, Cooperation and Collaboration in Human-Robot Teams, Applications of Social Robots
Abstract: This study investigated how displaying a robot’s attention heatmap while the robot point gesture at it influences human trust and acceptance of its outputs. We conducted an experiment using two types of visual tasks. In these tasks, the participants were required to decide whether to accept or reject the answers of an AI or robot. The participants could see the answers with an AI attention heatmap, the heatmap with AI pointing (displayed as a laser dot cursor), a robot attention heatmap with robot pointing (pointing at a certain location on the heatmap displayed on a tablet with a stick), or no heatmap. The experimental results revealed that the AI and robot pointing at their attention heatmaps lowered the participants’ acceptance of their answers when the heatmaps had low interpretability in a more difficult task. Also, the robot pointing at the heatmaps showed the possibility of increasing acceptance of its answer when the heatmaps had high interpretability in a more difficult task. In addition, the acceptance of the robot’s answers correlated with emotional trust in the robot. This study demonstrates that a robot pointing gesture at its attention heatmap could be used to control human behaviors and emotional trust in human-robot interactions.

Keywords: Cooperation and Collaboration in Human-Robot Teams, User-centered Design of Robots
Abstract: Increasingly, robots are adopted for routine tasks such as cleaning and disinfection of public spaces, raising questions about attitudes and trust of professional cleaners who might in future have robots as teammates, and whether the general public feels reassured when disinfection is carried out by robots. In this paper, we present the results of a mixed-methods user study exploring how trust and reassurance by both professional cleaners and members of the public is affected by the use of a UV-C disinfection robot and information about its performance after disinfecting a simulated classroom. The results show a range of insights for those designing and wishing to deploy UV-C robots: we found that trust and reassurance are affected by information about the UV-C robot’s task performance, with more information coinciding with significantly more agreement to be able to judge that the robot is doing a good job. However, care should be taken when designing information about task performance to avoid misinterpretation. Overall, the results suggest a generally positive picture regarding the use of UV-C disinfecting robots and that cleaning professionals would be happy to have them as their teammates; however, there were also some concerns regarding the effect on less-skilled jobs. Taken together, our results provide considerations to make UV-C robots welcomed by cleaning teams as well as to provide reassurance to space users.

Keywords: HRI and Collaboration in Manufacturing Environments, Evaluation Methods, Human Factors and Ergonomics
Abstract: Investigating application areas for utilizing robots to support human workers is a continuing concern within human robot collaboration. In this paper, we focus on surface repair and finishing processes in which robots support skilled workers in task execution. We conducted a user study which investigates novel, pen-based human robot interaction modalities for a collaborative spot repair processes in a Virtual Reality (VR) setting. Our findings show that participants preferred a test condition in which all interaction with the robot took place directly on the work piece. Furthermore, the interaction modality had an impact on objective performance indicators. These findings are discussed in the context of designing intuitive interfaces for collaborative robots in industrial settings.

Keywords: HRI and Collaboration in Manufacturing Environments, Human Factors and Ergonomics, Motion Planning and Navigation in Human-Centered Environments
Abstract: Close-proximity human-robot interactions can be improved through the optimization of task-centric factors or by prioritizing the user experience. Prior work has often explored these factors individually. In this paper, we conducted a within-subject study with 18 participants that compared a multi-objective robot motion adaptation method (CoMOTO) against methods that optimize distance from the user (UserAvoidant) or task performance (ShortestPath) in a close-proximity human-robot interaction task. In the task, the robot and participants worked on different tasks in an overlapping workspace. We show that while CoMOTO trajectories took a longer time, they caused significantly fewer interruptions compared to ShortestPath and generated shorter trajectories than UserAvoidant. CoMOTO was also perceived as significantly more intelligent, more trustworthy, and preferred by an overwhelming majority of the participants.

Keywords: HRI and Collaboration in Manufacturing Environments, Motion Planning and Navigation in Human-Centered Environments, Cooperation and Collaboration in Human-Robot Teams
Abstract: This paper addresses human-robot collaboration (HRC) challenges of integrating predictions of human activity to provide a proactive-n-reactive response capability for the robot. Prior works that consider current or predicted human poses as static obstacles are too nearsighted or too conservative in planning, potentially causing delayed robot paths. Alternatively, time-varying prediction of human poses would enable robot paths that avoid anticipated human poses, synchronized dynamically in time and space. Herein, a proactive path planning method, denoted STAP, is presented that uses spatio-temporal human occupancy maps to find robot trajectories that anticipate human movements, allowing robot passage without stopping. In addition, STAP anticipates delays from robot speed restrictions required by ISO/TS 15066 speed and separation monitoring (SSM). STAP also proposes a sampling-based planning algorithm based on RRT* to solve the spatio-temporal motion planning problem and find paths of minimum expected duration. Experimental results show STAP generates paths of shorter duration and greater average robot-human separation distance throughout tasks. Additionally, STAP more accurately estimates robot trajectory durations in HRC, which are useful in arriving at proactive-n-reactive robot sequencing.

Keywords: HRI and Collaboration in Manufacturing Environments, Virtual and Augmented Tele-presence Environments, Linguistic Communication and Dialogue
Abstract: Collaborative robots seamlessly share the space with humans in production scenarios such as those involved in smart manufacturing and agriculture, thus raising several human safety concerns. Since a collaboration between humans and robots is performed through communicative acts, applying accurate techniques for understanding them is of the utmost importance to guarantee the overall safety of the human. A preliminary classification of the communicative acts into categories is required to increase the accuracy of adopted methods and the promptness of the response. This paper evaluates a speech communicative act classification methodology in the challenging scenario of precision agriculture using Virtual Reality (VR). Our proposal can easily be applied to any production scenario involving collaborative robots.

Keywords: HRI and Collaboration in Manufacturing Environments, Cooperation and Collaboration in Human-Robot Teams, Degrees of Autonomy and Teleoperation
Abstract: Robots have become ubiquitous tools in various industries and households, highlighting the importance of human-robot interaction (HRI). This has increased the need for easy and accessible communication between humans and robots. Recent research has focused on the intersection of virtual assistant technology, such as Amazon’s Alexa, with robots and its effect on HRI. This paper presents the Virtual Assistant, Human, and Robots in the loop (VAHR) system, which utilizes bidirectional communication to control multiple robots through Alexa. VAHR’s performance was evaluated through a human-subjects experiment, comparing objective and subjective metrics of traditional keyboard and mouse interfaces to VAHR. The results showed that VAHR required 41% less Robot Attention Demand and ensured 91% more Fan-out time compared to the standard method. Additionally, VAHR led to a 62.5% improvement in multi-tasking, highlighting the potential for efficient human-robot interaction in physically- and mentally-demanding scenarios. However, subjective metrics revealed a need for human operators to build confidence and trust with this new method of operation.

Keywords: Soft Sensors and Actuators, Soft Robot Materials and Design, Soft Robot Applications

Keywords: HRI and Collaboration in Manufacturing Environments, Human Factors and Ergonomics
Abstract: In the emerging phase of industrialization, Industry 5.0, humans will be working alongside advanced technologies such as Artificial Intelligence (AI) and robots to improve the manufacturing process. As a result, it is crucial to evaluate trust in the robot from a human perspective in order to provide a safe environment and balance workloads. Relevant trust indicators in the industrial context include proximity between human and robot, as well as risk associated with robot’s performance. In this study, a chemical industry scenario was developed, where a robot assists a human in mixing chemicals. An experiment was conducted for analysing how proximity and risk impact the trust level of the participants. According to the results, there was a higher average trust score in the low proximity (i.e. robot not close to the human) and low risk sections compared to the high proximity and high risk sections of the experiment, respectively. Moreover, statistical analysis indicates that risk had a higher impact on trust than proximity. The findings of this study encourage further research in this area since tools such as AI could be used to control the robot’s behavior according to the level of trust between the human and the robot.

Keywords: HRI and Collaboration in Manufacturing Environments, Machine Learning and Adaptation, Detecting and Understanding Human Activity
Abstract: Assemble-to-Order (ATO) has become a popular production strategy for the increasing demand for mass-customized manufacturing. In order to facilitate a flexible and automated Human-Robot-Collaboration system for ATO, we propose a Learning from Demonstration (LfD) framework based on 2D videos in this paper. We initially combine temporal hand motions with spatial hand-object interactions to detect assembly actions. Therefore, an assembly graph can be constructed using classified action sequences. Compared to previous studies on task planning for robots, our graph-based semantic planner can directly learn the demonstrated task structure and thus produce more detailed assistive robot actions for more effective collaboration. We validate our approach by applying it to a real-world ATO problem. The results demonstrated that our proposed system can produce actions adaptively in response to varying human action sequences, as well as guide human assembly when the robot is not involved. Our approach also shows generalizability to unseen human action sequences.

Keywords: Monitoring of Behaviour and Internal States of Humans, Machine Learning and Adaptation, Child-Robot Interaction
Abstract: To make social robots effective in education, they need to be autonomous both in terms of assessing the student's engagement state as well as intervening effectively in soft real-time when necessary. Hidden Markov Model (HMM) is an interpretable machine learning technique for modeling temporal data that is commonly used post-hoc to analyse latent learning processes. In this paper, we contribute by proposing an HMM-based intervention methodology for assessing and classifying the state of the student as either productive or unproductive in soft real-time. The system identifies and tracks states and patterns not conducive to learning, and a robot intervention is triggered whenever a too-high non-productive engagement is detected. In a pilot study with 22 children, we evaluate this methodology in terms of both 1) the effectiveness of the interventions on the students' learning gains and on behaviors found conducive to learning, and 2) the students' perception of the robotic interventions. Results suggest that the robot interventions have a positive effect on the post-test scores relative to the baseline robot, although there isn't a significant difference in the learning gains. Moreover, interventions that try to induce reflective behaviors are most effective in inducing the required learning behavior, followed by communication-inducing interventions. Lastly, students' perception of intervention usefulness does not reflect their actual effectiveness.

Keywords: Anthropomorphic Robots and Virtual Humans, Monitoring of Behaviour and Internal States of Humans, Affective Computing
Abstract: Everyday life scenarios where non-expert users (e.g., customers) are confronted with frontline service robots will become more and more likely. In particular, misunderstandings and incidents may occur during these interactions because of wrong expectations of the robot's capabilities. Current applicable laws are based on technological assumptions from prior decades unsuitable to modern robotics and AI. The new AI Act as a part of the solution to this is still in development. In addition to the pure legal view, a technological viewpoint may be beneficial for establishing a fitting, trustful, and, thus, acceptable technology liability law. This work contributes to this by empirically evaluating the service robot non-expert user's liability expectations, the use of robots, and well-being. The results in a DIY store environment significantly show that the store deploying the robot should be liable if an incident happens. Further, we examined that even a minor simulated incident affected the participants' emotions and moods. Consequently, this influences their perception of liability while not mitigating their acceptance of frontline service robots.

Keywords: Monitoring of Behaviour and Internal States of Humans, Evaluation Methods, Assistive Robotics
Abstract: Trust is a crucial part of human-robot interactions, and its accurate measurement is a challenging task. We introduce Trusty, a handheld continuous trust level measurement device and investigate its validity by analysing the correlation between its measurements and self-reported trust scores. In a study with 29 participants, we evaluated the effectiveness of the device with an autonomous wheelchair in a mobile navigation task. The participants collaborated with an autonomous wheelchair to deliver packages to predefined checkpoints in an unstructured environment, and the performance of the wheelchair was manipulated to be either under a good-performing condition or a bad-performing condition. Our first finding reveals a notable influence of wheelchair performance on self-reported trust. Participants interacting with a good-performing wheelchair exhibited increased trust levels, as evidenced by higher scores on post-experiment trust questionnaires and verbal self-reported trust measures. Additionally, our study proposes Trusty as a continuous measurement tool for assessing trust during HRI, demonstrating its equivalence to self-report measures and traditional questionnaire scores.

Keywords: Monitoring of Behaviour and Internal States of Humans, Detecting and Understanding Human Activity, Curiosity, Intentionality and Initiative in Interaction
Abstract: This paper explores whether experiential knowledge of computer vision from interacting with daily apps (e.g., Instagram, Zoom, etc.) can be leveraged to improve users’ expectations of robotic capabilities. We evaluate users’ ability to predict when computer vision apps might fail and if they can apply their experience to reason about computer vision in robotic systems. We show that although users can reliably predict computer vision app capabilities and functionality, they tend to ascribe human-level knowledge to those apps and do not reliably correlate app functionality with similar robotic tasks. We propose that experiential knowledge gained through interaction with software apps is a potential way to “calibrate” user expectations of the function and failure states of complex systems.

Keywords: Multimodal Interaction and Conversational Skills, Social Touch in Human–Robot Interaction
Abstract: This work presents a framework for a visual-auditory attention-driven robot eye-head gaze movement, which combines visual and auditory inputs to determine the direction of gaze movement for a social robot. The framework computes the most salient changes in position by considering both visual and auditory cues. The proposed system was implemented on Haru, a tabletop social robot, where eye-head gaze movement was controlled using visual input from a camera positioned above the eyes and auditory input from a seven-channel microphone. This allowed for eye movement on a two-dimensional flat screen and body rotation towards the person who is speaking. This framework provides a representation of the robot's attentional gaze that leverages both visual and auditory cues, resulting in more natural and responsive coordinated eye-head gaze movements of the social robot. The potential benefits include improved communication, increased engagement, and a stronger sense of connection with the robot.

Keywords: Multimodal Interaction and Conversational Skills, Machine Learning and Adaptation, Applications of Social Robots
Abstract: Gaze cues, which initiate an action or behaviour, are necessary for a responsive and intuitive interaction. Using gaze to signal intentions or request an action during conversation is conventional. We propose a new approach to estimate gaze using a neural network architecture, while considering the dynamic patterns of real world gaze behaviour in natural interaction. The main goal is to provide a basis for robot or avatar to communicate with humans using multimodal natural dialogue. Currently, robotic gaze systems are reactive in nature but our Gaze-Estimation framework can perform unified gaze detection, gaze-object prediction and object-landmark heatmap in a single scene, which paves the way for a more proactive approach. We generated 2.4M gaze predictions of various types of gaze in a more natural setting (GHI-Gaze). The predicted and categorised gaze data can be used to automate contextualized robotic gaze-tracking behaviour in interaction. We evaluate the performance on a manually-annotated data set and a publicly available gaze-follow dataset. Compared to previously reported methods our model performs better with the closest angular error to that of a human annotator. As future work, we propose an implementable gaze architecture for a social robot from Furhat robotics

Keywords: User-centered Design of Robots, Robot Companions and Social Robots
Abstract: Mental health challenges became more prevalent during the COVID-19 pandemic, especially among older adults. Consequently, we witnessed an uptake of new technologies, including social robots to address these challenges. However, we observed limited inclusion of older adults in the design process to design these technologies to cater user needs during the pandemic. To address this gap, we conducted a co-design workshop with 17 older adults and explored their emotional challenges after the COVID-19 pandemic. They evaluated the current social robot designs available in the literature and elicited the design preferences for a social robot to address their current emotional challenges. Our results based on thematic analysis show that the impact of the pandemic on older adults' emotional challenges is persisting, and the companionship of a social robot is preferred to enhance their mental well-being. We also show that older adults preferred an animal-like robot design embodied with soft skin possessing a medium size. These findings highlighted older adults' design choices of a social robot and affirmed their potential to support older adults' mental well-being.

Keywords: Monitoring of Behaviour and Internal States of Humans, Detecting and Understanding Human Activity, Applications of Social Robots
Abstract: In recent years, communication robots have been employed to assist workers. However, it is known that users experience embarrassment when interacting with a robot in a public space, which may hinder their use. Previous studies investigated methods to reduce embarrassment when using robots and the factors that cause embarrassment. Although these studies have investigated the embarrassment experienced by users through only a single interaction, they have not investigated the embarrassment influenced by users' past experiences. Therefore, in this study, we investigated changes in embarrassment and the factors causing embarrassment through repeated interactions with robots in public spaces. We conducted experiments in which the same participants used a robot in a public space multiple times and continuously experienced embarrassment through repeated interactions with the robot. The results show that repeated interactions with the robot reduce embarrassment, and that embarrassment is influenced by two factors: understanding of the user's behavior from surrounding people and the user's previous experience with the interaction.

Keywords: Cooperation and Collaboration in Human-Robot Teams, Machine Learning and Adaptation, HRI and Collaboration in Manufacturing Environments
Abstract: Humans can perform seamless object handovers with little to no effort. These handovers are characterized by an early movement onset that anticipates the handover location and a smooth velocity profile with minimal trajectory corrections. Replicating these characteristics in an object handover task between humans and robots presents a significant modeling challenge. In this paper we implement a Gaussian Process prediction model to serve as a robotic surrogate of human inference, and investigate how this model affects the kinematics of a human giver handing an object to the robot. Additionally, we analyze how the resulting robot kinematics compare to those of a human, and gauge human comfort through subjective reporting. Human giver kinematics during human-robot handover compared closely to human-human giver kinematics with respect to movement speed, movement timing, movement smoothness, and handover distance. Notable differences were observed in reach time and receiver peak transport velocity. When asked how well four attributes of their human-robot handovers (receiver competence, handover comfort, handover naturalness, handover safety) compared to those attributes in human-human handovers, subjects gave mean scores ranging from 4.43 (naturalness) to 5.13 (safety) on a 7 point Likert scale.

Keywords: Cooperation and Collaboration in Human-Robot Teams, Motion Planning and Navigation in Human-Centered Environments, HRI and Collaboration in Manufacturing Environments
Abstract: This paper studies the perception of robot intelligence for a manipulator that shares its workspace with a human operator, while the two execute independent tasks. Four different motion planning algorithms were employed to plan the robot motion in four subsequent experimental sets for 48 participants, and the order with which these algorithms were used was changed, obtaining all possible combinations. While guaranteeing safety, each of the four algorithms exhibited a different level of adaptability and reactivity, in terms of real-time motion planning abilities and of speed reduction based on heart rate feedback. We analyzed how perceived intelligence was influenced by the employed algorithm, by the order of execution and by the previous experience of the participants working with robots. In conclusion, perceived intelligence resulted being significantly lower in the first experimental set compared to the following ones, which showed a positive effect of habituation on perceived intelligence. On the other hand, neither the use of different motion planning algorithms nor the previous participants' experience significantly influenced perceived intelligence.

Keywords: Programming by Demonstration, HRI and Collaboration in Manufacturing Environments, Motion Planning and Navigation in Human-Centered Environments
Abstract: Tool manipulation is a crucial skill for robots to perform intricate tasks, and learning from demonstration methods can provide an effective means for robots to learn these skills. However, the process of collecting human demonstration data can be challenging and may lead to information loss, requiring a large number of demonstrations to learn the human's policy. In this work, we propose a novel framework for collecting information-rich human demonstration data for learning complex tool manipulation skills. Our framework can accommodate data collection from multiple modalities such as speech, gesture, motion, video, and 3D depth data. Additionally, the framework actively queries the human expert to improve the information content of the data. We showcase the effectiveness of our method in collecting demonstration data for a complex granular media transport task and performing the task on a real robot.

Keywords: Anthropomorphic Robots and Virtual Humans, Innovative Robot Designs, Novel Interfaces and Interaction Modalities
Abstract: In this paper, we propose a modular soft-rigid hybrid (MSRH) approach for designing a highly anthropomorphous and dexterous robotic hand. This MSRH approach allows the robotic hand to possess an inherently soft and compliant interface suitable for pHRI while maintaining the structural rigidity of the robot with the usage of rigid skeletons. We share the details of the design rationale, fabrication and assembly methods, and evaluation of the first prototype. Even though the presented prototype is scaled to be 125% of the average human hand, the mechanical components only weigh less than 450 g. The modular design approach allows the MSRH hand to have low manufacturing costs and a short lead time. Creation of the presented prototype costs less than 100 CAD and can be built in three days from scratch with two-person labour. The usage of pneumatic soft robotic actuators also provides flexibility over choosing the number of controlled DOF and joint coupling. This hence provides a new angle to tackle spatial constraint that normally arises in robotic hand design with increased anthropomorphism. Lastly, the dexterity of the proposed hand is demonstrated by evaluating the hand using taxonomies highly relevant to replicating tasks humans perform daily.

Keywords: Programming by Demonstration, Detecting and Understanding Human Activity, Assistive Robotics
Abstract: We address the problem of teaching a service robot to detect doors and drawers in indoor environments. We propose a robust and accurate method in which a human demonstrates to the robot how to open doors and drawers that the robot is expected to operate in its future use. The proposed algorithm creates a model of a door or drawer from a sequence of RGB-D images and inserts it into an environment map. The model contains information about the size of the door panel or drawer front, as well as the position and orientation of the joint axis. This augmented environment map is then used by the robot to detect the target object in its environment and estimate its state.

Keywords: Anthropomorphic Robots and Virtual Humans, Human Factors and Ergonomics, User-centered Design of Robots
Abstract: This paper reports a failed attempt to predict deviations of human arm movements from task elicited by perception of simultaneous robot arm movements (motor in-terference) using subjective ratings of anthropomorphism. This motor interference effect is known to result from unconscious imitation of movements of a human-like counterpart due to mirror-neuron activation. A study is conducted, with test participants and a robot conducting vertical and horizontal arm movements in front of each other in all combinations with and without a robot head being present in a between-subjects design. Perceived anthropomorphism is acquired using the evaluated tests HRIES and Godspeed. A correlation analysis on the dependency of motor interference on anthropomorphism did not reveal significant results of the constructs targeting human-like appearance, intelligence or movement behavior, which were expected to represent relevant predictor parameters. Further, scatter plots do not show a clear pattern. Only the likability Godspeed sub-scale is significant, which however may not be related to the targeted effect. Additionally, the ‘consciousness’ item within the ‘anthropomorphism sub-scale of Godspeed is significant, however, with unclear effect. So, no evidence could be found, that subjective assessments of anthropomorphism with existing tests may be suitable for prediction of motor interference and further research is required.

Keywords: Cooperation and Collaboration in Human-Robot Teams, Assistive Robotics, Degrees of Autonomy and Teleoperation
Abstract: Robotic solutions, in particular robotic arms, are becoming more frequently deployed for close collaboration with humans, for example in manufacturing or domestic care environments. These robotic arms require the user to control several Degrees-of-Freedom (DoFs) to perform tasks, primarily involving grasping and manipulating objects. Standard input devices predominantly have two DoFs, requiring time-consuming and cognitively demanding mode switches to select individual DoFs. Contemporary Adaptive DoF Mapping Controls (ADMCs) have shown to decrease the necessary number of mode switches but were up to now not able to significantly reduce the perceived workload. Users still bear the mental workload of incorporating abstract mode switching into their workflow. We address this by providing feed-forward multimodal feedback using updated recommendations of ADMC, allowing users to visually compare the current and the suggested mapping in real-time. We contrast the effectiveness of two new approaches that a) continuously recommend updated DoF combinations or b) use discrete thresholds between current robot movements and new recommendations. Both are compared in a Virtual Reality (VR) in-person study against a classic control method. Significant results for lowered task completion time, fewer mode switches, and reduced perceived workload conclusively establish that in combination with feedforward, ADMC methods can indeed outperform classic mode switching. A lack of apparent quantitative differences between Continuous and Threshold reveals the importance of user-centered customization options. Including these implications in the development process will improve usability, which is essential for successfully implementing robotic technologies with high user acceptance.

Keywords: Multifingered Hands, Grippers and Other End-Effectors, Soft Robot Applications
Abstract: Anthropomorphic robotic hands are seeking to achieve key features such as multi-degree-of-freedom motion ability, bi-directional actuation, high adaptability, and sufficient stiffness. In this research, we propose a 10 active degrees-of-freedom anthropomorphic robotic hand with a soft-rigid hybrid structure and positive-negative pneumatic actuation. The fingers of the hand utilize pneumatic actuators composed of soft bellows to actuate a rigid skeleton for bending motion with unidirectional-stretchable nylon elastic fabric as a strain limiting layer. The positive-negative pneumatic actuation mimics the flexors and extensors of the human hand to implement bi-directional finger actuation. We present a prototype implementation of this robotic hand and provide a preliminary evaluation. The experimental results show that the combined positive-negative pneumatic actuation can extend the workspace of the finger and increase the finger stiffness compared to using only positive pneumatic actuation. The proposed robotic hand has good comprehensive performance (generated a blocking force of 7.5 N, scored 7 in the Kapandji test, achieved 32 of the 33 grasp postures from the Feix taxonomy, and could implement 5 human grasping strategies) compared to other robotic hands in literatures.

Keywords: User-centered Design of Robots, Robots in Education, Therapy and Rehabilitation, Assistive Robotics
Abstract: Bimanual humanoid assistive robots can be a valuable tool to improve access to physical therapy and multidimensional physical status monitoring for older adults living at home. However, at present, there is no implementation of end-effector robot-assisted rehabilitation in assistive social robots. Therefore, this paper illustrates the first steps of a user-centered design approach to develop such a robot for upper limb treatments and rehabilitation. Based on observation of geriatric rehabilitation and expert interviews with physical therapists, an online survey was conducted with 87 physical therapists. The first part of the questionnaire aimed to better understand the context of use, current practices, and goals of geriatric rehabilitation. Our findings suggest that integrating exercises that combine physical and cognitive skills and aim to improve specific activities of daily living (ADLs) are critical. Secondly, a KANO analysis was conducted to prioritize 20 potential features for the robot. Among these features, assist-as-needed (AAN) control for physical exercises, voice control for general settings, and the capability to perform treatments while the patient is seated or lying down were identified as essential or "must-have" features. Third, the possibility of an autonomously conducted weekly assessment of patients' active range of motion (ROM) and weekly to monthly muscle function testing could allow the best possible monitoring of their functional status. Generally, applying a user-centered design approach allows an interdisciplinary team tasked with developing assistive robots to establish a common objective, based on which an initial prototype can be designed in the next step.

Keywords: User-centered Design of Robots, Assistive Robotics, Innovative Robot Designs
Abstract: In this paper, we present ALH-E (ALternative Healthcare for Expressing ache), an assistive robot with a deformable and flexible interface for pain communication. It consists of two components: a squeezable device for inputting the patient’s pain intensity and a flexible output device that expresses the pain by twisting–bending movements in response to the input signals. The interconnectivity between the devices allows for communication of pain intensity between patients and caregivers, anywhere and anytime. A field study in the hospital (dental clinic and orthopedic) was conducted to verify the usability and effectiveness of ALH-E in pain communication. Our field study results demonstrated the unique advantages of ALH-E over conventional methods, providing significant assistance to patients and caregivers.

Keywords: User-centered Design of Robots, Motion Planning and Navigation in Human-Centered Environments
Abstract: In this paper, we explore how robots can properly explain failures during navigation tasks with privacy concerns. We present an integrated robotics approach to generate visual failure explanations, by combining a language-capable cognitive architecture (for recognizing intent behind commands), an object- and location-based context recognition system (for identifying the locations of people and classifying the context in which those people are situated) and an infeasibility proof-based motion planner (for explaining planning failures on the basis of contextually mediated privacy concerns). The behavior of this integrated system is validated using a series of experiments in a simulated medical environment.

Keywords: Social Intelligence for Robots, Robotic Etiquette, Linguistic Communication and Dialogue
Abstract: Social robots will inevitably confront social or moral norm violations. While researchers have identified preliminary strategies for when and how robots should respond in such situations, it is not well understood how humans will make sense of these robot behaviors. We used qualitative, narrative-based methods inspired by design fiction to better understand how humans appraise these interactions. Our narrative survey invited participants to share their assumptions and expectations, analyze scenarios, and make suggestions. Our results highlight key situational and psychological factors that characterize norm-sensitive robot interactions, and suggest clear insights for the development of socially competent robot teammates.

Keywords: User-centered Design of Robots, Personalities for Robotic or Virtual Characters
Abstract: Non-anthropomorphic robots can be cost-effective and efficient choice in certain context in comparison to social or humanoid robots. However, introduction of non- anthropomorphic robots can evoke uncertainty and anxiety due to novelty of technology. The goal of this paper is to explore personality design space for non-anthropomorphic well- ness robots in office environment to foster acceptance among users. Through Participatory Design approach, we explored appropriate personalities for a well-being robot, which would detect employees’ sitting posture and suggest small wellness interventions. We addressed the following research questions: (i) How can personalities be designed and integrated to non- anthropomorphic wellness robots to promote users’ acceptance? (ii) How do the users perceive designed personalities of non- anthropomorphic wellness robot in the office context? We conducted one contextual inquiry (n=5) and one co-design workshop (n=15) followed by evaluation (n=5) in IT office environment with office employees . As a contribution to the paper, we present personalities and design implications for non-anthropomorphic wellness robot in the office context. Our contribution will serve as a guideline for designers to explore and expand their knowledge on designing robot personalities for non-anthropomorphic robots in the context.

Keywords: Detecting and Understanding Human Activity, Human Factors and Ergonomics, Machine Learning and Adaptation
Abstract: The drone industry is diversifying and the number of pilots increasing rapidly. In this context, flight schools need adapted tools to train pilots, most importantly with regard to their own awareness of their physiological and cognitive limits. In civil and military aviation, pilots can train on realistic simulators to tune their reaction and reflexes, but also to gather data on their piloting behavior and physiological states, helping to improve their performance. As opposed to cockpit scenarios, drone teleoperation is conducted outdoors in the field, with only limited potential from desktop simulation training. This work aims to provide a solution to gather pilot behavior in the field and help them increase their performance. We combined advanced object detection from a frontal camera with gaze and heart rate variability measurements. We observed pilots and analyzed their behavior over three flight challenges. We believe this tool can support pilots both in their training and in their regular flight tasks.

Keywords: Medical and Surgical Applications, Assistive Robotics, Evaluation Methods
Abstract: Cooperative robots for intraocular surgery allow surgeons to perform vitreoretinal surgery with high precision and stability. Several robot structural designs have shown capabilities to perform these surgeries. This research investigates the comparative performance of a serial and parallel cooperativecontrolled robot in completing a retinal vessel-following task, with a focus on human-robot interaction performance, user experience, and procedural safety. Our results indicate that despite differences in robot structure and control gain parameters, the two robots exhibited similar levels of performance in terms of robot-to-patient interaction force and average operating time. These findings have implications for the development and implementation of surgical robotics, suggesting that both serial and parallel cooperative-controlled robots can be effective and safe for this type of task.