Keywords: Linguistic Communication and Dialogue, Assistive Robotics, Multi-modal Situation Awareness and Spatial Cognition
Abstract: Navigating unfamiliar environments presents significant challenges for blind and low-vision (BLV) individuals. In this work, we construct a dataset of images and goals across different scenarios such as kitchens or outdoor navigation. We then investigate how grounded instruction generation methods can provide contextually-relevant navigational guidance to users in these instances. Through a study involving sighted users, we demonstrate that large pretrained language models can produce correct and useful instructions perceived as beneficial for BLV users. We also conduct a survey and interview with 4 BLV users and observe useful insights on preferences for different instructions based on the scenario.

Keywords: Linguistic Communication and Dialogue, Cooperation and Collaboration in Human-Robot Teams
Abstract: This study (N = 317) investigated the influence of verbal communication (social vs. functional) on the acceptance of robot recommendations in non-moral, somewhat moral, or very moral decision-making situations. The robot’s communication style had no impact on the participants (1) being confident in their decision, (2) perceiving the robot’s recommendation as helpful, and (3) making a decision dependent on the robot’s recommendation. However, all three aspects were strongly influenced by the morality of the decision situation demonstrating higher algorithm aversion in moral contexts.

Keywords: Long-term Experience and Longitudinal HRI Studies, Child-Robot Interaction, Robots in Education, Therapy and Rehabilitation
Abstract: Previous studies on the social acceptance of domestic robots have primarily focused on personal assistance. The present study broadens the scope by considering multi-user interactions in natural home settings. Specifically, we investigated how parent-child storytime may differ when a robot is present versus when it is not. Given that family storytime can serve both utilitarian purposes (e.g., learning) and hedonic purposes (e.g., bonding), we asked: (1) How may the use of the robot influence parents’ attitudes toward adopting children’s robots in the home? and (2) How may the presence of a storytelling robot affect parent-child interactions? Twenty-one families in the Midwestern United States received a zoomorphic robot for their children's storytime. Data were collected through pre-post tests, diary entries, and interviews. We found that parents' intention to adopt children’s robots decreased after seven days of the robot's presence. While data analysis is still in progress, we aim to provide an in-depth understanding of families' use of children’s robots in the home and propose guidelines for designing social robots for families.

Keywords: Machine Learning and Adaptation, HRI and Collaboration in Manufacturing Environments
Abstract: Recently, as the demand for automation in the textile industry has increased, there has been a growing need for the application of robots and vision systems. However, challenges such as the flexibility of textile materials and light reflection issues have made automation implementation and human-robot collaboration difficult. In this study, we developed a machine vision system which assists to connect unrecognized cut outlines. We utilized CAD images to remove noise caused by factors such as light reflection, and it is combined them with a grid to connect unrecognized cut outlines to consider the target shape. Additionally, we developed an assisting system for calculating gripping points based on the center of mass. We validated the accuracy and speed of cut outline recognition and gripping point determination when applied to the on-site robot system

Keywords: Machine Learning and Adaptation, Motion Planning and Navigation in Human-Centered Environments
Abstract: Many navigational problems in robotics necessitate a well defined cost map related to the environment. Traditional techniques in creating these involve manual specification of terrain costs based on some context known to the human. However, this becomes intractable with large numbers of terrain types. Preference learning offers a unique way of tackling this type of problem by inferring a reward function through trajectory queries. However, offline preference learning suffers from the variability of the initial dataset, which limits the amount of information that can be gained from query responses and introduces a higher degree of cognitive burden on the human. In this paper, we propose to utilize recent advancements in preference learning surrounding the use of generative models, specifically variational autoencoders as they utilize a lower dimensional latent space useful for clustering and inferring similarity or dissimilarity, to combat analogous or insufficient trajectory sets towards robotic navigation through learned terrain weights.

Keywords: Medical and Surgical Applications, Innovative Robot Designs
Abstract: Endoscopic retrograde cholangiopancreatography (ERCP) is a minimally invasive technique for treating biliary and pancreatic diseases. It requires inserting a guide wire into the bile duct or pancreatic duct to enable further procedures. However, this is challenging because the duct openings are at an obtuse angle to the direction of the instrument entering the duodenum. The guide wire may fail to adjust its position and posture, causing mis-insertion and tissue damage. To address this issue, we proposed a flexible surgical robot with a concentric design that combines magnetic-driven and tendon-driven methods. The NiTi wires work as both the backbone and the driving element of the outer tube, and a flexible inner part equipped with several magnets works as a guide wire that can bend in random directions. By using two steerable segments, our robot has higher flexibility and a lower chance of tissue damage. Then, we analyse the dexterity of the proposed mechanism and performed a phantom experiment. The result shows that it can perform insertion with different postures and adapt to the natural configuration of the natural cavity of a human.

Keywords: Multi-modal Situation Awareness and Spatial Cognition, Machine Learning and Adaptation, Motion Planning and Navigation in Human-Centered Environments
Abstract: Scene graphs have been proven to be useful for various scene understanding tasks due to their compact and explicit nature. However, existing approaches often neglect the importance of maintaining the symmetry-preserving property when generating scene graphs from 3D point clouds. This oversight can diminish the accuracy and robustness of the resulting scene graphs, especially when handling noisy, multi-view 3D data. This work, to the best of our knowledge, is the first to implement Equivariant Graph Neural Network in semantic scene graph generation from 3D point clouds for scene understanding. Our proposed method, ESGNN, outperforms existing state-of-the-art approaches, demonstrating a significant improvement in scene estimation with faster convergence. ESGNN demands low computational resources and is easy to implement from available frameworks, paving the way for real-time applications such as robotics and computer vision.

Keywords: Multimodal Interaction and Conversational Skills, Multi-modal Situation Awareness and Spatial Cognition, Novel Interfaces and Interaction Modalities
Abstract: Attention attraction in human-robot interaction (HRI) involves strategically guiding an individual's focus to specific stimuli to enhance engagement and responsiveness. On the other hand, attention shifting is defined as the ability to redirect a person's focus away from the current task through dynamic interventions like swiftly moving objects across their field of vision. Traditional methods, such as pointing, touch, and voice have been somewhat effective but lack the subtlety and precision of gaze-based interactions. While naturally effective in indicating interest, gaze cues often assume a mutual gaze, which limits their effectiveness in dynamic environments or when the robot is out of the person's field of view.

This paper proposes a system integrating augmented reality (AR) devices into the HRI setup to address these limitations. The system allows the robot to place virtual visual markers, visible through an AR headset. These markers not only serve as dynamic cues to attract the human’s attention but can also be used to seamlessly shift attention, independent of the robot’s physical position. By using AR technology, the robot can guide the user's focus to specific areas or objects, enhancing the interaction experience and expanding the potential applications of attention control in HRI.

Keywords: Non-verbal Cues and Expressiveness, Detecting and Understanding Human Activity, Creating Human-Robot Relationships
Abstract: This paper introduces an enhanced hand gesture recognition system utilizing inductive sensors, specifically developed for the identification of American Sign Language (ASL) gestures. The system is built into a regular glove with conductive threads by sewing coils onto various areas, including the fingers, wrist, palm, and ulnar regions. To increase the system's sensitivity, a data acquisition system measures a tank circuit created by each sewn coil and external components. The effectiveness of the system is rigorously evaluated on a sample of 10 participants. The identification of the 26 ASL letters is achieved using a random forest machine learning algorithm. To enhance the diversity and applicability of the machine learning model, a data augmentation scheme is utilized. The outcomes reveal a notable level of accuracy, achieving 98.91% accuracy with 5-fold cross-validation and 98.82% accuracy with leave-one-subject-out cross-validation.

Keywords: Novel Interfaces and Interaction Modalities
Abstract: Human-robot interaction (HRI) research plays a pivotal role in shaping how robots communicate and collaborate with humans. However, conducting HRI studies, particularly those employing the Wizard-of-Oz (WoZ) technique, can be challenging. WoZ user studies can have complexities at the technical and methodological levels that may render the results irreproducible. We propose to address these challenges with HRIStudio, a novel web-based platform designed to streamline the design, execution, and analysis of WoZ experiments. HRIStudio offers an intuitive interface for experiment creation, real-time control and monitoring during experimental runs, and comprehensive data logging and playback tools for analysis and reproducibility. By lowering technical barriers, promoting collaboration, and offering methodological guidelines, HRIStudio aims to make human-centered robotics research easier, and at the same time, empower researchers to develop scientifically rigorous user studies.

Keywords: Novel Interfaces and Interaction Modalities, Multi-modal Situation Awareness and Spatial Cognition, Motion Planning and Navigation in Human-Centered Environments

Keywords: Robot Companions and Social Robots, Anthropomorphic Robots and Virtual Humans, Curiosity, Intentionality and Initiative in Interaction
Abstract: This study employs both qualitative and quantitative approaches to explore perceptions of sex robots. Utilizing grounded theory, a thematic analysis revealed three primary themes: uncanny, artificial, and curious. Following this, an exploratory factor analysis was conducted to evaluate the dimensionality of these themes. Regression analyses were then performed to examine the influence of individual differences on these dimensions. The findings indicated that females perceived sexbots as more uncanny, older individuals perceived sexbots as more artificial, Americans were more curious about sexbots than the British, and males were more curious about sexbots, particularly the female sexbots.

Keywords: Robots in Education, Therapy and Rehabilitation, Personalities for Robotic or Virtual Characters, Non-verbal Cues and Expressiveness
Abstract: People can respond to feedback and guidance in different ways, and it is important for robots to personalize their interactions and utilize verbal and nonverbal communication cues. We aim to understand how older adults respond to different cadences of verbal and nonverbal feedback of a robot exercise coach. We conducted an online study of older adults, where participants evaluated videos of the robot giving feedback at different cadences for each modality. The results indicate that changing the cadence of one modality affects the perception of both it and the other modality. We can use the results from this study to better design the frequency of the robot coach's feedback during an exercise session with this population.

Keywords: Social Touch in Human–Robot Interaction, Social Presence for Robots and Virtual Humans, Creating Human-Robot Relationships
Abstract: Understanding prosociality toward robots furthers our insight into robot acceptance and perception. This study (N=310) adopts a third-person perspective to investigate outcomes of the Dictator Game paradigm. Participants were tasked with evaluating the shared allocation of four distinct resources (money, electronic tools, food, and access to electricity). Results reveal that only 27% of answers indicate a preference for non-sharing. Our findings also show a tendency towards greater sharing of electronic tools and lesser sharing of food compared to access to electricity. Notably, perceptions of robot trustworthiness do not influence people’s sharing judgments, while anthropomorphic perception does play a significant role. Participants expressed greater confidence in their judgments regarding sharing food with robots, a resource where participants preferred non-sharing. This suggests that individuals most likely distinctly categorize robots when contemplating resources necessary for sustaining life. Our study indicates an emerging societal expectation regarding the types and extent of resources to be shared with robots, thus contributing to the delineation of evolving robot social norms.

Keywords: User-centered Design of Robots, Applications of Social Robots, Robotic Etiquette
Abstract: This study investigates the impact of the Internet of Robotic Things (IoRT) on user perceptions when the system encourages users’ behavior in accordance with social norms. Utilizing a 2 (Relationship between humans: hierarchical vs. flat) x2 (IoRT mode: hierarchical vs. flat) mixed-participants design, the research explores how different modes of IoRT operation—flat and hierarchical—affect perceived usefulness, service evaluation of the IoRT, and perceived sociability and familiarity towards counterparts. Results indicated that the flat mode was rated more favorably than the hierarchical mode in terms of usefulness and service evaluation. Additionally, sociability and familiarity ratings towards the interaction counterpart significantly varied based on the relationship type and IoRT mode, with the flat mode enhancing these perceptions particularly in flat relationship conditions.

Keywords: User-centered Design of Robots, Storytelling in HRI, Child-Robot Interaction
Abstract: This study is co-designing and evaluating with potential end users a storytelling-based Participatory Design (PD) methodology for use in Human-Robot Interaction (HRI). It will facilitate the means for roboticists and other stakeholders to establish a rich picture of how people currently conceive of robots and to develop these conceptions into a common design parlance.

Keywords: Assistive Robotics, Human Factors and Ergonomics, Androids
Abstract: This paper investigates patient perceptions towards the potential use of humanoid assistive robots in rehabilitation hospitals. We ran sessions with a demo of a humanoid robot performing assistive tasks for the participant. After the demo the participants were interviewed. The findings indicate an overall openness and acceptance of the concept of humanoid robots in healthcare justified both with facilitation of daily activities for the patients and the potential relief of workload for the healthcare staff. However, certain core concerns were raised regarding the suitability of humanoid assistive robots to individual patient needs and phases of recovery, as well as issues related to safety, privacy, and implementation dilemmas. This paper discusses the findings attending to literature, identified research gaps, and aims to provide recommendations for the deployment of robotics in real-world hospital settings attending to patients’ viewpoints.

Keywords: Creating Human-Robot Relationships, Applications of Social Robots, HRI and Collaboration in Manufacturing Environments
Abstract: Addressing multi-label action recognition in videos represents a significant challenge for robotic applications in dynamic environments, especially when the robot is required to cooperate with humans in tasks that involve objects. Existing methods still struggle to recognize unseen actions or require extensive training data. To overcome these problems, we pro- pose Dual-VCLIP, a unified approach for zero-shot multi-label action recognition. Dual-VCLIP enhances VCLIP, a zero-shot action recognition method, with the DualCoOp method for multi-label image classification. The strength of our method is that at training time it only learns two prompts, and it is therefore much simpler than other methods. We validate our method on the Charades dataset that includes a majority of object-based actions, demonstrating that – despite its simplicity – our method performs favorably with respect to existing methods on the complete dataset, and promising performance when tested on unseen actions. Our contribution emphasizes the impact of verb-object class-splits during robots’ training for new cooperative tasks, highlighting the influence on the performance and giving insights into mitigating biases. Dataset splits and code are publicly available on the project's repository https://github.com/hsp-iit/DualVCLIP.

Keywords: Robots in Education, Therapy and Rehabilitation, Multimodal Interaction and Conversational Skills, Assistive Robotics
Abstract: Robot systems in education can leverage Large language models' (LLMs) natural language understanding capabilities to provide assistance and facilitate learning. This paper proposes a multimodal interactive robot (PhysicsAssistant) built on YOLOv8 object detection, cameras, speech recognition, and chatbot using LLM to provide assistance to students' physics labs. We conduct a user study on ten 8th-grade students to empirically evaluate the performance of PhysicsAssistant with a human expert. The Expert rates the assistants' responses to student queries on a 0-4 scale based on Bloom's taxonomy to provide educational support. We have compared the performance of PhysicsAssistant (YOLOv8+GPT-3.5-turbo) with GPT-4 and found that the human expert rating of both systems for factual understanding is same. However, the rating of GPT-4 for conceptual and procedural knowledge (3 and 3.2 vs 2.2 and 2.6, respectively) is significantly higher than PhysicsAssistant (p < 0.05). However, the response time of GPT-4 is significantly higher than PhysicsAssistant (3.54 vs 1.64 sec, p < 0.05). Hence, despite the relatively lower response quality of PhysicsAssistant than GPT-4, it has shown potential for being used as a real-time lab assistant to provide timely responses and can offload teachers' labor to assist with repetitive tasks. To the best of our knowledge, this is the first attempt to build such an interactive multimodal robotic assistant for K-12 science (physics) education.

Keywords: Social Intelligence for Robots, Affective Computing, Detecting and Understanding Human Activity
Abstract: Social networking services (SNS) provide a rich source of user-generated emotion-expressed text. However, deciphering the emotion from these texts, often marked by vernacular expressions and abbreviations, poses significant challenges. This study introduces a novel approach to enhance emotion detection by converting SNS informal texts into everyday language using ChatGPT, resulting in a generative large language model (LLM). The study uses ten publicly available emotion datasets and a unique tweet dataset, augmented through ChatGPT. Three models were trained for comparison: one using original texts (n=408,359), another with ChatGPT-augmented texts (n=408,359), and the last with a combination of both (n=816,718). The four transformer models, RoBERTa, BERT, DistilBERT, and XLM-RoBERTa, trained with the combined dataset outperformed those trained solely on original texts, indicating that converting SNS vernacular text into everyday language improves emotion detection. The study provides significant insights for enhancing emotion analysis in human-robot interaction and other fields reliant on accurate emotion detection, demonstrating the potential of LLMs in natural language processing (NLP) data augmentation.

Keywords: Detecting and Understanding Human Activity
Abstract: Multiple modalities can boost accuracy in the difficult task of Sign Language Recognition (SLR), however, each modality does not necessarily contribute the same quality of information. Current multi-modal approaches assign the same importance weightings to each modality, or set weightings based on unproven heuristics. This paper takes a systematic approach to find the optimal weights by performing grid search. Firstly, we create a multi-modal version of the RGB only WLASL100 data with additional hand crop and skeletal pose modalities. Secondly, we create a 3D CNN based weighted multi-modal sign language network (WMSLRnet). Finally, we run various grid searches to find the optimal weightings for each modality. We show that very minor adjustments in the weightings can have major effects on the final SLR accuracy. On WLASL100, we significantly outperform previous networks of similar design, and achieve high accuracy in SLR without highly complex pre-training schemes or extra data.

Keywords: Child-Robot Interaction, Applications of Social Robots, Robot Companions and Social Robots
Abstract: Parent-Child Interaction Therapy (PCIT) is a well established method used by psychologists to improve and to maintain a healthy and good parent-child relationship. PCIT requires trained and certified therapists, and the therapy sessions are conducted in a special room, but participants can also practice and later perform sessions in their homes with our without therapist supervision. In order to facilitate the therapy for the parents, here we present a possible approach to introduce social robotics into the PCIT therapy. We developed a behavior model to control a mobile robot and interact with both the parent and the child during a therapy session with the goal of stimulating interaction and communication between the parent and child. The behavior of this social robot depends on the contents of the conversation between the participants, which is analyzed in real-time. For speech analysis we have developed a framework for integrating various already available components to achieve real-time Speech-to-Text transcription, and also developed our own Natural Language Understanding (NLU) engine. The evaluation of the conversation is performed by this NLU engine, which was inspired by the Dyadic Parent- Child Interaction Coding System (DPICS) used in the PCIT therapy. Depending on the evaluation results, our mobile robot can exhibit different behaviors, which were inspired by Etho-Robotics, such as approaching, getting attention, playing fetch with a toy. Besides constructing a functional real-time human conversation analysis system and a behavior model, we also performed preliminary HRI experiments with a real mobile robot controlled by the proposed model in real-time.

Keywords: Applications of Social Robots, Child-Robot Interaction, Motivations and Emotions in Robotics
Abstract: Research has shown the potential of social robots to support learning in science, technology, and language. We contribute to this field by exploring how robots can support music learning. We report on a within-subjects experiment where 50 young learners practiced the piano in the presence of a robot assuming a non-evaluative and a self-assessment enhancing role implemented in a Wizard-of-Oz fashion. We examined whether the robot can make piano practice more fun, and whether initiating self-assessment to support self-regulated learning is a useful strategy for the robot. We collected quantitative self-report data to assess fun, learning, interest, engagement, and effort. We found a direct positive effect of fun on learning in the context of musical instrument practice. Path modeling showed a positive influence of having fun on learners' attitudes, interests, and learning outcomes in music education, particularly with the self- assessment robot role exhibiting superiority.

Keywords: Applications of Social Robots, Cognitive Skills and Mental Models, Robots in art and entertainment
Abstract: Social robots are increasingly finding applications in sectors such as industry, retail, and healthcare. They employ a combination of verbal and non-verbal cues in order to allow an empathetic and efficient interaction with the humans. In contrast to traditional robots, social robots possess contextual awareness, enabling intelligent responses to human interactions. Within this context, in this paper we propose a framework for social robots, integrating advanced audio and video analytics capabilities with a novel engagement algorithm, designed and developed to facilitate effective communication in multi-user settings. Furthermore, other than conversation skills, the proposed social robot is enhanced with the capability to interactively play games with the human. The effectiveness of the proposed system was tested in real-world scenarios, during two trade shows in Italy, providing valuable insights into its performance and adaptability to different contexts and different audiences.

Keywords: Applications of Social Robots, Creating Human-Robot Relationships, Social Intelligence for Robots
Abstract: With more robots being deployed in the world, users will likely interact with multiple robots sequentially when receiving services. In this paper, we describe an exploratory field study in which unsuspecting participants experienced a ``person transfer'' -- a scenario in which they first interacted with one stationary robot before another mobile robot joined to complete the interaction. In our 7-hour study spanning 4 days, we recorded 18 instances of person transfers with 40+ individuals. We also interviewed 11 participants after the interaction to further understand their experience. We used the recorded video and interview data to extract interesting insights about in-the-field sequential human-robot interaction, such as mobile robot handovers, trust in person transfer, and the importance of the robots' positions. Our findings expose pitfalls and present important factors to consider when designing sequential human-robot interaction.

Keywords: Applications of Social Robots, Interaction with Believable Characters, Philosophical Issues in Human-Robot Coexistence
Abstract: Science fiction has long promised a future within which robots assist humans in many facets of their daily lives, and robot technology is advancing at a pace which suggests that the necessary technology already exists, or may exist, in the near future. But, once the technology is in place, how accepting will humans be to autonomous machines performing tasks traditionally performed by humans? Are we designing and developing robots that are human centric? In a study involving 357 undergraduate students, we found that acceptance of robots was dependent upon previous exposure to different forms of technology (i.e., robots, avatars, video games). Men were more likely to have previous exposure to technology, and were therefore more likely to accept robots and avatars in different tasks compared to women. Enhancing the acceptability of robots by both men and women will require an increased exposure to technology, and women may require additional experience with technology to close the technology acceptance gap.

Keywords: Applications of Social Robots, Linguistic Communication and Dialogue, Ethical Issues in Human-robot Interaction Research
Abstract: Humanlike design of social robots can potentially reproduce societal biases. This paper presents an online video experiment (n = 194) examining the stereotyping effects of speech act (assertive vs. affiliative speech) on people’s evaluation of gendered robots (masculine vs. feminine) in terms of warmth, competence, and discomfort. Results show that feminine robots are rated higher in competence than masculine robots, regardless of speech act, and assertive robots are rated higher in competence than affiliative robots, regardless of robot gender. Additionally, women rate robots as more competent than men do. The results for warmth and discomfort are insignificant. This study emphasizes the need for theory-driven experiments addressing robot gendering and highlights the importance of avoiding the reinforcement of gender bias in social robot design.

Keywords: Applications of Social Robots, Multimodal Interaction and Conversational Skills, Detecting and Understanding Human Activity
Abstract: Recent models for Visual Speech Recognition (VSR) have shown remarkable progress over the last few years. They have however been applied mainly to datasets such as Lip Reading Sentences 3 (LRS3), LRS2 or Lombard GRID, but not yet on social robots. As social robots struggle to recognize speech in more challenging acoustic and crowded environments, we believe such models are promising tools for real-time interaction with users. This paper presents a feasibility study focusing on integration of speech recognition (SR) using mixed modalities - audio, visual (lip-reading) and audio-visual - in social robots. To this end, this paper contributes a pipeline to detect an active speaker based on lip movement, post-processing of audio and video footage and inferencing it with the state-of-the-art Auto-AVSR model. In a user study (N=26), we evaluated the feasibility of audio, visual and mixed modality speech recognition on a Pepper robot. We demonstrate the feasibility of using singular and mixed modalities with speech-to-text inference in natural interaction. The results show that it is feasible to deploy such models on social robots in a controlled, noiseless and non-interactive environment. Additionally, the results revealed that informing participants to emphasize their lip movements significantly improved text-to-speech inference results. Our work provides initial insights into the benefits and challenges of using VSR, ASR and AVSR for HRI.

Keywords: Cooperation and Collaboration in Human-Robot Teams, Creating Human-Robot Relationships
Abstract: Robots are inherently imperfect, and collaborating with an error-prone robotic teammate can deteriorate perceptions of trust and the willingness of users to continue working with the robot. Evidence-based trust repair strategies can be implemented into a robot’s design to mitigate the decline of trust in human-robot relationships following errors. It is not yet clear what trust repair strategies are most effective. To address this shortcoming, this study investigates two novel trust repair strategies: offered and automatic physical repair. A between-subjects repeated measures study was performed to determine the extent to which each type of physical trust repair was successful in restoring participants’ perceptions of trust. The results indicated that, where the no-repair condition experienced a significant decrease in trust score, only the automatic repair was consistently successful in bypassing the trust decline. Detailed analysis showed that participants from the offered repair condition did not view the robot as providing the appropriate information, meaning that the offer itself may have confused them. Participants’ response rate to the Multi- Dimensional Measure of Trust also revealed that users were less willing to associate moral terms with robotic teammates, though this hesitancy may reduce over time. These results contribute to research on human-robot trust repair by uncovering that physical repair is effective when it is automatic, but not when it is offered. This finding will help to further elucidate what repair strategies work to mitigate trust decline and thus help inform robot design.

Keywords: Cooperation and Collaboration in Human-Robot Teams, Detecting and Understanding Human Activity, Machine Learning and Adaptation
Abstract: This paper demonstrates the effectiveness of learning based models for accurate, and reliable robot to human handoffs in various HRI scenarios. Specifically we bench marked a neuromorphic spiking neural network and a time series k-nearest neighbors classifier against traditional hand crafted force threshold methods. These models use linear force in the x, y, and z direction, as well as torque about the x, y, and z axis at the end effector of the robot arm to make handoff predictions. This paper demonstrates that these learning based methods are more robust to noise which occurs during operational use. We applied our algorithms to both stationary handoffs (stationary robot) and moving handoffs (robot walking). We believe that our evaluation is the first to examine walking handoffs. We evaluated all models in tests which determined the accuracy, precision, recall, f1, and average execution time for handoff events, noise events, and no event tests. We find that the SLAYER spiking neural network model performed the best across both walking and stationary handoffs for the majority of the evaluation criteria. Our results suggest that neuromorphic spiking neural networks are strong contenders for applications in time series, event based HRI applications.

Keywords: Cooperation and Collaboration in Human-Robot Teams, Detecting and Understanding Human Activity, Machine Learning and Adaptation
Abstract: Using a dual-task paradigm, we explore how robot actions, performance, and the introduction of a secondary task influence human trust and engagement. In our study, a human supervisor simultaneously engages in a target-tracking task while supervising a mobile manipulator performing an object collection task. The robot can either autonomously collect the object or ask for human assistance. The human supervisor also has the choice to rely on or interrupt the robot. Using data from initial experiments, we model the dynamics of human trust and engagement using a linear dynamical system (LDS). Furthermore, we develop a human action model to define the probability of human reliance on the robot. Our model suggests that participants are more likely to interrupt the robot when their trust and engagement are low during high-complexity collection tasks. Using Model Predictive Control (MPC), we design an optimal assistance-seeking policy. Evaluation experiments demonstrate the superior performance of the MPC policy over the baseline policy for most participants.

Keywords: Cooperation and Collaboration in Human-Robot Teams, Detecting and Understanding Human Activity, Motion Planning and Navigation in Human-Centered Environments
Abstract: While robots have advanced in understanding their environments, collaborative tasks demand a deeper comprehension of human intentions to mitigate uncertainty. Anticipatory and proactive behaviours are pivotal in enhancing Human-Robot Interactions (HRI), yet literature often conflates these terms. This study elucidates the distinction between anticipation and proactivity, offering clear definitions and exemplifying their implications through a handover scenario. Through a user study with 24 volunteers performing a total of 72 experiments, we have found that humans are able to distinguish both behaviours and that there is a statistically significant increase in the anthropomorphism of the robot when it behaves proactively. Additionally, both anticipation and proactivity show statistically significant increases in multiple aspects of effective HRI (fluency, comfort, performance, etc.). However, no clear preference for either has been detected.

Keywords: Cooperation and Collaboration in Human-Robot Teams, Detecting and Understanding Human Activity, Non-verbal Cues and Expressiveness
Abstract: To enable a comprehensive human-robot interaction, it is essential to refer to human-human collaboration and decode complex non-verbal communication aspects that are essential for adaptive decision-making and task success. Indeed, for a robust collaboration, it is useful to understand the intricacies and complexities of human communication during human-human interaction and to compare it with the human-robot interaction case. We study this communication exchange and information flow by evaluating the leader/follower behavior during physical interaction with different agents and different control types, focusing on non-verbal cues, to identify collaborative or competitive attitudes. To achieve this, we consider a dynamic task of collaboratively catching a falling object, which, by its nature, favors non-verbal communication channels. Multiple subjects performed the same task with different collaborative agents (i.e., human and robot) and with different control modalities, to evaluate the leadership roles and their implication on task success (successfully catching the object while minimizing impact forces). We analyze how the impact force minimization induced by the velocity matching optimal planner affects the catching success rate. The information flow is analyzed, and the leadership roles are identified. Further qualitative data is gathered from questionnaires and compared with respect to the analytic results.

Keywords: Cooperation and Collaboration in Human-Robot Teams, Embodiment, Empathy and Intersubjectivity, Linguistic Communication and Dialogue
Abstract: This paper investigates how conversational agents that are embedded in smart environments should present themselves socially. In an online study, we simulate a future space habitat in which “astronauts” (participants) interact with one or more agents to complete several tasks related to science, maintenance, and inventory. We examine effects of agent affiliation (affiliation with a user, affiliation with a domain, or affiliation with all users and all domains) and narrative perspective (first- vs. third-person references to parts of the environment) on mental models of the smart environment as one or multiple entities, trust, performance, and social variables. Our findings suggest that in this type of setting, interacting with a single agent may increase mental demand, and that agents that speak about embodied interaction in third person are perceived as more trustworthy and competent than agents that speak in first person.

Keywords: Monitoring of Behaviour and Internal States of Humans, Affective Computing, Human Factors and Ergonomics
Abstract: In the modern manufacturing industry, workers are still required to manually perform complex, repetitive, and physically demanding tasks. Collaborative robotics has emerged to assist human workers, reducing physical strain and monotony, while increasing productivity and safety. Nonetheless, cobots still struggle to match the dexterity of human hands and they lack the ability to understand natural language and interpret human needs, leading to worker frustration and stress. Psychological stress is a critical issue in industrial workplaces, as it affects both workers' well-being and productivity. This work addresses the issue of operators' well-being in the context of human-robot collaboration within industrial settings. We propose a multimodal approach to detect psychological stress during an industrial assembly task. Data are collected from 12 participants while performing both autonomous and robot-assisted assembly tasks. Our approach combines physiological signals (ECG, EMG, EDA), facial action units (AUs), and voice features to classify stress levels. The extracted features are combined using a late fusion approach involving the use of a self-attention layer. The results demonstrate the effectiveness of our model in predicting stress levels with a weighted F1-score of 0.81. This research paves the way for the development of more empathetic and human-aware robotic partners, capable of adapting their behavior to improve collaboration and operator well-being.

Keywords: Detecting and Understanding Human Activity, Curiosity, Intentionality and Initiative in Interaction, Affective Computing
Abstract: This study examines how education level and self-perceived tech savviness influence individuals' reliance on AI technologies in their daily lives. We surveyed participants (N=53) with varying educational backgrounds to assess their AI perceptions (comfort, trust, usefulness, ease of use) and habit formation around AI features. Results reveal that individuals with higher education, especially those with Master's degrees, exhibit significantly stronger reliance on AI tools. Additionally, self-reported tech savviness impacts reliance on AI-powered recommendations. These findings emphasize the interplay of formal education and self-confidence in shaping AI adoption. Designing inclusive AI interfaces and creating educational interventions tailored to diverse backgrounds are crucial for promoting AI use and unlocking its full potential for all users.

Keywords: Detecting and Understanding Human Activity, Evaluation Methods, Child-Robot Interaction
Abstract: Identifying gaze targets in videos of human-robot interaction is useful for measuring engagement. In practice, this requires manually annotating for a fixed set of objects that a participant is looking at in a video, which is very time-consuming. To address this issue, we propose an annotation pipeline for automating this effort. In this work, we focus on videos in which the objects looked at do not move. As input for the proposed pipeline, we therefore only need to annotate object bounding boxes for the first frame of each video. The benefit, moreover, of manually annotating these frames is that we can also draw bounding boxes for objects outside of it, which enables estimating gaze targets in videos where not all objects are visible. A second issue that we address is that the models used for automating the pipeline annotate individual video frames. In practice, however, manual annotation is done at the event level for video segments instead of single frames. Therefore, we also introduce and investigate several variants of algorithms for aggregating frame-level to event-level annotations, which are used in the last step in our annotation pipeline. We compare two versions of our pipeline: one that uses a state-of-the-art gaze estimation model (GEM) and a second one using a state-of-the-art target detection model (TDM). Our results show that both versions successfully automate the annotation, but the GEM pipeline performs slightly (≈ 10%) better for videos where not all objects are visible. Analysis of our aggregation algorithm, moreover, shows that there is no need for manual video segmentation because a fixed time interval for segmentation yields very similar results. We conclude that the proposed pipeline can be used to automate almost all the annotation effort.

Keywords: Detecting and Understanding Human Activity, Human Factors and Ergonomics
Abstract: The fluency of handover between two agents is important to ensure safety and success of handover. In this work, we study the relationships between the timings of giver and receiver motions in human-human handover interactions, to inform human-robot handover. We use giver and receiver hand trajectories from the Human-Object-Human (HOH) handover dataset to study movement during the pre-handover phase, prior to the point of transfer. We find that human receivers adopt a largely proactive behavior, and plan and start motion early in the pre-handover phase. We also find that human receivers spend much of their motion moving in coordination with the giver, rather than after the giver has reached the transfer point. Further, we find that human receivers may predict future movement of the giver from early giver motion, and adjust their start times accordingly to ensure coordinated grasp at transfer. Our findings suggest that robot receivers should adopt a predictive giver-aware approach to plan motion early, and robot givers should recognize that human receivers may expect giver behavior to be human-like and predictable.

Keywords: Detecting and Understanding Human Activity, Innovative Robot Designs, Anthropomorphic Robots and Virtual Humans
Abstract: Infants at risk for developmental delays such as Cerebral Palsy, can often be distinguished through observation of their motion and postural stability. In an effort to provide more accessible and quantitative metrics for the early detection of such impairments, this study focuses on the development of a 6-degree of freedom (DOF) robotic infant simulator. The robotic simulator provides insight into the relationship between infant motion and center of pressure (COP), a common measure that is capable of distinguishing impaired infants through quantification of postural stability. We focus on the impact of a 2 DOF motion in the trunk on COP through two experiments where we 1) Compare the impacts on COP of the limbs and the trunk, and 2) Compare the robot's COP to a real infant's. Our results indicate that motion in the trunk plays a strong part in replicating an infant's COP. Additionally, we can observe that changes in COP exhibit clear and repeatable patterns with respect to motion in each DOF. Future directions suggest a look at how different motions in the trunk affect infant COP.

Keywords: Detecting and Understanding Human Activity, Machine Learning and Adaptation, Applications of Social Robots
Abstract: Home robots are envisioned to provide in-home assistance for older adults and other people who may need help with daily tasks. To gather information for inferring user status, robots typically require cameras to detect human subjects, track their positions, and recognize their activities or poses. However, having cameras in personal spaces, such as homes, could pose privacy concerns and risks due to the potential misuse or compromise of personal image data. It can also lead to psychological unease and feelings of insecurity, stemming from the fear of being watched and recorded. To address this issue, this paper proposes a method for preserving privacy based on physically obstructing the robot's camera image and computer vision methods for detection and tracking of humans in these obstructed images. We present a hardware platform that includes a semi-transparent physical layer in front of the robot's cameras to obtain privacy-preserving shadow images, and a software framework that uses a pre-trained EfficientNet, retrained with a newly-collected dataset of human shadow images for detecting and tracking human subjects. The testing results reveal that the network achieves reliable accuracy in detecting humans from various distances and angles, and it can be applied to a new subject that it has never seen before. Finally, the algorithm is implemented in a gaze-based human-robot interaction scenario, demonstrating its ability to track humans in real time while preserving privacy.

Keywords: Robots in Education, Therapy and Rehabilitation, Applications of Social Robots, Assistive Robotics
Abstract: This study explored how a table-top robot as a recipient of explanations can be applied to the education of students with autism spectrum disorder and intellectual disability. Four 17-year-old participants were recruited from a special needs school. We observed their engagement in assigned tasks and any significant interactions with the robot during the three personalized learning sessions in the school environment. During sessions without a robot, each student was asked to solve arithmetic questions related to purchasing items. During sessions with a robot, each student was required to not only solve questions but also explain the processes or results of simple calculations to the robot. We examined the behavioral, affective, and cognitive engagement of each participant during personalized learning sessions. The results showed that participants solved problems faster, more accurately, or with fewer distractions in the with-robot conditions compared to the task-only conditions. It suggests that this approach, where students explain what they know to a robot, might be effectively adopted for students in special needs schools. Future research should examine the role of robots more comprehensively, acknowledging the limitations of this study, such as the measurement of only three sessions.

Keywords: Robots in Education, Therapy and Rehabilitation, Applications of Social Robots, Robot Companions and Social Robots
Abstract: The landscape of education with social robots is evolving, especially within the realm of music education. However, past studies have focused on children as music learners and such verbal behaviors as praise. Therefore, it remains unknown whether existing studies are effective for adult learners in musical education as well as how to effectively design the non-verbal behaviors of robots. This study investigates the effective behaviors of social robots by comparing three kinds of listening behaviors: none, nodding (simple listening), and enjoying (affective listening). We developed a music education support system that consists of a social robot and a MIDI keyboard and conducted an experiment with adult participants. Our experimental results described the advantages of affective-listening behavior during music education over simple listening and non-listening based on gender.

Keywords: Robots in Education, Therapy and Rehabilitation, Applications of Social Robots
Abstract: Social robots have become increasingly prevalent in the context of one-on-one tutoring, serving as effective educational aids. In response to this trend, the present study was devised to conduct a comparative analysis between human tutors and robot tutors. Additionally, the study aims to investigate how varying previous knowledge in robots influence students' tendencies for help-seeking. By examining the performance and physiological signals of participants, this research seeks to provide valuable insights into the effectiveness of social robots in educational contexts. 21 participants were divided into three groups, each seeking assistance from a human tutor (HT), seeking help from a robot without any prior knowledge of robots (RT1), and seeking help from a robot after gaining some understanding of its capabilities (RT2). Our results demonstrated that participants sought more help from robot than from human and participants in Group RT2 performed better than participants in Group RT1. However, participants experienced greater cognitive load when interacting with a robot tutor compared to interacting with a human tutor. Future work could focus on developing interventions to alleviate students' cognitive load during interactions with robot tutors.

Keywords: Robots in Education, Therapy and Rehabilitation, Assistive Robotics, Applications of Social Robots
Abstract: Stroke-induced aphasia, an acquired speech impairment, poses significant challenges to individuals' communication abilities. Our robot-assisted speech training app aims to facilitate home-based, self-administered training for individuals with aphasia to complement their speech therapy. We evaluated our app in a single-session study with four individuals with aphasia in a rehabilitation setting. Each participant had a different severity grade of the condition, including one case of global aphasia. Our findings suggest an overall positive user experience, with indications that the training facilitated by the robot is suitable for individuals across all severity grades of aphasia, though exercise customization is crucial. Participants, even those with limited technical experience, adapted to using the system by themselves quickly. However, usability issues specific to this diverse target group were noted, such as the length of explanations and the abundant use of robot gestures, which will be addressed in future iterations.

Keywords: Robots in Education, Therapy and Rehabilitation, Child-Robot Interaction, Applications of Social Robots
Abstract: Social robots are being introduced in a variety of educational domains with great success. These social robots are often designed by drawing inspiration from practices held by human teachers. Contingent mediations are a prime example of a high-quality teaching practice that supports better outcomes in human-human teaching. This can inform the design of social robots. Current research on robot use in education has focused on curriculum-level contingent mediations where the difficulty level of subsequent tasks are adjusted to the current capabilities of a learner. However, task-level contingent mediations that provide support for students’ learning during a specific task remain unexplored. This research investigates whether patterns of task-level mediations differ between a robot, virtual, and human agent, as well as their effects on learning outcomes. To investigate these research questions, we designed instruction that utilizes contingent mediational flows, based on formative assessment data, to be delivered by a human, robot, and virtual agent to teach grade 3-5 children science words. We identified 23 unique instructional patterns. Then, we compared these patterns across teaching agents and their effects on learning outcomes. Overall, our study demonstrated that high-quality contingent mediations support children’s science vocabulary in learning regardless of the teaching agent.

Keywords: Robots in Education, Therapy and Rehabilitation, Child-Robot Interaction, Applications of Social Robots
Abstract: Parental estrangement can cause long-term harm to socio-emotional skills, mental health and wellbeing. This is pronounced for children who experienced trauma. “Robo- Therapists” have been used to train children to express emotions, but such robots are often “human replacements” and tend to be too costly to deploy widely. We propose e-MoBo, a low-cost, non-humanoid “Robo-Mediator” of our own design to train children’s socio-emotional skills towards creating stronger relationships with others. We report on the developing e-MoBo and results of a field study with the current prototype and 14 neurotypical children, ages 5-9, as we prepare to deploy e-MoBo in a residential center with traumatized children. Our results suggest that e-MoBo, compared to a control, helps children express emotions, offering the promise of a low-cost, nonanthropomorphic, Robo-Mediator meeting children’s needs.

Keywords: User-centered Design of Robots, Ethical Issues in Human-robot Interaction Research, Robotic Etiquette
Abstract: Robots powered by Artificial Intelligence require continuous sensing to function and interact autonomously with their environment. This may create concerns for their users. Here, we describe a study in the context of an office environment equipped with autonomous delivery robots. Our study aims to understand the employees' current comprehension of the data captured by the robots, the nature of their concerns and ways to mitigate those. We found that the employees have little knowledge about data capture and that providing such knowledge can go either way, reassuring or generating new concerns which are often contextual. We qualitatively analysed reasons for having or not having concerns. Our findings include that trust in the employer is an important factor limiting them.

Keywords: User-centered Design of Robots, Evaluation Methods, Robot Companions and Social Robots
Abstract: Despite advances in areas such as the personalization of robots, sustaining adoption of robots for long-term use in families remains a challenge. Recent studies have identified integrating robots into families' routines and rituals as a promising approach to support long-term adoption. However, few studies explored the integration of robots into family routines and there is a gap in systematic measures to capture family preferences for robot integration. Building upon existing routine inventories, we developed Family-Robot Routines Inventory (FRRI), with 24 family routines and 24 child routine items, to capture parents' attitudes toward and expectations from the integration of robotic technology into their family routines. Using this inventory, we collected data from 150 parents through an online survey. Our analysis indicates that parents had varying perceptions for the utility of integrating robots into their routines. For example, parents found robot integration to be more helpful in children's individual routines, than to the collective routines of their families. We discuss the design implications of these preliminary findings, and how they may serve as a first step toward understanding the diverse challenges and demands of designing and integrating household robots for families.

Keywords: User-centered Design of Robots, HRI and Collaboration in Manufacturing Environments, Human Factors and Ergonomics
Abstract: A user-centered method of parameterizing robotic system is indispensable to enable workers with lower qualification levels the access to robotic systems. In this paper, different ways and types of communication are investigated to increase the usability during the programming of a movable robotic cell utilized for mounting processes inside a production line. Thereto, the entire usability circle given by the ISO 9241-210 is applied. Using the feedback of 13 participants from an initial formative evaluation inside a real production line, usability problems and six requirement categories were identified. Applying the prototypical design, we evaluated four possibilities of multimodal and bidirectional communication utilizing audio, haptic and visual channels. The results from the final usability study in the laboratory reveal that haptic plus visual input combined with visual plus audio feedback is assessed as most intuitive. Pre-experience with robots showed no significant influence on usability, acceptance and perceived safety in most modalities.

Keywords: User-centered Design of Robots, Human Factors and Ergonomics, Motion Planning and Navigation in Human-Centered Environments
Abstract: The field of robotics has shown significant advances in autonomous systems, particularly in robot navigation. Since the decisions made during navigation can be difficult for human operators to understand, research aims to provide explanations that improve human-robot interaction (HRI). However, generating and designing such explanations with the intention of improving robot-related human factors is still an ongoing research challenge. This paper addresses this challenge by investigating the impact of different explanation types on a set of human factors in the context of robot navigation. For this purpose, we conducted a user study that examined the impact of six different explanation types on commonly used human factors, including trust, satisfaction, situation awareness, likeability, understandability, and perceived usefulness. Additionally, the study provides indications of their general applicability for robot navigation explanations through creation of sum ranks across the observed human factor metrics. The results show that depending on the chosen explanation type, a significant impact on the measured factors can be observed. While constraint-based explanations are generally rated highly across all factors, apologetic explanations are not perceived well across all measured human factors. Our results provide insights into the impact of explanation types used for robot navigation scenarios on robot-related human factors, and also provide practical insights for designing explanations for robot navigation scenarios.

Keywords: User-centered Design of Robots, Human Factors and Ergonomics
Abstract: This work illustrates the testing of the Spot Robot performed at the training camp of Civil Protection and ANPAS (National Association of Public Assistance) in Foligno. The camp simulates the aftermath of an earthquake with different types of collapsed buildings. We teleoperated the quadruped Spot robot in different areas of the camp where Spot needs to address different challenges. The focus of the testing was not on the objective performance of the robot but on how the robot was subjectively perceived by rescuers of ANPAS and Civil Protection. Initially, we formulated and tested two hypotheses to check if locomotion in some areas is perceived better than in other areas and if there are perceivable differences when the robot is using different types of locomotion gaits. Then, we conducted unstructured interviews with participants who observed the robot in action to describe their rescue procedures and give us suggestions and opinions on what operations they expect the robot might perform.

Keywords: User-centered Design of Robots, Innovative Robot Designs, Robot Companions and Social Robots
Abstract: In the pursuit of enhancing the quality of life, research in domestic robotics is rapidly evolving, offering a broad spectrum of services that not only provide practical aid but also contribute to emotional well-being and social interaction. However, existing domestic robots often lack essential features necessary for continuous interaction with humans and their surroundings, such as stability, mobility, safety, size, and physical interaction capabilities. Addressing this need, this paper introduces the development of a prototype robot tailored for harmonious coexistence with humans. This robot utilizes its own heavy equipment, such as batteries and actuators, to achieve an intrinsically balanced structure along with a comprehensive set of features optimized for seamless integration into human-centric environments. Focusing on the mechanical design of the robot, we evaluated its performance through a series of experiments, highlighting its mechanical advantages, addressing its limitations, and proposing solutions to enhance its performance for future applications.

Keywords: Cognitive Skills and Mental Models, Assistive Robotics, Applications of Social Robots
Abstract: Persuasion is defined as the process of changing people’s attitudes and behaviours and social assistive robots are used to favour such changes through a variety of mechanisms. While human-robot deception is considered to present philosophical and psychological issues, it still may have positive consequences. In particular, prosocial deception can be beneficial and have positive influences, such as increasing trust. This work presents an exploration of robot deception and its effects on people's changes in behaviour and trust in a social assistive robot. In particular, we explore the effects of different types of deception states (superficial, external, and hidden states) on people's compliance with a social robot during an assistive game scenario. We collected the responses of 63 participants to evaluate their perception of trust in the robot, and their perception of the robot's deceiving behaviours. Our results showed that the deceiving behaviours of the robot affected people’s trust and that superficial state deception has higher negative effects on people's perception of the robot and trust in it compared to the other two deceptive states.

Keywords: Human Factors and Ergonomics, Monitoring of Behaviour and Internal States of Humans, Cognitive Skills and Mental Models
Abstract: Recently, electroencephalographic (EEG) signals have been used to design and enhance human-robot interaction (HRI). In particular, error-related potentials (ErrPs) have been leveraged since very few years. These potentials can be used to provide feedback to the robot about any mismatch between the user’s expectations and the robot’s behavior, during interaction tasks. In this process, the correct classification of ErrPs is crucial, which, in turn, relies on the reliability of the process for extraction and selection of signal features. In this work, we consider an extensive list of possible features and perform a statistical analysis to assess their discriminative power for ErrP analysis. The aim is to reduce the number of features used for classification while retaining the most relevant ones only. Overall, the outcome of our study shows that some parameters have relevant importance compared to others, (i.e. temporal features, frequency features, signal processing features, and some wavelet transform coefficients), while some of the features used in existing works are not useful since they have low discriminative power.

Keywords: Novel Interfaces and Interaction Modalities, Cooperation and Collaboration in Human-Robot Teams, HRI and Collaboration in Manufacturing Environments
Abstract: This article investigates mixed reality (MR) to enhance human-robot collaboration (HRC). The proposed solution adopts MR as a communication layer to convey a mobile manipulator's intentions and upcoming actions to the humans with whom it interacts, thus improving their collaboration. A user study involving 20 volunteers demonstrated the effectiveness of this MR-focused approach in facilitating collaborative tasks, with a positive effect on overall collaboration performances and human satisfaction.

Keywords: Creating Human-Robot Relationships, Social Intelligence for Robots, Robot Companions and Social Robots
Abstract: The growing implementation of robots in societal contexts necessitates a deeper exploration of the dynamics of trust between humans and robots. This exploration should expand beyond traditional viewpoints that primarily emphasize the influence of robot performance. In the burgeoning area of social robotics, fine-tuning a robot's personality traits is increasingly recognized as a crucial element in shaping users' experiences during human-robot interaction (HRI). Research in this field has led to the creation of trust scales that encompass various trust dimensions in HRI. These scales include aspects related to performance as well as moral dimensions. Our previous study revealed that breaches of moral trust by robots impact human trust more negatively than performance trust breaches, and humans take retaliatory approaches in response to morality breaches by robots. In the present study, our main aim was to explore if trust loss and retaliation tendencies differ based on the identity of the teammates following the violations of these different trust aspects. Through multiple versions of an online search task, we examined our research questions and found that breaches of morality by robotic teammates cause a significantly higher trust loss in humans compared to human teammates. These findings highlight the importance of a robot's morality in determining how humans view a robot's trustworthiness. For effective robot design, robots must meet ethical and moral standards, which are higher than the ethical and moral standards expected from humans.

Keywords: Social Presence for Robots and Virtual Humans, Applications of Social Robots, Evaluation Methods
Abstract: People’s ascription of intentional agency to robots necessitates understanding how, when, and why people attribute robot behavior to underlying intentional states. While many studies explored mind attribution to robots including its determinants and consequences, little attention has been given to the attribution of second-order mental states, such as a robot’s beliefs about people’s intentions during interactions. In an online study (n = 155), participants watched a video of a humanoid robot tracking a ball hidden under one of two cups. 19% of participants predicted that the robot could correctly locate the ball after it was displaced twice by a person deliberately distracting the robot, indicating implicit attribution of second-order beliefs to the robot. These implicit attributions influenced participants’ actions in a subsequent interactive game with a virtual counterpart of the robot but did not affect their explicit assessments of the robot’s second-order reasoning. In contrast, observing the robot demonstrate second-order reasoning by correctly identifying the ball’s location affected participants’ explicit attributions but not their behavior in the interactive game. This reveals a complex interplay between implicit and explicit attribution processes in how people interpret robot behavior.

Keywords: Applications of Social Robots, Non-verbal Cues and Expressiveness, Creating Human-Robot Relationships
Abstract: Every individual lives in daily lives in own unique tempo, called Personal Tempo. Tempo is also highly important in dialogue situations, and it is thought that if the tempo can be matched with the conversational partner, it will lead to smoother communication with a higher level of comprehension. Spoken-dialogue systems have been used in many situations, and by personalizing dialogue on the basis of the user's tempo, it is thought that dialogue will be able to make it easier to speak and make people want to speak. Previous research has focused on methods for encouraging users to change their tempo to be in tune with the tempo of their dialogue partner. However, a conversation that differs from the user's tempo can be stressful and burdensome for the user in the process of tuning in.Therefore, we define personal tempo as speech speed, which is the number of moras divided by the duration of speech and propose a speech-speed control method for spoken-dialogue systems. We implemented our method in a spoken-dialogue system that synchronizes speech with the user. We verified the effectiveness of the proposed method by analyzing its impact on the comprehension of speech and user impressions of the spoken-dialogue system. The results indicate that significant differences were obtained with the proposed method between impression and comprehension of the speech content.

Keywords: Applications of Social Robots, Social Intelligence for Robots, Multimodal Interaction and Conversational Skills
Abstract: Autistic individuals, or individuals diagnosed with autism spectrum disorder (ASD), often experience challenges in social relations and may experience increased stress or anxiety. Recent studies highlight the potential of socially assistive robots (SARs) as tools for robot-assisted interventions. However, current human-robot interaction (HRI) designs lack consideration for the diverse anxiety issues in individuals with ASD and the need for personalized, empathetic assistance. The question that persists is whether the most recent individualized and modular intervention program for autism can also be effectively implemented through the use of SARs. This study presents a novel HRI framework that incorporates psychological factors into a modular intervention aimed at reducing user’s anxiety levels. The developed system integrates techniques such as human pose estimation and motion imitation while also leveraging conversational agents. During the user study, users engaged in interactions with a humanoid robot equipped with multiple intervention modules, and their anxiety and perceptions of the robot’s empathy were assessed through questionnaires in a within-participants study design. Results show the implemented system effectively mitigated users’ anxiety by assessing the differences between pre-session and post-session scores. Additionally, our findings suggest that the observed change in empathetic scores may be a contributing factor to the reduction in anxiety levels. The study offers significant insights into the use of SARs for mental health support in autistic adolescents.

Keywords: Applications of Social Robots, Social Intelligence for Robots, Non-verbal Cues and Expressiveness
Abstract: Successfully implementing robots to support senior adults requires their acceptance. Leveraging nonverbal communication could enhance the ease and intuitiveness of accepting robot assistance. However, it is essential to see how different age groups understand nonverbal communication cues to understand the dynamics between different user groups and assistive robots. Our research specifically delves into the intuitive understanding of handshaking gestures across multiple interactions, focusing on seniors (between 70 and 97) and young (21 and 26) adults. Through a combination of observations and open-ended surveys, we conducted a video observation and thematic analysis. Interestingly, our findings indicate no significant differences between the two age groups, except for reactions and interaction time variables. Furthermore, we report on possible motivations behind the initial reactions in the two age groups, familiarity, and ways to improve the overall Human-Robot Interaction experience potentially.

Keywords: Applications of Social Robots, User-centered Design of Robots, Human Factors and Ergonomics
Abstract: The deployment of social robots in public spaces has received increased interest over the past years. To execute various tasks, these robots need to process a wide array of personal data to offer services that are tailored to users’ requirements. While much research has been carried out regarding the creation of explainable content, little research has dealt with how data transparency - as a way to address uncertainty and concerns regarding the handling of personal data - is conveyed towards users. To examine the impact on trust, performance, and robot evaluation of modalities of transparency declarations, we conducted a virtual reality (VR) supported laboratory experiment with N = 53 participants who interacted with a robot in a public setting (a library). The interaction between users and robots is accompanied by information on the handling of users’ personal data using three different modalities (via posters, the robot’s tablet, or verbally). The results imply that, while all modalities are understandable and perceived as useful, there is no preference for any modality. This research contributes to HRI research by examining different modalities for transparency declarations in an effort to foster understandable and transparent processing of data.

Keywords: Applications of Social Robots, User-centered Design of Robots, Multimodal Interaction and Conversational Skills
Abstract: Urinary Tract Infections (UTIs) have the potential to lead to severe health implications if not detected and treated promptly. The Feather project aims to develop a system which will provide early identification of UTIs through regular interactions with an individual at home. This paper utilizes a participatory design workshop to investigate the use of Socially Assistive Robots (SARs) for this purpose, and understand 1) the appropriate robot embodiment, and 2) the relevant social interactions to identify UTI indicators. Two of the five explored embodiments were found to facilitate a suitable balance between agency and autonomy for this task, with the telepresence robot creating a preference due to a need for strong navigation capabilities. Participants further devised a series of multimodal interactions to identify a range of UTI indicators. These early investigatory design insights lay the groundwork for introducing SARs to this novel area of application, from which we plan to build upon to develop a working system.

Keywords: Applications of Social Robots
Abstract: Dexterous manipulation in robotics, particularly with high degrees of freedom (DoF) devices like the 24-joint Shadow Hand, confronts complexities in search space and execution precision. Humans, however, manipulate objects effortlessly, thanks to their innate grasp of physics. Inspired by this, we introduce the Physics Representation Learning (PRL) framework for robotic hand manipulation. PRL uses physics principles for action conceptualization, such as aligning axes at specific angles (Figure 1). It is a natural language that describes the physics laws, thus bridging the gap between high-level semantics planning and joint-level execution through inverse kinematics (IK). Built on top of the physics-informed action space, PRL deploys a Reinforcement Learning (RL) network with expert demonstrations. Its effectiveness was validated in Nvidia Isaac Sim on four tasks: hammering, unscrewing, sweeping, and pinching. Results show that PRL significantly outperforms conventional joint-control algorithms and RL without demonstrations, underscoring the benefits of physics-based action representations for complex tasks.

Keywords: Cooperation and Collaboration in Human-Robot Teams, Evaluation Methods, Computational Architectures
Abstract: Human-Autonomy Teaming (HAT) is a multi-disciplinary domain with a diverse set of research needs and goals stemming from fields such as computer science, robotics, and human factors. This melting pot of fields generates a unique challenge in that there exist many disjoint research methods (measures and tasks) that cause issues with knowledge transfer and comparison between researchers. One way to address this issue is by providing researchers with a testbed containing a standardized suite of analysis tools and tasks that allow direct comparison between different approaches. Therefore, this study attempts to bring the HAT community together in a collaborative discussion to collect and organize their research needs for the future development of these testbeds. Specifically, through thematic analysis, our work reveals three emergent prongs that underpin testbed needs of HAT experts: task, AI, and technical requirements. Also, we organize our thematic analysis by priority to suggest possible paths for HAT testbed development to maximize its immediate and continued utility. Our research indicates that the HAT community places significant importance on both the pre-established, standardized functions available within the testbed and the freedom to tailor and develop their unique tasks or AI solutions.

Keywords: Cooperation and Collaboration in Human-Robot Teams, HRI and Collaboration in Manufacturing Environments
Abstract: The efficiency of collaborative mobile robot applications is influenced by the inherent uncertainty introduced by humans' presence and active participation. This uncertainty stems from the dynamic nature of the working environment, various external factors, and human performance variability. The observed makespan of an executed plan will deviate from any deterministic estimate. This raises questions about whether a calculated plan is optimal given uncertainties, potentially risking failure to complete the plan within the estimated timeframe. This research addresses a collaborative task planning problem for a mobile robot serving multiple humans through tasks such as providing parts and fetching assemblies. To account for uncertainties in the durations needed for a single robot and multiple humans to perform different tasks, a probabilistic modeling approach is employed, treating task durations as random variables. The developed task planning algorithm considers the modeled uncertainties while searching for the most efficient plans. The outcome is a set of the best plans, where no plan is better than the other in terms of stochastic dominance. Our proposed methodology offers a systematic framework for making informed decisions regarding selecting a plan from this set, considering the desired risk level specific to the given operational context.

Keywords: Cooperation and Collaboration in Human-Robot Teams, HRI and Collaboration in Manufacturing Environments
Abstract: Visual planning methods are promising to handle complex settings where extracting the system state is challenging. However, none of the existing works tackles the case of multiple heterogeneous agents which are characterized by different capabilities and/or embodiment. In this work, we propose a method to realize visual action planning in multi-agent settings by exploiting a roadmap built in a low-dimensional structured latent space and used for planning. To enable multi-agent settings, we infer possible parallel actions from a dataset composed of tuples associated with individual actions. Next, we evaluate feasibility and cost of them based on the capabilities of the multi-agent system and endow the roadmap with this information, building a capability latent space roadmap (C-LSR). Additionally, a capability suggestion strategy is designed to inform the human operator about possible missing capabilities when no paths are found. The approach is validated in a simulated burger cooking task and a real-world box packing task.

Keywords: Cooperation and Collaboration in Human-Robot Teams, Human Factors and Ergonomics, User-centered Design of Robots
Abstract: In robotics, understanding human interaction with autonomous systems is crucial for enhancing collaborative technologies. We focus on human-swarm interaction (HSI), exploring how active robot groups of varying sizes affect operators' cognitive and perceptual reactions over distinct durations. We analyze the impact of different numbers of active robots within a 15-robot swarm on operators' time perception, emotional state, flow experience, and task difficulty perception. Our findings indicate that managing multiple active robots when compared to one active robot significantly alters time perception and flow experience, leading to a faster passage of time and increased flow. More active robots and extended durations cause increased emotional arousal and perceived task difficulty, highlighting the interaction between the number of active robots and human cognitive processes. These insights inform the creation of intuitive human-swarm interfaces and aid in developing swarm robotic systems aligned with human cognitive structures, enhancing human-robot collaboration.

Keywords: Cooperation and Collaboration in Human-Robot Teams, Interaction with Believable Characters, Social Presence for Robots and Virtual Humans
Abstract: In this paper, we set out to explore how people judge the personality of a non-anthropomorphic virtual agent during group interactions. Using a Wizard of Oz (WoZ) based approach, we observed that people judged the acceptability of a virtual agent's behavior from a tool-based lens, that is, if this robot and its behavior are useful to the team or not. Furthermore, we found that while people were able to acknowledge the virtual agent's personality and recognize its identity through social cues, the tool-based framing impacts these perceptions into a normative judgment of the robots' utility. We present two case studies that we think highlight this tool-based interpretation of robotic personality: robots expressing non-factual opinions and robots expressing humor. Finally, we suggest that researchers should consider the impact of this tool-based framing on people's perceptions of a robot's identity when designing robots for social interaction.

Keywords: Cooperation and Collaboration in Human-Robot Teams, Monitoring of Behaviour and Internal States of Humans
Abstract: Human–robot work collaboration can be comple- mentary where both the robot and employee use their strengths to complete the task at hand. But who is ultimately responsible for the task? Employees may assume more or less responsibility when working with robots, which can have a direct impact on job outcomes. In this paper the authors use the job characteristics model as a framework to investigate changes in job characteristics and their impact on job responsibility and worker outcomes as a result of robot implementation in restaurants. The results identify changes in task significance, job autonomy, and feedback from the job as important predictors of changes in experienced responsibility toward work. The study also found a significant positive relationship between job responsibility and motivation to work.

Keywords: Detecting and Understanding Human Activity, Machine Learning and Adaptation, Novel Interfaces and Interaction Modalities
Abstract: Hand gesture recognition has been playing an important role in robotic applications, which allows robots to communicate with humans in an effective way. However, it typically desires to process high-dimensional data, such as images or sensor measurements. To address the computational challenges due to the data growth, it is desirable to select most relevant features during recognition by reducing the redundancy of the data. In this paper, we propose a novel feature selection approach based on the separable nonnegative matrix factorization (NMF) framework for hand gesture recognition. In particular, we adopt a nonconvex regularization term, i.e., the ratio of matrix nuclear norm and Frobenius norm. The proposed method reduces the data dimension by utilizing the data low-rankness in an adaptive way. To address the nonconvexity of the proposed model, we reformulate it by introducing an auxiliary variable and then apply the alternating direction method of multipliers (ADMM). Furthermore, a variety of numerical experiments on binary and grayscale hand gesture images demonstrate the efficiency of the proposed feature selection approach in improving the quality of factorization and its potential impact on robotic applications.

Keywords: Detecting and Understanding Human Activity, Machine Learning and Adaptation, Virtual and Augmented Tele-presence Environments
Abstract: In recent years, the shortage of the workers in the machine inspection is a serious problem. To reduce the workload, the automation of the tasks such as recording the process is necessary, which needs to recognize the actions. In the environment which the machines are installed, it is difficult to install the thirdview sensor that captures the workers’ actions. Rather than the third-view sensor, the wearable sensor, especially, the egocentricview sensor is more appropriate to capture them during the task. These actions in the machine inspection often can be represented by the relationship among the hands, the gaze, and the objects. In this paper, a method for (1) constructing the graph representing the relationship among them in the egocentric view, and (2) processing the graphs for recognizing the actions. To confirm the validity of the proposed method, the egocentric view data capturing the actions during the machine inspection is collected. The experimental results show that the proposed method is competitive with the state-of-the-art models of video-based recognition. Furthermore, the proposed method is more robust against the

individual difference than the video-based models.

Keywords: Detecting and Understanding Human Activity, Monitoring of Behaviour and Internal States of Humans, Cognitive Skills and Mental Models
Abstract: When collaborating with humans in a shared task, a robot must be able to estimate the shared goal and monitor the tasks completed by its partners to adapt its behavior. Our contribution is a lightweight, hierarchical task recognition system that enables the robot to estimate shared goals and monitor human tasks. This recognition system is integrated into a robotic architecture to take advantage of the semantic knowledge flow available and builds upon our previous work on action recognition. We demonstrate the mechanisms of our recognition system and how we improved it to handle missing information using a kitchen scenario. This also enables us to showcase its usability from the perspective of other agents, using Theory of Mind.

Keywords: Detecting and Understanding Human Activity, Monitoring of Behaviour and Internal States of Humans
Abstract: Pedestrian flow simulation is important to predict congestion in an urban area and prevent a crowd accident. Many studies have used the machine learning-based pedestrian flow model, which require a measurement of pedestrians to obtain training data.Not only the positional data of each person but also pose and body shape information is useful because it enables the model to learn pedestrian features implicitly including age, gender and social relationship. In this study, we present a video motion capture method that estimates correspondences of an unspecified number of pedestrians in different camera images using the body shape feature of Skinned Multi-Person Linear (SMPL) model. Simultaneously optimizing the correspondence, position, pose and body shape, we can find the same person in multiple cameras and reconstruct their poses and body shapes. We quantitatively compare the result of the method with that of an optical motion capture and qualitatively evaluate the method using an open dataset of pedestrians.

Keywords: Detecting and Understanding Human Activity, Motion Planning and Navigation in Human-Centered Environments, Non-verbal Cues and Expressiveness
Abstract: In human-robot collaboration, legible intent of the robot is critical to success as it enables the human to more effectively work with and around the robot. Environments where humans and robots collaborate are widely varied and in the real world are most often cluttered. However, prior work in legible motion utilizes primarily environments which are uncluttered. Success in these environments does not necessarily guarantee success in more cluttered environments. Furthermore, the prior work has been primarily performed based on results from robot-human studies and the problem has not been studied from the prospective of what people do to express intent to each other. Therefore, this work addresses a gap in current research into legible robot arm motion in the following ways: first we perform a human-human study in order to establish the factors which humans use to express their intent through body language, and second we perform the study in a cluttered and varied environment. Through the study we showed that the primary factors which people considered are: timing, kinematic parameters, hand gestures, object proximity, etc. The results also showed that legibility is correlated with perceived safety, perceived social intelligence, the collaborator’s contribution, and trust which further speaks to the importance of legible motion. Future work will utilize the pose data extracted from the study's video recordings to develop a model for legible motion.

Keywords: Detecting and Understanding Human Activity, Non-verbal Cues and Expressiveness, Novel Interfaces and Interaction Modalities
Abstract: The human gaze is an important cue to signal intention, attention, distraction, and the regions of interest in the immediate surroundings. Gaze tracking can transform how robots perceive, understand, and react to people, enabling new modes of robot control, interaction, and collaboration. In this paper, we use gaze tracking data from a rich dataset of human motion (THÖR-MAGNI) to investigate the coordination between gaze direction and head rotation of humans engaged in various indoor activities involving navigation, interaction with objects, and collaboration with a mobile robot. In particular, we study the spread and central bias of fixations in diverse activities and examine the correlation between gaze direction and head rotation. We introduce various human motion metrics to enhance the understanding of gaze behavior in dynamic interactions. Finally, we apply semantic object labeling to decompose the gaze distribution into activity-relevant regions.

Keywords: Robots in Education, Therapy and Rehabilitation, Child-Robot Interaction, Assistive Robotics
Abstract: Social Narratives (SNs) have shown promising benefits for helping children navigate diverse social situations. The use of SNs in Robot-Assisted Play (RAP) can offer creative ways to address social and communication challenges of children with functional needs. In this study, we evaluate the initial use of social stories in robot-assisted therapy to help 17 children with diverse conditions such as Down Syndrome (DS), Autism Spectrum Conditions (ASC), and speech disorders (SD) understand and act on real-world situations while interacting with the social robot Furhat. We developed educational stories based on six common professions to show how each professional works and interacts with people. We conducted a user study as part of their daily intervention in a rehabilitation center over two weeks. Overall results demonstrate that there were no significant differences in socio-emotional outcomes among children grouped by their diagnosis, age, and other characteristics. However, their communication skills played a significant role in their performance; in particular, children with verbal abilities had higher completion time in activities compared to their non-verbal counterparts. This result was supported by two therapists who reported higher engagement with the robot among verbal children. We suggest that the SN-inspired intervention could be used to support children with diverse special needs, without causing any negative impacts on their learning.

Keywords: Robots in Education, Therapy and Rehabilitation, Creating Human-Robot Relationships, Storytelling in HRI
Abstract: In 1997, the very first tour guide robot RHINO was deployed in a museum in Germany. With the ability to navigate autonomously through the environment, the robot gave tours to over 2,000 visitors. Today, RHINO has become an exhibit itself and is no longer functional. In this paper, we present RHINO-VR, an interactive museum exhibit using virtual reality (VR) that allows museum visitors to experience the historical robot RHINO in operation in a virtual museum. Compared to static exhibits, RHINO-VR allows users to overcome the fear of damaging the exhibit while familiarizing them with the basic concepts of mobile robotics. In the virtual environment, the user is able to interact with RHINO in VR by pointing to a location to which the robot should navigate and observing the corresponding actions of the robot. To include other visitors who cannot use the VR, we provide an external observation view to make RHINO visible to them. We evaluated our system by measuring the frame rate of the VR simulation, comparing the generated virtual 3D models with the originals, and conducting a user study. The user-study showed that RHINO-VR improved the visitors' understanding of the robot's functionality and that they would recommend experiencing the VR exhibit to others.

Keywords: Robots in Education, Therapy and Rehabilitation, Creating Human-Robot Relationships
Abstract: Technology-supported learning is an integral part of everyday learning. Adaptive tutoring systems are already widespread, but they often lack the social component of learning. Social robots can provide personalized tutoring as well as social interaction and can thus address this challenge. In this contribution, we present a model to personalize the behavior of a social robot tutor for higher education which has both a theoretical as well as an empirical foundation: Theoretically, it is based on the well-known Self-Determination Theory (SDT) that assumes that intrinsic motivation is strongly related to the fulfillment of three basic human needs, namely the need for autonomy, competence, and relatedness. Empirically it is based on the results of four studies with university students that demonstrate that particularly adaptation on a social level is beneficial. The final model considers the personalization of the robotic tutor's behavior based on the current learning content and its social behavior and also considers a long-term perspective for multi-session learning. Thereby each of the three layers of SDT is addressed in multiple ways. This is one of the first models that contain a long-term perspective and is particularly designed for higher education.

Keywords: Robots in Education, Therapy and Rehabilitation, Degrees of Autonomy and Teleoperation, User-centered Design of Robots
Abstract: Recent work on Socially Assistive Robotics in Therapy has revealed a dual-cycle model, with the vast majority of prior work on Socially Assistive Robotics narrowly focused on the human-robot interaction, termed the “inner cycle”. In contrast, little attention has been paid to the activities performed before and after the interaction, termed the “outer cycle”,in which authoring and evaluation also take place. Authoring and evaluation are activities that are key sources of invisible labor for Therapists who serve as Care Wizards (i.e., SAR teleoperators). In this work, we consider the outer cycle needs of Care Wizards in another key Socially Assistive Robotics domain, Special Education, with a careful eye toward the barriers to entry and invisible labor that may manifest in this domain, and how those barriers and invisible labor might be subverted and mitigated. Our interviews with six Care Wizards who teleoperate robots in Special Education contexts reveal new insights surrounding these stakeholders’ needs. Our key insights are that (1) support systems are necessary for SAR adoption; (2) currently invisible Care Wizard labor may be indirectly compensated; and (3) training must be personalized to specific Care Wizards.

Keywords: Robots in Education, Therapy and Rehabilitation, Medical and Surgical Applications, Machine Learning and Adaptation
Abstract: Advanced by rich perception and precise execution, robots possess immense potential to provide professional and customized rehabilitation exercises for patients with mobility impairments caused by strokes. Autonomous robotic rehabilitation significantly reduces human workloads in the long and tedious rehabilitation process. However, training a rehabilitation robot is challenging due to the data scarcity issue. This challenge arises from privacy concerns (e.g., the risk of leaking private disease and identity information of patients) during clinical data access and usage. Data from various patients and hospitals cannot be shared for adequate robot training, further compromising rehabilitation safety and limiting implementation scopes. To address this challenge, this work developed a novel federated joint learning (FJL) method to jointly train robots across hospitals. FJL also adopted a long short-term memory network (LSTM)-Transformer learning mechanism to effectively explore the complex tempo-spatial relations among patient mobility conditions and robotic rehabilitation motions. To validate FJL's effectiveness in training a robot network, a clinic-simulation combined experiment was designed. Real rehabilitation exercise data from 200 patients with stroke diseases (upper limb hemiplegia, Parkinson's syndrome, and back pain syndrome) were adopted. Inversely driven by clinical data, 300,000 robotic rehabilitation guidances were simulated. FJL proved to be effective in joint rehabilitation learning, performing 20% - 30% better than baseline methods.

Keywords: User-centered Design of Robots, Machine Learning and Adaptation, Anthropomorphic Robots and Virtual Humans
Abstract: Robots that make use of multi-fingered or fully anthropomorphic end-effectors can engage in highly complex manipulation tasks. However, the choice of a suitable grasp for manipulating an object is strongly influenced by factors such as the physical properties of an object and the intended task. This makes predicting an appropriate grasping pose for carrying out a concrete task notably challenging. At the same time, current grasp type predictors rarely consider the task as a part of the prediction process. This work proposes a learning model that considers the task in addition to an object's visual features for predicting a suitable grasp type. Furthermore, we generate a synthetic dataset by simulating robotic grasps on 3D object models based on the BarrettHand end-effector. With an angular similarity of 0.9 and above, our model achieves competitive prediction results compared to grasp type predictors that do not consider the intended task for learning grasps. Finally, to foster research in the field, we make our synthesized dataset available to the research community.

Keywords: User-centered Design of Robots, Machine Learning and Adaptation
Abstract: It is crucial that users are empowered to use the functionalities of a robot to creatively solve problems on the fly. A user who has access to a Reinforcement Learning (RL) based robot may want to use the robot's autonomy and their knowledge of its behavior to complete new tasks. One technique is for the user to take control of some of the robot's action space through teleoperation, allowing the RL policy to simultaneously control the rest. We formalize this type of shared control as Partitioned Control (PC). However, this may not be possible using an out-of-the-box RL policy. For example, a user's control may bring the robot into a failure state from the policy's perspective, causing it to act unexpectedly and hindering the success of the user's desired task. In this work, we formalize this problem and present Imaginary Out-of-Distribution Actions, IODA, an initial algorithm which empowers users to leverage their expectations of a robot's behavior to accomplish new tasks. We deploy IODA in a user study with a real robot and find that IODA leads to both better task performance and a higher degree of alignment between robot behavior and user expectation. We also show that in PC, there is a strong and significant correlation between task performance and the robot's ability to meet user expectations, highlighting the need for approaches like IODA. Code is available at https://github.com/AABL-Lab/ioda_roman_2024

Keywords: User-centered Design of Robots, Novel Interfaces and Interaction Modalities, Assistive Robotics
Abstract: Augmented reality multi-user interfaces facilitate communication, coordination and collaboration among teams. Moreover, these interfaces can help to align the team's perceptions and expectations under a shared mental model. This model is a psychological construct that represents the common knowledge, beliefs, and understandings held by team members. In this paper, we study to what extent, if any, the combination of Augmented Reality multi-user interfaces and shared mental models affects human-robot trust. To this end, we developed an Augmented Reality multi-user interface to perform a user study (N=37) comparing non-dyadic human-robot interactions with a quadruped robot exhibiting low reliability (Group 3), against dyadic interactions while the robot exhibited high-reliability (Group 1) or low-reliability (Group 2). We made this comparison using validated trust questionnaires relevant to HRI. Our results, obtained via Bayesian data analysis methods, show differences in the distribution of answers between groups 1 and 2. Notably, this difference is smaller between groups 1 and 3, which suggests that the combination of shared mental models and multi-user interfaces holds promise as an effective way to manage and calibrate human-robot trust.

Keywords: User-centered Design of Robots, Robot Companions and Social Robots, Multimodal Interaction and Conversational Skills
Abstract: The sources of a person's ikigai---their sense of meaning and purpose in life---often change as they age. Reflecting on past and new sources of ikigai may help people renew their sense of meaning as their life circumstances shift. Building on insights from an initial Wizard-of-Oz robot prototype, we describe the design of an autonomous robot that uses a semi-structured conversation format to help older adults reflect on what gives their life meaning and purpose. The robot uses both pre-determined (scripted) and Large Language Model (LLM) generated questions to personalize conversations with older adults around themes of social interaction, planning, accomplishments, goal setting, and the recent past. We evaluated the autonomous robot with 19 older adult participants in a lab setting and at two eldercare facilities. Analysis of the older adults' conversations with the robot and their responses to an evaluative survey allowed us to identify several design considerations for an autonomous robot that can support ikigai reflection. Interweaving simple yet detailed pre-determined questions with LLM-generated follow-up questions yielded enjoyable, in-depth conversations with older adults. We also recognized the need for the robot to be able to offer relevant suggestions when participants cannot recall events and people they find meaningful. These findings aim to further refine the design of an interactive robot that can support users in their exploration of life's purpose.

Keywords: User-centered Design of Robots, Robot Companions and Social Robots
Abstract: To facilitate easy, seamless, and dynamic interactions between humans and social robots, it is important that the robot’s physical appearance gives clear cues to its affordances. However, little is known about what underlying concepts and assumptions shape laypeople’s perception and understanding of different robotic embodiments. To explore what robot-inexperienced users expect robots in different application domains to look like, we drew on Research through Design principles to pilot a tangible design kit with which laypeople could prototype robot designs. 27 participants with no background in robotics were asked to design robots for four different application domains. These participants were then further interrogated about their design choices in structured qualitative interviews. The resulting designs primarily ranged from mechanical to humanoid in appearance, though several animal-like entertainment robots were also created. The inductive thematic analysis of participant interviews revealed complex opinions on anthropomorphism in robotic designs and stressed social robots' potential for customization and accessibility. Our findings provide qualitative insight into the beliefs that underlie laypeople’s understanding of robotic design in different contexts. Furthermore, we piloted a design toolkit that allows laypeople at any level of creative ability to design and critically reflect on robotic concepts and embodiments.

Keywords: User-centered Design of Robots
Abstract: How a robot moves is among the first things an observer notices when they encounter a robot. While considerable research has investigated the perception of robot body language, no studies yet, to our knowledge, have explored the social effects of how a robot moves through space (its gait) on people's first impressions of a robot. To this end, here we performed two complementary experiments online (n=98) and in-person (n=26), with the objective of determining the extent to which a quadruped robot's gait influences a) what animal people perceived it to be; and b) its social attributes in terms of warmth, competence, causing discomfort and zoomorphism. Results differed depending on whether participation was online or in-person, with gaits influencing participants' perception more markedly when they encountered the robot in-person. Online, most participants saw the robot as a dog for every gait except one, while in-person participants reported more varied responses. Participants in both studies rated the more active, "bouncy" gait as warmer and less discomforting. In-person participants also consistently rated all gaits as warmer, more competent, less discomforting and more zoomorphic than did online participants. The study supports findings that in-person exposure and embodiment affect a robot's social perception and further suggests that gait may have a limited effect as well.

Keywords: Linguistic Communication and Dialogue, Creating Human-Robot Relationships, Evaluation Methods
Abstract: The need for robots to explain their failures grows as the variety and number of robots deployed in public, homes, and work environments increases. This paper extends our prior work utilizing explanation templates by comparing those Templated explanations to Generative explanations created by a Large Language Model. Our study surprisingly reveals that Templated explanations result in similar or higher perceived intelligence and trust while also being more understandable. Through our findings, we aim to provide considerations for effective robot explanation systems, ultimately enabling people to be able to understand and provide assistance to robots that have encountered unforeseen circumstances.

Keywords: Linguistic Communication and Dialogue, Creating Human-Robot Relationships, Long-term Experience and Longitudinal HRI Studies
Abstract: The use of dialogue agents with a strategy that focuses beyond a static depth level and gradually deepens the conversation over time can nurture relationships with users. Thus, this study aimed to develop a dialogue agent capable of gradually deepening conversations over time and consequently examine its effectiveness. First, we constructed a topic depth estimation model capable of estimating the depth of any given topic. Subsequently, we developed a dialogue agent designed to gradually deepen conversation topics as interactions progress, tailored for operations on users' smartphones. To evaluate the effectiveness of the developed agent, we conducted a 10-day dialogue experiment with 26 participants. The results indicated that the proposed agent elicited deep self-disclosure from users without significantly increasing psychological reluctance compared to the control condition. Moreover, the participants perceived a deeper relationship with the agent and were less likely to consider it as a non-human entity. These findings suggest that the proposed dialogue agent was capable of effectively engaging in deeper conversations over time, thereby enhancing human-agent interactions.

Keywords: Linguistic Communication and Dialogue, Novel Interfaces and Interaction Modalities, Applications of Social Robots
Abstract: This work introduces a robotics platform which comprehensively integrates multi-step action planning, natural language understanding, and memory to interactively perform service tasks in accordance with variable needs and intentions of users. The proposed architecture is built around an AI agent, derived from GPT-4, which has been embedded in an embodied system. Our approach utilizes semantic matching, plan validation, and state messages to ground the agent in the physical world, enabling a seamless merger between communication and behavior. We demonstrated the advantages of this system with an HRI study comparing mobile robots with and without conversational AI capabilities in a free-form tour-guide scenario. The increased adaptability of the system is measured along five dimensions: flexible tasking planning, interactive exploration of information, emotional friendliness, personalization, and increased overall user satisfaction.

Keywords: Linguistic Communication and Dialogue, Novel Interfaces and Interaction Modalities, Multimodal Interaction and Conversational Skills
Abstract: With current state-of-the-art automatic speech recognition (ASR) systems, it is not possible to transcribe overlapping speech audio streams separately. Consequently, when these ASR systems are used as part of a social robot like Pepper for interaction with a human, it is common practice to close the robot's microphone while it is talking itself. This prevents the human users to interrupt the robot, which limits speech-based human-robot interaction. To enable a more natural interaction which allows for such interruptions, we propose an audio processing pipeline for filtering out robot's ego speech using only a single-channel microphone. This pipeline takes advantage of the possibility to feed the robot ego speech signal, generated by a text-to-speech API, as training data into a machine learning model. The proposed pipeline combines a convolutional neural network and spectral subtraction to extract overlapping human speech from the audio recorded by the robot-embedded microphone. When evaluating on a held-out test set, we find that this pipeline outperforms our previous approach to this task, as well as state-of-the-art target speech extraction systems that were retrained on the same dataset. We have also integrated the proposed pipeline into a lightweight robot software development framework to make it available for broader use. As a step towards demonstrating the feasibility of deploying our pipeline, we use this framework to evaluate the effectiveness of the pipeline in a small lab-based feasibility pilot using the social robot Pepper. Our results show that when participants interrupt the robot, the pipeline can extract the participant's speech from one-second streaming audio buffers received by the robot-embedded single-channel microphone, hence in near-real time.

Keywords: Linguistic Communication and Dialogue, Social Intelligence for Robots, User-centered Design of Robots
Abstract: Navigating through a new indoor environment can be stressful. Recently, many places have deployed robots to assist visitors. One of the features of such robots is escorting the visitors to their desired destination within the environment, but this is neither scalable nor necessary for every visitor. Instead, a robot assistant could be deployed at a strategic location to provide wayfinding instructions. This not only increases the user experience but can be helpful in many time-critical scenarios e.g., escorting someone to their boarding gate at an airport. However, delivering route descriptions verbally poses a challenge. If the description is too verbose, people may struggle to recall all the information, while overly brief descriptions may be simply unhelpful. This article focuses on studying the optimal length of verbal route descriptions that are effective for reaching the destination and easy for people to recall. This work proposes a theoretical framework that links route segments to chunks in working memory. Based on this framework, an experiment is designed and conducted to examine the effects of route descriptions of different lengths on navigational performance. The results revealed intriguing patterns suggesting an ideal length of four route segments. This study lays a foundation for future research exploring the relationship between route description lengths, working memory capacity, and navigational performance in indoor environments.

Keywords: Linguistic Communication and Dialogue
Abstract: For robots to effectively collaborate with human partners, they need to be able to understand what instructions and/or statements mean in the context of their environment. To ground objects in a dynamic world, robots need an understanding of how both spatial and temporal relationships evolve as a function of time in the environment. However, it is computationally intensive to classify, track, and predict the motion of all objects. Approaches based on Language-Guided Temporally Adaptive Perception (LGTAP) utilize information embedded in the instruction to selectively classify objects to construct minimal but sufficiently detailed models of the environment for symbol grounding. Such methods, however, fail when the instruction refers to the future state of the environment as it lacks any notion of whether to and for how long a future prediction is necessary to ground the instruction. This prompts a reformulation of LGTAP that can selectively utilize information from past observations to accurately predict the future state of objects. This paper describes a novel approach for LGTAP for instructions that may refer to the past, present, and/or future state of the environment by closing the loop around symbol grounding and adaptive perception. A detailed analysis of a grounding problem that refers to the future state of the environment, a corpus-based analysis of performance, and a physical demonstration of natural language understanding is presented along with a description of this novel architecture.

Keywords: Anthropomorphic Robots and Virtual Humans, Computational Architectures, Medical and Surgical Applications
Abstract: Humanoid robots are promising approach to automating patient interviews routinely conducted by medical staff. Their human-like appearance enables them to use the full gamut of verbal and behavioral cues that are critical to a successful interview. On the other hand, anthropomorphism can induce expectations of human-level performance by the robot. Not meeting such expectations degrades the quality of interaction. Specifically, humans expect rich real-time interactions during speech exchange, such as backchanneling and barge-ins. The nature of the patient interview task differs from most other scenarios where task oriented dialogue systems have been used, as there is increased potential of engagement breakdown during interaction. We describe a dialogue system architecture that improves the performance of humanoid robots on the patient interview task. Our architecture adds a nested inner real-time control loop to improve the timeliness of the robot’s responses based on the notion of “stance”, an elaboration of the concept of a “turn”, common in most existing dialogue systems. It also expands the dialogue state to monitor not only task progress, but also human engagement. Experiments using a humanoid robot running our proposed architecture reveal improved performance on interview tasks in terms of the perceived timeliness of responses and users’ impressions of the system.

Keywords: Anthropomorphic Robots and Virtual Humans, Linguistic Communication and Dialogue, Sound design for robots
Abstract: People regularly interact with synthetic speech systems that vary in vocal characteristics. Three studies explored whether anthropomorphic (i.e., humanlike) and gendered (i.e., masculine/feminine) features of voices affected fundamental judgments of warmth, competence, and discomfort. Study 1 used vocal recordings with semantic content removed. Natural voices were judged as warmer, more competent, and produced less discomfort than machine-like voices. Feminine voices were judged as warmer and elicited less discomfort than masculine voices. Study 2 used unfiltered versions of the same stimuli and found similar negative impacts of machine-like features of social judgments, but the effects of voice gender disappeared. Study 3 examined whether gendered effects returned when gender-ambiguous semantic content was labeled as stereotypically feminine (braiding hair) or stereotypically masculine (tying rope). Stereotype consistency between task label and voice gender mattered for discomfort ratings of feminine voices. In sum, semantic content and the consistency between content and stereotypical expectations affected judgments of gendered voices. Additional considerations regarding gendered voices and speech content should be explored in future research.

Keywords: Anthropomorphic Robots and Virtual Humans, Linguistic Communication and Dialogue, User-centered Design of Robots
Abstract: This paper aims to mitigate impression loss due to response delays in spoken dialogue systems. The system addressed in this paper consists of connected ASR, NLP, and TTS AI services, and large response delays due to processing time and network latency in each module may give a bad impression to users. Therefore, we propose three techniques to deal with the inevitable delays: filler insertion, parallel speech processing, and prompt editing. To evaluate the effectiveness of the proposed method, measurements and subject experiments were conducted. The results showed that filler insertion had statistically significant improvements of 1.1 points on a Likert scale in the impression of response delay. Parallel speech processing reduced delay by 0.6 seconds. Prompt editing was also effective in reducing delay through response length suppression.

Keywords: Anthropomorphic Robots and Virtual Humans, Programming by Demonstration, Machine Learning and Adaptation
Abstract: Dexterous in-hand manipulation is a unique and valuable human skill that requires sophisticated sensorimotor interaction with the environment while respecting stability constraints. Incorporating these constraints into motion generation is essential for a robotic platform to achieve reliable in-hand manipulation skills. Explicitly modeling these constraints can be challenging, but they can be implicitly modeled and learned through experience or human demonstrations. In this article, we propose a learning and control approach based on motion primitive dictionaries for acquiring in-hand manipulation skills from human demonstrations. To achieve this, we define an optimization process that combines motion primitives to generate robot fingertip trajectories for moving an object from an initial to a desired final pose. Based on our experiments, we found that our approach allows the robot to handle objects like a human would, while also adhering to stability constraints, without requiring explicit formalization. In other words, the proposed motion primitive dictionaries learn and implicitly embed the constraints crucial to the in-hand manipulation task.

Keywords: Robotic Etiquette, Anthropomorphic Robots and Virtual Humans, Interaction Kinesics
Abstract: This paper proposes an energy consumption estimation in the trajectory planning task of an industrial robot (IR) arm manipulator. Human-robot collaborative scheduling has become integral to modern manufacturing. Effective scheduling of human-robot cooperation is crucial for enhancing production efficiency. However, energy-efficient trajectory planning in human-robot interaction remains an unexplored area. To address this, a 6-DoF (Degree of Freedom) robot arm, available in the laboratory, was utilized to assess the energy consumption of this specific robot type. However, his approach can be applied to any robot, whether it's a human-robot collaboration, a mobile robot, or any other articulated robot arm manipulator. For this goal the electrical and mechanical losses of a 6-DoF robot arm manipulator from the company Universal Robots is modeled and its power losses cost function (PLCF) is derived correspondingly to calculate the electromechanical energy losses. The objective of this paper is to estimate the energy losses in a trajectory planning task in cartesian space for a tool center point (TCP) of the robot, starting from an initial point and resting at a final point. This goal is achieved by examining and analyzing the kinematics and dynamics followed by measuring the velocity and acceleration related terms of each motor at each joint of the UR3e robot arm in a laboratory setup. The results estimate the changes of energy losses in the overall trajectory following of the robot manipulator and presents the results of the estimated and measured reduction terms.

Keywords: Cooperation and Collaboration in Human-Robot Teams, Multimodal Interaction and Conversational Skills, Novel Interfaces and Interaction Modalities
Abstract: Effective communication between robots and human operators is crucial for seamless collaboration in industrial settings. We aim to capture the requirements of collaborative assembly procedures in the context of the Industry 5.0 paradigm and explore the feasibility of various multi-modal feedback systems, including visual cues, sound effects, and virtual eyes. To achieve this, we present a novel research platform designed to investigate human-robot interaction (HRI) strategies in collaborative assembly tasks. The platform uses LEGO bricks to simulate real-world assembly processes. An experiment is designed in which human and robot collaboratively build a structure, allowing us to investigate potential communication interfaces between them. A preliminary user study provides first insights into the perception of the robot's visualized intentions and actions by the user. The platform setup is not intended to be a fixed system, but rather a starting point for further investigation and future studies in the field of HRI.

Keywords: Cooperation and Collaboration in Human-Robot Teams, Non-verbal Cues and Expressiveness
Abstract: Robotics researchers are increasingly exploring how robots can support human groups and teams. Action teams experience high workload, must work quickly, and must make decisions under uncertainty. Robots in these teams must be designed and contextualized to not contribute to errors or interrupt human team workflow. We conducted a study where human-robot action teams collaborated with a mobile manipulator (Stretch) in an escape room paradigm to better understand: 1) how a robot’s actions influenced intra-team dynamics, and 2) how attitudes towards the robot affected human-robot teaming. Our behavioral analysis highlights the effect of a robot’s functional and social behaviors on its acceptance within teams, how human teams adapt to perceived robot capabilities, and the significance of the robot’s nonverbal cues in shaping human expectations. These insights offer valuable implications for designing effective human-robot interactions in dynamic environments where team success can save lives.

Keywords: Cooperation and Collaboration in Human-Robot Teams, Robot Companions and Social Robots, Non-verbal Cues and Expressiveness
Abstract: When social robots are joining human teams it is relevant to not only study the social dynamics within these human-robot mixed groups but also to explore whether outsiders perceive them as groups. Perceiving higher or lower groupness of mixed groups potentially influences which expectations individuals have towards the group, its human and importantly its robot members. We examined how much outsiders perceive differently composed human-robot groups as a group. In a between-subjects video-based online study, participants rated videos of group interactions with diverse group compositions (i.e., 1 robot with 3 humans, 2 robots with 2 humans, and 3 robots with 1 human) whose members either did or did not wear a visual group token (i.e., the same wristband) regarding the perceived groupness. Group composition and visual token influenced perceived groupness with a group of 1 human and 3 robots being less perceived as a group. Visual tokens enhanced group perceptions. This research contributes to our comprehension of how people perceive human-robot groups.

Keywords: Cooperation and Collaboration in Human-Robot Teams, Virtual and Augmented Tele-presence Environments, Novel Interfaces and Interaction Modalities
Abstract: In this paper, we introduce an innovative robotic nursing assistance system with multilateral multimodal human-robot (MMHR) collaboration, enabling nursing robots to be assisted by remote and on-site operators. Through our augmented reality (AR) interfaces, the remote and local operators can monitor the robots’ operations, issue task and action commands, and facilitate collaborative assistance and information exchange via AR cues or verbal communication. Our preliminary user study evaluated the usability of the prototype system and validated the efficacy of our MMHR collaboration in a representative nursing assistance task scenario. The results show significant improvements in overall task efficiency for the remote operator and reveal human strategies and rationales in the spontaneous multilateral human-robot collaboration.

Keywords: Cooperation and Collaboration in Human-Robot Teams
Abstract: Prior research has suggested that equalizing participation may benefit group performance and group cohesion. Robot-moderated groups have largely focused on improving member participation by focusing on the least performing member and do not consider the frequency of interaction or type of interaction. We introduce a robot moderator that varies its frequency and interaction types to observe the impact on groups in terms of performance and group cohesion. We investigate this in user studies across four conditions for equalizing participation. Leveraging Bayesian statistical methods that can evaluate evidence both for and against the null hypothesis, we find evidence that neither personalizing robot actions nor balancing the target of the robot's assistance affected user experience in the group (as measured by performance, group cohesion, and variance of participation). We find a lack of evidence for equalization of participation impacting performance and group cohesion. Additionally, we also find positive evidence against the correlation of equalized participation and group cohesion in our task and weak evidence against equalized participation correlating with performance. In addition to guiding future researchers regarding robot behaviors that may not be effective in affecting groups, this work is an important negative result suggesting that equalizing participation may not be adequate to improve group performance and cohesion in all tasks.

Keywords: Cooperation and Collaboration in Human-Robot Teams
Abstract: In today’s era, robots are increasingly being developed to collaborate with human teammates in order to form highly efficient teams. However, it is not uncommon for teammates to prioritize their personal goals over assisting others in order to meet deadlines. Sometimes, the most successful teams are characterized by the ability of each team member to prioritize their own individual or shared objectives while maintaining consistency and cooperation with one another. To gain insights into human perceptions of robot teammates who prioritize their own tasks over helping human partners, we conducted a research experiment involving (n=317) participants. Through careful observation of various instances of prioritization, our findings indicate that humans are required to put in additional effort in tasks when the robots ruthlessly prioritize their own. In addition to that the humans feel more neglected by robots in verbal prioritization conditions, but view them as more intelligent and value their reliability and competitive capabilities in other scenarios.

Keywords: Medical and Surgical Applications, Assistive Robotics, Machine Learning and Adaptation
Abstract: Functional electrical stimulation is a promising technique for restoring arm function to those with paralysis from a high spinal cord injury. While simple controllers are easy to implement, model-based controllers are likely better equipped to leverage the arm's kinematic and dynamic complexity, particularly for the high variations associated with functional arm movement. One modelling technique for a model-based controller is Gaussian Process Regression. Previous simulation work has shown promise leveraging whole-arm error data to identify the arm's various subsystems, but used perfect simulated data. We asked caregivers to correct a robotic arm's movement as simulated muscles generated torque. The simulated muscles were controlled as if they were electrically stimulated human arm muscles. This study demonstrates non-expert caregivers' ability to collect this error data via whole-arm corrections, and provides insight into their ability to improve arm subsystem models made with Gaussian Process Regression. Despite significant error in caregivers' ability to provide force corrections to hold the robot in a static configuration, these corrections were leveraged to significantly improve muscle models; the muscles that improved the most were the ones primarily used to move the physiologically actuated robot.

Keywords: Medical and Surgical Applications, Degrees of Autonomy and Teleoperation, Human Factors and Ergonomics
Abstract: This paper describes the safety and efficacy of an autonomous robotic system to collect ultrasound (US) images of the peripheral vasculature of 31 human participants, while also assessing their trust and comfort with the procedure. The procedure used a custom restraint mechanism and robotic arm guided by RGB-D imaging to collect clinically meaningful US images of human vasculature in the peripheral forearm safely and autonomously. All initial presses and scanned trajectories were executed under a safety force threshold (13N), included vasculature in imaging (from trajectory selected by non-clinician), and had a full scan completion success rate of greater than 80%. Participants indicated increased trust and perception of safety in the robotic system after the procedure. The positive findings suggest that careful attention to patient safety and well-designed patient/robot interactions can positively affect human-robot interaction and change the perception of robotic systems in medical contexts.

Keywords: Medical and Surgical Applications, Degrees of Autonomy and Teleoperation, Machine Learning and Adaptation
Abstract: In this work, we present a learning from demonstration solution for automating a surgical blunt dissection task. In addition to learning motion trajectories, our goal is to learn variable impedance behaviors that enable the robot to interact safely and compliantly during the task. To that end, we propose a teaching interface using bilateral teleoperation, which allows the natural transfer of human motions and impedance behaviors skills to robots. The demonstrated profiles are captured with Dynamic Movement Primitives and Gaussian Mixture Models, which subsequently provide the robot a reference motion plan, and a stiffness adaptation policy, during physical interaction. Experimental validation on real robot hardware shows the effectiveness of the proposed approach in terms of ensuring successful task execution, as well as safety compared to stiff high-gain control.

Keywords: Embodiment, Empathy and Intersubjectivity, Anthropomorphic Robots and Virtual Humans, Assistive Robotics
Abstract: Social robots and virtual agents can provide a low-threshold social interface for the independent self-health monitoring of older adults, thereby supporting their self-care and empowering them to take charge of their own health. In this work, we present a study on the interface preference of 35 older adults for a health monitoring system which they will be evaluating for eight weeks in their own homes. We found that participants who prefer the social robot have a higher affinity for technology interaction (ATI) score and are more likely to use assistance systems in their daily lives, while participants with a slightly lower ATI score prefer the virtual agent. Participants cited more personality and an interest in robots as reasons for the robot preference, and space and flexibility as reasons to prefer the virtual agent. These results underscore the importance of a personalized introduction of social technologies for health monitoring into the daily lives of older adults.

Keywords: Embodiment, Empathy and Intersubjectivity, Motivations and Emotions in Robotics, Creating Human-Robot Relationships
Abstract: Past research with participants in the United States showed that, in competitive group tasks, they have more positive attitudes and behaviors toward robots on their team over humans in another team. Here we present a study in which two Japanese students and two robots, placed in a randomly assigned group, compete with another student-and-robot team in a digital game. We explored participants’ moral behavior towards the robots, measured through their assignment of loud noise blasts to human and robot participants, and their perceptions of and attitudes towards the robots. We then compared this data to that which was collected within the United States. Results indicated that participants in Japan favored their ingroup humans and robots over outgroup agents and differentiated ingroup members more than outgroup members, as within the US. Japanese participants also anthropomorphized robots more than US participants and treated them more positively than US participants.

Keywords: Embodiment, Empathy and Intersubjectivity, Philosophical Issues in Human-Robot Coexistence, Ethical Issues in Human-robot Interaction Research
Abstract: Though some matters of consensus have begun to crystallize, scholars in human-robot interaction have thus far reasoned about artificial identity under many different definitions. Many of these seemingly disparate perspectives may, however, be unified into one coherent model through a synthesis of contemporary scientific and Buddhist philosophy of identity. Under this model, artificial and human identity are modeled equivalently under an assumption that there is no "unchanging essence" which constitutes an agent's identity, but rather that identity may be defined as the sum of overlapping aggregates subject to change through time. The model reckons with the idea that much of what is conceived of as identity may be arbitrarily ascribed, artificial boundaries, but that these boundaries often constitute substantial social and psychological realities. This thinking is congruent with contemporary philosophical perspectives across disciplines from biology to cognitive science. The model may serve as a useful tool for reasoning about identity in complex, dynamic situations and provide a firm foundation for work which utilizes artificial identity. The model may even offer one or two possible answers to the question: Who is a robot?

Keywords: Robots in Education, Therapy and Rehabilitation, Multimodal Interaction and Conversational Skills, Non-verbal Cues and Expressiveness
Abstract: Different people respond to feedback and guidance in different ways. Their preferences may even change depending on their mood, fatigue, physical health, etc. We present a robot exercise coach that provides both verbal and nonverbal feedback. We first introduce an exercise evaluation method where the camera feed from the robot is used to evaluate how well people perform exercises. We then present a multi-modal feedback controller that uses the exercise evaluation to respond with verbal and nonverbal feedback in different styles (firm and encouraging). Our user study found that participants have significantly different performances and subjective experiences with the different styles. We also found differences in how participants with different preferences for the styles perform with the different styles. These results show that varying feedback styles has an impact and builds the basis for a robot that adapts its style in real-time to personalize to the individual.

Keywords: Robots in Education, Therapy and Rehabilitation, Philosophical Issues in Human-Robot Coexistence, Child-Robot Interaction
Abstract: Educational robotics is increasingly merging into school curricula. Understanding the subjective perceptions of robots, particularly among children, requires nuanced approaches. In this paper, we employed a qualitative coding analysis method to explore how children of different ages conceptualise robots through drawings, and how prior experiences with robotics influence their perceptions. Our findings reveal that the perception of robotics is influenced by cognitive development stages, which is in turn affected by age, and by educational robotics. The latter plays a significant role in shaping perceptions of robots, fostering positive attitudes and aiding cognitive development, particularly in first-grade students. Our insights can inform both teachers to better tailor their educational robotics activities for different age groups and robot designers themselves. For instance, our findings highlight the importance of emotional expression and the preference for humanoid robots among primary school children.

Keywords: Robots in Education, Therapy and Rehabilitation, Robots in art and entertainment, Cognitive Skills and Mental Models
Abstract: With the increasing prevalence of robots in everyday life, there is a growing need for the general public to understand the capabilities and limitations of these technologies. For example, in the case of sidewalk delivery robots, humans ranging from nearby pedestrians to intended end users may miscalculate robot states, plans, and capabilities without this type of intuition. This paper explores the effectiveness of a relatively brief video-based intervention on enhancing public understanding of current delivery robots. The presented study (N = 100) assessed participants' perceptions of their own knowledge and characteristics of the studied robotic system before and after watching the video. Results indicated significant changes in participants' confidence levels, robot trust ratings, robot competence ratings, and attachment to the robots. The study can help to inform future techniques for improving the general public's understanding of day-to-day robotic technologies, including rapid and relatively entertaining tactics like the video compilation considered in this work.

Keywords: Robots in Education, Therapy and Rehabilitation, Social Touch in Human–Robot Interaction, Monitoring of Behaviour and Internal States of Humans
Abstract: Devices that deliver deep pressure sensations (DPS) are common, but how well do these systems actually work, and are DPS experiences different across devices? To help address these questions, we previously designed a portable and fast-acting soft-robotic DPS alternative: the AID Vest. In this work, we evaluate the AID Vest's effect on individuals with moderate or high anxiety specifically. We conducted a study with N=10 participants, providing experiences with a weighted blanket and the AID Vest, and measuring biosignals, one-shot self-reports, and exploratory continuous self-reports related to these experiences. The results show reductions in established biosignal and self-report methods for measuring anxiety for both DPS experiences. The continuous self-report results were mixed, but may be useful for future hypothesis generation. This work shows more positive AID Vest effects compared to our past work on convenience population users, and our results can inform others with interest in DPS applications such as anxiety management.

Keywords: Motivations and Emotions in Robotics, Robots in Education, Therapy and Rehabilitation, Social Presence for Robots and Virtual Humans
Abstract: This study investigates how the level of physical anthropomorphism in conversational agents affects their persuasiveness. The experiment compared robots with low and high levels of anthropomorphism, akin to human appearance. Results indicated that while sophisticated physical embodiment did not universally enhance persuasiveness, it did have an interaction effect based on issue involvement. For low involvement issues, persuasiveness increased with higher anthropomorphism. In contrast, for high involvement issues, persuasiveness remained constant across different levels of anthropomorphism but dropped when robots closely resembled humans, possibly due to trust in high-tech robots. The findings highlight the importance of digital agent’s persuasive power and ethics in employing and designing robots.

Keywords: Art pieces supported by robotics, Robots in art and entertainment
Abstract: We present a lightweight, real-time gesture responsive framework designed to investigate dynamics of human-robot interaction in live dance performances, merging aspects from choreography and robotics. In particular, it is tailored for artists to integrate human-sized humanoid robots into their dances seamlessly, offering an intuitive solution without the complexities of mastering robot control systems. Unlike existing interaction methods relying on wearable sensors or predetermined music cues, our framework, integrated in a task-space controller, enables the robot to dynamically respond to the dancer's movements, generating unpredictable yet artistically meaningful gestures without the need for body-mounted sensors. This design choice emphasizes the artistic intention behind the improvisation, illustrating that it is not merely about creating movements spontaneously but rather a way of expression in intuition guided by perception. We assess our framework on the HRP-4 humanoid robot and the result has been successfully demonstrated in a public human-robot dance performance at the Arts Center of Enghien-les-Bains's 23/24 season launch event, aiming to contribute to the exploration of co-creation between human and human-sized humanoid robot in improvised dance performances.

Keywords: User-centered Design of Robots, Applications of Social Robots, Creating Human-Robot Relationships
Abstract: Compliance is when a human positively responds to a request or a recommendation given by a system. For example, when prompted, providing your thumbprint for an automated biometric scanner at the airport or starting to watch a new TV show on a streaming service ‘we think you will love’. In trust-related research, compliance is frequently used as a behavioural measure of trust. When evaluating the compliance-trust association in experimental settings, typically, the participants agree, when asked, that they complied because they trusted the system. We developed three scenarios in instantaneous settings where compliance with an instruction delivered by a robot would typically be ascribed to trust. However, rather than asking, ‘Did you trust?’, we asked, ‘Why did you comply?’ In a thematic analysis of responses, we discovered robot design characteristics and sources not related to the design that persuade humans to comply with instructions delivered by a robot.

Keywords: User-centered Design of Robots, Applications of Social Robots, Human Factors and Ergonomics
Abstract: Many research works currently focus on algorithms designed to generate explanations and then evaluate their effect on user trust and understanding of robots. Even though some projects attempt to design understandable interfaces, they usually serve as extra features for solutions that already exist. In this paper, we suggest a user-centric approach to design explainable robot systems from the very beginning. In particular, we provide a participatory design approach that places emphasis on the cooperative design of an understandable and intuitive interface between the user and the robot system. We suggest turning the attention to the robot's functionality and autonomous behaviours development after this interface has been established. We exemplify how to apply the proposed framework in a geriatric unit at an intermediate care centre.

Keywords: User-centered Design of Robots, Applications of Social Robots, Linguistic Communication and Dialogue
Abstract: Robotic coaches have been recently investigated to promote mental well-being in various contexts such as workplaces and homes. With the widespread use of Large Language Models (LLMs), HRI researchers are called to consider language appropriateness when using such generated language for robotic mental well-being coaches in the real world. Therefore, this paper presents the first work that investigated the language appropriateness of robot mental well-being coach in the workplace. To this end, we conducted an empirical study that involved 17 employees who interacted over 4 weeks with a robotic mental well-being coach equipped with LLM-based capabilities. After the study, we individually interviewed them and we conducted a focus group of 1.5 hours with 11 of them. The focus group consisted of: i) an ice-breaking activity, ii) evaluation of robotic coach language appropriateness in various scenarios, and iii) listing shoulds and shouldn'ts for designing appropriate robotic coach language for mental well-being. From our qualitative evaluation, we found that a language-appropriate robotic coach should (1) ask deep questions which explore feelings of the coachees, rather than superficial questions, (2) express and show emotional and empathic understanding of the context, and (3) not make any assumptions without clarifying with follow-up questions to avoid bias and stereotyping. These results can inform the design of language-appropriate robotic coach to promote mental well-being in real-world contexts.

Keywords: User-centered Design of Robots, Assistive Robotics, Applications of Social Robots
Abstract: The seamless integration and acceptance of humanoid service robots in society requires a general understanding of those robots. An overall transparent design can be a means to establish this. To guide the implementation of effective transparency mechanisms, we propose the novel Transparency Design Model, which builds on the concept of Interaction Transparency from our previous work. This model encompasses a wide range of transparency aspects aimed to improve the overall transparency of humanoid service robot designs. To demonstrate the applicability of our model and gather initial insights on its impact on robot perception, we conducted a user study involving 50 participants. The study compared a neutral and transparent robot behavior within a complex sales service scenario. Our findings indicate that the transparent design significantly improve the robot’s likeability, anthropomorphism, and benevolence. Additionally, we observed positive trends in attractiveness, stimulation, and novelty as measured by the User Experience Questionnaire. These measurable effects highlight the important role of Interaction Transparency. The results also imply that a holistic system analysis in multiple complex and application-dependent interactions is worthwhile as the service content influences the perception and effects of the individual design steps. Further, there is a need to develop a new scale for measuring Interaction Transparency. Lastly, future research could benefit from employing multi-modal assessment tools.

Keywords: Wearable Robotics, Biologically-Inspired Robots, Mechanism Design
Abstract: This paper introduces a “rigidable exosuit” inspired by bow mechanics, which can dynamically alter its frame rigidity based on the tension of the attached string. The exosuit affords posture assistance (rigid phase) or unrestricted movement (flexible phase) as required by tasks. In the flexible phase, relaxed strings enable free shoulder and waist movements, reducing entanglement likelihood. Conversely, during strength-demanding tasks (rigid phase), pulled strings provide waist support. The tension in the string enables seamless transitioning between rigid and flexible configurations as required by the task at hand. The waist posture support prototype features a carbon fiber reinforced plastic bow structure and a tension control mechanism utilizing a twisted string actuator. Experiments using the prototypes demonstrated that movements in the flexible phase were comparable to those in the unworn states. During the rigid phase, while performing lifting tasks, the erector muscle electromyography signal decreased by over 20% and the motion angle by approximately 40% after wearing the device. The exosuit thus has significant potential to enhance human performance across various physical activities and implications in wearable robotics. By providing seamless transitioning between rigid and flexible configurations, the exosuit can aid support workers in diverse occupational settings, ultimately improving safety and productivity.