Keywords: Mobile Robots, Identification and Estimation in Mechatronics, Sensor Integration, Data Fusion
Abstract: This paper studies mobile robot localization with unknown noise covariance. The AprilTag is used as landmarks and observed using the onboard camera. The system model is created based on the mobile robot motion and AprilTag measurements. The unknown measurement noise covariance is considered as a random matrix satisfying an inverse Wishart distribution. A variational Bayesian estimator is proposed to estimate the robot pose, AprilTag locations, and measurement noise covariance, where the Rao-Blackwellized estimator is developed for robot pose and AprilTag location estimation, and variational Bayesian approximation is adopted for measurement noise covariance estimation. Simulations and experiments are conducted to validate the effectiveness of the proposed method.

Keywords: Planning and Navigation, Mobile Robots, Transportation Systems
Abstract: Scheduling collision-free paths for a large number of robots in dense environments with high efficiency is achieved in this work. We propose an algorithm, OMPP (One-way Multi-robot Path Planning), using a new topological skeleton representation of the dense environment by introducing the particular collision-free traffic rules. We propose the integer programming technique based on the topological skeleton graph to tackle the multi-robot path planning optimization problem using distance metrics. We realize two practical achievements in solving multi-robot path planning problems in dense environments: collision-free robotic path generation and an efficient solving process. We have performed numerous simulations. According to the extensive simulation data, our algorithm suggests a higher overall performance in dense environments than the existing representative algorithms.

Keywords: Mobile Robots, Control Application in Mechatronics, Modeling and Design of Mechatonic Systems
Abstract: A control Lyapunov function and control barrier function-based quadratic program (CLF-CBF-QP) is proposed to safely navigate a nonholonomic mobile robot (NMR) facing a moving obstacle. The conventional CLF-CBF-QP realizes safety and stability conditions using distance-based CLF and CBF without considering the practical limitations of the steering angle. Hence, the NMR may execute undesirable avoidance maneuvers such as u-turns. This work introduces a safety condition on the steering angle to maintain efficient, smooth avoidance maneuvers. The safe steering condition keeps the NMR steering angle within a predefined envelope of the reference steering angle. Moreover, one can improve the response time of the control system by enforcing exponential decay rates on the CLF and CBF. However, fixed decay rates lead to aggressive maneuvers around moving obstacles, where the CLF and CBF may produce conflicting conditions. Hence, this work proposes to optimize the decay rates of the CBF and CLF in real-time to improve the NMR performance during obstacle avoidance maneuvers. Lastly, the effectiveness of the proposed control method is verified by simulating two case studies where an NMR i) overtakes and ii) avoids head-on collision with a moving obstacle.

Keywords: Mobile Robots, Control Application in Mechatronics, Modeling and Design of Mechatonic Systems
Abstract: Towing a mobile platform with a tether fixed to the external environment is an effective way to traverse rough terrain. A flexibly bendable telescopic arm that can carry the tether at its arm tip was developed in a previous study. This study is focused on the rough terrain traveling method using a compact elastic telescopic arm installed on a mobile platform. Furthermore, by fusing the linear and bending mechanisms, we developed an elastic telescopic arm that is compact and can perform extension and bending motions on the same body. The results of the arm operation experiment showed that the extension-to-contraction ratio was improved by 16%, and the tip position control accuracy was improved by 70% compared to the previous study. Finally, the effectiveness of the proposed method was demonstrated by a step-climbing experiment, wherein, the step height was more than the wheel diameter of the mobile platform.

Keywords: Mobile Robots, Walking Machines, Robot Dynamics and Control
Abstract: Lateral walking of the newly introduced flatworm-like multiple pedal wave locomotory robot called WORMESH-II is presented in this paper. Use of multiple traveling waves is the significance of WORMESH-II, and it has been discussed previously. Pedal wave locomotion of WORMESH-II displayed a relative motion between the ground and contact points that is not efficient for locomoting in gravel and sandy terrains. Hence, it needed to develop a locomotion gait with joint mechanism and flexibility capable for multi-mode locomotion to use in such landscapes. In this study, lateral walking locomotion gait which facilitates WORMESH-II to move perpendicular to the wave propagation direction has been introduced. The locomotion features are similar to the statical walking of a multi-leg robot. The proposed locomotion gait was verified and tested successfully respectively using dynamic simulations and the prototype of WORMESH-II. Since there is no relative motion between the ground and contact points of the robot, the suggested lateral walking locomotion is better for locomoting in sandy and gravel surfaces.

Keywords: Tele-operation, Mobile Robots, Control Application in Mechatronics
Abstract: This paper proposes a decentralized adaptive impedance control approach to the bilateral control of a team of mobile robots under time-varying delays. A master-slave framework is developed and a decoupled approach is taken to ensure stability and allow for different formation controllers to be implemented. A novel decentralized method of estimating the center of formation is formulated. An adaptive impedance controller is proposed where the impedance parameters are functions of the error between the estimated center of formation and desired center of formation. In comparison to traditional tank-based passivity control for the teleoperation channels, this approach provides more accurate stiffness following and fewer open parameters that must be tuned. The purpose of the force feedback is to reflect the information of the environment to the operator to provide a transparent representation of the environment and the formation performance. This feedback encourages the operator to command the team in a motion that maintains formation and maneuvers around obstacles. Simulations and experiments with a Phantom Omni haptic device and three TurtleBot3 mobile robots are conducted to validate the proposed framework in the presence of time-varying delays.

Keywords: Mobile Robots, Robot Dynamics and Control, Control Application in Mechatronics
Abstract: As a single-track mobile platform, bikebot (i.e., bicycle-based robot) has attractive navigation capability to pass through narrow, off-road terrain with high-speed and high-energy efficiency. However, running crossing step-like obstacles creates challenges for intrinsically unstable, underactuated bikebots. This paper presents a novel autonomous bikebot control with assistive leg actuation to navigate crossing obstacles. The proposed design integrates the external/internal convertible-based control with leg-assisted impulse control. The leg-terrain interaction generates assistive impulsive torques to help maintain the navigation and balance capability when running across obstacles. The control performance is analyzed and guaranteed. The experimental results confirm that under the control design, the bikebot can smoothly run crossing multiple step-like obstacles with height more than one third of the wheel radius. The comparison results demonstrate the superior performance than those under only the velocity and steering control without leg assistive impulsive actuation.

Keywords: Mobile Robots, Modeling and Design of Mechatonic Systems
Abstract: This paper proposes a stability indicator, “Static Mobility Indices (SMI)” for multi-agent robotic mobile systems. The SMI consists of several index values to describe the quasi-static stability of the whole system composed of homogeneous modules with mobility function such as wheels or legs.

Those index values are computed with the configuration characteristics, “radius” and “diameter” in terms of graph theory for instance, of the interconnected modules. The computation scheme of SMI is defined and formulated with respect to a three-axis coordinate system, “HoneyComb by Projection (HCP),” over a two-dimensional discrete space.

A series of experiments using a distributed legged mobile robot prototype was performed with the selected gaits and system configurations to examine the significance of the proposed indicator. The results of the experiments suggested that the obtained SMI appropriately described the status of the system stability that is profitable to improve the robustness and the adaptability of the system.

Keywords: Modeling and Design of Mechatonic Systems, Underwater robotics
Abstract: Soft actuators, such as dielectric elastomer (DE) and ionic polymer metal composite (IPMC), can generate large deformation under electric stimuli, which makes them promising in bio-mimetic robotic jellyfish applications. Although dielectric elastomers and ionic polymer metal composites (IPMCs) have been used in jellyfish-inspired robots respectively, there is no research which combines both of them to develop a jellyfish robot capable of two-dimensionally (2D) maneuvering capability. In our previous work, a jellyfish robot fabricated with DE can achieve effective locomotion. However, the robot requires a big bell to provide propulsion and can only move vertically. The jellyfish-inspired robot developed in this paper exhibits contracting muscle-like behaviour using a DE membrane to generate a periodic contraction on its eight fins to provide a thrust force which propels the robot to transit through underwater. The robot utilizes an IPMC to generate a bending moment which directs the heading angle of its swimming. This paper presents the design, modeling, experimental characterization of the 2D maneuverable jellyfish robot. The preliminary results show the jellyfish-inspired robot can swim underwater effectively in the vertical direction with different sinusoidal input signals, the direction of the robot can be changed by bending the IPMC. The average speed of the robot is about 4.8 mm/s when a sinusoidal signal with 5 kV amplitude and 1.4 Hz frequency is applied to the DE actuator. The maximum average heading angle change can reach to 3.02 degree by actuating IPMC without voltage applied to DE.

Keywords: Modeling and Design of Mechatonic Systems, Actuators in Mechatronic Systems
Abstract: The origami principle of folding planar sheets into functional three-dimensional devices promises a future with increased compactness and reconfigurability of biomedical robots. To inspire new origami-based biomedical robots, we highlight two categories of origami patterns that are applicable to building compact actuators, deployable stents, and minimally invasive surgery devices. Due to the requirements in system integration, biocompatibility, and payload capability, the conventional paper-based origamis are converted to physical robots by replacing the zero-thickness facets and zero-width folds with thick panels and flexible hinges, respectively. Therefore, the physical origami-based robots become inhomogeneous, making it more complex and challenging to analyze their mechanics. So far, no such model can analyze the mechanics of any physical origami robot. Herein, we propose a computational model that discretizes the continuous structures into truss and spring elements to fulfill this goal. Based on the model, we simulate the identified origami structures under various boundary conditions to investigate their application in biomedical scenarios. This work is expected to accelerate the design iteration of new functional biomedical origami robots.

Keywords: Modeling and Design of Mechatonic Systems, Actuators in Mechatronic Systems, Humanoid Robots
Abstract: Modern anthropomorphic prosthetic and robotic hands are not capable of meeting desired values in all the five criteria of size, weight, strength, speed and dexterity. A key reason for this is that existing actuators combined with transmissions with constant reduction ratio are unable to achieve performance that addresses all of the above criteria. Continuously Variable Transmissions (CVTs), which can produce any value within a range of reduction ratios, are a promising alternative that have the potential to provide the desired improvements in the five criteria when combined with Brushless DC (BLDC) motors. However, existing CVTs that can produce sufficient output torque and speed capabilities are not small and lightweight enough for practical use in an anthropomorphic prosthetic or robotic hand. In this paper, we present the design of a novel linkage-based active CVT that has the potential to overcome the limitations of existing CVTs and thereby achieve the desired values in all five criteria. The CVT is theoretically shown to be capable of producing the maximum torque output of the mean maximum male index finger metacarpophalangeal joint of 4.5 Nm across the joint’s approximate range of motion of 90°. With appropriate BLDCs and planetary gearboxes, this CVT is small and lightweight enough to meet all target specifications needed for use in anthropomorphic prosthetic and robotic hands that achieve strength comparable to the human hand as well as a suitably high speed and dexterity.

Keywords: Design Optimization in Mechatronics, Modeling and Design of Mechatonic Systems, Flexible Manipulators and Structures
Abstract: Abstract— This research paper will describe a new design of pulley guided wire-driven continuum robot arm structure. Continuum manipulators in the last couple of decades gaining more attention from the robotics community and a widening application horizon. However, wire-driven continuum manipulators have a problematic issue with the friction between the driving wire and robot rigid parts. Thus, such a drawback has a significant impact on the robot payload capacity and puts stress on the driving motors. Therefore, in this research paper, we proposed a new wire-driven continuum robot arm design that seeks to minimize friction effect to gain a smooth transmission of driving force.

Keywords: Modeling and Design of Mechatonic Systems, Robot Dynamics and Control, Rehabilitation Robots
Abstract: Stroke and spinal cord injury are becoming ever more prevalent in the United States. Recent research has shown that rehabilitation robots have the potential to positively impact the rehabilitation process by providing a platform for repetition-based movement therapy. To advance the field, future research directions for robotic rehabilitation are focused on advanced model-based control algorithms, and the combination of robotics with cutting-edge neuromodulation technologies. These approaches necessitate devices that not only feature kinematic and dynamic properties well-suited for model-based control, but also require devices that allow for easy placement of sensors and electrodes on the limb when inserted in the robot. With these design goals in mind, we present the MAHI Open Exoskeleton (MOE), a four degree of freedom robot with an open mechanical structure and simplified dynamics, combined with open-source software, that together lay the groundwork for advanced model-based control. The dynamic properties of each joint were characterized and compared against other recently developed rehabilitation robots. Open-source software was developed for the robot, which provides users with both low-level and application-level interfaces to implement a variety of control strategies. Dynamic equations were developed and implemented into a real-time simulation with a visualization, including a seamless interface to the developed software library. Impedance control and model predictive control were implemented and compared to the simulation, proving the value of the new designs.

Keywords: Virtual Reality and Human Interface, Modeling and Design of Mechatonic Systems, Sensors and Sensing Systems
Abstract: Uncoupled stability, the condition by which the user is not in contact with the haptic device, is arguably a stringent stability condition for haptic simulation systems. Uncoupled stability of haptic systems simulating linear mass-spring or viscoelastic virtual environments have been analyzed. In this paper, we analytically and experimentally evaluate uncoupled stability for simulating mass-damper-spring virtual environments when only position or when position and velocity are available. In addition, the effect of using a linear combination of position and velocity in deriving acceleration estimate is also studied. Experimental results in a one degree-of-freedom device showed that the highest stiffness values are obtained when the acceleration is equally derived from position and velocity. This work will shed light on the interaction of the three dynamic components for virtual environment rendering.

Keywords: Fixture and Grasping, Actuators in Mechatronic Systems, Modeling and Design of Mechatonic Systems
Abstract: Automation of the food handling process is not an easy problem. Grasping irregular-shaped objects with varying stiffness by robot hands is challenging, especially when the objects frequently change in a short time like Bento box packing lines. This paper proposes a robot finger with an automatic grasping completion mechanism by balancing the force from the objects and the restoring force of installed springs. This mechanism makes the gripper grasp various objects without any setting changes or complicated control algorithms for detecting objects’ shapes and positions even though handling objects may change frequently. We present of the design a robot hand with four proposed fingers and show the effectiveness of the hand by experiments thirteen kinds of food.

Keywords: Flexible Manipulators and Structures, Actuators in Mechatronic Systems, Robot Dynamics and Control
Abstract: This article combines nonlinear estimation and control methods for precise bending angle control in soft pneumatic actuators driven by a pressure source and single low-cost on/off solenoid valve. First, a complete model for the soft actuator is derived which includes both the motion and pressure dynamics. An Unscented Kalman filter (UKF) is used to estimate the velocity state and filter noisy measurements from a pressure sensor and an embedded resistive flex sensor. Then, a feedback linearization approach is used with pole placement and linear quadratic regulator (LQR) controllers for bending angle control. To compensate for model uncertainties and improve reference tracking, integral action is incorporated to both controllers. The closed-loop performance of the nonlinear estimation and control approach is experimentally evaluated with a soft pneumatic network actuator. The simulation and experimental results show the UKF provides accurate state estimation from noisy sensor measurements. The results demonstrate the effectiveness and robustness of the proposed observer-based nonlinear controllers for bending angle trajectory tracking.

Keywords: Neural Networks, Actuators, Flexible Manipulators and Structures
Abstract: This paper proposes to use a reservoir computation approach to model the non-linear dynamic behaviour of a novel electroactive soft actuator. The soft actuator is fabricated as a unimorph cantilever beam, in which the active layer is a mat of electrospun aligned nanofibers of the P(VDF-TrFE-CTFE) electrostrictive polymer integrated into a PDMS silicone matrix. The passive layer consists of kapton, while the soft electrodes are made of conductive carbon powder. The non-linear dynamic response of three specimens of the soft actuator, when stimulated by both DC and complex electric fields of varying frequency, is modeled by means of an echo state network. The proposed architecture is able to achieve a normalized root mean square error of 0.429 for the tip deflections and of 0.265 for the blocking forces, when compared to experimental data.

Keywords: Flexible Manipulators and Structures, Actuators in Mechatronic Systems, Modeling and Design of Mechatonic Systems
Abstract: Tendon driven continuum manipulators have advantages such as being able to deform around and through their environment, but this typically comes at the cost of reduced precision and load bearing capabilities. This paper presents a strong, compliant manipulator system using a rigid-link tendon driven manipulator controlled by a novel compliant drive mechanism. Numerical simulations show that this new manipulator can help to address the stiffness/compliance trade-off by exploiting a compliant drive and a stiff manipulator at the same time. A proof-of-principle real-world prototype demonstrates how the manipulator's compliance enables conformation to the shape of a soft and compressible object while its stiffness can support an object more than 15 times its own weight.

Keywords: Flexible Manipulators and Structures, Modeling and Design of Mechatonic Systems, Compuational Models and Methods
Abstract: This paper investigates the design of a compliant robotic finger with beam flexure joints to act as antagonists against a known actuator load. Stiffness based flexure design equations are derived from a static analysis of the equivalent joint torques caused by end effector loading and the desired end effector pose. The beam flexure stiffnesses are then compared against a finite element analysis for a range of flexure lengths. Two 3D printed compliant finger prototypes are designed using this methodology and tested experimentally. The results indicate that the simple beam flexures have appropriate stiffnesses and the finger mechanism is able to achieve the desired pose at loads near the designed-for linear actuator forces.

Keywords: Design Optimization in Mechatronics, Compuational Models and Methods, Flexible Manipulators and Structures
Abstract: The design, fabrication, and assembly of mesoscale robots can require expensive manufacturing equipment with multiple fabrication and assembly steps. This work presents the design of a monolithic transmission through the use of compliant mechanisms. The transmission mechanism is modelled using finite element analysis and is used to determine critical parameters for the conversion of small linear displacement to large rotational motion. Three monolithic prototypes are fabricated simultaneously via additive manufacturing, allowing for multiple designs to be investigated. The three transmission designs are then experimentally tested to determine the effectiveness and accuracy of the mechanism.

Keywords: Fixture and Grasping, Modeling and Design of Mechatonic Systems, Flexible Manipulators and Structures
Abstract: An underactuated mechanism, which has less input than the active degree of freedom, is often used for grippers such as robot hands. The use of this mechanism can contribute to the reduction of actuators, that is, the reduction in size and weight. However, many of the conventional underactuated mechanisms are equipped with ordinary springs at the joints and are driven by a motor and they tend to be bigger and bulkier. Therefore, we have developed a compact underactuated tendon-driven mechanism using a lightweight shape memory alloy (SMA) actuator and a superelastic SMA sheet with an elastic range that cannot be achieved by conventional metal leaf springs. It has a range of motion of over 140 deg and can lift a weight of 60 g by energizing the SMA actuator. Using the shape memory effect and the superelastic effect of SMA, it is possible to exert a large force while keeping the mechanism compact and lightweight with a length of 13 mm and a weight of 0.14 g.

Keywords: Humanoid Robots, Flexible Manipulators and Structures, Actuators
Abstract: This manuscript covers the development workflow for the prototype of a novel synergy driven soft robotic hand. By embedding the main human grasping synergy to individual soft silicone fingers, we are able to produce a functional under-actuated hand with fingers composed uniquely of soft materials and no electronic parts. An iterative design workflow based on specialized software for soft robots is employed, defining the target response for the individual fingers through a human-in-the-loop optimization process. The result is a low-cost compliant hand capable of replicating a programmed human grasp synergy using few mechanical components. The use of simulation reduced the need of produced prototypes, requiring only an initial characterization model, slashing waste of un-recyclable materials and production costs. Grasping quality was demonstrated with a variety of objects, with a success rate of 80% for the test items and grip strength comparable to similar compliant hands.

Keywords: Mobile Robots, Machine Vision, Robot Dynamics and Control
Abstract: This paper describes a stereo image-based visual servoing system for trajectory tracking by a non-holonomic robot without externally derived pose information nor a known visual map of the environment. It is called trajectory servoing. The critical component is a feature-based, indirect Simultaneous Localization And Mapping (SLAM) method to provide a pool of available features with estimated depth, so that they may be propagated forward in time to generate image feature trajectories for visual servoing. Short and long distance experiments show the benefits of trajectory servoing for navigating unknown areas without absolute positioning. Empirically, trajectory servoing has better trajectory tracking performance than pose-based feedback when both rely on the same underlying SLAM system.

Keywords: Machine Learning, Neural Networks, Image Processing
Abstract: Kendo is becoming popular worldwide nowadays. This research aims to develop a high-speed vision system to capture Kendo motions clearly and predict the intent attacking target of the trainee. We proposed a method to increase the joints detecting frequency by combining the joints tracking algorithm with OpenPose. For motion prediction, we applied LSTM method to achieve real-time predicting. In the end, we got 66% accuracy for basic Kendo moves prediction.

Keywords: Machine Vision, Fixture and Grasping, Service Robots
Abstract: Factory automation has been growing rapidly in production sites. On the other hand, it is mentioned that inspection work is still often done manually in factories where many dangers lurk. Although, safety can be ensured by substituting robot automation for many tasks that require dexterity such as opening and closing valves. Therefore, more accurate object recognition and pose estimation of the target object is required. In this paper, the task of automatically opening and closing a globe valve using a robot arm from different angles is considered. A system was developed and introduced to make a precise object recognition and pose estimation using an RGB-D camera attached to the robot arm. The system consists of three main sub-systems for object recognition, pose estimation, and arm-control. By using an RGB-D camera, it was possible to make color-based segmentation of the globe valve to detect the valve area using the Color Signature Of Histograms of Orientations (CSHOT) algorithm by detecting features. Then apply singular value decomposition to the precise position of the valve. By considering the angle of the robot arm to the globe valve, from 0° to 30° was taken as the front mode, and from 30° to 90° was taken as the side mode. Detecting the objects from the side is novel in this research area. RANdom SAmple Consensus (RANSAC) algorithm was used to verify the pose estimation of the front mode of the valve. Centroid connection and Pose Integration Hough-Voting methods were used to verify the pose estimation of the side mode. From the results, it can be concluded that the system is accurately detecting object recognition and pose estimation of a globe valve from the side mode to the front mode.

Keywords: Machine Vision, Image Processing, Neural Networks
Abstract: Smart digital advertising requires face detection as the initial stage to recognize the person attribute by localizing human facial areas. This technology tends to operate with CPU-based systems. The deep Convolutional Neural Network approach has demonstrated excellent accuracy for face detection work. However, this architecture involves heavy computations and parameters because it uses many filter operations. It causes deep architecture to slow down the detector speed. Moreover, a practical application entails using a detector that can operate in real-time. The recent CPU-based face detectors operate slowly in an integrated system. This study proposes a faster face detector to robustly predict the human faces area using efficient architecture. The architecture consists of a light backbone to discriminate distinctive features and a four detection module to predict multiple faces. In order to bridge the three prediction layers, it implements a high-level transition module with a cheap operation. It also offers a new light attentive block to efficiently highlight typical facial features at each detection module. As a result, this detector achieves excellent performance and outperforms other low computing detectors. The proposed detector can fast operate at 112 frames per second on a Core I5 CPU and at 11 frames per second on a Lattepanda device, faster than other competitors.

Keywords: Image Processing
Abstract: Adequate visualization of the retina is an important factor in assisting ophthalmologists in diagnosing various eye conditions and diseases. Retinal images pertaining to the posterior surface of the eye are captured using specialized devices called fundus cameras. Shortages of such devices are prevalent in areas without sufficient funding due to their expensive costs and, therefore, introduce a barrier to proper eye care. The advent of portable, low-cost smartphone-based fundus cameras has presented a promising solution to address this issue. However, images captured from recently developed smartphone-based portable fundus cameras and ophthalmoscopes have a lower field-of-view and quality compared to expensive tabletop fundus cameras. This paper proposes a method to enhance the diagnostic capacity of such smartphone-based fundus cameras and ophthalmoscopes by stitching multiple retinal images into a mosaic with a greater field-of-view of the retina. The key steps include feature detection, feature correspondence, image warping, and image blending. The method is implemented as a smartphone app intended to be used with low-cost portable fundus cameras and ophthalmoscopes. It successfully generates high-quality mosaics that retain the landmark structures and other notable clinically relevant features present in the original images while maintaining low profile image boundaries. The proposed method is validated by comparison with a standard stitching method on a success rate metric corresponding to the success and quality of image warping and blending. The dataset used for the validation includes 16 retinal fundus mosaicking image sets captured by smartphone-based retinal imaging devices. The comparison results indicate the proposed method is much more effective in stitching retinal fundus images than standard stitching techniques.

Keywords: Identification and Estimation in Mechatronics, Machine Vision, Image Processing
Abstract: In visual servoing (VS) tasks, the target object with complex shape brings the challenge to extract effective visual information used for robotic control. Appropriate image features describing the overall contour of the object are critical to implement such tasks. In this work, we propose a B-spline features-based contour VS method. The quasi-uniform B-spline curve is employed to construct image features by control points from contours. With the good shape description capability and concise mathematical expression, B-spline features handle the object with complex shape in a visually intuitive and efficient way. Moreover, to improve the VS system’s robustness to the dynamic environment and temporary occlusion, a real-time estimate framework composed of B-spline features estimator (BFE) and B-spline features predictor (BFP) is proposed. BFE achieves the optimal estimation of current image features and corresponding depth based on adaptive extended Kalman Filter (AEKF), where control points are regarded as observations. BFP mainly tackles the problem of object occlusion, where the perspective projection invariance of B-spline features and Lie algebra model are introduced to predict the occluded features through nonlinear optimization. The effectiveness of the proposed method is validated on VS tasks for objects with complex shapes by simulations and experiments.

Keywords: Biomechatronics, Underwater robotics
Abstract: Dynamic modeling has been capturing attention for its fundamentality in precise locomotion analyses and control of underwater robots. In this article, we focused on the three-dimensional dynamic modeling and the experimental validation of multiple motion patterns of underwater robots in large-scale parameter space. A three-dimensional dynamic model of an active-tail-actuated robotic fish with a barycentre regulating mechanism was first built by combining Newton's second law for linear motion and Euler's equation for angular motion. Then, the model parameters were determined by three-dimensional computer-aided design (CAD) software SolidWorks, HyperFlow-based computational fluid dynamics (CFD) simulation, and grey-box model estimation method. Finally, both kinematic experiments with a prototype and numerical simulations were applied to validate the dynamic model's accuracy mutually. Based on the dynamic model, multiple three-dimensional motions, including rectilinear motion, turning motion, surfacing motion, and spiral motion, were analyzed. The experimental and simulation results demonstrated the effectiveness of the proposed model in evaluating the trajectory, attitude, and motion parameters such as the velocity, turning radius, turning angular velocity of the robotic fish.

Keywords: Human -Machine Interfaces, Fuzzy Logic
Abstract: In human-robot interaction, accurate estimation of human intention can improve the interaction and enhance the stability of the interaction process. However, in the past research process, the research on human intention was relatively inaccurate or performed in a single point-to-point (PTP) task, which cannot meet most application scenarios. This paper proposes a fuzzy variable admittance system based on human-robot interaction. In this method, not only the direct intention is represented by the product of the Cartesian velocity and the interaction force, but the angular constraint with the axis represents the indirect intent, and fuzzy rules integrate the two purposes. Finally, to verify the performance of this method, PTP and trajectory tracking experiments are performed using a FRANKA PANDA robot.

Keywords: Biomechatronics, Modeling and Design of Mechatonic Systems, Actuators in Mechatronic Systems
Abstract: Swinging-generating high impact by maximizing the speed of motion-is essential in sports activities. However, designing prosthetic hands suitable for swinging is still a challenge. Herein, we propose a swing-dedicated prosthetic hand that adopts the anatomical features of humans for an efficient swing. Our design, inspired by the joint arthrokinematics and tendon routing of fingers, enables a diagonal power squeeze grip robust to impact while increasing the reach of the clubhead. The swing speed is further increased by radio/ulnar deviation of the wrist and its nonlinear stiffness change achieved by the passive clutch mechanism. We evaluated our prosthetic hand through golf and found that it sustained high impact with diagonal grip, and clubhead speed increased by 19% at 90 rpm, with the radio/ulnar deviation nonlinearly correlated with arm angle. Our prosthetic hand design will contribute to improving amputees quality of life by allowing them to participate in sports activities.

Keywords: Biomechatronics, Human -Machine Interfaces, Modeling and Design of Mechatonic Systems
Abstract: This study presents a method for trip detection using inertial sensors, and a controller for balance recovery in the sagittal plane after trip using exoskeleton devices. Trip detection was performed using a computed root mean square of the lower back angular acceleration in a sagittal plane and a fixed threshold resulting in mean detection times of 51 ± 16 ms. The foot placement estimator-based controller was designed for bilateral, pneumatic, constant-torque output exoskeleton devices to provide assistance to reposition the foot to prevent a fall. At the moment of detection, foot placement estimation (FPE) for recovery is calculated, and a human-inspired response (lift/lower) is chosen for the perturbed leg. A pendulum model is used to determine the necessary torque injections to achieve FPE. Sensitivity analysis of this pendulum model is performed to influence the outputs chosen. The controller is validated through simulation results of the applied strategies showing successful fall prevention.

Keywords: Identification and Estimation in Mechatronics, Sensor Integration, Data Fusion, Biomechatronics
Abstract: This paper presents an Error-State Kalman Filter (ESKF) for state estimation in a 2-DOF robotic prosthetic ankle. The filter estimates the ankle angle in inversion-eversion (IE), external-internal (EI), and dorsiflexion-plantarflexion (DP), using measurements from two low-cost magnetic, angular rate, and gravity sensor modules (MARGs), also known as 9-axis Inertial Measurement Units (IMUs). To this end, we transformed raw MARG measurements into body frames and modeled the states and constraints of the 2-DOF robotic prosthesis in an Error State Kalman Filter (ESKF). Experimental tests showed that the proposed ESKF provided better results than the Madgwick filter, a commonly used attitude estimator. The proposed filter is developed for ankle prostheses requiring direct angle measurement and can be expanded to an online evaluation of ankle angle in humans.

Keywords: Identification and Estimation in Mechatronics, Rehabilitation Robots, Medical Robotics/Mechatronics
Abstract: Understanding human motion is of critical importance for health monitoring and control of assistive robots, yet many human kinematic variables cannot be directly or accurately measured by wearable sensors. In recent years, invariant extended Kalman filtering (InEKF) has shown a great potential in nonlinear state estimation, but its applications to human poses new challenges, including imperfect placement of wearable sensors and inaccurate measurement models. To address these challenges, this paper proposes an augmented InEKF design which considers the misalignment of the inertial sensor at the trunk as part of the states and preserves the group affine property for the process model. Personalized lower-extremity forward kinematic models are built and employed as the measurement model for the augmented InEKF. Observability analysis for the new InEKF design is presented. The filter is evaluated with three subjects in squatting, rolling-foot walking, and ladder-climbing motions. Experimental results validate the superior performance of the proposed InEKF over the state-of-the-art InEKF. Improved accuracy and faster convergence in estimating the velocity and orientation of human, in all three motions, are achieved despite the significant initial estimation errors and the uncertainties associated with the forward kinematic measurement model.

Keywords: Identification and Estimation in Mechatronics, Mobile Robots, Automotive Systems
Abstract: In this paper, the extrinsic parameter estimation of a camera mounted on an intelligent vehicle is addressed. The trifocal tensor is utilized to construct vision dynamics which relates image coordinates, velocity signals, and extrinsic parameters. Artificial visual patterns such as chessboards and planar reference objects used in homography based methods are no longer required. An auxiliary tensor decouples the rotational extrinsic parameters from the translational ones. A key frame strategy is adopted to deal with the field of view constraint and an unknown distance is eliminated from the vision dynamics to counter the scale change caused by key frame switching. The Lyapunov method is used to design nonlinear observers, which estimate the extrinsic parameters at each time step based all collected valid historical data.

Keywords: Mobile Robots, Machine Vision, Sensor Integration, Data Fusion
Abstract: An occupancy grid map considering only geometric information is often used for autonomous mobile robots. In the outdoors, there are various areas we do not want autonomous robots to enter, such as grass areas. These areas are not reflected in the occupancy grid map because geometric information is not sufficient to distinguish these areas. In this work, we attempt to add semantic information of the ground surface to a prior occupancy grid map for recognizing traversable regions. We create a semantically segmented bird's eye view (BEV) using semantic segmentation and inverse perspective mapping (IPM) and then a apply one-sided truncated Gaussian filter and binary Bayes filter to deal with the uncertainty of semantic segmentation and IPM. We tested our method on an approximately 1-km route at the University of Tsukuba and found that the recognition accuracy is highest if we apply these two filters together.

Keywords: Mobile Robots, Robot Dynamics and Control, Planning and Navigation
Abstract: Multi-robot system has great potential than one single robot due to the flexibility. By adjusting formations of the system, complex tasks can be accomplished. However, formation control may conflict with obstacle avoidance in clutter environments. Therefore, we propose a distributed motion controller for the multi-mobile robot system to balance formation control and obstacle avoidance. Caging behavior is designed for the system to achieve desired formations. For obstacle avoidance, the artificial potential filed algorithm is improved, in which the force caused by formation control is considered. Thus, conflicted forces can be avoided and tangential directional forces can be enrolled. Simulation results demonstrate the system can keep desired formations in multi-obstacle environments. Moreover, the proposed algorithms is distributed, and the priority between the two tasks can be adjusted.

Keywords: Mobile Robots, Machine Vision, Control Application in Mechatronics
Abstract: Autonomous Mobile Manipulators (AMMs) transport between locations and execute manipulation tasks such as object handling. As the manipulator resides on the mobile platform, the performance of manipulation is often limited by the position accuracy of the mobile platform. The robot can often be poorly positioned for dynamically balanced robots or those without rigid tracks. As a result, there exists a high uncertainty on the relative position between the target and AMM's parking position. This article contemplated the above scenario and proposed a cascaded visual object localization architecture to achieve vast-area search and progressive localization precision. We demonstrated a three-stage cascaded design utilizing the Convolutional Neural Network for illumination-robust localization in the final stages. Our field tests showed that given approximately 60×20 cm² position variations of the target object, the proposed system located the target object and performed pick-and-place with a success rate of 95%. The method was demonstrated on a two-wheeled mobile robot; however, it can also be applied to other mobile platforms that lack positioning accuracy due to rugged terrains or other reasons. The entire localization module is light and operates on a Raspberry Pi 4.

Keywords: Mobile Robots, Artificial Intelligence in Mechatronics, Wed-based Control of Robotic and Automation Systems
Abstract: In this article, we proposed an autonomous mobile robot (AMR) capable of conducting multiple deep learning inferences by a computing network as a Cyber-Physical System (CPS). In view of industry 4.0, a five-layered CPS architecture including Component, Intelligence, Cyber, Configuration, and Deployment was devised. Mechatronic control of the two-wheeled self-balancing AMR presents the first challenge to embedded computing, whereas multiple deep learning modules for visual navigation and localization further escalate the stake. To meet the challenge of real-time and affordable computing, we constructed an embedded cluster based on Raspberry Pi4s and Intel NCS2 for simultaneous intelligent inference and data exchange. We tested the performances of three networking methods, Network File System (NFS), Socket Server with Multiple Clients (SSMC), and ROS TOPIC (ROS-T), and reported the comparison. We also conducted a series of field tests on autonomous corridor navigation, localization, object delivery, and object pick-and-place to verify the performance of the self-balancing AMR. The results showed high success rates with sufficient spatial precision, stability, and robust loop closure. The web-based console realizes the functions of the digital twin and provides connectivity to higher-level management systems.

Keywords: Mobile Robots, Vehicle Technology, Artificial Intelligence in Mechatronics
Abstract: This article tackled some adverse influences on the localization problem in challenging indoor corridor/aisle environments. We proposed Automatic Topological Localization Network (ATopNet), suitable for autonomous mobile robots (AMRs) based on low-resolution RGB vision. We provided experimental evidence that ATopNet enables AMRs to conduct autonomous routing along indoor corridors with sufficient spatial precision and accuracy that allows AMRs to conduct the following manipulation works. To effectively enhance the performance of the deep Convolutional Neural Network (ConvNet) for visual localization of AMR, we developed automatic training augmentation schemes for the pose, illumination, speed, and occlusion enhancements. We derived the kinematic theories correlating view-frames and observer’s poses, based on which thousands of pose-explicit images can be automatically generated and annotated for ConvNet training. In addition, the strategy of recruiting a conditional Generative Adversarial Network for illumination image generation was proposed. The impact of vehicle speed on localization performance when trained on static images was also tested and discussed. A series of experiments were conducted to verify the efficacy of the proposed methods. We demonstrated ATopNet performing real-time robot localization with a Raspberry Pi4+NCS2. Another prominent feature of ATopNet is that all the procedures are fully automated and require minimal manual operation.

Keywords: Sensors and Sensing Systems
Abstract: Inhalers are commonly used to treat asthma and chronic obstructive pulmonary disease. Regular and correct usage of inhaler is necessary for effective use. However, approximately 70% of patients do not use their inhalers as directed. This is due to the lack of understanding about the medication and misunderstanding of directions. Assessment of patients' inhaler techniques are usually conducted in person. However, doctors or pharmacists have no objective information regarding how patients use their inhalers at home. Therefore, monitoring daily inhaler use is necessary for precise medical treatment. This paper proposes an inhalation monitoring device using an inertial measurement unit (IMU). IMU is used to measure a patient's inhalation motion. Incorrect inhalation usage can be determined by comparing the measurement data with the correct usage. The experimental results show the utility of the proposed device.

Keywords: Intelligent Sensors, Neural Networks, Modeling and Design of Mechatonic Systems
Abstract: Measuring the ripeness of fruits is one of the key challenges to enable optimal and just-in-time strategies across the fruit supply chain. In this paper, we study the performance of a tactile sensor to estimate the ground truth of the stiffness of fruits, with kiwifruit as a case study. Our sensor configuration is based on a three-beam cantilever arrangement with piezoresistive elements, enabling the stable acquisition of sensor readings over independent trials. Our estimation scheme is based on the compact feed-forward neural networks allowing us to find effective nonlinear relationships between instantaneous sensor readings and the ground truth of stiffness of fruits. Our experiments using several kiwifruit specimens show the competitive performance frontiers of stiffness approximation using 25 compact feed-forward neural networks, converging to MSE loss at 1e-5 across training-validation-testing in most of the cases, and the utmost predictive performance of a pyramidal class of feed-forward architectures. Our results pinpoint the potential to realize robust fruit ripeness measurement with intelligent tactile sensors.

Keywords: Sensors and Sensing Systems, Biomechatronics, Human -Machine Interfaces
Abstract: Evaluating the influence of human-centered devices, such as exoskeletons on internal body forces, is important in order to make safe and effective designs. In-vivo measurements are typically difficult to obtain, however, instrumented artificial limbs that approximate human limbs may be appropriate test-beds. In this paper, a pneumatic sensing chamber (PSC) is designed and characterized for sensing biomechanical joint forces in an artificial tibiofemoral limb. The flexible soft PSC was 3D printed using the soft material Ninjaflex (NinjaTek, USA) and extensive characterization tests (drift, ramp, cyclic) were conducted on a universal testing machine. A specific interface was created to tightly fit the PSC to the non-uniform bone surfaces of the femur and tibia bones, and additional testing conducted with the PSC embedded in the artificial lower leg setup. In general, we found that PSC pressure output followed the UTM forces closely with a linear relationship for the static tests (RMS error of 0.82 kPa/1.68N between experimental data and linear model) with little drift over extended hold tests. Cyclic tests showed the PSC could follow the dynamic force signal closely, although some hysteresis was found between the loading and unloading cycles. Our results show that, overall, a reliable sensor can be designed, characterized and calibrated in the application of identifying biomechanical joint pressures and forces.

Keywords: Sensors and Sensing Systems, Intelligent Sensors
Abstract: This paper describes a prototype of a novel Permanent Magnetic Elastomer (PME) sheet based skin sensor for robotic applications. Its working principle is to use a Hall effect transducer to measure the change of magnetic field. PME is a polymer that has Neodymium particles distributed inside it, after strong magnetization for anisotropy, the PME acquires strong remanent magnetization that can be comparable to that of a permanent magnet, in this work, we made improvement of the strength of the magnetic field of PME, so it achieved magnetic strength as high as 25 mT when there is no deformation. When external forces apply on the sensor, the deformation of PME causes a change in the magnetic field due to the change in the alignment of the magnetic particles. Compared with other soft magnetic sensors that employ similar technology, we implemented linear regression method to simplify the calibration, so we focus on the point right above the magnetometer. An MLX90393 chip is installed at the bottom of the PME as the magnetometer. Experimental results show that it can measure forces from 0.01--10 N. Calibration is confirmed effective even for shear directions when the surface of PME is less than 15 x 15 mm.

Keywords: Sensors and Sensing Systems, Novel Industry Applications of Mechatroinics, Intelligent Sensors
Abstract: Construction machinery automation has drawn more and more research interests in recent years. To realize the automatic operation of a hydraulic excavator, a key technology is the effective pose estimation of its multiple actuators. In this work, a new measurement method of real-time pose estimation is developed by using machine vision and artificial neural network (ANN) to overcome the limitations of existing methods. A high-resolution camera is installed on the excavator cab to capture the pose image with a constant view angle even when the excavator swings. Images are processed in hue, saturation and value space to extract the centroid pixel coordinate (CPC) of the markers fixed on the actuator joints. Taking the centroids as featured points, the mapping relations between the pixel coordinates and the joint angles are derived by training the ANN. A number of experiments are implemented under various conditions and the results show that the proposed method has good real-time performance and robustness. This work can provide basis for the feedback control of the actuators in the excavator.

Keywords: Sensor Integration, Data Fusion, Intelligent Process Automation, Mobile Robots
Abstract: This paper discusses the creation of a map of multi- floor buildings using elevators in autonomous and manual driving modes. Two disadvantages of existing Simultaneous Localization And Mapping (SLAM) algorithms are inability to detect elevation change and drift inside reflective environments such as many modern elevators. Therefore, we integrate the LeGO-LOAM SLAM algorithm with air pressure data collected by a barometric pressure sensor to create a map of a multi-story building in real-time without losing track of robot’s movement inside an elevator. To achieve this, we developed an elevator detection module to detect elevators using a depth camera and locate them in floor maps. In autonomous driving mode, after exploring and mapping one floor, the robot autonomously navigates to the detected elevator, takes it to another floor, and starts mapping the new floor without losing track of the robot’s position despite sudden changes in the environment during this process. The manual driving mode is subsequently added to evaluate the performance of the system and the accuracy of the generated map. The experimental results show that the proposed algorithm is capable of mapping multiple floors autonomously and manually with minimal drift.

Keywords: Medical Robotics/Mechatronics, Flexible Manipulators and Structures, Modeling and Design of Mechatonic Systems
Abstract: Flexible endoscopes have been widely adopted in clinical practices, which enable surgeons to observe the inside of the body through natural orifices. However, observing lesions inside the narrow space of the human sinus with existing flexible endoscopes is still challenging due to their large occupied motion space and limited dexterity. In this paper, we propose a dexterous hybrid-structure hand-held robotic endoscope (HHRE) for sinus inspection, which composes of a 2 degrees-of-freedom (DOFs) proximal flexible section and a 2-DOF distal articulated wrist structure. Compared with conventional hand-held flexible sinus endoscopes, the proposed HHRE has a larger visual coverage inside the maxillary sinus and smaller occupied motion space due to the 2-DOF wrist structure. Our specially designed flexible section and distal articulated wrist structure offer enhanced torsional stiffness to minimize unpredictable view rotations during the diagnostic inspection. The HHRE can be used as a standalone hand-held device or installed on a robotic platform with gravitational force compensation to minimize surgeon’s fatigue and improve the usability of the endoscope. Various benchtop experiments and a user study have been performed to validate the effectiveness, reachability, and visual coverage of the HHRE in the maxillary sinus. The user study results show that the proposed platform can potentially improve the usability of the endoscope and reduce surgeons’ workload in performing inspections of the maxillary sinus.

Keywords: Medical Robotics/Mechatronics, Biomechatronics, Identification and Estimation in Mechatronics
Abstract: In low-dose-rate permanent-seed (LDR-PS) implant brachytherapy, it is crucial to predict the movement of internal target points (planned radioactive seed locations) under the effect of external forces. Accurate prediction of the target locations is critical for precise seed implantation, as inaccurate seed implantation diminishes the effectiveness of radiotherapy. Therefore, developing a model to simulate tissue dynamics is necessary. All physics-based tissue models have model-reality mismatches due to unmodeled dynamics, a problem which should be addressed. In this work, we propose the KF-ADMM method as a solution, which compensates for a portion of unmodelled dynamic terms existing in the alternating direction method of multipliers (ADMM)-based projective dynamics (PD) tissue simulator through Kalman filtering. This method provides accurate predictions of the location of inner tissue points with an error of around 0.8 mm. Experiments on a breast tissue phantom are performed to evaluate the efficacy of the proposed approach. According to the results, the accuracy of tissue deformation is enhanced by 52 % on average, and the convergence rate is accelerated compared to an ADMM-based PD tissue simulator.

Keywords: Rehabilitation Robots, Actuators in Mechatronic Systems
Abstract: Work-related back pain has become a global challenge. In addition, the number of people with back pain is increasing, and this has become a worldwide challenge. Therefore, a variety of lower back assist devices have been

developed to reduce the burden on the lower back during work. However, if they are too heavy because of energy sources and actuators, or take too much time to wear, the overall workload cannot be reduced. In this study, we develop a pneumatic passive lower back assist device with an assistive force adjustment function that is user-friendly, lightweight, and easy to wear for various tasks. This study describes the outline of the developed lower back assist device, estimation of the assist force, and a simple lifting experiment to assess the lower back assist device we developed.

Keywords: Rehabilitation Robots, Medical Robotics/Mechatronics, Control Application in Mechatronics
Abstract: Ultrasound-based state assessment of the human muscle during rehabilitation and its integration into a hybrid exoskeleton comprising an FES system and a powered orthosis are emerging research areas. This paper presents results from the first experimental demonstration of a hybrid knee exoskeleton that uses ultrasound-derived muscle state feedback to coordinate electrical motors and functional electrical stimulation (FES). A significant contribution of the paper is to integrate a real-time ultrasound image acquisition and processing framework into a recently derived switching-based feedback control of the hybrid knee exoskeleton. As a result, the contractility response of the quadriceps muscle to the FES input can be monitored in vivo in real-time and estimate FES-induced muscle fatigue changes in the muscle. The switched controller's decision-making process can then use the estimated muscle fatigue to compensate or replace the FES-stimulated muscle power with an electrical motor, thus avoiding extensive stimulation of the fatigued muscle. The experimental results suggest a potential application in the rehabilitation of neurological disorders like spinal cord injuries and stroke.

Keywords: Rehabilitation Robots, Human -Machine Interfaces, Design Optimization in Mechatronics
Abstract: Lower-limb exoskeletons with proper assistance can improve the mobility of people with leg impairments. Prior studies have shown that tethered ankle exoskeletons can reduce the metabolic cost during walking and running, but few mobile ankle exoskeletons can achieve similar performance. The additional weight of the actuation system and compromised control performance decrease the assistance efficiency. Here we developed a mobile exoskeleton that can provide plantarflexion torque assistance at the ankle. The actuation system is switchable between a single- and a dual-motor ones to adapt to different users and tasks. The single-motor system can achieve lighter weight and better transparency while the dual-motor one can provide higher assistance. The single-motor exoskeleton system has a 0.95 Kg weight worn on the leg with another 1.15 Kg power belt mounted on the waist. The dual-motor system can provide maximal 100 N·m assistive torque and reduce average 32.5% of soleus muscle activity in preliminary tests. A real-time controller was implemented in an integrated embedded system and achieved high torque control performance. The closed-loop bandwidths of the single- and the dual-motor systems were 13.0 Hz and 15.3 Hz, respectively. The average root-mean-squared torque tracking error was 2.6% of peak torque during walking and running. The mobile ankle exoskeleton can assist for 3 hours of continuous active walking or running with an integrated 4500 mAh battery, making it promising in investigating assistance strategies off the treadmill.

Keywords: Image Processing, Machine Vision
Abstract: Feature extraction and matching are the basis of many computer vision problems, such as image retrieval, place recognition and visual odometry. In this paper, we present a novel RGB-D feature with Texture and Geometric information (TG). It consists of a keypoint detector and a feature descriptor, which is accurate, efficient and robust to scene variance. In the keypoint detection, we build a simplified Gaussian image pyramid to extract the texture feature. Meanwhile, the gradient of the point cloud is superimposed as the geometric feature. In the feature description, the texture information and spatial information are encoded in relative order to build a discriminative descriptor. We also construct a novel RGB-D benchmark dataset for RGB-D detector and descriptor evaluation under single variation. Comprehensive experiments are carried out to prove the superior performance of the proposed feature compared with state-of-the-art algorithms. The experimental results also demonstrate that our TEG can achieve better performance especially on accuracy and the computational efficiency, making it more suitable for the real-time applications, e.g. visual odometry.

Keywords: Compuational Models and Methods, Image Processing, Mobile Robots
Abstract: Simultaneous localization and mapping (SLAM) is a widely used technique in autonomous mobile robots. This study deals with the estimation of localizability, which indicates the reliability of localization at each location on the occupied grid maps created by SLAM. There are several approaches to estimate localizability, this paper proposes a method using local map correlation. Our method represents the localizability using a covariance matrix of a Gaussian distribution, not just a scalar value. The simulation experiment results showed that the uncertainty of localizability increased at locations where degeneration is likely to occur, suggesting that localizability could be estimated appropriately.

Keywords: Mobile Robots, Sensor Integration, Data Fusion
Abstract: Advanced tasks such as planning and scene interaction for autonomous robots require a detailed instance-level semantic map of the environment. To this end, this paper proposes a new volumetric instance-level semantic mapping approach, in which BlendMask is introduced as the instance segmentation algorithm for RGB images. As a result, improvements in the quality and speed of the instance segmentation are observed. Specifically, the geometric segmentation for the depth image and the instance segmentation results are fused together to construct the geometrically and semantically unified instances. Then, cross-frame instances are tracked and matched through data association. Based on the above, a global instance-level semantic map is constructed. Comparative experiments on public datasets are conducted to show that the proposed instance-level semantic mapping approach can effectively improve the instance segmentation effect and the quality of the constructed instance-level semantic map.

Keywords: Service Robots, Machine Learning, Machine Vision
Abstract: In many social scenarios,human emotion governs external behavior, and behavior reflects intention.If a social robot can recognize users’ interaction intention through observable behaviors, it can respond in a personalized way, thus exhibiting "natural" behavior. In this paper, we propose a new method for social human-robot interaction, where the robot obtains the facial expression and body action of the people only from RGB videos to recognize whether humans intend to interact with it or not.Through feature extraction and fusion, a computational model of emotion is established to further identify the interlocutors'intention. We create a real unconstrained dataset to evaluate our method.The result shows that our method which combines facial expression and body action features, is more efficient than single-cue based methods.

Keywords: Human -Machine Interfaces, Tele-operation, Machine Vision
Abstract: Remotely operating a robot in Augmented Reality (AR) is a challenging problem due to the limited information of the environment around the robot. The current AR teleoperation interface lacks the collision checking between the virtual robot model and the environment. This work aims to overcome that problem by using depth mesh generation to reconstruct the environment from a single pair of RGB and Depth images. By presenting the generated mesh with the virtual manipulator model in AR, we introduce three collision-aware features, i.e., collision checking, AR guidance, and ray casting distance calculation, to support the operator in the manipulation task. The reconstruction can be done instantly on the smartphone, allowing the system to be used on mobile AR applications. We evaluate our system with the pick-and-place task. The accuracy of the reconstruction is enough for the user to succeed in the operation. In addition, the collision-aware features reduce the task completion time, lower workload, and enhance the system's usability.

Keywords: Control Application in Mechatronics, Modeling and Design of Mechatonic Systems
Abstract: Industrial robots are widely used in many applications with structured and deterministic environments. However, the contemporary need requires industrial robots to intelligently operate in dynamic environments. It is challenging to design a safe and efficient robotic system with industrial robots in a dynamic environment for several reasons. First, most industrial robots require the input to have specific formats, which takes additional efforts to convert from task-level user commands. Second, existing robot drivers do not support overwriting ongoing tasks in real-time, which hinders the robot from responding to the dynamic environment. Third, most industrial robots only expose motion-level control, making it challenging to enforce dynamic constraints during trajectory tracking. To resolve the above challenges, this paper presents a jerk-bounded position control driver (JPC) for industrial robots. JPC provides a unified interface for tracking complex trajectories and is able to enforce dynamic constraints using motion-level control, without accessing servo-level control. Most importantly, JPC enables real-time trajectory modification. Users can overwrite the ongoing task with a new one without violating dynamic constraints. The proposed JPC is implemented and tested on the FANUC LR Mate 200id/7L robot with both artificially generated data and an interactive robot handover task. Experiments show that the proposed JPC can track complex trajectories accurately within dynamic limits and seamlessly switch to new trajectory references before the ongoing task ends.

Keywords: Robot Dynamics and Control, Control Application in Mechatronics, Machine Learning
Abstract: Sample-based trajectory optimisers are a promising tool for the control of robotics with non-differentiable dynamics and cost functions. Contemporary approaches derive from a restricted subclass of stochastic optimal control where the optimal policy can be expressed in terms of an expectation over stochastic paths. By estimating the expectation with Monte Carlo sampling and reinterpreting the process as exploration noise, a stochastic search algorithm is obtained tailored to (deterministic) trajectory optimisation. For the purpose of future algorithmic development, it is essential to properly understand the underlying theoretical foundations that allow for a principled derivation of such methods. In this paper we make a connection between entropy regularisation in optimisation and deterministic optimal control. We then show that the optimal policy is given by a belief function rather than a deterministic function. The policy belief is governed by a Bayesian-type update where the likelihood can be expressed in terms of a conditional expectation over paths induced by a prior policy. Our theoretical investigation firmly roots sample based trajectory optimisation in the larger family of control as inference. It allows us to justify a number of heuristics that are common in the literature and motivate a number of new improvements that benefit convergence.

Keywords: Actuators, Actuators in Mechatronic Systems, Motion Vibration and Noise Control
Abstract: The rapid development of the soft robotics has led to the emergence of many novel soft actuators, among them, dielectric elastomer actuators (DEAs) have demonstrated the clear advantages of large actuation strains, fast response speeds and high energy densities. When actuated at its resonance, the DEA can exhibit a significantly boost in its output stroke/power and energy efficiency, thus enabling dynamic soft robotic applications with high power density demands. As a result, increasing researches have been dedicated to improve the outputs of the DEAs and to expand their functionalities. In this work, we present a bistable DEA (BDEA) that exploits the inherent bistability to broaden the resonant bandwidth, increase the output and expand the actuation programmability. Through extensive experiments, we show that proposed BDEA can exhibit a significantly broader resonant bandwidth and potentially higher power output and energy efficiency than a monostable counterpart that is widely adopted in the literature. The developed BDEA also exhibits three oscillation modes and a control strategy is proposed to achieve switching between these oscillation modes. Robust programmable actuation is demonstrated in experiments in this work. The outcomes of this work can offer guidelines for developing next-generation highly programmable DEAs and multi-functional soft robotic systems.

Keywords: Motion Vibration and Noise Control, Control Application in Mechatronics, Mechatronics in Manufacturing Processes
Abstract: In this paper, we present the sensitivity function shaping method and its experimental validations for the non-collocated active damping system in a ram-type milling machine. While the non-collocated configuration is often required for the practical reasons in the application of the active damper to the machine structure, such configuration can be challenging to obtain the high-gain control under stable conditions. We analyze the frequency response of the non-collocated active damping system where the opposite-phase-mode dynamics limit the achievable control gain. Utilizing the understanding on the non-collocated plant, we shape the sensitivity function to effectively reduce the closed-loop compliance of the dominant mode dynamics under varying ram postures. We experimentally validate the resultant active damping performance, where the peak compliance of the dominant mode is reduced by a maximum of 61%. The chatter suppression performance due to the enhancement in the dynamic stiffness is also experimentally validated in rough-machining tests using our active damper and the proposed control method.

Keywords: Planning and Navigation, Wed-based Control of Robotic and Automation Systems, Mobile Robots
Abstract: People with severe disabilities are limited in their options for independent mobility. Amyotrophic Lateral Sclerosis (ALS), Spinal Muscular Atrophy (SMA), Spinal Cord Injury (SCI), cerebral palsy, multiple sclerosis and muscular dystrophy etc., seriously affect the extremities and make the operation of commercially available wheelchairs and Powered Wheelchairs (PWs) through a joystick User Interface (UI) difficult or near impossible. Symptoms affecting individuals can vary in severity and consistency, reducing the effectiveness of singular control modalities. This paper presents Smart Wheelchair Assistant for Autonomous Navigation (SWAAN), an add-on device for existing PW platforms that provides an alternative set of Human-Machine Interface (HMI) modalities for user selection. SWAAN offers users control through either speech recognition, teleoperation or autonomous navigation, beneficial to people with different levels of disabilities. This addresses the constrained application of singular control modalities and minimises the physical and cognitive effort required for Activities of Daily Life (ADL). SWAAN demonstrates successful autonomous navigation through outdoor urban environments at The University of Sydney (USYD).

Keywords: Control Application in Mechatronics
Abstract: According to the well-known loop shaping method for the design of controllers, the performance of the controllers in terms of step response, steady-state disturbance rejection and noise attenuation and robustness can be improved by increasing the gain at lower frequencies and decreasing it at higher frequencies and increasing the phase margin as much as possible. However, the inherent properties of linear controllers, the Bode's phase-gain relation, create a limitation. In theory, a complex-order transfer function can break the Bode's gain-phase relation; however, such transfer function cannot be directly implemented and should be approximated. This paper proposes a reset element and a tuning method to approximate a Complex-Order Controller (CLOC) and, through a simulation example, shows the benefits of using such a controller.

Keywords: Robot Dynamics and Control, Artificial Intelligence in Mechatronics, Machine Learning
Abstract: Cooperative multi-agent manipulation systems allow to extend on the manipulative limitations of individual agents, increasing the complexity of the manipulation tasks the ensemble can handle. Controlling such a system requires meticulous planning of subsequent subtasks, queried to the individual agents, in order to execute the master task successfully. Real-time (re)planning is essential to ensure the task can still be achieved when subtasks execution suffers from uncertainty or when the master task changes intermittently requiring real-time reconfiguration of the plan. In this work we develop a supervisory control architecture tailored to the cooperation of two robotic manipulators equipped with standard pick-and-place facilities in the plane. We control the planar position and orientation of an object using two underactuated manipulators so that only the position of the object can be controlled directly. The desired orientation follows from the accumulation of alternating relative angles. A time-invariant policy function is trained using deep reinforcement learning, which can determine a finite sequence of pick-and-place maneuvers to manipulate the object to a desired configuration. Two policy architectures are compared. The first uses the kinematic model to determine the final step, whilst the second policy makes this decision itself. The more information is given to the policy the easier it trains. In return, it becomes less adaptable and loses some of its generalisability.

Keywords: Robot Dynamics and Control, Control Application in Mechatronics, Design Optimization in Mechatronics
Abstract: Many nonlinear model predictive controls (NMPCs) are still suffering from the stability and the computational cost for mechanical and mechatronic systems. In order to overcome these situations, the paper provides a simple but general fast search method of design parameters for NMPCs. By applying the proposed method, the computational cost is reduced since the number of the (stable or unstable) closed-loop simulations is decreased and also each closed-loop simulation runs faster. First, we assume SI units for the objective function as well as the constraints. Second, we propose a fast search method of design parameters for NMPCs by introducing a new on-line nondimensionalization for the objective function instead of the standard nondimensionalization for the physical dynamics. Finally, the effectiveness of the proposed method is confirmed by numerical and real experiments using an actual mechatronic system. Almost 98 % reduction of the total search time is achieved to find a good design parameters for the closedloop stability. Remarkably, the proposed method is applicable to many NMPCs and is not restricted to a specific one.

Keywords: Robot Dynamics and Control, Identification and Estimation in Mechatronics, Control Application in Mechatronics
Abstract: The drivetrain flexibility of industrial robots limits their accuracy. To open up new areas of application for industrial robots, an increased dynamic path accuracy has to be obtained. Therefore, this paper addresses this issue by a gain-scheduled drive-based damping control for industrial robots with secondary encoders. For this purpose, a linear parameter-varying (LPV) model is derived as well as a system identification method is presented. Based on this, a gain-scheduled drive-based LPV damping control design is proposed, which guarantees stability and performance under variation of the manipulator configuration. The control performance of the approach is experimentally validated for the three base joints of a KUKA KR210-2 industrial robot. The approach realizes a trade-off between ease of implementation and control performance as well as robustness.

Keywords: Robot Dynamics and Control, Mobile Robots, Modeling and Design of Mechatonic Systems
Abstract: Tracked vehicle dynamic model is important for analyzing the dynamics of tracked vehicles and designing controllers for tracked vehicles. In this paper, a new view of the general theory for skid steering of tracked vehicles is put forward. Based on the general theory, a dynamic track force model used for 3 degrees of freedom (DOF) vehicle model is proposed for time-variant velocity. Simulation shows that the model requires moderate computation and that it can model tracked vehicles more precisely than the vehicle model based on general theory. The simulation result of the dynamic model in steady states is consistent with that of the general theory.

Keywords: Mobile Robots, Robot Dynamics and Control
Abstract: This study combines feedback linearization control theory and the backstepping method to design a transformation controller for a wheeled mobile robot to switch the robot between two-wheeled and four-wheeled modes. A dynamic model of the robot is obtained using the Lagrange method. A linearized external model and a nonlinear internal model are obtained through Input-output feedback linearization strategy. The resulting models are used to design two controllers. In the external model, applied linear optimal control theory, we designed a linear quadratic regulator with integral action to control the robot pose. In the internal model, an adaptive nonlinear controller with integrator based on backstepping method is proposed for robot position control. Finally the effectiveness of the proposed approach is demonstrated using the simulation results.

Keywords: Underwater robotics, Robot Dynamics and Control
Abstract: Robotic fish have received increasing attention in the last few decades, as they hold strong promise in a myriad of applications. Efficient and precise control of these robots, particularly accurate trajectory control, has become essential in many of these applications. This work proposes a dual-loop backstepping-based trajectory tracking control approach for a robotic fish actuated by rowing pectoral fins. While rowing pectoral fin-based locomotion is important for maneuvering, the range constraints of fin movement pose significant challenges in the control of robotic fish, including potentially preventing the robot from generating the thrust needed to maneuver in a desired direction. To overcome these challenges, we propose a dual-loop controller, designed based on an averaged dynamic model of the robot. In particular, an outer-loop backstepping-based controller finds the needed force and moment inputs for the robot to track the desired trajectory, while the inner loop determines the optimal fin-beat parameters such that the resulting fin-generated forces and moment are close to their desired values. Experimental results are presented to show the efficacy of the proposed control scheme, where the robot is commanded to track a trajectory with variable linear and angular velocities. Comparing to a well-tuned PID controller, the proposed control scheme shows a distinct advantage in tracking desired orientations in addition to tracking desired positions. This paper is complemented with a video: url{https://youtu.be/iAH3D5rEZ7M}

Keywords: Sensors and Sensing Systems, Opto-Mechatronic Sensors, Medical Robotics/Mechatronics
Abstract: With the rising demand for flexible strain sensors for soft robots, optical sensing solutions have been proven to be electrically safe, stable and precise. This study presents a highly stretchable optical waveguide sensor that extends existing solutions to two degrees of freedom while maintaining low manufacturing complexity and part count. Through casting in a 3D-printed mould, a polymer waveguide is manufactured, featuring a semi-divided core cross section. The light from a near-infrared LED light source is guided to two phototransistors connected to each chamber. The connection between both core chambers makes the power throughput amplitude and ratio at the two outputs dependent on the strain amplitude as well as its location or direction. The proposed waveguide is experimentally compared to single and dual core designs in four different strain modes, to sense elongation, local deformation amplitude and position across length, twisting angle and direction as well as bending amplitude and direction in a soft finger. While influences of the manual manufacturing process are apparent, the results verify that the presented waveguide sensor can be effectively applied in these four strain scenarios. We also demonstrate that information about superimposed strain states can be obtained from the time signal.

Keywords: Sensors and Sensing Systems, Medical Robotics/Mechatronics, Human -Machine Interfaces
Abstract: Evaluating the softness of the cervix is required during obstetric and gynecological internal examinations to predict the risk of miscarriage and premature birth. In a previous study, we previously developed a wearable pressure distribution sensor to measure the softness of the cervix during an internal examination. However, the sensing operation required precise control to achieve high accuracy. Although wearable sensors are sufficiently flexible to reflect manual operation by humans, they cannot maintain high sensitivity. In particular, appropriate operation is difficult during internal examination because the cervix is not visible to the clinician. Thus, we attempt to solve this problem by providing feedback on the operation to adjust the sensing operation. In this study, we focused on finger orientation and investigated effective feedback for operational adjustment and softness evaluation. First, we compared the two feedback methodologies using visual and tactile stimuli. The comparison results showed that the perceptual substitute visual method was significantly faster and more accurate in adjusting the orientation. Based on these results, we conducted an experiment to simulate palpation with perceptual substitute visual feedback to verify the effectiveness of the softness evaluation. The results showed that perceptual substitute visual feedback could improve sensor sensitivity and achieve a high-accuracy softness evaluation.

Keywords: Sensors and Sensing Systems, Mobile Robots, Planning and Navigation
Abstract: The detection of radiation field is a significant task for nuclear emergency robots. When the radiation field is known, appropriate path planning can be made to reduce unnecessary radiation damage to the robots. However, when the radiation sources are unknown, like under nuclear accident environments, path planning of robots is difficult to make manually. Thus, a planning strategy for radiation field detection is proposed of nuclear emergency robot equipped with gamma camera. The strategy can increase the amount of useful data from proper selection of detection poses of gamma camera, and reduce the radiation dose on the search path. To get more radiation information from the next detection pose, Fisher information matrix (FIM) is applied to assess candidate poses. To protect equipment from radiation damage, dose accumulation is combined with a path planning algorithm for safer path selection. Experiments show that the proposed selection strategy has better performance in radiation field detection tasks compared with manual pose selection.

Keywords: Opto-Mechatronic Sensors, Actuators
Abstract: This paper presents a modular actuator with an integrated force sensor for the utilization in an active optics support system for primary mirrors in the meter-class. The actuator uses a commercial stepper motor together with an arrangement of helical springs. The force measurement of the individual actuators required for the active optical system is based on an optical proximity sensor which measures the distance between the components connected by the springs. This way, the force sensor is highly integrated into the actuator and for the developed system a precise force measurement with an RMS error of 4.5 mN over a range of 24 N is achieved.

Keywords: Intelligent Sensors, Opto-Mechatronic Sensors, Sensors and Sensing Systems
Abstract: This work presents the design, alignment algorithm and validation of an optical scanning color sensor system capable of precisely aligning itself to a sample without additional external sensors, improving the repeatability of the color measurement. The proposed system design enables to scan the optical path of the color sensor by a fast steering mirror (FSM), such that the angle of incidence can be varied within ±3°. A color characterisation is performed to determine the parameter of the CIELAB color space, which can be used to correctly align the color sensor with respect to the sample. An alignment algorithm automatically determines the correct measurement distance and angle of incidence, based on the measured brightness value L*. Experimental results show that the alignment algorithm can precisely align the sensor with a deviation of 9.3 µm and 8 mdeg. To evaluate the repeatability of the system, ten measurements with various start positions are performed, resulting in a maximum color difference of 0.0741 between the measurements. Additionally, the measurement data acquired during the alignment process can be used to characterize the surface finish.

Keywords: Sensor Integration, Data Fusion, Robot Dynamics and Control
Abstract: Autonomous robots on construction sites are designed with the objective to take over time-consuming tasks while also relieving human construction workers. However, to operate safely, they must reliably detect obstacles and avoid collisions. The proposed approach targets this problem for a SCARA-like granular-fill insulation distributing robot, using ultrasonic distance sensors. Mounting the sensors on the robot’s links allows to observe the workspace at a sufficient rate and accuracy. A microcontroller reads out the sensor data and transfers it to the PLC in charge. Furthermore, an efficient sensor error detection principle for the developed sensor suite is demonstrated. Experiments conducted in a laboratory environment prove the functionality of the system. Collision avoidance with various obstacles commonly found on construction sites is realized reliably over a multitude of test runs. The system dependably detects and reacts to obstacles of varying sizes and materials in a safety margin of 300mm within at least 160ms, sufficient for applications in building construction sites.

Keywords: Control Application in Mechatronics, Novel Industry Applications of Mechatroinics, Actuators
Abstract: Water, air pollution, and waste management are key for solving present global sustainability challenges. Waste water treatment processes can be improved by application of kinetic compressors in water aeration. Air pollution can be considerably reduced by the use of industrial heat pumps, where kinetic oil-free compressors are again the key driving technological solution. High-speed motors offer increased efficiencies while bearingless technology can offer a higher reliability and lower long-term costs. For high-speed actively levitated rotors, the main challenges arise in the robust control of uncertain dynamics. In rotating systems, inherent disturbances relate to the rotational frequency. Furthermore, unbalance forces, runout, and unbalanced magnetic pull always impair high-speed rotors. In this work, a centralized robust control of a bearingless compressor suitable for waste water processing or a heat pump is presented. The effect of synchronous disturbances with respect to changes in rotational frequency in the closed loop is studied.

Keywords: Control Application in Mechatronics, Modeling and Design of Mechatonic Systems, Mechatronics in Manufacturing Processes
Abstract: Polygon mirror (PM) based scanners suffer from scan line wobbles caused by facet variant pyramidal errors. The active error correction method uses a fast steering mirror (FSM) to adjust the laser reflection angle actively such that the wobble error can be compensated on the scan surface. This method requires the FSM to follow a fast varying and repetitive reference, which is challenging for the traditional feedback and feedforward control due to limited dynamic bandwidth and model accuracy. This paper proposes to use iterative learning control (ILC) for controlling the FSM, which significantly improves the correction effectiveness and reduces the line wobble for PM scanning systems.

Keywords: Control Application in Mechatronics, Actuators, Neural and Fuzzy Control in Mechatronics
Abstract: In this paper, we propose a new control method based on fuzzy PID and linear support vector classifier (SVC) for previously developed mechanism that consists of a magnetorheological fluid (MRF) piston, a fly-wheel, and two springs. The integration of the three kinds of mechanism enables three different mechanical impedance (variable impedance mechanism: VIM): viscosity, inertia, and elasticity. PID control is proven a good control method for many actuators, however, specific characteristics in VIM such as strong nonlinearity have to be taken into consideration for better performance. Here, we propose a control strategy based on fuzzy PID and linear SVC method for the VIM. Experiments show that suitable operation mode for a given target value and desired operation characteristics can be achieved, also, performance of the VIM system is improved in terms of both accuracy and response time. The proposed control strategy is proven feasible.

Keywords: Design Optimization in Mechatronics, Control Application in Mechatronics
Abstract: For hard disk drive systems (HDD), designing a robust controller that guarantees favorable disturbance rejection performance is crucial in increasing the storage. However, the design process is arduous. To decrease the design effort, employing convex optimization to sequentially design the loop shaping filter is presented in this paper. Optimization constraints are the robust performance and the system's stability. The optimization objective is selected as the 2-norm of the error signal. The satisfying initial condition for optimization is selected based on the RCBode plot. Finally, the effectiveness of the proposed design procedure is validated through a numerical case study.

Keywords: Control Application in Mechatronics, Actuators in Mechatronic Systems, Compuational Models and Methods
Abstract: Automation in hydraulic applications increases a lot. This yields to higher requirements for the dynamic properties of the supply for the hydraulic energy consisting of pressure and volume flow. For this purpose, firstly a detailed mechanical model of the axial piston pump including a torque balance on the swashplate as well as as the modeling of the hydraulic actuation is performed, which is then used for a model based calculation of a feedforward and feedback controller. The nonlinear model is simplified to obtain a differentially flat output to implement a flatness based feedforward controller and to use linear control techniques in the flat coordinates. For the feedforward controller a smooth trajectory together with its derivatives must be calculated. The feedforward and feedback controller together with the trajectory generation is implemented on the real system and the performance is reviewed. At first, the performance is validated with a physical simulation model and the tracking behaviour of the system output to a reference trajectory is examined. Then, the behaviour of the real system is examined and results for exemplary movements are illustrated. These results show a shorter delay-time for the supply volume flow due to the feedforward controller and a shorter settling time for the error of the system pressure to a given reference due to the model based feedback controller. The volume flow supply is more even hence the movement of the equipment is smoother.

Keywords: Software Design for System Integration, Control Application in Mechatronics, Robot Dynamics and Control
Abstract: Passivity is a sufficient condition for system stability and is particularly convenient in loop-control design. This paper presents an architecture for composing and structuring loop controllers out of generic components with preserved passivity properties. This allows handling passivity at a structural level by monitoring the energy flows. A use case shows how this architecture preserves the passivity and, in consequence, the stability of a robotic system operating under detrimental conditions. This effort facilitates generating loop-control software which passivity is unaffected by its composition and/or operation conditions.

Keywords: Modeling and Design of Mechatonic Systems, Actuators, Actuators in Mechatronic Systems
Abstract: A type of scissor-mechanism based actuator with characteristic of bidirectional contraction is designed and studied in this paper. The actuator consists of a soft elastomeric skin, a scissor-mechanism skeleton, and shape memory alloy (SMA) springs for tuning the contracting directions. The actuator is actuated by air pump, and it contracts under vacuum pressure and restores under positive pressure. The two contracting directions are perpendicular to each other, and can be controlled through slightly adjusting the opening angle of the skeleton actuated by SMA springs before applying the negative pressure. This bidirectional contraction characteristic enables the actuator to shorten or elongate under negative pressure. To studied the bidirectional contraction characteristic of the actuator, a model to estimate the critical opening angle of the skeleton between two contracting directions is built through force analysis. Through model validation and experiments, the bidirectional contraction characteristic of the actuator is fully studied. The results show that the actuator can contract in two directions under negative pressure through control the opening angle as model prediction. The new design and bidirectional contraction characteristic in this paper offer valuable insight for developing novel soft actuators.

Keywords: Modeling and Design of Mechatonic Systems, Actuators, Actuators in Mechatronic Systems
Abstract: The valve mode is the most common operational mode of magnetorheological fluid (MRF) devices in engineering applications. Magnetorheological (MR) valves have been studied upon their self-sensing capabilities resulting from the electromagnetic induction phenomenon. In parallel, studies have been carried out regarding the microstructural properties and the analytical modeling of MRFs used in valves. This paper presents the design and fabrication of a microfluidic system in the form of a magnetic circuit, consisting of a MRF flow channel, with dimensions close to those of previously studied miniaturized MR valves. The channel is enclosed in ferromagnetic and transparent sheets. The sheets facilitate the magnetic field distribution, created by a coil. The channel allows the microscopical MRF particles observation and their correlation with the MRF magnetic and rheological properties when the self-sensing phenomenon occurs.

Keywords: Modeling and Design of Mechatonic Systems, Design Optimization in Mechatronics
Abstract: The opposed-piston cam engine (that is OCPE) comprehensively adopts the structural characteristics of the opposed piston engine and the cam type piston engine. Its unique cam power transmission mechanism significantly improves the power density and energy density of the system, and it also cause different characteristics of force transmission and contact that are different from traditional engines. For the toroidal double-acting cam power transmission mechanism, its structure and working principle are analyzed, and the mathematical model of the cam space working surface of the power transmission mechanism is established. Based on the Hertz Line Contact Theory, the mathematical model of the internal contact stress of the power transmission mechanism is established, and the changing law of the cam space curvature and the internal contact stress of the power transmission mechanism are calculated based on different cam profiles, and the changing of contact stiffness of different parts is calculated. The research results show that changing the profile parameters of the cam can significantly change the internal contact stress distribution of OCPE, which provides theoretical support for improving the internal operating environment and prolonging the working life of OCPE.

Keywords: Modeling and Design of Mechatonic Systems, Micro-Electro-Mechanical Systems, Actuators
Abstract: This article presents a circuit model that is able to capture the full nonlinear behavior of an asymmetric electrostatic transducer whose dynamics are governed by a single degree of freedom. Effects such as stressstiffening and pull-in are accounted for. The simulation of a displacement-dependent capacitor and a nonlinear spring is accomplished with arbitrary behavioral sources, which are a standard component of circuit simulators. As an application example, the parameters of the model were fitted to emulate the behavior of an electrostatic MEMS loudspeaker whose finite-element (FEM) simulations and acoustic characterisation where already reported in the literature. The obtained waveforms show good agreement with the amplitude and distortion that was reported both in the transient FEM simulations and in the experimental measurements. This model is also used to predict the performance of this device as a microphone, coupling it to a two-stage charge amplifier. Additional complex behaviors can be introduced to this network model if it is required.

Keywords: Modeling and Design of Mechatonic Systems, Intelligent Process Automation, Hybrid intelligent systems
Abstract: Solve social issues and create new value through a cyber-physical system (CPS) is expected. In industry, it is thought that new value can be created by a control system utilizing a CPS. To realize that control system, the following three technologies are required. (i) Technology for utilizing data obtained from physical space. (ii) Modeling technology for expressing a physical-space object in cyber-space. (iii) Control technology utilizing a model expressing a physical-space object.

This paper presents a design scheme of database-driven model predictive control (DD-MPC) system based on alarm management including those three technologies. This system can detect the occurrence of a modeling error, issue an alarm, and execute database-driven modeling (DD-Modeling) based on the alarm. These functions enable the model to update at appropriate times and maintain good control performance.

The proposed method consists of the alarm filter and DD-MPC. The alarm filter monitors the error level between the controlled object and the model, and issues an alarm if the error exceeds a threshold. The alarm filter has an adaptive threshold that is adjusted with DD-Modeling. DD-MPC consists of the model of the controlled object estimated by DD-Modeling and MPC using this estimated model. DD-Modeling estimates the model of the controlled object when an alarm is issued. Using the control system with these functions, the model can be updated at an appropriate time, and the desired control performance can be obtained, even though the characteristics of the controlled object change during operation.

The effectiveness of the proposed method was verified by a numerical simulation. By applying DD-MPC based on alarm management, the following were confirmed. (a) Modeling errors could be detected by an alarm filter with an adaptive threshold. (b) The model and the threshold were adjusted by DD-Modeling when an alarm was issued. (c) By updating the model based on the alarm, good control performance could be obtained.

Keywords: Modeling and Design of Mechatonic Systems, Control Application in Mechatronics, Motion Vibration and Noise Control
Abstract: A nonlinear model of the permanent magnet synchronous motor (PMSM) to represent current harmonics when using Active Noise Cancellation (ANC) with electrical drives is proposed. Furthermore, a scheme to compensate for these harmonics is presented. These measures reduce the problem of unwanted harmonics when injecting a sinusoidal signal into a PMSM in standstill to cancel a narrowband disturbance with ANC. The parameters for the stated model are identified with the help of high frequency signal injection. A proportional resonant controller is used together with an input/output linearisation of the model in order to reduce the harmonics. Experiments show a good reduction especially of the dominating third harmonic for two injected frequencies.

Keywords: Human -Machine Interfaces, Hybrid intelligent systems, Rehabilitation Robots
Abstract: In recent years, intelligent external devices (such as mobile robots, wheelchairs, and robotic arm exoskeletons) driven by brain computer interfaces (BCIs) have increasingly appeared in people's field of vision and have gradually expanded from helping end users with disabilities to assist normal human beings. This review focuses on the comparison of latest brain-device interaction methods and common control strategies based on 22 published representative studies in IEEE during the last 5 years. Corresponding approaches of electroencephalography (EEG) signal preprocessing, feature extraction, and classification are summarized. Moreover, the review concludes the challenges and the future directions of current brain-controlled intelligent devices.

Keywords: Medical Robotics/Mechatronics, Flexible Manipulators and Structures, Machine Vision
Abstract: This paper presents a concept of a model-based localization algorithm for deformable systems. The proposed approach considers a real-time co-simulation to predict deformations. The simulated deformations are incorporated in a state-of-the-art simultaneous localization and mapping (SLAM) algorithm to enable precise localization in an endoscopic, non-rigid environment. To demonstrate the potential of the proposed model-based SLAM algorithm, simulation results of a synthetic environment are discussed. Therefore, an ideal model and accurate knowledge of the acting forces are assumed. This work is intended as proof of concept to provide fundamental ideas and the framework to apply a model-based localization algorithm in a real-world endoscopic surgery environment. The code of the implemented framework is publicly accessible.

Keywords: Medical Robotics/Mechatronics, Virtual Reality and Human Interface, Robot Dynamics and Control
Abstract: The catheter that integrates electronic components for sensing and actuation presents some limits due to the misleading information consequently inducing delays and instability. Besides, the relatively high cost and the fixed dimension, strongly hold back their miniaturization for an implementation in new biomedical applications like targeted delivery. We propose a haptic architecture combined Virtual Reality (VR) for the magnetic navigation of catheter inside a vascular network that facilitates basic navigation with a high precision. VR was incorporated into the system, allowing the user to control the catheter from a more intuitive standpoint and provide a description of the nature, texture and intensity of collision in real-time. The kinematics, dynamics and the contact forces derived respectively from the modified Denavit-Hartenberg (DH), Euler-Lagrange and springdamper are used to establish the closed loop inverse dynamic control of the virtual model which is validated to support the real prototype being controlled in open loop.

Keywords: Medical Robotics/Mechatronics, Design Optimization in Mechatronics, Actuators in Mechatronic Systems
Abstract: Recent advances in integrated circuits and micromachining have enabled the integration of battery-powered micro-actuators in miniaturized drug delivery systems. However, the power/energy management system that treats current overloading remains sub-optimal. Overloading not only deteriorates the actuators' long-term performance but also attenuates battery capacity. In this work, we are proposing a simple yet powerful solution to manage current overloading to maximize battery-powered system lifetime. The proposed solution consists of a digitally programmable soft starter and DC-DC converter that can dynamically balance the trade-off between inrush current amplitude and motor starting speed as well as maximally minimize the continuous current draw from a battery. Our experimental results show that the proposed soft starter alone can enhance the battery capacity by 18% and together with the DC-DC converter, they can increase the drug delivery cycles by 33% without sacrificing the system's output performance.

Keywords: Micro/Nano Manipulation, Medical Robotics/Mechatronics, Actuators in Mechatronic Systems
Abstract: Guiding magnetic particles through the cochlear canal in the human anatomy presents a challenge for disease treatment. Having the particularity of being operated remotely, magnetic particles are innovative solutions. However, the use of hard microrobots and the complexity of navigation in automatic mode present a real obstacle to the integration of these methods in the medical field. Faced with this, the choice of the actuation mode, the type of microrobot and the navigation strategy are therefore of paramount importance for guidance and navigation in the cochlear canal. For this, we have developed in this work a strategy of navigation and control of soft magnetic microrobot (biocompatible) using haptic assistance allowing to give all the control to the doctor. The navigation strategy is tested in an in-silico model based on the finite element method, then we use a human scale artificial cochlea to demonstrate the effectiveness of this strategy and the interest of integrating the doctor into the control loop.

Keywords: Artificial Intelligence in Mechatronics, Mobile Robots, Machine Learning
Abstract: As the world is moving towards more personalized and customized manufacturing, the manufacturing system needs to adapt. One method can be to integrate industry 4.0 concepts in reconfigurable manufacturing systems (RMS). This allows the manufacturing system to become more self-sustaining and flexible at the same time. There is however, a lack of research on how to integrate industry 4.0 concepts such as industrial Big Data (IBD) into RMS. This paper looks at how IBD techniques can be used on an RMS for classification and how to collect data from an RMS. A case study where five different movable platforms are identified with an accuracy of more than 85% is showcased.

Keywords: Learning and Neural Control in Mechatronics, Robot Dynamics and Control, Machine Learning
Abstract: Online Gaussian processes (GPs), typically used for learning models from time-series data, are more flexible and robust than offline GPs. Both local and sparse approximations of GPs can efficiently learn complex models online. Nevertheless, these approaches assume that all signals are relatively accurate and that all input-output data are available for learning without misleading data. Besides, the online learning capacity of online GPs is limited for high-dimension problems and long-term tasks in practice. This paper proposes a sparse online GP (SOGP) with a forgetting mechanism to forget distant model information at a specific rate. The proposed approach combines two general data deletion schemes for the basis vector set of SOGP: The position information-based scheme and the oldest points-based scheme. We apply our approach to learn the inverse dynamics of a collaborative robot with 7 degrees of freedom under a two-segment trajectory tracking problem with task switching. Both simulations and experiments have shown that the proposed approach achieves better tracking accuracy and predictive smoothness compared with the two general data deletion schemes.

Keywords: Flexible Manipulators and Structures, Learning and Neural Control in Mechatronics, Robot Dynamics and Control
Abstract: In the field of soft robotics, Dielectric Elastomer Actuators (DEAs) represent a compact and efficient alternative to bulky pneumatic drives. Both soft robots and DEAs show highly nonlinear behaviours due to their intrinsic large deformations, making the design of dynamic controllers highly challenging. At present, the problem of position control of DEA soft robots has not yet been explored in the literature. In this paper, we directly account for the nonlinearities of a DEA-driven soft robotic structure using an Adaptive Dynamic Programming (ADP) approach. We develop for the first time a closed-loop optimal controllers for position regulation of a DEA soft robot, by approximating the solution of the Hamilton-Jacobi-Bellman equation with Neural Networks (NNs). We extend an existing model-based ADP approach to deal also with asymmetric input constraints. Simulation studies asses the improvements in positioning performance of the proposed approach, in comparison to traditional strategies.

Keywords: Artificial Intelligence in Mechatronics, Hybrid intelligent systems, Mechatronics in Manufacturing Processes
Abstract: Predictive models can be integrated in the sensing and monitoring methodologies of mechatronic systems in operation. When systems change or are subject to varying operating conditions, adaptivity of the models is needed. The goal of this paper is to enable this adaptivity by presenting a framework for continual learning. The framework aims to transfer- and remember information from previously learned systems when a model is updated to new operating conditions. We achieve this by means of the following three key mechanisms. We first include physical information about the system, heavily regularizing the model output. Secondly, the usage of epistemic uncertainty, used as an indicator of the changing system, shows to what extend a transfer is desired. Last but not least the usage of a prior within a Bayesian framework allows to regularize models further according to previously obtained information. The last two principles are enabled thanks to the use of Bayesian neural networks. The methodology will be applied to a camfollower system in a simulation environment, where results show that previously trained systems are better remembered with an increase of 72% compared to normal training procedures.

Keywords— Bayesian, neural networks, catastrophic forgetting, continual learning, hybrid modelling, physics inspired, mechatronic system

Keywords: Mechatronics in Manufacturing Processes, Machine Vision, Artificial Intelligence in Mechatronics
Abstract: Additive Manufacturing (AM), known as 3D Printing (3DP), has been widely implemented in the industry due to its advantage of free design and high efficiency in rapid prototyping. Recently, 3D Construction Printing (3DCP) has become an emerging topic. As a multidisciplinary subject, it involves a complex system that consists of multiple sub-systems, the uncertainty of material behaviour also bring risks to the printing quality. Hence, the main issue of 3DCP lies in the online monitoring and control of the printing process quality. This paper presents a combined approach for online 3D construction material printing quality monitoring and adaptive compensation for printing layer. The width of freshly printed filament can be adopted to characterize the printing quality. A vision system based on Deep Learning is developed to detect Automatically the filament width deviation. Based on the results of the vision system, a real time adaptive compensation of the nozzle travel speed is realized to maintain the quality deposit performance. The proposed approach is validated by on site printing experiments.