ETHOS: A Robotic Encountered-Type Haptic Display for Social Interaction in Virtual Reality

We present ETHOS (Encountered-Type Haptics for On-demand Social interaction), a dynamic encountered-type haptic display (ETHD) that enables natural physical contact in virtual reality (VR) during social interactions such as handovers, fist bumps, and high-fives. The system integrates a torque-controlled robotic manipulator with interchangeable passive props (silicone hand replicas and a baton), marker-based physical-virtual registration via a ChArUco board, and a safety monitor that gates motion based on the user’s head and hand pose. We introduce two control strategies: (i) a static mode that presents a stationary prop aligned with its virtual counterpart, consistent with prior ETHD baselines, and (ii) a dynamic mode that continuously updates prop position by exponentially blending an initial mid-point trajectory with real-time hand tracking, generating a unique contact point for each interaction.

The developed system is first applied to the recreation of physical interpersonal interactions with a virtual avatar - specifically, an object handover, fist bump, and high five. Such interactions are well-suited for dynamic encountered-type haptics, as they require the real-time coordination of two entities with short-duration intermittent contact. These interactions will also be easily recognizable and carry familiar social context for the user to enhance the naturalness of their experience. Two interaction control strategies were developed: the static strategy presents a stationary object for the user to interact with whereas the dynamic strategy leverages online pose tracking of the user’s hands to continuously update a target contact point, a weighted average of an initial trajectory and the user’s current hand position, creating a unique contact point for each interaction.

To evaluate the colocation between the virtual and physical interactions, two metrics were evaluated. The first, spatial alignment, evaluates the performance of our fiducial-based registration approach by comparing the stationary virtual-physical alignment to a ground-truth motion capture system. The second metric, interaction latency, characterizes the time difference between a physical and corresponding virtual event to evaluate the impact of delay on interaction efficacy.

In this work, we introduced ETHOS (Encountered-Type Haptics for On-demand Social interaction), extending the capabilities of ETHDs beyond static props toward dynamic, socially meaningful contact in VR. By equipping a torque-controlled manipulator with customized physical props, we realized novel interpersonal interactions—handover, fist bump, and high five—that are intuitive, repeatable, and grounded in everyday social behaviour. The results in this work provide the first quantitative evidence that high-fidelity interpersonal interactions are feasible in VR through an ETHD framework.