A model-based system for real-time articulated hand tracking using a simple data glove and a depth camera. IEEE/CVF International Conference on Computer Vision 11532–11541 (IEEE, 2021). Physics-based human motion estimation and synthesis from videos. Constraining dense hand surface tracking with elasticity. International Conference on 3D Vision (3DV), 333–344 (2020). Grasping field: learning implicit representations for human grasps. In IEEE International Symposium on Circuits and Systems (ISCAS), 1–5 (2018). An EMG gesture recognition system with flexible high-density sensors and brain-inspired high-dimensional classifier. Perspectives of users for a future interactive wearable system for upper extremity rehabilitation following stroke: a qualitative study. IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, 1–11 (2019). Vision-based action understanding for assistive healthcare: a short review. Multi-functional soft strain sensors for wearable physiological monitoring.

Review of wearable sensor-based health monitoring glove devices for rheumatoid arthritis. AI enabled sign language recognition and VR space bidirectional communication using triboelectric smart glove. A survey of vision-based human action evaluation methods. Learning the signatures of the human grasp using a scalable tactile glove. Augmented tactile-perception and haptic-feedback rings as human–machine interfaces aiming for immersive interactions. A simple, inexpensive, wearable glove with hybrid resistive-pressure sensors for computational sensing, proprioception, and task identification. Soft modular glove with multimodal sensing and augmented haptic feedback enabled by materials’ multifunctionalities. Gesture recognition using a bioinspired learning architecture that integrates visual data with somatosensory data from stretchable sensors. Interactive hand pose estimation using a stretch-sensing soft glove. A survey on hand pose estimation with wearable. Sign-to-speech translation using machine-learning-assisted stretchable sensor arrays. A wearable biosensing system with in-sensor adaptive machine learning for hand gesture recognition. Learning human–environment interactions using conformal tactile textiles. We demonstrate accurate tracking of dexterous hand movements during object interactions, opening new avenues of applications, including accurate typing on a mock paper keyboard, recognition of complex dynamic and static gestures adapted from American Sign Language, and object identification. We report a data augmentation technique that enhances robustness to noise and variations of sensors. We use multi-stage machine learning to report average joint-angle estimation root mean square errors of 1.21° and 1.45° for intra- and inter-participant cross-validation, respectively, matching the accuracy of costly motion-capture cameras without occlusion or field-of-view limitations. The sensor yarns have a high dynamic range, responding to strains as low as 0.005% and as high as 155%, and show stability during extensive use and washing cycles. Here we report accurate and dynamic tracking of articulated hand and finger movements using stretchable, washable smart gloves with embedded helical sensor yarns and inertial measurement units. Capturing realistic hand movements is challenging because of the large number of articulations and degrees of freedom. Accurate real-time tracking of dexterous hand movements has numerous applications in human–computer interaction, the metaverse, robotics and tele-health.

Sticking up middle finger drawing