TY - GEN
T1 - Hand Orientation Detection Based on Disparity Maps from Stereo Imagery
AU - Setiawan, Dion
AU - Yuniarno, Eko Mulyanto
AU - Purnomo, Mauridhy Hery
N1 - Publisher Copyright:
© 2024 IEEE.
PY - 2024
Y1 - 2024
N2 - Manipulating objects is one of the most basic human interactions. This involves holding, shifting, lifting, and passing objects to other people. These interactions can be replicated for human assistive robot systems in robotics, which can help the elderly with their daily tasks. Many researchers have developed 2D image-based approaches to produce hand landmark detection to recognize hand shapes and gestures. However, there are shortcomings in the resulting data output, namely that it does not provide the characteristics of the distance and orientation of the hand being detected. As a result, it cannot be utilized as input to control the robot manipulator's movements to get closer to the region of the human hand. We have designed a hand position and orientation recognition system with a stereo camera to anticipate its disparity map, which includes distance prediction data for every map pixel, to overcome this challenge. The disparity map is projected onto a point cloud using a hand landmark detection technique, and the points that match the hand landmarks are chosen. We employ the Singular Value Decomposition (SVD) mathematical technique to ascertain the hand position and orientation. We verified the sequence of tasks completed to determine the position and orientation of the stereo camera. Furthermore, our approach is user-friendly and can be easily applied across a range of simpler systems, achieving a processing speed of approximately 10.03 FPS.
AB - Manipulating objects is one of the most basic human interactions. This involves holding, shifting, lifting, and passing objects to other people. These interactions can be replicated for human assistive robot systems in robotics, which can help the elderly with their daily tasks. Many researchers have developed 2D image-based approaches to produce hand landmark detection to recognize hand shapes and gestures. However, there are shortcomings in the resulting data output, namely that it does not provide the characteristics of the distance and orientation of the hand being detected. As a result, it cannot be utilized as input to control the robot manipulator's movements to get closer to the region of the human hand. We have designed a hand position and orientation recognition system with a stereo camera to anticipate its disparity map, which includes distance prediction data for every map pixel, to overcome this challenge. The disparity map is projected onto a point cloud using a hand landmark detection technique, and the points that match the hand landmarks are chosen. We employ the Singular Value Decomposition (SVD) mathematical technique to ascertain the hand position and orientation. We verified the sequence of tasks completed to determine the position and orientation of the stereo camera. Furthermore, our approach is user-friendly and can be easily applied across a range of simpler systems, achieving a processing speed of approximately 10.03 FPS.
KW - 3D Hand Tracking
KW - Elderly Assistive Robots
KW - Interaction
KW - Stereo-Vision
UR - http://www.scopus.com/inward/record.url?scp=85199471320&partnerID=8YFLogxK
U2 - 10.1109/CIVEMSA58715.2024.10586620
DO - 10.1109/CIVEMSA58715.2024.10586620
M3 - Conference contribution
AN - SCOPUS:85199471320
T3 - CIVEMSA 2024 - IEEE International Conference on Computational Intelligence and Virtual Environments for Measurement Systems and Applications, Proceedings
BT - CIVEMSA 2024 - IEEE International Conference on Computational Intelligence and Virtual Environments for Measurement Systems and Applications, Proceedings
PB - Institute of Electrical and Electronics Engineers Inc.
T2 - 2024 IEEE International Conference on Computational Intelligence and Virtual Environments for Measurement Systems and Applications, CIVEMSA 2024
Y2 - 14 June 2024 through 16 June 2024
ER -