Development of a Visual-Tactile Fingertip Sensor and an Object Manipulation Method using a Multi-Fingered Robotic Hand

In this paper, a novel visual-tactile sensor is proposed and an object manipulation method of a grasped object by a multi-fingered robotic hand is proposed by detecting a contact position using the visual-tactile sensor. The visual-tactile sensor is composed of a hemispheric fingertip part made of soft silicone with hollow inside and a general USB camera located at the inside of the fingertip to detect the displacement of many point markers embedded in the silicone. The deformation of each point marker due to a contact force is measured and a contact position is estimated in stable through a novel method of creating virtual points in figuring out the point clouds. Not only to demonstrate the estimation performance of the new visual-tactile sensor itself, but its practical usefulness in a grasping and manipulation task is also illustrated as well. By using a contact position obtained from the proposed sensor and the position of each fingertip obtained from kinematics, the position and orientation of a grasped object are estimated and controlled. Its performance is demonstrated through numerical simulation results.