Researchers at Cornell University and the University of Wisconsin, Madison working together at Cornell’s SciFi Lab developed FingerTrak, a potentially breakthrough wearable that tracks human hand motion in 3D. The SciFi Lab is a new interdisciplinary lab that’s part of the University’s Computing and Information Science department. The SciFi lab focuses on developing intelligent sensing systems. The research team published a paper that describes FingerTrak in the Proceedings of the Association for Computing Machinery on Interactive, Mobile, Wearable, and Ubiquitous Technologies.
According to the research team, FingerTrak overcomes the limitations of past wrist-mounted sensing devices that rely on cameras to capture finger positions. FingertTrak has four miniature thermal-sensing cameras that capture silhouette images of the entire hand by focusing on wrist positions. FingerTrak is relatively lightweight compared to previous technologies and allows free movement by the wearer. The device uses machine learning with a deep neural network to patch together the multiple silhouette images to form a total 3D image of the hand. According to SciFi Lab director Cheng Zhang, Assistant Professor of Information Science, the greatest discovery was that wrist contour positions provided sufficient data to “predict the entire hand posture.” The ability to position the thermal sensors on the band to capture wrist positions rather than cameras focused on all parts of the hand is a practical advantage for designers and for wearers.
Researchers at the SciFi Lab wrote that the initial use of FingerTrak is in sign language translation. The device could also be used in health care applications such as monitoring fine motor skills for people with related disorders. We would add that it might also become a user input device for human/machine interface applications, including controlling wearables and mobile devices.
