“If these walls could talk.” We tend to think of augmented reality (AR) as a technology that helps sighted users see more than is really there in their surroundings. But researchers are finding novel applications for AR to help those who have lost their vision as well.
A recent series of experiments by researchers at the California Institute of Technology (Caltech) used off-the-shelf consumer hardware to create a system that allows blind users to navigate unfamiliar settings safely and with confidence. The system relies on a Microsoft HoloLens that scans the scene in front of the user, and renders a 3D map of the space. The researchers used the Unity game development platform, which supports head tracking and text to speech generation. They created a system in which objects can “talk” to the user, making it easy to walk to a chair in a room. The system is also able to warn the user of obstacles along the intended path. Perhaps most impressive was an experiment where users were asked to navigate an unfamiliar two-story building. The system would constantly adjust the “target” to a few feet ahead of the subject so that they followed a programmed path (much like a GPS route). The voice would say “follow me” as the user progressed along the path, and would call out turns, obstacles, and other useful information such as the presence of handrails on a stairway.
The researchers ran blind test subjects through a series of activities. They provided no training on the system before the exercises. Using the system, the subjects were able to walk directly to a chair located in a random location in a room; it took them far longer to find the chair using just their cane. And all the subjects were able to quickly complete the long navigation exercise (with paths shown in the illustration above).
This is a remarkable demonstration of how relatively low-cost devices can be put to new uses and solve significant real world problems. If such a wearable system could become widely available, it would be a boon to millions of people with limited or no vision, giving them new freedom and confidence to move freely about in the world.
