By: Guy Tal Hauser
Technology has come a long way in helping the disabled and the blind. From apps to devices, they’re finding new ways to assist those in need by giving them tools that compensate in functionality. Microsoft’s HoloLens was recently used to effectively make objects talk to a blind person in order to help them navigate a space.
Researchers at Caltech used the Microsoft mixed reality headset to create what they’re calling a Cognitive Augmented Reality Assistant or CARA. CARA is able to identify objects in any room and call out their names. But it gets even cooler. It would be one thing to tell a blind person “There’s a chair in front of you,” or other such description, but using spatialized sound, CARA makes it sound as if the object itself is calling out its name.
For example, if there is a chair in front of the blind person and set off to the right, it will say, “Chair,” but it will sound like it’s coming directly from the chair so they can identify where and how far away the object is. As the blind person gets closer to it, the voice gets higher in pitch.
When a blind person steps into a room for the first time the objects will begin calling themselves out by name from left to right. They’ll call out object names like lamp, laptop, table, door, etc. You can also activate spotlight mode where it’ll identify any objects you point your head towards. So, if you “look” in the direction of a picture a desk it’ll say, “Picture” and “Desk.” It will include any object in the direction your facing that falls within that spotlight. For a more targeted description, i.e., one object, there is target mode.
Navigating a single room isn’t enough, especially if this is going to have more widespread application. The system also includes a navigation or virtual guide that calls out “follow me” to the user as they walk through the building towards their destination.
Professor Markus Meister and his team would like to see this software-meets-hardware solution exported beyond their lab. “We’d like to see this device used and offered at the entrance of large spaces, more or less the way you’d adopt the audio guide when you walk into a museum,” Meister says.
About Guy Tal Hauser
