AI joins the Seeing Eye team. Bailey’s job is safe — for now

By Sophia Ricciardelli

Guide dogs have been helping blind people navigate the world for decades, forming bonds with their handlers that would be difficult for technology to replicate. Artificial intelligence, however, may make a pretty good teammate.

The Seeing Eye, a Morris Township nonprofit that has been training guide dogs since 1929, recently launched a pilot program to equip clients with Meta smart glasses to give them more information about the world around them as they navigate with their dogs.

For about six months, the organization has been testing the technology with its graduates. Next month, it will begin offering classes to new students who want to learn to use the glasses alongside a guide dog.

Kyle Street, 32, has been involved with The Seeing Eye since getting his first dog at 17 and has worked there for the past year and a half. Smith is now on his third dog: Bailey, a black Lab-golden retriever mix that has been by his side for nearly seven years.

Street grew up visually impaired and lost his remaining vision in 2020. He always knew he wanted a Seeing Eye dog.

He said the relationship between a handler and a guide dog is more of a partnership than people may realize. “We really are a team,” he said. “I’m the brain; he’s the eyes. And so we’re making all these decisions together.”

That partnership takes time to build. He said it took about six months for him and Bailey to develop the intuitive rhythm they have today. “He’ll be like, ‘Oh, you want to go in this coffee shop? No, not right now? Okay,'” Street said. “We just read each other’s body language so well.”

On a recent weekday, Street had his first session with Bailey, the Meta glasses, and Seeing Eye instructor Bill Lang. The mission? Walk from The Seeing Eye training center on Maple Avenue in Morristown to 7-Eleven and buy an energy drink.

On the way there, Street used BlindSquare, a GPS app designed for blind users that announces things like intersections and buildings he passes, as well as the direction he’s walking in. The technology itself isn’t new, but now it can connect directly to the speakers in his Meta glasses. Bailey is responsible for getting Street safely from corner to corner, but Street still has to know where he’s going and give Bailey commands at each intersection.

They had never taken this particular route before. As Street and Bailey got closer to the 7-Eleven, where the surrounding area was less familiar, the glasses came more into play.

Once they reached the store, Street called Aira, a paid service that connects him with a trained agent who can see through the glasses’ camera and describe Street’s surroundings. The agent helped him locate the 7-Eleven’s door handle, find the refrigerator stocked with energy drinks, and navigate to the Celsius display.

But Aira couldn’t identify the flavor Street wanted. Whether it was an issue on their end or a connectivity problem on his, Street wasn’t sure. He hung up and tried the Meta glasses instead.

“Hey Meta, what flavor is this?” he asked while holding up the can. All he heard was the bloop-bloop-bloop loading sound. Lang eventually stepped in, and Street called Aira back for help checking out.

During moments of navigation, Bailey and Aira’s roles can become less distinct. Street needed Aira to tell him where the cash register was, but Bailey still was responsible for safely guiding him there.

A stack of bottled-water cases blocked their path. They paused while Bailey figured out the best way around the obstacle.

They made it to the register, where Street paid with the Aira agent talking him through the process. On the way out, with a person blocking their path and Street listening to two guides rather than one, Bailey seemed unsure where to go.

“Eventually I was like, all right, there’s just too much going on here,” Street said. “I didn’t tell (Aira) this, but I was like, ‘I don’t need you to tell me how to get out of here. My dog knows how to get out of here.’”

He gave Bailey the command to go outside. The dog surged forward and led him straight to the exit.

LISTENING FOR NATURE’S CALL

Understanding his dog’s capabilities and needs means knowing when to set the technology aside, Street said.

On the way there, Bailey stopped walking and started sniffing around the curb. Street flipped up the glasses.

“Bailey’s acting weird. I have to pay attention to every move that he’s making so that I can understand what he’s trying to tell me,” he said.

In this case, the message was pretty simple: Bailey needed a bathroom break.

Outside the store, Street gave the Meta glasses another try with the Celsius can. This time, it worked: “Orange.”

Internet access definitely can be a challenge, he said. “I’ve been in the back of restaurants and tried to ask it to read a menu to me, and there just wasn’t good enough cell reception.”

Loading sounds aside, he’s excited about where the technology is headed.

“A lot of people that I talk to might complain about the lag time,” he said. “But I’ve already seen an improvement from Gen 1 to Gen 2, and it’s only gonna get better.”

Street has had the glasses for a couple of years now, and he said he appreciates the extra layer of independence they provide. Shortly after getting them, he took his young son to a local ice cream shop.

“The menu’s not online,” he said. “But I was able to tell Meta to take a picture and read me the flavors, and I was able to tell him what the flavors were. That was a really cool moment.”

Before incorporating the glasses into their work with a guide dog, participants will need to be comfortable using the technology on its own. The Seeing Eye plans to ship glasses directly to participants’ homes so they can learn how to use them before arriving for training.

Lang, who has worked with the Seeing Eye for five years, said becoming familiar with the glasses first is extremely important because the additional information they provide can become distracting or interfere with the subtle communication between a handler and his or her dog.

“If you hear one of the operators on the phone tell you, ‘You’re gonna have to turn left up ahead,’ and you start to make a left turn before asking your dog if it’s safe to do so, that could be a problem,” he said.

Seeing Eye dogs are trained through praise and learn to adapt to their handlers, which means repeatedly bypassing the dog could inadvertently teach it new habits. “It is possible to untrain a Seeing Eye dog,” Lang said.

Street said he would be hesitant to introduce the technology too early in a new guide dog partnership. Building trust with a dog takes time, and he worries that a new handler could become overly reliant on information coming from the glasses before learning when to trust the dog.

“AI is still an evolving tool, and it hallucinates. It told me I had a $3 bill one time,” Street said. Dogs, on the other hand, “have been keeping us safe for over 100 years.”

When it comes down to it, he sees a pretty clear role for each.

“Bailey’s here to get me from point A to point B. He’s still the most reliable service to do that,” Street said, “whereas the Meta glasses really expand the experience once you get to where you’re going.”