AI Avatars Help Autistic People Hone Social Skills On Demand

Autistic individuals will soon be able to practice and improve their social skills through realistic conversations with avatars powered by generative AI technology.

Autism is a developmental disorder that affects how people interact and communicate, often hindering diagnosed individuals in social situations. In many cases, autistic individuals struggle to initiate conversations, respond to the initiations and non-verbal cues of others, maintain eye contact, and take on another person’s perspective.

The web-based Skill Coach application developed by Israeli startup Arrows lets users engage in conversations in a variety of scenarios – from chatting with a stranger in a café to making small talk with a work colleague. 

Not a single scenario is scripted, and the avatar’s responses vary in each encounter. Users can also set themselves a specific goal, such as learning how to handle disagreements, before the start of a conversation.

Throughout every conversation, the AI platform identifies what it sees as conversational missteps and provides users with real-time recommendations through textual feedback, as well as visual cues that appear discreetly on the screen. These cues include symbols to indicate that the user should elaborate on their most recent remarks (a plus sign) or ask a followup question (a question mark). 

And as soon as the conversation ends, the Skill Coach app will provide detailed textual feedback such as whether a response was rude and the user should be less direct next time.

The AI was trained using a leading, evidence-based international program that aims to enhance the social skills of autistic adults. In the coming months, the web app will be tried out for the first time by early adopters who struggle with social interaction.

Family Ties 

The inability to interact and form social bonds can lead to anxiety and depression. And according to clinical psychologist Dr. Jana Rundle, individuals with autism may be three times more likely to have episodes of depression compared to the general population. 

Arrows founder Eran Dvir found that this was the case with his 17-year-old autistic daughter, who struggled to practice and improve her social skills outside of scheduled speech therapy sessions. 

“When they are children, [autistic people] usually have a shadow teacher or an integrator who helps facilitate their social lives,” he tells NoCamels, referring to professionals who help neurodivergent youngsters.

“But what happens is when they grow up, it’s no longer appropriate to always have [another person supporting them 24/7], and the problems are still there.”

Seeking a solution for his daughter while studying for a Master’s in human-computer interaction, Dvir had the idea to guide autistic adults through social situations using tech.  

He initially explored integrating this kind of technology into augmented reality glasses, where the aforementioned visual cues would offer social recommendations in real time.

Though the technology was tested on dozens of autistic individuals, he says that the hardware for augmented reality glasses has not advanced enough, and needs several more years to fully reach its potential. 

And so Arrows, which is based in Tel Aviv, has since pivoted to developing its Skill Coach software – at least for now.

A Virtual Guide

The company’s software centers on visual feedback cues and was created with the assistance of the Arrows’ scientific director – an Israeli psychologist and expert in autism whose identity is still under wraps. 

The realistic avatars that facilitate the conversations were created by an Israeli startup that specializes in AI-generated photos and videos, and the generative AI was developed using a range of software from several Israeli companies.

The startup is also already collaborating with the Israeli offices of IT giant Cisco to embed Arrows’ visual cues into video conferencing platforms. This will allow autistic people to receive real-time feedback during video conversations.

He stresses that this does not replace the need for a psychologist who helps autistic individuals with social interactions face-to-face, but allows people on the spectrum to practice their social skills at will – something that isn’t possible with scheduled in-person sessions. 

The company is planning to conduct a pilot by next year, which will focus on social interactions in the workplace.  

Once the pilot has been completed successfully, a subscription to the software will be available to both medium and large companies trying to boost diversity and inclusion. Therapists will also be able to offer the platform to patients for use when they are not in session.

“Because the conversation is with a virtual avatar, it is not only a safe space for the individual to practice, but it is also available for them at all times,” explains Dvir.  

Arrows was one of the winners of the 2021 HackAutism event, a yearly program where ​​entrepreneurs, experts and investors come together to develop 10 viable, sustainable projects that address the challenges experienced by autistic children and adults and their families.

The startup has thus far raised over $150,000 in funding, most of which was received as a grant from the Israel Innovation Authority. 

There are other coaching services for autistic individuals – both in-person and virtual – but Arrows’ says its platform with its real-time feedback sets it apart. 

“I think that our experience and our excellent team, with the uniqueness of the visual cues and the evidence-based program, makes us more competitive right now,” says Dvir. 

“And our focus on autistic individuals is a niche that not many other software companies are trying to cater to.”

Subscribe to NoCamels weekly newsletter and get our top stories