Finding A Voice: How AI Is Helping Children With Autism

App alerts parents to key moments in home therapy sessions

Children with autism are learning to speak for the first time, thanks to an AI-powered app.

Parents record videos of structured therapy sessions at home, then receive practical guidance based on sophisticated computer analysis.

It’s like having a therapist there in the room. Instead, through deep learning, the app is able to identify the moments, overlooked by the parents, when a child attempts to verbalize, like during activities they enjoy.

Trained therapists intervene at this point and suggest detailed ways in which parents could implement strategies in future scenarios. 

For example, when a mother blows on her child’s hands to dry them, and pauses for a moment, the AI will suggest she could have encouraged him to verbalize and say the word “blow”.

The app is designed to make the most of the early years of a non-verbal child – up to the age of five – when they are best able to acquire language skills.

It uses deep learning to recognize the points at which a non-verbal child is most likely to try and communicate, and when parents should reinforce their attempts. 

The program is based on Pivotal Response Treatment (PRT), an evidence-based behavioral treatment for autism that increases the child’s motivation to learn and initiate communication, and improves their social skills.

Parents are often unable to access human PRT intervention because of long waiting lists, lack of insurance coverage, and the fact that it’s expensive.

SocialMind Autism was founded in Israel by Dr Keren Sagiv Friedgut, a licensed pediatrician of 20 years who first learned PRT to help her two autistic kids communicate.

The program is structured into 22 lessons that target areas of communication development, including motivation, response to multiple cues, and initiation of social interactions. 

“There is a critical window of opportunity for children with autism up to the age of five that can improve their lives and developmental trajectory significantly – and science has known how to do it for more than four decades,” she says.

“But these therapies are not accessible to the children. There are long waiting lists, and once you get into therapy, you don’t get the best stuff science has to offer.

“So what I do is train the adults – therapists or parents – to provide the best developmental opportunities throughout the day and make these kids reach their full potential.”

Each lesson teaches a parent part of the strategy with an animated scene, and prompts them with questions to help it sink in.

They also watch videos of parents who have already participated in the program interacting with their kids, and rate their performance.

The most effective and innovative part of the app is the feedback parents receive on five key moments, each time they submit a video.

“What makes it very accessible is that you videotape yourself in a therapy session or interacting with the child at home, you upload it to the platform, and there’s AI that recognizes if you implemented it right or wrong, so it helps standardize the whole training system, and makes it cheaper.

“My algorithm can see if the child just vocalized, or really had a communication attempt. And then I can see if the adult reinforced the attempt, or what usually they do, which is give them another demand,” she tells NoCamels.

“That’s very intuitive, but it’s not helping the child, because it’s very hard for him to communicate. And if you give another demand instead of understanding him and reinforcing him, then he feels that communication is even harder than he thought it is.”

The feedback arrives within three days. Parents are asked whether they successfully implemented a certain method, and they are shown 10 seconds from their video demonstrating whether they did or didn’t.

After answering, the user receives the complete feedback from trained therapists and alumni.

“Our user has to think about whether they actually implemented the method correctly before they get the answer, and that really accelerates their learning curve, because you see moments of yourself and your child, and it’s much better than someone telling you what to do. You have to at least try to figure out if you did it right or wrong.”

SocialMind Autism offers training to parents who scored really well on the modules and want to join the community, so they can help improve the algorithm and give feedback to those that are struggling. 

Parents progress from one lesson to the next, but can repeat parts of the curriculum, and continue to upload videos and receive feedback to help their children improve if they are still delayed in speech and communication.

“Sometimes we encourage parents to go back to a previous module. Even if you’re good at what you’re doing, you need feedback on more scenarios and need to understand how to use these skills for other kinds of developmental goals or objectives.”

SocialMind Autism plans to eventually introduce a curriculum that helps parents potty train their autistic children, play with peers, and manage tantrums. 

“These are all strategies that are evidence-based, but are not available to the public. So like we did with language and communication we want to do with other other topics that children need to develop.”

“There are studies, even from Stanford, that parents learned these therapies faster than professionals, because they’re very highly motivated.

“You don’t need to be a professor in speech therapy or psychology to do it. It’s hard for everyone to change their behaviors, but it’s not an academic thing. It’s a motivation thing. And once it worked with my kids, I became a certified therapist and began training other parents remotely.”

Dr Sagiv Friedgut first founded her own telehealth service that she managed for seven years, where she went over videos parents sent in and gave them real-time, one-on-one feedback over Skype. 

Over time, she began to wonder how she could scale this treatment – and that’s how SocialMind Autism was founded. 

“Two per cent of young children are diagnosed with autism. And as a pediatrician, I see those numbers,” she says. “Every time there was a diagnosis, it was painful for me that this child would just go into a waiting list and he won’t get the best chance to progress. That was my drive.”

SocialMind Autism has so far helped over 60 parents teach their children to be more communicative. It is also selling its platform and training therapists at child therapy centers, hospitals, and clinics. 

The SocialMind Autism app is available to any parent who speaks and understands English or Hebrew.

There are many other methods of teaching PRT. Spectrum AI uses algorithms to train and supervise therapists in the US, but not parents. There are also companies that offer pre-recorded online training videos for parents. And experts, specialists, and providers offer direct PRT training, but they are not accessible to everyone due to cost. 

But Dr Sagiv Friedgut says that no other company provides video feedback to parents.

SocialMind Autism aims to train more therapists and parents, and increase its AI capabilities. Eventually, Dr Sagiv Friedgut would like to license the service to other countries, and expand to disabilities beyond autism.

Subscribe to NoCamels weekly newsletter and get our top stories