Hands-Free: Driverless Software That Thinks Like A Human

AI system will go live on buses in Tel Aviv and Nahariya next year

I’m sitting in one of the only driverless cars that’s allowed on Israel’s road. It’s waiting at a junction, scanning the traffic until it spots a gap. Then the steering wheel turns, untouched by human hand, and the Kia Niro edges out onto a busy street in Haifa.

On the screen in front of me I can see feeds from five roof-mounted cameras. Rectangles of different colors indicate vehicles, pedestrians, trees, lamp posts and other obstacles.

We’re doing 15km/hour along a narrow street when a driver cuts us up. Although we’re driving autonomously, we have Amir, the “safety driver” just in case (and as a legal requirement).

His hands are hovering over the steering wheel, but he doesn’t intervene. When I replay a video of our drive later, I see the system had already marked the car with a red rectangle car five seconds before our encounter and slowed down. It has eyes everywhere and human interventions are a rarity.

The software that’s powering our drive is unlike almost any other currently being developed. It is mapless.  Every other autonomous driving system in the world, except for Tesla, guides vehicles with high-definition (HD) maps.

Imagry, an Israeli startup, takes a different route.  Its developing software that learns how to drive like a human, replicating our reactions and decisions – but not our impatience, our risk-taking or our tendency to become distracted.

It is constantly refining that software, and very soon it will be integrated into cars, buses and more.

The AI Driver has over 100,000 miles of driving experience and has analyzed over 150 million still images, taken from video, of on-the-road situations over the last four years.

Instead of following a set of rules, designed to cover every eventuality, like other systems, it learns, like humans, from experience.

The car I’m in is gathering data, and storing it in an over-sized computer in the trunk, to be analyzed in detail by a team of data annotators.

“They mark the car’s mistakes and tell the system how to correct them,” says Eran Ofir, the company’s CEO. “It’s like a child learning how to avoid similar mistakes in the future.”

The system is built on neural networks, a series of algorithms that  mimic the way our brains work.

“The code we write is not what’s called rule-based code, meaning it’s not built like a decision tree, in which we are trying to predict episodes on the road,” says Ofir.

“Instead, it’s like a black box, we see the input, we see the output, and the system teaches itself. There is a computer vision system that needs to identify and classify objects as is as it is it drives in real time. 

“Everything needs to happen in in a few milliseconds, and then respond accordingly to what’s happening on the road.”

He says vehicles that use standard HD mapping, rather than his mapless system, don’t “think” for themselves. So they can freeze if it encounters flooding, road construction or other unexpected events.

“You can see videos on YouTube channels of a car stopping, signaling to the control center that they’re stuck, and then the control center remotely takes over, and brings them to a safe place,” says Ofir. 

Imagry is also testing its software on roads in San Jose, California, USA and Frankfurt, Germany, and says it can go off-road because it doesn’t rely on maps.

It needs to learn foreign road signs and to observe local traffic customs, such as “right on red” in the US. And so far it’s also limited to countries that drive on the right, and needs adapting for the UK and elswhere.

The Imagry car has its own dedicated “autonomous car” parking space on a road near the company’s offices and drives around the city every day, gaining more and more experience. Every day the team of annotators back at headquarters analyze and improve its performance, ahead of two trials involving electric buses that are due to start very soon.

In September, Imagry received funding from the Israel Innovation Authority to carry out an innovative pilot, starting in January 2023, running an autonomous, driverless shuttle at the Sheba Medical Center, in Tel Aviv.

And earlier this month, Israel’s Ministry of Transport announced that it would take party in one of four autonomous bus trials across the country. Imagry’s technology will be used by bus company Nateev Express on its number 5 route in the northern city of Nahariya in a six-month trial due to start by June 2023.

The buses will be marked as autonomous and they’ll have a safety driver sitting at the wheel. Passengers will press a button or use an app to stop the bus – but will no longer be able to shout for the driver to stop as the bus pulls away.

Imagry says its tech is road-ready but regulations are slowing things down. Governments are cautious about allowing truly autonomous vehicles  – with no safety drivers – onto their roads.

In the meantime, Imagry is refining its technology for the day when that does happen.  Next on its agenda is urban valet parking.

If you struggle to find on-road parking in a crowded city, your car will drop you at home and find a free space all by itself, no matter how far it needs to go. When you next need the car, you’ll simply send a message asking it to come and collect you.

Subscribe to NoCamels weekly newsletter and get our top stories