AI Makes High-Quality 3D Images From Single Line of Instruction

Are there no limits to the ingenuity of AI? It can now create a high-definition 3D image almost instantly, from a single sentence of simple instructions.

“A modern bedside lamp with an egg-shaped marble body and a cone-shaped fabric shade,” for example.

Press the “generate” button and 35 seconds later you have your lamp. Want to change colors, tweak the design or alter the shape? Give the AI another prompt and the job’s as good as done.

So how does 3DFY achieve such high quality? “We are using fundamentally different technology,” says Dahan. “We traded off the quality for or the type of content that can be created.

This is 3DFY, an Israeli startup, in action. Demand for 3D images is growing exponentially – for gaming, online retail, virtual and augmented reality (like Apple’s new Vision Pro) and the Metaverse.

But until now it’s been an expensive and labor-intensive process. Creating the bedside lamp can take anything from a few hours to several days, depending on the complexity.

3D artists today have to scan an existing object, using hundreds of photographs from all angles, or a 3D scanner.

“We have developed a novel AI pipeline that can generate high quality 3D content, using text prompts,” says Eliran Dahan, the company’s CEO. “And in the near future, also images.”

So there’s no longer the need for an actual, existing object as a starting point. The 3D image can be something that exists only in your imagination. (You can try it here)

It’ll come as no surprise that 3DFY, a small startup based in Haifa, isn’t alone in using AI to turn text into 3D images.

It’s up against tech giants OpenAI (developers of ChatGPT), Google and Nvidia (with 2023 revenue of almost $27 billion) who are all developing their own ways of doing it.

But while they’re chasing breadth, says Dahan – the ability to produce a 3D image of any object on earth, or beyond – 3DFY goes for depth.

It limits itself to a very narrow, and seemingly odd, range of objects – table lamps, sofas, tables, ottomans, cutlery and swords – but does them to such a high standard that it’s an attractive proposition for businesses that want 3D images of those particular objects.

Rivals produce a “cool toy,” says Dahan. “They can create pretty much any piece of content, they can do all kinds of wacky things. But whatever they produce is very low quality from a technical standpoint, meaning that files themselves 3d models, they cannot be edited, their materials look very poor.

3DFY encourages professionals to play with its service for free to see how it works, and to pay only if they want to own and download the image.

“However, the 3D models are very expensive, so using our web service at just a few dollars per one model, is very affordable.”

If the model’s not exactly what you want, the user can tweak the text prompt – the instructions they provide – until they achieve perfection.

“Our solution is category-bound, meaning that we can only generate 3D models from certain categories, for example, all kinds of furniture categories,

“And we have some gaming categories, like shields and swords, and working on axes, and stuff like that.

“Now that we have nailed down the technology, we are expanding the category selection further and further.

“I’m the last one to claim that I can do stuff that OpenAI or Google researchers cannot do. However, with the current technology they’re using I think it’s unlikely they’ll achieve this, so they’d need to pivot to fundamentally different technology.”

Both Dahan and Tal Kenig, the company’s vice president (research and development) have a background in medical imaging. They went to the same high school, and both studied at The Technion – Israel Institute of Technology, in Haifa.

“We’re both 3D natives, having worked in radiology and medical imaging of the human body. We tend to think in 3D,” says Dahan.

“We met a common acquaintance in the 3D modelling field and we learned that everything was done manually. To be honest, we didn’t believe it a first. We realized it costs a lot of money to do it that way. And we realized there is like a really, super high demand for 3D models.

“We understood immediately that there is, you know, contemporary technology that could be applied to this problem. We set out to develop an AI technology that can generate very high-quality 3D models.”

Their focus is on machine learning and computer graphics, but they say most large companies neglect the graphics side.

“What we’re aiming to do is to be the largest 3D model generation company in the world.  We now have seven categories.

“Think about what’s going to happen when we have 700 categories. You would find pretty much every item in the whole wide world.

“You could furnish entire, you know, houses or environments with you know, streetlights, trees, houses, cars, shoes, glasses, whatever.”

Subscribe to NoCamels weekly newsletter and get our top stories