Crazy Idea: AI-Based 3D Model Generation For Consumers

I watched an interesting video about face generation and had another crazy idea: generating objects. 

The video describes research undertaken to generate 2D images of a particular type of objects. Basically, the researchers designed a system that could be trained on a large set of images of a single type of object, say, horses, and then was able to automatically generate an image of a horse in incredible detail — but of a horse that never existed. 

Don’t believe me? Check out the images at top. 

They are not real people. They were generated by a system, StyleGAN 2. 

There have been significant advancements in image recognition over the past few years. In this image you can see how an image recognition system can examine the image and build a proper sentence describing what’s happening in the scene. 

The image was from this video, which continues on to show how images of this type can be generated automatically by new algorithm. 

Another video shows the process in a bit more detail. They are apparently able to generate highly realistic images of cars, churches, horses, cats (of course) and human faces. 

Faces are incredibly complex and yet the algorithm manages to create images that are essentially unrecognizable as fake to the human eye. 

Fascinating stuff, to be sure. But then I had a thought. 

One of the key problems that faced those promoting the consumer 3D printing revolution a few years ago was the issue of 3D content. While a consumer could purchase a 3D printer, and with some significant training could occasionally operate them, there was nothing in particular to do with the devices. What would a consumer 3D print? 

Yes, there were some online repositories of 3D printable objects, but one quickly tires of producing yet another plastic dragon for the bookshelf. There needed to be a “killer application” for consumer 3D printing, and none ever appeared. Those companies manufacturing the 3D printers eventually pivoted into more profitable markets. 

But then there’s these videos. 

The researchers chose to train their algorithms on popular image scenes, such as cats and faces. But what if they were to train them on mechanical objects? 

What if you had a system that could recognize, say, bolts. A system that could determine dimensions, threads, head style, and other attributes. Surely this would be possible as bolts are not nearly as varied as human faces. 

If you could do this, then presumably the system could generate images of bolts on demand. Perhaps a hex-head M10 bolt? Or a square head wood screw? 

Now, imagine further: instead of generating a 2D image of the bolt, this hypothetical system could generate a 3D model. 

A printable 3D model.

Finally, imagine this type of system being developed not just for bolts, but for each of a series of common objects like handles, adapters, covers and countless others objects. 

That might be a solution to the 3D content problem we’ve all been seeking for years. It would take a considerable amount of work to produce, obviously, but if researchers are able to produce the faces above, I cannot imagine why this would not work on bolts or handles. I think it’s just that no one has tried. 

Will someone secure some venture capital and attempt to do this? 

Has someone already done so, but not yet announced the work? 

I would not be surprised if this is in a lab somewhere. 

Via Arxiv (PDF)