AI Meets 3D Scanning: How Scan to 3D Could Transform the 3D Printing Process

I’ve realized there is yet another way to leverage AI for 3D printing that might be easy to do.

AI tools are exploding. There are now countless small utilities, such as the amusing Memecam, which accepts an image and generates a usually appropriate and funny meme. There are plenty of practical applications as well, with one being the “spaghetti detector” systems that now automatically monitor 3D print jobs for failures.

Text to 3D and Reverse Engineering Challenges

There are even several projects working on the idea of “text to 3D”, similar to the “text to image” applications that have become very popular. I’ve written previously on the “text to 3D” concept, which would revolutionize CAD and 3D design.

However, text to 3D is a pretty big challenge, but there could be one that might be a lot easier to create: Scan to 3D.

Introducing Scan to 3D

What do I mean by “Scan to 3D”? It’s really a way to break down the complex workflow of reverse engineering, which normally looks like this:

• Scan a part that is to be 3D printed
• Import the rough scan into a CAD tool
• Rebuild the design by “tracing” over the scan
• Throw away the scan
• Export the CAD design as a 3D model
• Print it

Most of these steps are pretty straightforward, except for the “rebuild the design” step, which can take a very long time if the geometry is complex. In fact, you have to be a pretty handy CAD operator in order to do that step at all, which is almost always the stumbling block for reverse engineering.

The Role of AI in Scan to 3D

It goes something like this:

• Hey, I have this random part, can you print me a copy?
• Do you have a 3D model?
• No, how do I make one?
• You have to hire someone to do it because you don’t know how, and it’s expensive
• Ok, forget it.

I can’t tell you how many times this scenario has unfolded before my eyes, and it’s a problem.

The reason for all this trouble is that 3D scanners are not as accurate as required. Consider the capture of a supposedly flat surface by almost any 3D scanner:

As you can see, it really isn’t flat in the capture. There are bumps and wiggles. This is why the reverse engineer must do their work: they replace that wobbly surface with a perfectly flat plane in the CAD tool.

Adapting AI Technology for 3D Scanning

That’s the core of this workflow issue. If only the scanner could produce perfectly smooth surfaces.

But hold on, maybe this is actually possible!

In the AI-image processing space, there have been many new advancements in recent weeks. This is done using a tool called “ControlNet” to constrain the image generation process to match an input image.

Here’s an example, where someone decided to make the Wendy’s logo “real”:

It’s certainly amazing technology, but could it be adapted for 3D scan use?

Imagine a system that is trained with images of a wide variety of surfaces and their corresponding scan results. Such a system just might be able to automatically smooth out those rough 3D scans in seconds.

Potential for Quick 3D Printable Models

It may thus be possible to literally 3D scan an object and have a completely 3D printable model ready in minutes.

That is, only if someone is actually working on this concept. If you are, please give us a shout, we’d like to talk to you.