Popular 3D Scanner, Polycam, Uses AI to Complete the 3D Picture

A second look at Polycam, in a conversation with company cofounder Elliott Spelman.

Polycam, the most popular scanning app on the Apple store—and perhaps in the world—has found an ingenious way to use AI in its scanning. Polycam users will be able to create a totally immersive view with the ease of creating a panorama. The normal process for getting an immersive view involves moving the camera up and down while rotating, what another iPhone-using scanner refers to as a “painting the wall” approach. But Polycam is able to take the panorama and guess the rest, as it were.

You can get a full 360-degree view of an outdoor scene in less than a minute.

Polycam uses a process it has named “stable diffusion complete,” which is based on “stable diffusion,” the technology similar to one used by media sensation, DALL-e, to generate fanciful pictures, except stable diffusion uses your computer rather than the cloud. To use stable diffusion, users must have a GPU and 8GB of VRAM. While DALL-E and stable diffusion generate images, inpaints and outpaints (more on that later), Stable Diffusion Complete appears to generate a picture based on the image itself, essentially guessing what the rest of the sky or ground would look like, and turns what is normally a low-resolution thin strip of images stitched together and puts you in the center of a full resolution, 360-degree view about the vertical axis (that’s the yaw axis if you are flying a plane), where you can also have a full 180-degree up and down view (the pitch axis).

Meeting Elliott

If Elliott Spelman, cofounder of Polycam, doesn’t strike you as the typical Silicon Valley startup founder, it’s because he is not. Spelman is a designer by education with a master’s in fine arts from Stanford. Being at the helm of a cutting-edge imaging technology is fun for Spelman.

Spelman wants scanning to be fun for everyone—not just professionals with expensive LiDAR rigs. We watch Spelman’s short instructional video that demonstrates how simple it is to take a full 360-degree image.

Sure enough, that is all that is needed to record a full 360-degree immersive image. Polycam may have ruined panoramas forever. After seeing the big picture that Polycam can create, a breathtaking sky-to-ground and all-around seamless (almost) shot, could you ever go back to a thin, distorted, low-resolution image you got from Apple?

Read the rest of this story at ENGINEERING.com