This week Apple introduced a new version of their popular iPad Pro, something they tend to do periodically. 

I use an iPad, so of course I perused the specifications of the new machine, since I always pretend I intend to upgrade. As usual, the new device has upped the specifications in typical ways. However, to my great surprise, the new iPad Pro includes LiDAR!

What is LiDAR?

LiDAR stands for “Light Detection and Ranging” and, according to Wikipedia, is: 

“Lidar is a surveying method that measures distance to a target by illuminating the target with laser light and measuring the reflected light with a sensor. Differences in laser return times and wavelengths can then be used to make digital 3D representations of the target. The name lidar, now used as an acronym of light detection and ranging (sometimes, light imaging, detection, and ranging), was originally a portmanteau of light and radar. Lidar sometimes is called 3D laser scanning, a special combination of a 3D scanning and laser scanning. It has terrestrial, airborne, and mobile applications.”

This is essentially a sophisticated depth sensor, in addition to the normal imaging cameras beside it. The imaging cameras are a normal 12M pixel and a 10M pixel wide angle. 

The LiDAR sensor is an entirely new animal for Apple, and likely any consumer handheld device. Apple explains: 

“LiDAR (Light Detection and Ranging) is used to determine distance by measuring how long it takes light to reach an object and reflect back. It is so advanced, it’s being used by NASA for the next Mars landing mission. And it’s now been engineered to fit in the thin and light iPad Pro.

The custom-designed LiDAR Scanner uses direct time of flight to measure reflected light from up to five metres away, both indoors and out. It works at the photon level, operates at nanosecond speeds, and opens up tremendous possibilities for augmented reality and beyond.”

Traditionally LiDAR sensors have been large affairs, certainly of dimensions vastly bigger than your pocket. In the past they’ve been installed on aircraft to perform ground surface mapping, for example. But recently there apparently have been some small LiDAR sensors placed on the market and it seems that Apple must have selected one for their new iPad.

I am unable to find any specifications about this sensor. I do not know, for example, how many pixels it can create for a given 3D scene. One thing we do know is that it has an effective range of 5m, which is quite a bit farther than many handheld 3D scanners available today. 

LiDAR iPad Applications

What could this sensor be used for? It’s highly likely Apple intends to use the additional 3D information to enhance their VR and AR capabilities. 

In AR mode the iPad could display a live view of the surrounding area, but “annotate” it with any interesting information or graphics. However, to do so the software must have a good understanding of the three-dimensional construction of the scene, and that’s likely how LiDAR sensor data would be used. 

LiDAR Scanning Possibilities

My interest in this development is for 3D scanning applications.

It now seems that a very high-end mobile device, the new iPad Pro, includes a 3D depth sensor. Apple’s role is to create APIs in their software system to allow developers to gain access to the 3D data collected and make use of it in the applications they write. 

Many of the software tools accessing the 3D data will certainly be VR and AR applications, as that was Apple’s intention. It’s quite likely some developers have been using the new system quietly already with Apple’s help.

But another type of application could be built to use the LiDAR sensor: 3D scanners. 

It’s very possible — and probably already begun — that developers will create applications that are essentially 3D scanners. One could then simply call up the scanning app and then record a scene or object. The software could then process the data to transform it into a proper 3D model, ready for 3D printing. 

Geometric information will certainly be gathered from the sensor, but LiDAR provides only depth information; meaning a LiDAR-only device could produce texture-less 3D models. However, Apple has those twin imaging cameras that can be leveraged by the software to map a color texture on the geometric information. In other words, the iPad Pro, with appropriate software, could become a mobile color 3D scanner! 

I’ve seen no announcements from scanning providers regarding new apps, but they were likely as surprised as I was when the announcement came. But I’m sure they’re quite busy now. 

Via Apple