Turn Your iPad Into A 3D Scanner With The Structure Sensor

Even though augmented reality sounds very interesting, the concept is a lot more interesting on paper than in real life as it never managed to have the wow factor of other technologies. Augmented reality is mostly a gimmick and even when it’s not, there is not much use for it due to the fact that mobile cameras ‘see’ in two dimensions. Occipital, a scanning a computer vision company, aims to add the third dimension to your iPad’s camera, with its Structure Sensor product. The company raised money for the Structure Sensor via Kickstarter (managing to raise $1,290,439 with a goal of $100,000). The product shipped to backers in late 2013.

The Structure Sensor device has a price tag of $349 and is a camera very similar to Kinect; it is basically a 3D sensing camera (a 3D scanner) that clips on the back of your iPad (or any other tablet if you don’t mind a bit of hacking) and connects to your iOS tablet via the Lightning port. A 3D scanner is a device that analyzes a real-world object or environment to collect data on its shape and possibly its appearance like color. The data can then be used to construct digital, three dimensional models. 3D scanners are not just gimmicks; they are the true augmented reality as they can be used not only for entertainment but also in some industries.

The Structure Sensor comes with its own, internal battery and it can capture VGA pictures at up to 60 frames per second. If you are not a tech enthusiast that may not seem like much, but in fact 60 frames per second is a high standard that can guarantee that you will be able to 3D scan your room for example just by swinging your iPad around. The Structure Sensor will also allow you to play 3D augmented reality games (these games allow you to interact with real objects from your room / environment). With the Structure Sensor attached to your iPad (or other tablet) you will be able not only to scan 3D objects from your environment, but you will also be able to measure how far they are from the tablet, turn them into digital 3D objects or even measure the distance of a room.

Occipital, the company behind the Structure Sensor, is not an augmented reality game developer, nor an augmented reality apps developer. They are not a camera developer either, even though the Structure Sensor is basically a camera from a hardware point of view. Occipital’s goal is to make the Structure Software Development Kit that is bundled with the Structure Sensor 3D scanner, an  industry standard for camera makers and developers. That is a bold goal and it reminds us of James Cameron’s goal to make 3D an industry standard for the movie industry. Unlike 3D in movies though (which we found to be a gimmick in 90% of cases) 3D scanning is definitely more interesting; the Structure Sensor includes a 3D scanning feature as we mentioned above and it is a feature that Occipital demoed.

3D scanning is one of the simplest ways to use the Structure Sensor camera. Many stationary 3D scanners have started to appear in the 3D printer world, such as MakerBot’s Digitizer. And if you are interested in high detail, stationary scanners are the best available option, as the Structure Sensor doesn’t work properly with objects smaller than a medium sized plush toy while larger objects are overlaid with a low poly mesh that doesn’t allow too much detail. However, that’s the trade off for being cheap and very very fast. How fast? Well, it takes about 10 minutes to 3D scan a single object with MakerBot’s Digitizer while the Structure Sensor can 3D scan an object as fast as you can slowly walk around it.

The iPad’s camera will capture and overlay textures on models and the included software will allow you to seal gaps and then export the model either to use it as a 3D asset or for use with a 3D printer. For example, a statute will look very good as a 3D model after being scanned. A large plush toy will probably remind you of 3D graphics that video games used to have 15 years ago while a DSLR will look like a camera shaped blob after being scanned.

The 3D scanning feature of the Structure Sensor caught the attention of 3D System, a 3D printing company. 3D Systems and Occipital signed a deal that allows 3D Systems to commercialize a self branded version of Occipital’s Structure Sensor. And even though the augmented reality uses are in their early days, what the Structure Sensor offers is very interesting. For example, one of the demos is similar to augmented reality special effects apps; the demo will allow you to scan a part of your room and use it to throw balls for a virtual cat. If you throw the ball at a wall, the ball will bounce off and if you throw the ball under the table, the virtual cat will chase it.

Another demo scans the walls, floor and furniture inside your room, notes the dimensions (to scale) of each of them and then transforms it into a map. Of course, the idea of a virtual map is not new as Ikea already lets you test its (virtual) products inside a virtual map of your house. But with the Structure Sensor, you won’t need to guesstimate the scale. The Structure Sensor will even allow you to move and place scanned models of your own furniture together with new additions.

The platform is not perfect yet as it needs more polish. For example, the cat could go through chair legs (if they are not thick enough) and there are many similar glitches, but there is plenty of time for Occipital to iron out these bugs. Plus the company is still developing a way of getting its tracking mechanism to work with cylindrical objects. The Structure Sensor can be used in many ways and it comes with a SDK and many other resources that users will love; for example it comes with a normal USB cable (that you can use to hack it into working with other tablets than the iPad) and also a set of CAD plans that will help you 3D print your own accessories. It’s also worth noting that the ecosystem will dictate if the product fails or succeeds, so Occipital will depend on early adopters a lot.

Thankfully, the Structure Sensor is one of the most impressive mobile augmented reality projects in the past couple of years. The makers of the Structure Sensor got the idea for the project after trying Microsoft’s Kinect, a camera peripheral that allows users to interact with their console via a natural user interface (gestures and spoken commands). On their Kickstarter page the makers explain how they got the idea for the Structure Sensor: “In November 2011, we conceived of the idea for the Structure Sensor after playing with a Microsoft Kinect 3D sensor plugged into a high-end desktop CPU. We mapped a small indoor environment using the sensor and some prototype software. After tripping over the Kinect’s cord a few times, we realized that needing a high-end desktop CPU meant that this amazing technology was never going to make its way to everyday life. So we set out to create the Structure Sensor, and with it, a new chapter for mobile computer vision“.

You can already pre order the Structure Sensor from Occipital’s website; the finished product will ship this year, in April. What are your thoughts on the iPad mounted Structure Sensor from Occipital? Are you excited about its 3D scanning capabilities? Leave us a comment in the dedicated section below.