Machine vision has long been a cornerstone of industrial automation, with 2D imaging leading the charge. While 2D technology remains relevant for specific tasks, the future of machine vision lies in the third dimension. 3D sensing, capable of capturing the intricate depth and form of objects, is transforming industries and unlocking new possibilities.

Think of it this way: a 2D camera can read a QR code with ease, but it can’t guide a robot through a complex warehouse environment. In practice? 2D cameras just won’t enable robots to pick and place objects in the correct orientation or avoid obstacles. That’s where 3D vision comes in, providing the depth perception necessary for navigation and intricate manipulation.

From precise metrology to cutting-edge robotics, 3D vision is driving innovation across various sectors. Let’s delve into the world of 3D sensing technologies, as explained in the paper by our co-founder, Tomas Kovacovsky.  

A Spectrum of 3D Sensing Technologies

The market offers a diverse array of 3D sensors, each with its own strengths and weaknesses. Here are some of the most prevalent technologies used in industrial automation today:

• Laser Triangulation: This technique involves projecting a laser line onto an object and capturing the reflected light with a camera. By analyzing the deformation of the laser line, the sensor can reconstruct the object’s 3D profile.
• Photogrammetry: This method utilizes multiple 2D images taken from different angles to reconstruct a 3D model. It’s akin to how our two eyes perceive depth.
• Stereo Vision: Similar to photogrammetry, stereo vision employs two cameras to capture images from slightly different perspectives. By analyzing the disparities between the images, depth information can be extracted. Stereo vision can be passive (using ambient light) or active (using projected patterns).
• Structured Light: This technique projects patterns of light (like grids or stripes) onto an object. The distortions in these patterns reveal the object’s shape and depth. This can be achieved with a single frame or multiple frames for higher accuracy.
• Time-of-Flight: These sensors measure the time it takes for light to travel to an object and back. By analyzing the time-of-flight, the sensor can determine the distance to the object. Time-of-flight can be implemented as area scan sensors or in LiDAR systems which excel in long-range applications.

On The Quest for the Optimal 3D Sensor

While each technology boasts unique advantages, there’s no one-size-fits-all solution. The ideal 3D sensor depends on the specific application and its requirements. Factors like accuracy, speed, range, and cost all come into play.

Moving forward, we’ll delve deeper into the key parameters that differentiate 3D sensing technologies. We’ll explore the trade-offs involved in optimizing each parameter and provide a framework for comparing different technologies.

Choosing the Right 3D Vision System: Key Parameters to Consider

Let’s dive deeper into the crucial parameters that differentiate these technologies and help you choose the perfect 3D vision system for your needs.

1. Operating Volume: Size Matters

The operating volume refers to the 3D space within which the sensor can effectively capture data. This parameter is critical, as different applications demand vastly different operating volumes. A metrology system might only need to scan a small object (e.g., 100mm x 100mm x 20mm), while a bin-picking robot requires a much larger range (e.g., 4 – 5 m³).

To categorize 3D vision systems based on their depth of field, we can define five levels:

• Very small: up to 50 mm
• Small: up to 500 mm
• Medium: up to 1500 mm
• Large: up to 4 m
• Very large: up to 100 m

2. Data Acquisition and Processing Time

In 3D scanning, time is of the essence. “Data acquisition time” refers to the time it takes for the sensor to capture the necessary data. “Data processing time” is the time required to transform that raw data into a usable 3D model.  

Light: Capturing enough light is crucial for a high-quality 3D scan. This can be challenging in applications with limited acquisition time.

• Motion Capture: Some applications require scanning objects in motion (e.g., on a conveyor belt). “One-shot” technologies, which capture all the necessary data in a single frame, are ideal for such scenarios.  
• Real-time vs. Offline: Consider whether your application needs instant results (e.g., robotics, sorting) or can tolerate some delay (e.g., offline metrology).

Here’s how we can categorize data acquisition and processing time:

Data acquisition time:

• Very high: minutes and more
• High: ~5s
• Medium: ~2s
• Short: ~500 ms
• Very short: ~50 ms

Data processing time:

• Very high: hours and more
• High: ~5s
• Medium: ~2s
• Short: ~500 ms
• Very Short: ~50 ms

3. Resolution of 3D Sensors: Capturing the Details

Resolution refers to the system’s ability to capture fine details. High resolution is essential for applications involving small features within a large operating volume.

• The Light Trade-off: Increasing resolution often means reducing the amount of light reaching each pixel, which can impact image quality. 
• XY-Resolution: This refers to the resolution in the X and Y directions, often represented by the number of 3D points per measurement. We can categorize it as follows:
  
  • Very small: ~100k points
  • Small: ~300k points (VGA)
  • Medium: ~1M points
  • High: ~4M points
  • Extended: ~100M points
• Z-Resolution: This refers to the resolution in the Z-direction (depth). Some technologies are more scalable than others in achieving high Z-resolution.
  
  • Very small: >10 cm
  • Small: ~2 cm
  • Medium: ~2 mm
  • High: ~250 um
  • Very high: ~50 um

4. Robustness: Handling Challenging Environments

In 3D scanning, robustness signifies a system’s capacity to deliver high-quality data consistently across diverse conditions. This involves overcoming two primary challenges:

1. Ambient Light Interference:

Many 3D scanning systems rely on either external light sources or their own active illumination. However, ambient light (from sunlight, indoor lighting, etc.) introduces noise that can degrade data quality. This occurs because ambient light increases the intensity values recorded by the system’s sensors, leading to higher measurement noise.

A significant contributor to this noise is shot noise, a fundamental quantum phenomenon. Shot noise arises from the discrete nature of light, where photons arrive randomly at the sensor. The more photons hit a pixel, the higher the uncertainty in the measurement. This uncertainty, mathematically represented as the square root of the average photon count, becomes problematic when ambient light overwhelms the system’s active illumination. In essence, the signal-to-noise ratio decreases, jeopardizing the accuracy of the final 3D data.

While techniques like black-level subtraction attempt to mitigate the impact of ambient light, they offer limited effectiveness against shot noise. True robustness demands solutions that minimize the influence of ambient light at the hardware or data processing levels.

2. Material Variations:

The ability to accurately capture a wide range of materials is crucial for a robust 3D scanning system. Challenges arise from varying levels of light interaction:

• Diffuse, well-textured materials (e.g., rocks): These scatter light evenly, making them relatively easy to scan.
• Diffuse materials (e.g., white wall): These also scatter light, but their lack of texture can pose challenges for some systems.
• Semi-glossy materials (e.g., anodized aluminum): These materials exhibit a mix of diffuse and specular reflection, requiring careful consideration of light angles.
• Glossy materials (e.g., polished steel): Strong specular reflections can cause data loss or artifacts in the scan.
• Mirror-like surfaces (e.g., chrome): These pose the greatest challenge, as they reflect light almost perfectly, making it difficult to capture the surface geometry.

Robustness in this context necessitates advanced algorithms and scanning techniques that can handle interreflections, multiple scattering events, and varying levels of specularity to accurately reconstruct the 3D geometry of diverse materials.

Environmental Conditions Impacting Robustness:

The following conditions illustrate the range of ambient light challenges a robust 3D scanner should ideally handle:

1. Indoors, dark room: Minimal ambient light, ideal conditions.
2. Indoors, shielded operating volume: Controlled environment with minimal external light interference.
3. Indoors, strong halogen lights, and milk glass windows: Significant ambient light with potential for glare and uneven illumination.
4. Outdoors, indirect sunlight: Moderate ambient light with varying intensity.
5. Outdoors, direct sunlight: High-intensity ambient light poses significant challenges.

Enhancing Robustness:

Achieving true robustness requires a multi-pronged approach:

• Advanced sensor technology: Sensors with high dynamic range and sensitivity can better handle varying light conditions.
• Effective filtering techniques: Hardware and software filters can help to minimize the impact of ambient light.
• Robust algorithms: Algorithms that can account for interreflections and varying material properties are crucial.
• Adaptive illumination: Systems that can adjust their illumination based on the environment and material can improve data quality.

By addressing these challenges, 3D scanning systems can achieve a higher level of robustness, enabling accurate and reliable data acquisition across a wider range of applications and environments.

5. Weight: Keeping it Light

The weight and size of the 3D vision system can be a significant factor in certain applications, especially those involving mobile robots or limited mounting space.

Lightweight Design: Compact and lightweight systems offer greater flexibility and ease of integration.

We can categorize systems based on their weight:

• Very heavy: >20 kg
• Heavy: ~ 10kg
• Medium: ~ 3kg
• Light: ~ 1 kg
• Very light: ~ 300 g

6. Budget: Finding the Right Price/Performance Value

The cost of a 3D vision system can vary significantly depending on the technology, features, and level of support. It’s crucial to balance the cost of the system with the value it brings to your application.

We can categorize 3D vision systems based on their price:

• Very high: ~100k EUR
• High: ~25k EUR
• Medium: ~10k EUR
• Low: ~1000 EUR
• Very low: ~200 EUR

By carefully considering these parameters and understanding the trade-offs involved, you can choose the 3D vision system that best meets the unique needs of your application.

A Deep Dive into 3D Vision Technologies: Understanding the Options

Now that we’ve explored the key parameters, let’s examine the specific 3D vision technologies available and how they stack up. We can broadly classify these technologies into two groups: those based on triangulation and those utilizing the time-of-flight principle.

Time-of-Flight (ToF)

TOF systems measure the time it takes for light to travel to an object and back to the sensor. This simple principle enables two distinct approaches:

1. LiDAR 

These systems capture one or a few 3D points at a time, scanning the entire scene by changing the sensor’s position or orientation. LiDAR excels in long-range applications, like autonomous navigation, but can be slower for detailed close-up scanning.

2. Area Sensing ToF 

These systems use specialized image sensors to measure the time-of-flight for multiple points simultaneously. They are faster than LiDAR and suitable for dynamic applications, but often have lower resolution and struggle with interreflections (light bouncing between objects).

Triangulation-Based Systems

Triangulation-based systems rely on observing a scene from multiple perspectives to determine depth.

1. Laser Triangulation (Profilometry) 

This technique projects a laser line onto an object and captures the reflected light with a camera. By analyzing the deformation of the laser line, the sensor reconstructs the object’s 3D profile. It’s accurate and widely used but typically requires scanning motion.

2. Photogrammetry 

This method uses multiple 2D images taken from different angles to create a 3D model. It’s versatile and can leverage high baselines (distance between viewpoints) for improved accuracy, but it relies on sufficient object texture and computational processing.

3. Stereo Vision 

Stereo vision mimics human depth perception by using two cameras to capture images from slightly different perspectives.

• Passive Stereo Vision: Relies on existing object texture and ambient light. It’s suitable for non-measurement applications like people counting but struggles with textureless surfaces.
• Active Stereo Vision: Overcomes the limitations of passive stereo by projecting structured light patterns to create artificial textures. This enhances accuracy and robustness.

4. Structured Light 

This technique projects patterns of light (grids, stripes, etc.) onto an object and analyzes the distortions to extract 3D information.

• Multi-Frame Structured Light: Captures multiple frames with different patterns for high accuracy and resolution. It’s commonly used in static scenes and metrology applications.
• Single-Frame Structured Light: Encodes all necessary information into a single pattern for faster acquisition, making it suitable for dynamic applications.
• Advanced Structured Light: Photoneo’s innovative “Parallel Structured Light” technology combines the high resolution of multi-frame systems with the speed of single-frame acquisition, pushing the boundaries of 3D sensing.

Parallel Structured Light in 3D Sensing

Imagine a 3D camera that can capture crisp, detailed images of objects moving at 140 km/h. This is now a reality thanks to Parallel Structured Light, a groundbreaking technology that’s transforming machine vision.

Unlike traditional structured light systems that rely on sequential projection patterns, Parallel Structured Light utilizes a unique CMOS image sensor with a mosaic pixel pattern. Instead of modulating the projected light, this innovative approach modulates the sensor’s pixels themselves.  

Here’s how Parallel Structured Light works

1. Continuous Projection: A laser projector illuminates the entire scene continuously.  
2. Pixel Modulation: Individual pixels within the sensor are rapidly switched on and off during the exposure, creating unique coded patterns.  
3. Simultaneous Capture: These coded patterns are sampled simultaneously within a single frame, effectively “freezing” the moving object in time.  

This revolutionary technique enables high-quality 3D capture of dynamic scenes, opening up a world of possibilities for automation.  

Applications of Parallel Structured Light

• Robotics: Robots can now accurately perceive and interact with moving objects, enabling applications like:
  
  • Object Handling: Picking, placing, sorting, and manipulating items on fast-moving conveyor belts.
  • Bin Picking: Identifying and grasping randomly oriented objects in dynamic environments.  
  • Machine Tending: Loading and unloading parts from moving machinery.  
  • Palletization/Depalletization: Building and dismantling pallets with moving boxes.  
• Quality Control & Inspection: Inspecting products for defects in real-time, even on high-speed production lines. 
• Metrology: Accurately measuring moving objects for dimensional analysis.

Photoneo has pioneered this technology with their MotionCam-3D camera, capable of capturing moving objects at incredible speeds with high resolution (0.9 MP for moving objects, 2 MP for static). This breakthrough is empowering industries like manufacturing, automotive, logistics, e-commerce, and healthcare to automate complex tasks that were previously impossible.

Choosing the Right 3D Sensing Technology

Parallel Structured Light is a game-changer in the world of machine vision. By capturing dynamic scenes with unprecedented accuracy and speed, it’s unlocking a new era of automation and robotic capabilities. As this technology continues to evolve, we can expect even more innovative applications that will further transform industries and redefine what’s possible.

The ideal 3D vision technology depends on your specific needs and application. Consider the key parameters discussed earlier, including operating volume, speed, resolution, robustness, weight, and budget. Each technology offers a unique set of advantages and trade-offs.