Multimodal Laser Triangulation for Industrial Reverse Engineering and Quality Control
Introduction
Measurement can be regarded as a reversed engineering process that starts from the final product and produces a description resembling the original design. The difference between the original design and this description is that the former also contains tolerances, ranges within which the product’s dimensions must remain for the product to be accepted. For quality control to be as complete as the design of the product, it needs to be performed in three dimensions, similarly to the design. Three-dimensional quality control for its part needs three-dimensional shape measurement and reverse engineering.
A simple non-contact method called laser triangulation, or laser scanning, has been around for decades, but more industrial implementations have not emerged until relatively recently. Laser triangulation is based on the idea of actively illuminating the measured object using a structured light source (usually laser that produces one or many bright lines) and using a camera or similar sensor so that the diffusely reflected light is recorded and the crosssectional shapes of the object are revealed. Figure 1 illustrates the measurement principle, which relies on the ‘clean’ reflection of the laser light from the measured object. The laser, aligned at angle ?, illuminates the measured object, and the sensor sees the reflected light at angle ?. Along with sensor alignment angle a and baseline distance b, distance d can be solved.
Measurement configurations and their properties
Both the sensor and the laser can be placed in almost any angle for performing the measurements. There are four basic configurations into which all geometries can be divided. Namely, SVLD, SDLV, and SDLD which contains two subgroups. The configuration names here refer either to vertically or diagonally (subscript V or D) aligned sensor and laser (capital S and L).
SVLD-configurations in which the sensor is vertically aligned and the laser projector is aligned diagonally, have often been referred to as the most intuitive configuration because the sensor is aimed directly at the measured object. If the sensor is, for example, a camera, it can be used for other kinds of measurements as well – with minimal viewpoint-originated distortions. Inversely to the previous ones, in SDLV-configurations the laser is aligned vertically and the sensor is aligned diagonally. Some sources also mention these configurations as being “reversed”. An immediate advantage of these geometries is that height measurements are always performed at the same position x-directionally. Thus, the measurements are always evenly spaced, if constant scanning velocity and constant measurement frequency are assumed.
When both the sensor and the laser projector are aligned diagonally, we can distinguish between two subgroups: ones where the measurement ray and the sensor optical axis cross under the baseline (also called “specular” configurations) and ones where they cross outside the baseline distance (also called “away look” configurations). Specular configurations are suitable for measurements where objects are dark or matte, only low laser output power can be used, or the spectral responsivity of the sensor matches poorly with the laser wavelength. Away look configurations are more extreme cases of SDLV, which means that the measured surface has to be diffusively reflective or the sensing system has to be able to detect even the smallest amounts of stray light – preferably both. Multiple reflections can cause problems if there are vertical edges on the measured object.
 
In practice, the accuracy and data quality of the laser triangulation measurement is a result of a number of factors. Most of the problems are caused by shadow regions where the laser light is occluded behind the measured object where the camera cannot see it. This is the main cause of incomplete measurement data or ”holes” in the result data sets. Another reason for incomplete data sets are specular surfaces that cause nearly or all of the reflected light to bounce to some other direction than to camera. The problem gets worse as better surface qualities need to be measured. The problem is eased by looking at the surface more closely: bigger magnifications reveal smaller surface roughness causing at least some diffuse reflection. Also surface shape and reflectance variations and the finite minimal width of the laser line can cause additional troubles. As the structured light in practice always illuminates at least a small area patch of the surface, the underlying surface geometry and reflectance has a direct effect on the dimensions of the reflection to the camera. This causes errors in measured shapes near object edges, other sharp shape variations, and surface brightness changes.
The science and industry have discovered various methods to solve the abovementioned problems. The first solution is to use consecutive triangulation units with differing triangulation angles along a conveyor line. The obvious drawback with this solution is the amount of floorspace and hardware needed. Moreover, more time is required for the whole measurement chain to execute. During this time, nothing guarantees that the end product remains unchanged for the measurements to be comparable. Another conventional solution uses one laser projector and multiple cameras to view the illuminated cross-section. The total costs, however, can be high as cameras are rather costly units and, in addition, they also require their own data processing units. The final downside comes from the fact that the illuminated surface needs to be visible from several directions.
The third choice would be to use several laser projectors to illuminate different parts of the field of view of one camera, technique also sometimes called windowing. Laser projectors are cheaper than cameras and they require only a minimal number of additional components. This method usually enables combining up to three laser projectors to one field of view. The worst disadvantage is that a cross-section illuminated with one laser projector can be allowed to occupy only a portion of the field of view. Otherwise the view may become too confusing to solve: which illuminated part in the view has been illuminated with which projector? Therefore, one laser projector (triangulation setup) can only use a subset of the whole field of view of each camera, and measurement ranges are shortened or resolutions decreased.
It is quite obvious that no triangulation setup with one single measurement ray, viewed from one direction only, is able to measure all surfaces. Either the surface geometry occludes the ray and thus renders it invisible, or the surface is so ill-formed that no sensible assumptions can be made about the cross-section shape.
  
The multimodal approach
Multimodal laser triangulation is one which uses several laser line projectors illuminating one single camera view, each illuminant occupying the whole field of view. The multimodal setup allows measuring simultaneously with different triangulation angles at (nearly) one measurement point while using the full measurement range and resolution of the sensor. Using different triangulation angles also implies to measuring with multiple resolutions and being able to avoid most of the problems of the traditional triangulation system by comparing the measured results with each other and selecting the “most reasonable” data point at each position. The problem then reduces to analyzing the camera view and separating light coming from different light sources. The measurement data generated by different projector-sensor-combinations can be treated as separate measurements, or all data can be merged together to provide an optimal comprehensive view of the object measured. For the purposes of quality control, separate measurements usually suffice, whereas reverse engineering aims to recreate as accurate digital models of the original object as possible.
The basic three-dimensional laser triangulation system can be extended to a multimodal one at least in two ways. The first method uses a color camera and laser line projectors working at different wavelengths. Figure 2a shows an example of this type of system. Note that the lasers need not necessarily be red, green and blue as long as the camera is able to differentiate them clearly enough. The obvious advantage of this system is that all measurement rays can be on all the time. In theory, all resolutions are available for measurements the whole time. In practice, however, object colors similar to the measurement rays can occasionally limit the detectability of some rays. Moreover, three times more data is acquired per every scanning position than with traditional triangulation system as color images are needed for light source separation. Measuring polymer materials using this method is also risky, as they can change the reflected wavelength. If one is allowed to reduce the scanning directional resolution, it may be more beneficial to use single wavelength laser projectors in a time-shared manner. In such system the lasers can be modulated for example by using the camera synchronization signals. This way, the light source is switched between every frame outputted by the camera. Figure 2b shows an example of this type of system.
Results
The multimodal measurement was tested with real-world test parts, for example with a connection strip block (shown in Figure 3a). The part contains mostly very clear shapes; a low, flat surface on the bottom and almost equal-high edges around it. The material is also relatively easy to image as it is mostly light gray plastic. The block also contains two metal insert parts that both consist of a number of components. Common to the metal parts is that the surface reflectance changes rapidly at the boundary between them and the plastic material below; the light gray plastic reflects light diffusely, whereas all metal parts reflect light mostly specularly. Particularly the rounded corners of the metal parts cause strong reflections to all directions and increase the risk of either saturating the detector, seeing multiple reflections, or not seeing the reflection at all.
As can be seen from Figures 3b and 3c, the multimodal approach can clearly enhance the measurement results: Multiple viewpoints to the measured object decrease the possibility of occlusions and specular reflections that often disturb the measurement. Likewise, multiple measurement rays that can be freely positioned and oriented enable the selection of optimal measurement ranges and resolutions for each measurement task. This enables using the method also in fully automated production systems and for quality control of even smaller lot sizes. At the same time, the solution saves costly floor-space.
|
