Design of a machine vision systemIntroduction The design of a cost effective but still sufficient machine vision system is a complicated task. To make a thorough survey of the basic cause to use a machine vision system is extremely important. Only with a good understanding of the basic needs the right actions to fulfil the actual needs can be chosen. When this basic information is found the design of hardware and software used in the application can be started. A good understanding of the influence of lighting conditions, used hardware and used software algorithms is a must to be able to make the right decisions and choices during the design of a machine vision system. A high competence in the fields of electromagnetic radiation, optics, camera technology and image data processing can lead to effective machine vision solutions in applications where machine vision usually is not considered as an alternative or in applications considered impossible for a machine vision system. Basic survey of needs The use of a machine vision system is often caused by the demands of a customer, by the regulations of a quality standard or by authority regulations. In these cases the task of inspection or measuring is usually well defined, but the really big possibilities in the use of a machine vision systems are available when a machine vision system is used for solving a manufacturing problem or to improve and control the manufacturing process. To be able to understand and locate all the possibilities of machine vision in a production process, a systematic analysis made by an experienced, competent and independent machine vision specialist is needed. Even if the actual problem is known, there is usually a lot to gain if the situation is carefully analysed and the basic needs defined. Definition of needed actions, precision and accuracy When the basic need of the actual task is fully understood, the definition of needed actions to fulfil these needs can be made. The needed actions are totally dependent of the needs of the application. An inspection task where a missing part has to be detected is quite different from a measuring task where the dimension of the part has to be measured. In all cases the definition of the needed actions has to be made carefully to meet the requirements of functionality, reliability, precision and accuracy. Also the processing time and the overall system cost is to be considered when the final definition of needed actions is made. Definition of system hardware When the actions needed to meet the basic needs are defined, the definition of the hardware to be used can be started. In addition to the good knowledge of the imaging process, including camera objectives, camera sensors, transfer of image data and the creation of the digital image, the design of the lighting conditions is of extreme importance. With a proper design of lighting conditions and effective use of electromagnetic radiation the computational cost can be reduced and the reliability, precision and accuracy improved. Also the total hardware cost can be reduced when the required actions with required precision and accuracy can be reached using a smaller amount of components or low-cost components. Definition of software algorithms The definition of the software algorithms to be used in a machine vision system is, in addition to the basic needs of the application, mainly influenced by the quality of the received image data. The quality of the image data is strongly dependent of lighting conditions, external disturbances and the variations in shape and position of the object of interest. The knowledge of available image processing techniques, image transformations, measuring- and inspection tools and the effects of all adjustable parameters to the final result is of great importance in the selection of the algorithms. Also the computational cost of the used algorithm in relation to the basic needs is to be optimised. Off -the-shelf or in-house When the hardware and software definitions are made, the design process proceeds with an investigation of existing commercial machine vision systems. It is a demanding task to filter out the real characteristics of a machine vision system from the commercial information. To be able to evaluate a machine vision system, good knowledge of the hardware and software algorithms used in the system is needed. Depending on the actual needs in a particular application the best solution may vary from a complete commercial system to an in-house built and programmed system. As the use of commercial systems depends on their characteristics the use of in-house systems is a trade-off of the availability of suitable commercial systems and the in-house machine vision knowledge and available resources. Testing Regardless of how the final machine vision system is built, testing of the system is needed. One of the “Seven Deadly Sins of Machine Vision” is the lack of a careful and competent testing. It is very important to verify that the precision and accuracy values defined during the design process are fulfilled. Especially in cases where high accuracy is needed, the verification needs to be planned carefully. Also a long-term testing of the systems reliability in an as real as possible environment is needed before the system can be installed in a industrial environment. Example: 3D measuring of crate and bottles In the following an example of a real design and development process of a commercial machine vision system is presented. The system is a 3D measuring system used in a reverse vending machine manufactured by Bevesys Oy in Kouvola, Finland. The system measures a 3D point cloud of a crate and the bottles in the crate. The aim is to recognize the type of the crate and the type of each bottle in the crate. In this example we only discuss the 3D measuring while the actual recognition computation is left out. Basic survey of needs The basic task of this application is to recognize the type of the crate and the type of each bottle in the crate when the crate is brought to the reverse vending machine. The type of a crate is mainly defined by its height, width and length and the type of a bottle is defined by the height and diameter of its top (top = mouth or cap) and collar. The maximum size of an accepted crate is defined by maximum height 400 mm, maximum width 400 mm and maximum length 600 mm. The smallest difference to be detected for the bottle features (top height, top diameter, collar height and collar diameter) is 1.5 mm. The top diameter varies from 30 to 40 mm and the mouth thickness is about 2-5 mm. The collar diameter varies from 35 to 45mm and the collar width is about 3-6 mm. When the crates and bottles usually are made of plastics in different colours, the bottles can also be made of differently coloured glass. The maximum time allowed for the measuring and recognition task is defined by Bevesys to 5 seconds. As the measuring is a part of a reverse vending machine the long-term reliability must be very high. Definition of needed actions, precision and accuracy As the bottles in the crate only can be seen from above, the measuring must be done from above. This fact together with the physical features of the bottles leads to the requirement that the gap between measuring points in the horizontal plane (coverage) must not be greater than approximately 1 mm. This coverage is needed to ensure that enough points are measured to be able to distinguish between different bottle types. Due to the physical features of accepted bottles we get a minimum number of approximately 10 measuring points (usually much more) describing each feature. This means that we need an accuracy of approximate +-6 mm (6 sigma) for each point in order to get +-0.6 mm (6 sigma) accuracy for the bottle features. This accuracy should be preserved without need for frequent manual maintenance. Definition of system hardware As we are dealing with a commercial product, one of the main design criterions is the overall cost of used hardware and software components. This means that the measuring system must be designed using minimum number of low-cost components, still fulfilling the defined accuracy and reliability criterions. Another design criterion is the fact that there are patents protecting the measuring methods used in other reverse wending machines. The basic solution in this case is therefore to use two B/W CCD video cameras imaging the stopped crate. The optimal measuring geometry is obtained when the cameras sight of view direction lines will intersect perpendicular to each other. In this case the cameras will be placed as near as possible the optimal geometry, restricted by the reverse wending machines physical measures and the visibility of the collars of the bottles. The focal length of the optics of the cameras is chosen so that the whole measuring space defined (600x400x400 mm) is visible. In general there is two methods to define the points to be measured. One is to use the natural visible features of the object measured, another is to project a spot of electromagnetic radiation onto the surface of the object. In this case a laser is used as the radiation source to project a spot because the wavelength of the laser together with a suitable band pass filter makes it possible to reduce the disturbances of exterior electromagnetic radiation. To simultaneously fulfil both the coverage requirements and the processing time requirements the laser spot is optically divided into a matrix of spots. These laser spots are then moved using step motors to achieve the final coverage. Also the spot size should be optimised as it affects the coverage. The size should be big enough to fulfil the coverage requirements but still not too big because then the accuracy may be decreased.
In accordance with the cost design criterion a PC-computer was chosen as the computation platform. Also the needed image grabbers are low-cost PCI-bus based grabber cards utilizing the main memory as image buffers.
 Image 1: The result of a 3d measurement of a crate with 10 bottles Definition of software algorithms The basic 3D coordinate computation is based on photogrammetric algorithms. After the cameras have been fixed, their inner and outer orientation is computed using 3D control points (set up calibration). When then the corresponding point from each image is measured, the actual 3D coordinate can be computed using photogrammetric intersection (2D->3D transformation). The use of photogrammetric algorithms makes it also possible to maintain the achieved accuracy automatically, as the outer orientation of a camera can be recalculated using stable checkpoints (recalibration). Linux was chosen as operative system, not only due to the cost design criterion, but also because the possibilities to easily build efficient device drivers and handle operative system problems. Off-the-shelf or in-house All used hardware components belonging to the crate measuring part of the reverse vending machine, except two, are commercial off-the-shelf low-cost components. The measuring system consists of two B/W CCD video cameras with optics and filters, two image grabbers, one laser, two step-motors and one PC-computer. Specially made for this application is the step-motor controller and the diffractive lens used to divide the laser spot into a laser spot matrix. The basic photogrammetric calibration and intersection software used in this application is commercial and made by Mapvision Ltd. All other software is in-house made. The most critical parts, the fast image grabbing, synchronized with the movement of the laser matrix, and the fast reliable detection and measurement of the laser spots from both images are made in cooperation with Expericon Oy. To meet the expected time requirements the laser spot matrix is moved between every video field. This means that over 200 corresponding laser spots have to be identified and measured from two 8-bit digital images (768*288 pixels) in the time of 20 ms. The synchronization of the image grabbing and the movement of the laser matrix is handled by the image grabbers device driver, based on interrupts. The reliable detection of corresponding laser spots is utilizing a priori knowledge obtained during a teaching procedure. The detection of corresponding laser spots assumes that the laser spots are moved to approximate same positions from time to time. The detection of a laser spot involves a decision of the presence of the spot and in case of presence, the definition of the approximate position of the spot. The measuring of the final position of the spot is then based on the approximate position and the grey values of the pixels belonging to the spot. A special adaptive threshold algorithm is used to find the spot pixels from the image containing a lot of disturbances coming from reflections and artefacts due to the low-cost hardware. Testing There are a number of adjustments and thresholds that affect the final recognition of the type of the crate and the bottles. To find the best settings for these adjustments and thresholds extensive empirical tests are needed. The camera aperture size, camera gain, image grabber settings, laser effect adjustment, laser spot size and step-motor controller settings are examples of this kind of adjustments. Examples of critical software thresholds are the grey value noise level, the minimum allowable size of a laser spot in pixels and the maximum inaccuracy for a single point passed to the recognition computation. Also the long-term reliability of the system has to be verified. During these tests both the reliability of the software, hardware and the measuring accuracy is tested. A good and reliable long-term testing requires an automated testing environment. Conclusions
The design process has here been described as a sequential process, step by step. In reality it is more an iterative process where the described steps will be passed several times during the whole process. Usually also the selection of a solution based on hardware versus a solution based on software or the selection of a commercial versus in-house built system is difficult. In the end the main thing to keep in mind is the well-known rule of thumb: Keep it simple!
|
