Visual gripper: an accurate vision guided manipulator

Introduction

Two and three dimensional machine vision technology has been selected to be one key technical area of the studies at the Kemi-Tornio Polytechnic. Experiences and knowledge of the vision-based 2D and 3D measurements have been gained through the development and implementation of the camera calibration and measuring methods and also by solving dimensional assembly problems (see e.g. Machine Vision News 4 and 5).
Besides the accurate dimensional inspection, the 3D machine vision is more and more often needed also to control the robotics of the present-day assembly lines. The demand for high accuracy movements of a robotic manipulator has become one of the most important issues among light assembly applications, which is the fastest growing area of robotic automation. A stiff mechanical structure of a manipulator and utilisation of proper conventional control facilities might, in many cases, be too expensive and inflexible solution due to varying environmental conditions in the assembling site. The integration of a visual sensor to the manipulator is a promising option to control e.g. pick-and-place robot working on such a varying environment.

At the Kemi-Tornio Polytechnic the performance of different kind of motion mechanisms (linear servo, linear conveyor, SCARA-type of a robot) has been studied and prototype sensors and devices utilising these mechanisms has been constructed. In this article we introduce one of these devices, visual gripper, which is a robotic manipulator controlled accurately by using 3D visual sensor installed in the gripper of the manipulator (Fig. 1.).
 
 

Setup

The experimental setup consists of a SCARA type of a robot, a visual sensor integrated to gripper and PC. The visual sensor comprises two analogue JAI microhead cameras with 7.5 mm focal length and LED lighting system. The tilt angles of the cameras are about 23 degrees and the distance between the cameras (imaging base) is about 150 mm (Fig. 2.). Both the tilt angles and the imaging base can be manually adjusted.
The gripper is linear and air-operated having machined fingers. The lengths of the fingers are 100 mm. Besides the target area, the fingertips or alternatively a target object picked are visible to both cameras leading many advantages in the control of the gripper. E.g. the exact pose of the target object can be determined also after picking. Thus, the possible mechanical inaccuracies of the gripper will not impair the performance of the system.
Image capturing is performed with Integral Technologies Flash Bus frame grabber. The image processing and control algorithms of the robot are run in PC. The motion controller of the robot and the PC communicate via serial port.

Operation

The operation principle of the visual gripper can be abstracted as follows: First the difference between the present pose of the gripper and the target pose is solved. This determination is based on the 3D information obtained from the visual sensor. A motion command is generated from the solved pose difference and executed using the control structure of the robot. In the following the 3D measurement method of the visual sensor and control structure of the robot used are presented.
3D measurement method
The method of the 3D measurement is following. First the cameras are calibrated: the intrinsic parameters e.g. focal length, lens distortions etc. and the extrinsic parameters i.e. the positions and orientations of the cameras are determined. After that the actual measurement images are captured and features of interest are detected from both images. Using the detected image coordinates and the computed relation between the camera frames (see Fig. 3), the 3D lines-of-sight of an object point (point P in Fig. 3) can be determined (relative position and orientation of the cameras is assumed to stay constant during the operation). The intersection point of these lines define the object coordinates of the point P. Due to inaccuracies in the feature detection from the image and errors in the determination of camera poses, an actual intersection point can not usually be found. In our method least squares (LS) criterion is used to find the appropriate intersection point.
Control structure
A dynamic position-based look-and-move control structure is applied to the movement control of the visual gripper (Fig. 4). In the calibration phase the accurate poses of the visual sensor respect to the robot control frame and to the robot tool frame are determined. After these relations are determined, we can use the 3D information obtained from the visual sensor to control the SCARA-robot accurately in all four degrees-of-freedom of this robot type.

Experiments

The total accuracy of the visual gripper comprises of the accuracy of the visual sensor (the accuracy of the motion command) and of the accuracy of the robot movement (the accuracy of the execution of the motion command). We have performed some introductory experiments to find out both the accuracy of the visual sensor, the accuracy of the robot movement, and the total accuracy of the visual gripper. In the following these experiments are shortly introduced and results presented.
Performance of the Visual Sensor
The performance of the visual sensor was tested using an accurate reference object (accuracy of the object is better than 0,001 mm, see Machine Vision News 4). The object was measured at three separate height levels, called zero, bottom and top level, respectively. The distance between each of the separate levels was 2 mm. A manual coordinate meter with accuracy 0,002 mm was used to measure the height differences. Three tests were performed:
a) Height level test. The 3D coordinates were measured at each level. The height differences between the levels were calculated from the measured coordinates. These differences were compared to known ones and the deviations between them were examined. The results of the height level test are presented in Table 1. Maximum error in the height level determination was 0,023 mm. 
b) Dimension measurement test. All the possible dimensions in 3D at each height level from the object were measured. Deviations between the known and measured dimensions were studied. The results of the dimension measurement test are presented in Table 2. It can be suggested based on these results that our sensor is capable to measure 3D coordinates within accuracy, which is better than 0,015 mm.
c) Flatness test. A plane was fitted (least squares criterion was used) to the 3D coordinates measured at each height level and the deviations of the 3D coordinates from the corresponding LS plane were examined. The results of the flatness measurement test at the "Zero" level are presented in Fig. 2. The points marked with crosses are the actual, observed 3D object coordinates. Maximum deviation from the LS plane was 0,019 mm.
 
 

Robot movement

Besides a correct motion command from the visual sensor, also an accurate execution of the motion command is needed in order to achieve a certain target pose. This means that the robot motion itself should be accurate inside a specified region. We observed some unacceptable large inaccuracies in the X-Y plane motion of the robot used. Therefore a motion error compensation method based on the high repeatability of the robot was developed and examined. 
The compensation was carried out by aid of a calibrated camera attached to the robot end-effector. Camera was moved above the reference object and images were captured after each movement. The actual movement performed was determined from these images and compared to the movement information obtained from the motion feedback of the robot. The differences (dependent on the position of the robot) between these two movement data sets were compensated using linear interpolation.
The accuracy of the robot motion after compensation was determined using same method as was used in the compensation (accuracy of this method has been verified to be better than 0,005 mm). Movements of different lengths (10 - 86 mm) were performed above of a 70 x 50 mm target area and positioning accuracy was determined. Positioning errors at 40 destination positions are presented in Fig. 3. According to these results our robot is, after the compensation, capable to movements, whose accuracy is better than 0,040 mm.

Pick-and-place application

In order to examine the total accuracy of the visual gripper a pick-and-place application was designed and implemented. The components of the application are presented in Fig. 4. Target plate comprises two hexagonal holes in different heights and orientations. Target objects are hexagonal with a circle shape grasping hole in the middle. Three different sized target objects were machined. The differences between the diameters of the holes in the target plate and the diameters of the target objects are 0,500 mm, 0,100 mm and 0,050 mm. So, when the target object is centred into a hole of the target plate there is either +-0,25 mm, +-0,050 mm or +-0,025 mm margin for the inaccuracies of the visual gripper.
In the pick-and-place application tests a target object is picked and measured after picking. Then the robot is moved above the randomly positioned target plate and the pose of a target hole is measured. After that the target object is placed to the hole.
At this moment a target object with +-0,050 mm margin is picked and placed successfully. The image processing algorithms and the lighting equipment will be further developed. The final goal is to reach accuracy of +-0,025 mm.
 
 

Summary

In this article a visual gripper, a robotic manipulator controlled accurately using a 3D visual sensor, was introduced. The visual sensor based on two cameras and an LED lighting system is integrated with a gripper to the end-effector of the SCARA type of a robot. The control method and the results of the experimental tests of the visual gripper were also presented.
Both the motion accuracy of the robot and the accuracy of the visual sensor affect on the total accuracy of the visual gripper. It was reported that motion accuracy better than 0,040 mm was achieved using motion error compensation method developed. The accuracy of the visual sensor was reported to be better than 0,025 mm. A pick-and-place application, which can be used to examine the total accuracy of the gripper, was also presented. The current status of the assembly accuracy was reported to be better than +-0,050 mm.

Contact information:

Teuvo Heimonen
Kemi-Tornio Polytechnic
The Unit of Technical Education
Phone: +359 40 586 8108
Fax: +358 16 258 800
email: teuvo.heimonen@tokem.fi

.