Concept for a multi-purpose academic gripper

Received Dec 25, 2018 In “concept for a new precise academic gripper” the mechanical aspects of the new gripper were presented [3]. In this paper, the idea is improved and expanded towards a clever control part. This paper mainly focuses on the functional ideas and the state of research concerning the imaging tasks. Experiment and intermediate results of the imaging process are discussed. Different approaches of skin colour detection in diverse colour spaces have been tested. Skin colour detection is investigated whether skin colour can be monitored to improve work safety in terms of functional safety. To enhance versatility, several ideas are combined to form a clever gripping device. Keyword:


Introduction
Precise gripping of very different object sizes in different fields of automation becomes more and more essential.Existing gripper designs allow gripping of objects based on positional programming.Usually lots of mechanical aids as centering stations and mechanical stoppers are used to ensure that handled objects are located at defined locations and in defined positions for gripping.These mechanical aids allow to bring objects into positions which are mechanically well defined and repeatable.But these concepts do not allow to compensate mechanical bending of a robotic arm under load.These concepts also do not compensate for inexact robot respectively gripper positioning.To overcome these inherently difficulties of any robot system, various concepts have been published over the last decades proposing the use of cameras to locate parts and camera-based positioning data for gripping and placing [12].Most widespread concepts are fixed cameras above a conveyor, cameras in the upper corners of a robotic work cell [8], cameras attached to the robot's shoulder [13], cameras in the palm of a gripper [2] and cameras at the side of a gripper [1], [9] or in the palm plane of the gripper.All these concepts have in common that gripped objects block the view of the camera.No accurate camera guided placement of a gripped object is possible.

Goals of the concept
The main two goals are to simplify robotic programming by overcoming positional inaccuracies of the robotic system itself and to allow precise positioning without the need for centering stations and other mechanical helpers.This can be summarized as simplification of the gripping task.
The proposed concept allows to grip objects which are slightly displaced from a programmed pick-up position and allows to be positioning them with high accuracy relative to another object e.g. a tray or a peg-in-hole assembly position.The placement or assembly is controlled by visually obtained position data.A further goal is to minimize gripping pressure to the gripped objects.It is a well-known effect that strong gripping power can deform fragile objects when gripping these.Using the new gripping principle reduces this effect to a minimum.This results in the economic effect that the same gripper can be used to grip fragile objects as well as solid ones.The distinguishing decision which pressure force to use for gripping is omitted, because heavier objects which need more gripping force are gripped with higher forces than lightweight ones which need less pressure to be gripped.Also, elastic objects are gripped accurately by principle.All independent of their size and weight [11].

Working principle of the gripper under research
As the mechanical part of the gripping device was already discussed in [1], the description in this paragraph focuses on the optical and control principle.It has been demonstrated, that the mechanics are capable of achieving a resolution of approximately 2 micrometers, depending on the gear ratio used.To be able to use this high positioning resolution, an also high measurement resolution is necessary to generate the control commands at the intended high precision.
To achieve this, a set of cameras is used which is capable of delivering stereo images at different resolutions and speeds.The used cameras deliver up to 8 Megapixels per image and up to 60 images per second at resolution 640x480.It is necessary to be able to read images at very different resolutions and speeds from these devices, which is an essential criterion for the intended functionality as will be shown further.The cameras are used in a fixed stereo configuration attached directly to the gripper with the view through the open gripper from aside as shown in Figure 1.This results in the effect that gripped objects do largely not block the view to the area where the gripper can place objects to.This is symbolized in Figure 2. The green dashed lines indicate the field of view of the cameras and below the gripped object a tray with cut-outs for snug fit object placement is symbolized.Using the camera array for 3D triangulation allows to precisely navigate to grippable objects, measure the size of the object to grip and use this measurement data to close the gripper only as far as necessary so that the object is gripped but not smashed [10].The probing principle as described in references [4], [5] is used to determine the fine-tuning of the amount of gripper opening.This way the gripping force is self-adjusting to the friction of the material pairing as well as to the individual weight of the object.The cameras are also used to support functional safety at the gripper level: By detecting human skin colour in between the gripper fingers an emergency-stop function is in preparation to stop the movement of the gripper fingers before human body parts are grasped

Conceptual approach
The gripper under research is equipped with two cameras in an inwards tilted stereo configuration.The cameras themselves are inexpensive ELP cameras which connect to a PC via USB2.One connection per camera is used.These connections are data connections and power supply in one.Connecting both cameras to a single USB hub results in the effect that only one camera can be accessed by the OpenCV-python environment.Hence it is necessary to connect each camera to a separate USB controller.

PROS:
-The concept allows an unrestricted view of the cameras to the gripper surrounding.This can be used for safety functions as well as for intelligent collision avoiding.
-The concept allows 3D ranging with a gripped object.This is intended to be used for precise placement tasks.
-The gripping principle itself is highly versatile and allows e.g.bin-picking, as long as the view of the camera to the gripper fingers is not blocked.This can be avoided by using relatively flat bins.
-The concept can be extended to gain data of almost any optically measurable physical variable of the process the gripper is used in.CONS: -The obtainable gripping precision depends on camera resolution, image quality and calibration.Also lighting of the scene and the contrast at the edges of the objects have a large influence on the obtainable measurement and gripping precision.
-Gripping speed also depends on exactness of measurements.The closer the gripper fingers can be positioned to the object, the less probing cycles are needed until the object can be lifted.
-Susceptibility to vibrations might become a challenge.The taking of sharp pictures is only then possible, when the cameras do not move relative to the gripper and to the object.This also concerns the vibrations of the robot arm to which the gripper is attached.
-Largest disadvantage is the relatively high computational effort which is necessary for the calculation of the depth map.

The Control
To coordinate the different tasks of slip detection, skin colour detection and safety stop as well as gripping and releasing and further possible camera-based measurement tasks a multi-threaded python application is under development.This safety concept can be applied to many different gripper designs which have caused severe injuries in the past.According to statistics collected between 2005 and 2012 in Germany, 40% of all accidents with industrial robots injured the operators' hands.The presented concept is an attempt to reduce the risk of hand injuries and is not limited to the special mechanical design and forces of the gripper under research.[7].It will answer the question if this concept is suitable for production purpose and if the proposed safety concept based on skin colour detection is reliable enough to be used as a safety function in terms of the EU machinery directive.It will also answer the question whether the combination of OpenCV and Python is suitable for fast imaging processes.

Experimental results
Up to now, several different methods of skin colour detection were tested.Finding skin colour in RGB turned out to be inefficient due to the fact that all three values for the R-, G-, B-channels change with the lighting intensity.This makes it impossible to reliably detect skin colour as well as any other colour in this colour space.Doing the same test in other colour spaces resulted in similar results: many false positive pixels were detected or the skin colour pixels in the image were not detected at all.A working solution for skin colour detection is the algorithm proposed by [6]. Figure 3 shows three results of skin colour detection.Where the fixed values resulted in a large amount of false hits (whole picture is surrounded by green line-indicating "skin colour"), the more selective methods proved better.But each method still has a decent amount of false positives, especially with red and pink colours, as can be observed at the bag of beads in the lower centre of each image in Figure 3.
Figure 4 shows, although the resolution of the original images is only 1280x1024 pixels, that after the canny edge detection the inner and outer diameter of the tube stubs can be optically determined.The outer Diameter is approximately 4,5mm, the inner diameter ~2,6mm.The pegs of the peg plate have a length of 3mm and a diameter of 1,6mm at the top.For size comparison there is a 2cm edge length 3D-printed calibration cube in the lower left corner of the images.The red boxes show the magnified area around the gripper tips where grippable objects are expected to be located for gripping.Detectable edges and resolution appear to be within the expected theoretical magnitude.In the demonstrated configuration an optical resolution of approximately 60µm per pixel is expected at full resolution and at the shown working distance (at the gripper tips).

Conclusions
At low image resolutions very promising first results were obtained for skin colour detection as well as for the image preparation for 3D measurements.
-The reliability of the skin colour detection is suitable to reduce the risk of gripping human body parts.It can be used for functional safety purposes.
-The key to higher resolution is a very accurate camera calibration.The printed chessboard method which is widely used to calibrate cameras with OpenCV is good for low resolutions.The unevenness of the printed paper target makes this method inappropriate for high resolution imaging, which is vital for precise measurements.
-One of the most complicated parts of the proposed setup for optical measuring is the precision of the camera calibration.Especially the preparation steps for stereo triangulation work only when the images are properly undistorted.This gets more and more challenging with increasing camera resolution.
-The disparity calculation algorithms fail if camera calibration is not excellent.
-Timing at low resolutions is surprisingly less critical: The calculation of the algorithms run faster than expected.At higher resolutions it is expected that keeping calculation times within practically suitable limits gets challenging.
-The results of 3D imaging up to now are to be improved at higher resolutions.
-The gripper system in combination with the presented camera usage concept can be made at a fraction of the cost of classical systems.This makes it especially interesting for small companies with low production volumes and high demand for versatility.
-For use in mass production the concept is not ideal due to restricted acceleration and expected wear issues.
-High contrast between gripped objects and background can be adjusted by varying the colour of the background and using bright and diffuse lighting.Bright objects for instance can be detected better against a dark background where dark objects are better to measure against a bright background.Medium blue and white thin cardboard sheets worked well.This experimental section can be divided into subsections, the contents of which vary according to the subject matter of the article.It must contain all the information about the experimental procedure and materials used to carry out experiments.

Figure 2 .
Figure 2. Field of view and resulting optical positioning possibility A third USB connection is used to connect to the Arduino board which controls the gripper fingers and a fourth USB connection is attached to a RS232-converter to communicate with the robot control.

Figure 3 .
Figure 3. Example results of skin colour search.Left: fixed values, manually edited.Middle: Tool to selectranges with sliders in HSV colour space.Right: result of Algorithm by[6]

Figure 4 .
Figure 4. Undistorted and rectified images after edge detection