journal article·2024-08-29Open access

Integration of Stacked Object Recognition and Robotic Arm Gripping System Utilizing Mask Region-based Convolutional Neural Network and Red–Green–Blue Depth Images

Shu-Yin Chiang, Yu-Kai Zhuo

Sensors and Materials· Volume 36· Issue 8

Abstract

In this study, we explore the challenge of object recognition by robots in scenarios where individual objects cannot be identified due to stacking.We harness the capabilities of the light detection and ranging (LiDAR) camera as a sensor for object detection, leveraging its advanced sensing technology to simultaneously capture both color and depth images of objects.To address the issue of image overlap caused by stacking, the Mask region-based convolutional neural network (R-CNN) is employed for object recognition and segmentation.Additionally, through image mapping transformation, the positions of individual objects in the red, green, and blue (RGB) image are projected onto the depth image to extract their corresponding depth information.Given the disparity in camera positions between the color and depth cameras, occlusions and variations in depth mapping can result in missing depth values for certain objects.To mitigate this, various statistical methods are utilized to fill in these missing values and enhance the accuracy of the extracted depth information.Furthermore, by calculating the width and length of the rectangle of the rotated image, the angle with the smallest value is selected as the gripping angle of the object.Finally, leveraging the transformed coordinates and planned trajectory, the robotic arm executes the gripping task on stacked objects.This execution process validates the accuracy of the three-dimensional spatial information and showcases the functionality of an intelligent robotic arm.
Source: openalex_inverted_index · Quality: baseline

Research context

OpenAlex topics: Industrial Vision Systems and Defect Detection · Robot Manipulation and Learning

SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.

References

Reference metadata was not supplied for this record.