Visual recognition system for cleaning tasks by humanoid robots

In this study, we present a visual recognition system that enables a robot to clean a tabletop. The proposed system comprises object recognition, material recognition and ungraspable object detection using information acquired from a visual sensor. Multiple cues such as colour, texture and three-dimensional point-clouds are incorporated adaptively for achieving object recognition.Moreover, near-infrared (NIR) reflection intensities captured by the visual sensor are used for realizing material recognition. The Gaussian mixture model (GMM) is employed for modelling the tabletop surface that is used for detecting ungraspable objects.The proposed system was implemented in a humanoid robot, and tasks such as object and material recognition were performed in various environments. In addition, we evaluated ungraspable object detection using various objects such as dust, grains and paper waste. Finally, we executed the cleaning task to evaluate the proposed system's performance. The results revealed that the proposed system affords high recognition rates and enables humanoid robots to perform domestic service tasks such as cleaning.