Integrated Image Processing and Path Planning for Robotic Sketching

Since the beginning of the development of machine vision, researchers have realized its importance in the robotics field, as it provides a useful tool for both the environment detection and decision making during the automation process. At the same time, path planning for robots influences many in the robotics and automation field and it has remained active for both methodology research and system implementation. This research combines machine vision with robot path planning with an aim of programming-free robotic applications. Particularly as a proof of concept, a programming-free robotic sketching prototype is developed as a case study. Within the context, this paper consists of three parts. The first part covers the processing of a facial image taken by a webcam to identify the contours that represent the image; the second part converts these contours to paths for an industrial robot to follow; and the third part controls the robot adaptively for sketching including auto-generation of control codes and self-calibration. The developed prototype is a closed-loop system with networked camera and robot. Intelligent computation is applied to identify the contours of the image with minimum representation of points and with the correct sequence of points for each curve (path); the sequence of the output robot paths represents the near-optimal sequence to preserve the minimum travelling time for the robot. The robot control module can also retrieve the TCP of the robot for off-site monitoring. The ultimate goal of this research is future applications of robot path following, e.g. ad-hoc robotic cutting or welding where the paths can be specified by hand-drawings of an operator on the target workpiece with zero programming for the operator.