Development of Agricultural Work Automatic Record System by Using Kinect for Tacit Knowledge Extraction

This study aims for agricultural production systemized techniques centered on decision support with a computer that provides timely advice along the direction where an operator aims to by transforming the information related to tacit knowledge, growing conditions and growing environments of plants, and the like into data based on given rules and formats and by analyzing the data using techniques, such as data mining. We are conducting basic research and development of the Chinou robot as one of the core techniques. The Chinou robot is an agricultural robot that performs the support of agricultural tasks and the extraction of the tacit knowledge of a diligent farmer simultaneously, and it assumes the role of intermediating a farmer and the AI (Agri-Informatics) system. Among the functions of the Chinou robot, this paper investigated the automatic recording method of agricultural tasks using the Kinect sensor intended to extract the tacit knowledge from a diligent farmer. Consequently, we first conducted a recognition experiment of a farmer using the Kinect sensor and found that the coordinates of the joints can be calculated during basic agricultural tasks and, even in the field, the tasks performed in early morning and after late afternoon when the sunlight is weak, at night, and under artificial lighting could be detected. Next, we developed an automatic recording system for agricultural tasks where a diligent farmer detects the mildew of green pepper. The disease detection task was divided into four processes, and we could build a basic system to detect these processes. In particular, we investigated the method to calculate the line of sight that is important to extract the tacit knowledge in the future and evaluated the method. Although there was a limiting condition of no head rotation, a simplified calculation of the line of sight using the Kinect sensor became possible.