Agricultural worker behavioral recognition system for intelligent worker assistance

In the present study, we developed a worker behavioral recognition system for a single targeted worker to understand his three specific types of worker behavior for producing peas, namely, watering, seeding, and harvesting, performed along a furrow. The three behavior types were further classified into 14 behavior subtypes and six behavior categories. The 14 behavior subtypes were modeled by 14 hidden Markov models (HMMs): 10 for harvesting behavior, 2 for seeding behavior, and 2 for watering behavior. In the experiments, the targeted worker twice performed the three types of behaviors facing the left ridge and the right ridge along a single specific furrow within a range of 5-25 m from a pan-tilt-zoom camera. The watering and seeding behaviors were performed in the same field condition as the actual field. The harvesting behaviors were faithfully reproduced by the worker with his huge experience in the non-crop field. The recognition rates for watering and seeding were approximately 98% for the watering-left category, 97% for the watering-right category, 100% for the seeding-left category, and 94% for the seeding-right category. For harvesting, the recognition rates for five HMMs in the harvesting-left category ranged from 26% to 100%, and the overall recognition rates for five HMMs in the harvesting-right category ranged from 44% to 100%. Although the recognition rates of two HMMs were too low in harvesting categories, the behavioral recognition system achieved the robust responses to the harvesting behaviors by applying OR operation to the outputs of the HMMs.