Service robots: System design for tracking people through data fusion and initiating interaction with the human group by inferring social situations

•We developed a new system to allow a robot to determine whether it should approach a human group and interact with them by inferring what the current social situation is.•The proposed system tracks the positions of people through the fusion of different depth-related data within a long range and complex environment.•Social situations concerning the robot and the target group of people are inferred first and then the robot decides whether it should initiate interactions or not in the human–robot social domain.

In a situation where a robot initiates conversation with a group of people, questions such as “where is the people group?” and “whether the robot should approach them?” should be addressed. This paper develops a new system that enables a robot to determine whether or not it should approach the aforementioned human group and interact with them after identifying what the current social situation is. The system is mainly to fuse depth-related data to track the positions of a group of people, extract social cues of those people by using depth-related data and a decision network (DN) model, and the main challenge lies in understanding the social cues of the group and the current underlying social situation concerning the relation between the robot and the group. The social cues are based on Proxemics and F-formations, whereas the social situations are categorized as individual-to-individual, individual-to-robot, robot-to-individual, group-to-robot, robot-to-group, confidential discussion and group discussion. Our system proceeds as follows : once a group of people are detected and the social cues of that target group of people are extracted, the corresponding social situation is appropriately inferred, and in turn the robot decides whether it should initiate conversation with the group based on rules to be specified later. The conducted experimental results demonstrate the properness of the system design and the efficacy of the proposed method in recognizing the social cues among individuals of the group as well as the nature of the social situations concerning the group and the robot.