A force and thermal sensing skin for robots in human environments

Working together, heated and unheated temperature sensors can recognize contact with different materials and contact with the human body. As such, distributing these sensors across a robot's body could be beneficial for operation in human environments. We present a stretchable fabric-based skin with force and thermal sensors that is suitable for covering areas of a robot's body, including curved surfaces. It also adds a layer of compliance that conforms to manipulated objects, improving thermal sensing. Our open hardware design addresses thermal sensing challenges, such as the time to heat the sensors, the efficiency of sensing, and the distribution of sensors across the skin. It incorporates small self-heated temperature sensors on the surface of the skin that directly make contact with objects, improving the sensors' response times. Our approach seeks to fully cover the robot's body with large force sensors, but treats temperature sensors as small, point-like sensors sparsely distributed across the skin. We present a mathematical model to help predict how many of these point-like temperature sensors should be used in order to increase the likelihood of them making contact with an object. To evaluate our design, we conducted tests in which a robot arm used a cylindrical end effector covered with skin to slide objects and press on objects made from four different materials. After assessing the safety of our design, we also had the robot make contact with the forearms and clothed shoulders of 10 human participants. With 2.0  s of contact, the actively-heated temperature sensors enabled binary classification accuracy over 90% for the majority of material pairs. The system could more rapidly distinguish between materials with large differences in their thermal effusivities (e.g., 90% accuracy for pine wood vs. aluminum with 0.5  s of contact). For discrimination between humans vs. the four materials, the skin's force and thermal sensing modalities achieved 93% classification accuracy with 0.5  s of contact. Overall, our results suggest that our skin design could enable robots to recognize contact with distinct task-relevant materials and humans while performing manipulation tasks in human environments.