Towards context aware data fusion: Modeling and integration of situationally qualified human observations to manage uncertainty in a hard + soft fusion process

• We provide a framework for integrating human observation data into fusion systems.
• Error models are used to integrate qualified human observations into fusion systems.
• Error models rely on contextual variables that influence observational capabilities.
• Fusion system uses available observation metadata to align incoming soft data.
• System with uncertainty alignment yields more accurate level 1 and 2 fusion outputs.

This paper presents a framework for characterizing errors associated with different categories of human observation combined with a method for integrating these into a hard + soft data fusion system. Error characteristics of human observers (often referred to as soft data sensors) have typically been artificially generated and lack contextual considerations that in a real-world application can drastically change the accuracy and precision of these characteristics. The proposed framework and method relies on error values that change based upon known and unknown states of qualifying variables empirically shown to affect observation accuracy under different contexts. This approach allows fusion systems to perform uncertainty alignment on data coming from human observers. The preprocessed data yields a more complete and reliable situation assessment when it is processed by data association and stochastic graph matching algorithms. This paper also provides an approach and results of initial validation testing of the proposed methodology. The testing approach leverages error characterization models for several exemplar categories of observation in combination with simulated synthetic data. Results have shown significant performance improvements with respect to both data association and situation assessment fusion processes with an average F-measure improvement of 0.16 and 0.20 for data association and situation assessment respectively. These F-measure improvements are representative of fewer incorrect and missed associations and fewer graph matching results, which then must be considered by human analysts. These benefits are expected to translate into a reduction of the overall cognitive workload facing human analysts in situations where they are tasked with developing and maintaining situational awareness.