A Model-based Framework for Predicting Performance in Self-adaptive Systems

In recent years, the Department of Defense (DoD) has sought to develop military systems with increasing levels of autonomy. There has been an increase in requirements and desired capabilities that call for the semi-autonomous or autonomous performance of tasks. Military robot systems are an example of such complex systems. As the DoD develops these complex systems it is evident, based on recent research, that in order to achieve the desired capabilities the systems must adapt and learn to improve their performance and become more autonomous. However, it is cost prohibitive and impractical to evaluate self- adaptive systems in all possible scenarios and environments. As a result, it is desirable to improve our ability to understand how autonomous systems will perform in order to influence military acquisition decisions. Prior work has sought to characterize the environment or the performance of unmanned systems based on levels of autonomy and suggested that environmental complexity is a strong predictor of performance of mobile robot systems. However performance measures of unmanned systems dealing with complex and changing environments have been difficult to measure quantitatively because it is difficult to delineate the general operational domains of the unmanned systems or how to determine if an unmanned system satisfies capability specifications or expectations. This paper describes the development of a model-based framework for predicting the performance of self-adaptive systems, specifically a navigation task for mobile military robot systems. By developing a quantitative model of performance based on the complexity of the environment, including slope and vegetation, we can estimate the performance of a system in new regions based on performance in known regions. Using simulation and data from prior experiments, we demonstrate the ability to predict the performance in environments that have not been tested. In order to validate our model, we compare the model results to data from previous DARPA-led research experiments.