Benchmarking enhancements to a decision support system for global crop production assessments

The Office of Global Analysis/International Production Assessment Division (OGA/IPAD) of the United States Department of Agriculture – Foreign Agricultural Service (USDA-FAS) has been assimilating new data and information products from agencies such as the National Aeronautics and Space Administration (NASA) into its operational decision support system (DSS). The FAS mission is to improve monthly estimates of global production of major agricultural commodities and provide US Government senior decision makers and the public the most accurate, timely, and objective assessment of the global food supply situation possible. These estimates are ultimately captured as the US governments’ official assessments of world food supply for the commodity markets and policy makers. The goal of this research was to measure changes in the quality and accuracy of decision support information resulting from the assimilation of new NASA products in the DSS. We gathered both qualitative and quantitative information through questionnaires and interviews to benchmark these changes. We used an interactive project lifecycle risk management tool developed for NASA mission spaceflight design and quality assurance (DDP – Defect Detection and Prevention) to do this. In this case, we used it to (1) quantify the change in DSS Objectives attained after assimilation of new products, and (2) evaluate the effectiveness of various Mitigation options against potential Risks. The change in Objectives attainment was considered the most important benchmarking indicator for examining the effectiveness of the assimilation of NASA products into OGA/IPAD’s DSS. From this research emerged a novel model for benchmarking DSSs that (1) promotes continuity and synergy within and between government agencies, (2) accommodates scientific, operational and architectural dynamics, and (3) facilitates transfer of knowledge among research, management, and decision-making agencies.

Research highlights
► Enhancements to a complex global crop production decision support system were assessed.
► New satellite data products were tested by users to examine DSS enhancements.
► User feedback over time was analyzed with a benchmarking tool to quantify DSS change.
► Attainment of system requirements and risk mitigations were used as benchmarks.
► This new benchmarking model showed enhancements in the DSS and staff communication.