A benchmarking process to assess software requirements documentation for space applications

• We propose a benchmark to assess the quality of space software specifications.
• The benchmark includes 3 checklists, each composed by a large number of questions.
• Two checklists verify compliance with PUS standard and faulty behaviour.
• One checklist verifies the absence of typical documentation errors.
• We validate the benchmark using the concept of fault injection.

Poorly written requirements are a common source of software defects and, in application areas like space systems, the cost of malfunctioning software can be very high. This work proposes a benchmarking procedure for assessing the quality of software requirements that adopt the Packet Utilization Standard (PUS) defined by the European Cooperation for Space Standardization (ECSS) standards. The benchmark uses three checklists that aim at guaranteeing that the specifications comply with the PUS standard, consider faulty behaviour, and do not include errors typically found in this type of documents. The benchmark is defined for two services of the PUS standard: the telecommand verification and on board operating scheduling. A benchmark validation approach is also proposed in the paper. It uses the concept of fault injection to insert known errors in software requirements specification documents. The benchmark validation is performed through its application to three projects from different countries. Results show that our proposal provides a simple and effective way for identifying weaknesses and compare the degree of maturity of requirements documents.