Software generated device exception for more intensive device-related software testing: An industrial field study

• We propose a lightweight device exception testing method and a related automation tool, AMOS v3.0.
• The proposed method artificially triggers more realistic device exceptions in runtime.
• The proposed method monitors how software components handle exceptions in detail.
• AMOS v3.0 has been applied to the operational testing for the final mass production verification stage of the CAR-infotainment system in the automobile industry.
• We examine the meaning of a device exception testing through an industrial field study.

It is more important to properly handle exceptions, than to prevent exceptions from occurring, because they arise from so many different causes. In embedded systems, a vast number of exceptions are caused by hardware devices. In such cases, numerous software components are involved in these hardware device-originated exceptions, ranging from the device itself to the device driver, the kernel, and applications. Therefore, it takes a lot of time to debug software that fails to handle exceptions. This paper proposes a lightweight device exception testing method, and a related automation tool, AMOS v3.0. The proposed method artificially triggers more realistic device exceptions in runtime, and monitors how software components handle exceptions in detail. AMOS v3.0 has been applied to the exception testing of car-infotainment systems in an automobile company. The results based on this industrial field study have revealed that 39.13% of the failures in exception handling were caused by applications, 36.23% of the failures were caused by device drivers, and 24.64% were derived from the kernel. We conclude that the proposed method is highly effective, in that it can allow developers to identify the root cause of failure for exception handling.