Identifying and addressing the limitations of safety climate surveys

IntroductionThere are a variety of qualitative and quantitative tools for measuring safety climate. However, questionnaires are by far the most commonly used methodology.MethodThis paper reports the descriptive analysis of a large sample of safety climate survey data (n = 110,014) collected over 10 years from U.S. Naval aircrew using the Command Safety Assessment Survey (CSAS).ResultsThe analysis demonstrated that there was substantial non-random response bias associated with the data (the reverse worded items had a unique pattern of responses, there was a increasing tendency over time to only provide a modal response, the responses to the same item towards the beginning and end of the questionnaire did not correlate as highly as might be expected, and the faster the questionnaire was completed the higher the frequency of modal responses). It is suggested that the non-random responses bias was due to the negative effect on participant motivation of a number of factors (questionnaire design, lack of a belief in the importance of the response, participant fatigue, and questionnaire administration).ConclusionsResearchers must consider the factors that increase the likelihood of non-random measurement error in safety climate survey data and cease to rely on data that are solely collected using a long and complex questionnaire.Impact on IndustryIn the absence of valid and reliable data it will not be possible for organizations to take the measures required to improve safety climate.

► Surveys are by far the most commonly used methodology for evaluating safety climate.
► The analysis of the safety climate data of naval aviators demonstrated that there was substantial non-random response bias.
► The non-random responses bias was due to the negative effect on participant motivation of a number of factors.
► Researchers must cease the reliance on long and complex safety climate surveys and use a triangulation of research methods.