The impact of Test-First programming on branch coverage and mutation score indicator of unit tests: An experiment

BackgroundTest-First programming is regarded as one of the software development practices that can make unit tests to be more rigorous, thorough and effective in fault detection. Code coverage measures can be useful as indicators of the thoroughness of unit test suites, while mutation testing turned out to be effective at finding faults.ObjectiveThis paper presents an experiment in which Test-First vs. Test-Last programming practices are examined with regard to branch coverage and mutation score indicator of unit tests.MethodStudent subjects were randomly assigned to Test-First and Test-Last groups. In order to further reduce pre-existing differences among subjects, and to get a more sensitive measure of our experimental effect, multivariate analysis of covariance was performed.ResultsMultivariate tests results indicate that there is no statistically significant difference between Test-First and Test-Last practices on the combined dependent variables, i.e. branch coverage and mutation score indicator, (F(2,9)=.52F(2,9)=.52, p>.05p>.05), even if we control for the pre-test results, the subjects’ experience, and when the subjects who showed deviations from the assigned programming technique are excluded from the analysis.ConclusionAccording to the preliminary results presented in this paper, the benefits of the Test-First practice in this specific context can be considered minor.LimitationIt is probably the first-ever experimental evaluation of the impact of Test-First programming on mutation score indicator of unit tests and further experimentation is needed to establish evidence.