Assessment of the quality of meta-analysis in agronomy

A meta-analysis is a statistical treatment of a dataset derived from a literature review. Meta-analysis appears to be a promising approach in agricultural and environmental sciences, but its implementation requires special care. We assessed the quality of the meta-analyses carried out in agronomy, with the intent to formulate recommendations, and we illustrate these recommendations with a case study relative to the estimation of nitrous oxide emission in legume crops. Eight criteria were defined for evaluating the quality of 73 meta-analyses from major scientific journals in the domain of agronomy. Most of these meta-analyses focused on production aspects and the impact of agriculture activities on the environment or biodiversity. None of the 73 meta-analyses reviewed satisfied all eight quality criteria and only three satisfied six criteria. Based on this quality assessment, we formulated the following recommendations: (i) the procedure used to select papers from scientific databases should be explained, (ii) individual data should be weighted according to their level of precision when possible, (iii) the heterogeneity of data should be analyzed with random-effect models, (iv) sensitivity analysis should be carried out and (v) the possibility of publication bias should be investigated. Our case study showed that meta-analysis techniques would be beneficial to the assessment of environmental impacts because they make it possible to study between site-year variability, to assess uncertainty and to identify the factors with a potential environmental impact. The quality criteria and recommendations presented in this paper could serve as a guide to improve future meta-analyses made in this area.

► Eight criteria were defined for assessing the quality of meta-analysis carried out in agronomy.
► These criteria were used to assess 73 meta-analyses.
► No meta-analysis satisfied all the quality criteria.
► Recommendations were formulated and illustrated in a case study on nitrous oxide emission in legume crops.
► The proposed list of quality criteria can serve as a guide for future studies in this area.