The science of research: The principles underlying the discovery of cognitive and other biological mechanisms

Studies of cognitive function include a wide spectrum of disciplines, with very diverse theoretical and practical frameworks. For example, in Behavioral Neuroscience cognitive mechanisms are mostly inferred from loss of function (lesion) experiments while in Cognitive Neuroscience these mechanisms are commonly deduced from brain activation patterns. Although neuroscientists acknowledge the limitations of deriving conclusions using a limited scope of approaches, there are no systematically studied, objective and explicit criteria for what is required to test a given hypothesis of cognitive function. This problem plagues every discipline in science: scientific research lacks objective, systematic studies that validate the principles underlying even its most elemental practices. For example, scientists decide what experiments are best suited to test key ideas in their field, which hypotheses have sufficient supporting evidence and which require further investigation, which studies are important and which are not, based on intuitions derived from experience, implicit principles learned from mentors and colleagues, traditions in their fields, etc. Philosophers have made numerous attempts to articulate and frame the principles that guide research and innovation, but these speculative ideas have remained untested and have had a minimal impact on the work of scientists.Here, I propose the development of methods for systematically and objectively studying and improving the modus operandi of research and development. This effort (the science of scientific research or S2) will benefit all aspects of science, from education of young scientists to research, publishing and funding, since it will provide explicit and systematically tested frameworks for practices in science. To illustrate its goals, I will introduce a hypothesis (the Convergent Four) derived from experimental practices common in molecular and cellular biology. This S2 hypothesis proposes that there are at least four fundamentally distinct strategies that scientists can use to test the connection between two phenomena of interest (A and B), and that to establish a compelling connection between A and B it is crucial to develop independently confirmed lines of convergent evidence in each of these four categories. The four categories include negative alteration (decrease probability of A or p(A) and determine p(B)), positive alteration (increase p(A) and determine p(B)), non-intervention (examine whether A precedes B) and integration (develop ideas about how to get from A to B and integrate those ideas with other available information about A and B). I will discuss both strategies to test this hypothesis and its implications for studies of cognitive function.