I find it helpful to think in terms of two fundamentally different kinds of knowledge that we may want to advance through our research:
that which describes what is (either causal relationships or natural processes)
that which informs how to accomplish goals (usually called methods).
I refer to these as descriptive theory and design theory, respectively.
Each of these two kinds of knowledge requires different criteria for judging its quality. Descriptive theory is judged by its truthfulness, whereas design theory is judged by its usefulness. There are usually many ways to accomplish a goal. Saying one is more truthful than another is not helpful. Saying one is more useful than another is helpful. Truthfulness is assessed by the validity of the descriptive theory. Usefulness is assessed by the preferability of the design theory or methods.
A second point about the knowledge we want to advance through our research is that it is situational. For descriptive theory, the effects that we observe typically vary from one situation to another. Even for something as clear cut as the law of gravity, the speed (or acceleration) at which an object will fall depends on such situational factors as the density of the object and the density of the medium (usually air) through which it is falling. For design theory, the preferred method typically varies depending on a wide variety of situational factors, including the values by which we choose to judge preferability and a wide variety of environmental factors that influence the results of the methods. In learning contexts, those environmental factors typically include the nature of the learner, the content, the context, and the development constraints, as we describe in Volume III of Instructional-Design Theories and Models.
Since education (and training) is a profession, design theory is far more useful than descriptive theory. Our interest is in designing educational systems and instructional systems. So our research should seek those methods that are preferable to other methods under different situations.
But there is a third important point about design theory. It is related to the stage of development of the design theory or method. All technologies, whether hard (equipment) or soft (methods) – and indeed all systems – undergo a process of development which initially takes a long time to make small improvements in the preferability of the theory/method/technology/ system. (To simplify language, I'll use "method" to refer to all of them.) After a while, improvements start happening quite rapidly until an "upper limit" is approached, and the pace of improvements levels off, creating an S-curve (see figure on the right).
This is very important for researchers, because, if the method you are researching is low on its S-curve (that is, it's in the early stages of development), then "research to prove" might find that it is out-performed by an older method that is at the top of its S-curve, leading to rejecting any further development of the new method that could well have ended up superior to the old method. "Research to improve" is far more useful for a young method than is "research to prove."
In sum, for those of us interested in improving education or training, our research should:
be focused on improving design theory, which means we should be looking at usefulness rather than truthfulness, so we should be concerned with preferability rather than validity,
recognize and document the situational factors present in your study, and try to identify the ones that have the most impact on the preferability of the method.
if you are researching a relatively new method, be focused on research to improve the method rather than research to prove it is better than an alternative method.
Research to improve is often called design-based research or formative research. And the results of that research should be formulated into design theories, which not only describe a method, but also indicate the situations under which it is likely preferable to other known methods. Design theories are highly useful to practitioners, in contrast to descriptive theories, which are mostly useful to scientists and researchers.
The guidance I have developed for designing formative research studies can be found in the chapters on the right.
The Ends of Research
The purpose of research on education (and training) is to improve its quality. It is to provide guidance to practitioners. That kind of knowledge is what we call instructional-design theory. The four volumes of Instructional-Design Theories and Models provide many examples of instructional-design theories (or just "instructional theories"). Several chapters in those volumes also provide guidance for building and improving instructional theories.
I recommend that all researchers in the fields of education and training focus their research on improving existing instructional theories or developing new ones (new methods). These are the chapters that can help you to do so:
Reigeluth, C. M., & An, Y. J. (2009). Theory building. In C. M.
Reigeluth & A. A. Carr-Chellman (Eds.), Instructional-design
theories and models: Building a common knowledge base (Vol. III,
pp. 365-386). New York: Routledge.
Reigeluth, C. M., & Carr-Chellman, A. A. (2009). Understanding
instructional theory. In C. M. Reigeluth & A. A. Carr-Chellman
(Eds.), Instructional-design theories and models: Building a com-
mon knowledge base (Vol. III, pp. 3-26). New York: Routledge.
Reigeluth, C. M., & Carr-Chellman, A. A. (2009). Situational
principles of instruction. In C. M. Reigeluth & A. A. Carr-
Chellman (Eds.), Instructional-design theories and models:
Building a common knowledge base (Vol. III, pp. 57-68). New
Reigeluth, C. M. (1999). What is instructional-design theory and
how is it changing? In C. M. Reigeluth (Ed.), Instructional-design
theories and models: A new paradigm of instructional theory (Vol.
II, pp. 5-29). Mahwah, NJ: Lawrence Erlbaum Associates.
The S-curve of development for a method or technology
Guidance for Formative Research
Reigeluth, C. M., & Frick, T. W. (1999). Formative research: A methodology for improving design theories. In C. M. Reigeluth (Ed.), Instructional-Design Theories and Models: A New Paradigm of Instructional Theory (Vol. II). Mahwah, NJ: Lawrence Erlbaum Associates.
Reigeluth, C. M., & An, Y. J. (2009). Theory building. In C. M. Reigeluth & A. A. Carr-Chellman (Eds.), Instructional-design theories and models: Building a common knowledge base (Vol. III, pp. 365-386). New York: Routledge.