1991
Ernest R. House
| Abstract Disputes in educational research over the past few decades have resulted in part from an inadequate conception of the nature of science itself. Developments in the philosophy of science have led to a new understanding--scientific realism--that has promise of resolving many long-standing dilemmas. At the core of the "standard view" of science is the incorrect Humean notion of causation, which has had devastating effects on research in the social sciences. An adequate notion of causation may provide a framework for research that is at once scientific, that incorporates the perceptions and intentions of participants, and that advances critical values such as social justice. |
In my view none of the current paradigms offers an adequate understanding of where we have been and where we should go. We may be able to have an approach to educational research that incorporates the key criteria of all the current positions--an approach that is scientific, that incorporates the perspectives of participants, and that leads to social justice--but which is superior to any of these positions. That position is the new philosophy of science called scientific realism. This philosophy of science has been in development for the last twenty years and differs dramatically from that with which we are familiar. Scientific realists argue that we have misunderstood the nature of science itself.
The basis for this new understanding has emerged from two major strands in philosophy, from an anti-monistic strand which has focused on the social character of science and scientific change, including the work of Popper, Lakatos, Feyerabend, Sellars, and Kuhn, and from an anti-deductivist strand which has focused on the role of models, analogies, and stratification in scientific explanation, derived from the work of Scriven, Polanyi, Toulmin, Hesse, Harre', and Bhaskar. Kuhn's (1962) work is familiar, as is Scriven's contribution to evaluation, but not his earlier work that opened a new line of reasoning in the philosophy of science (Scriven 1956, 1962, 1966, 1970). Harre' has also been a seminal thinker (Harre' 1970, Harre' 1972, Harre' and Secord 1973, Harre' and Madden 1975).
There are various versions of this philosophy of science, and to simplify my presentation I shall focus on the work of Bhaskar (1978, 1979, 1986, 1989), which has received much recent attention. An explication has already appeared in psychology (Manicas and Secord 1983, 1984; Manicas 1987). (One could also justifiably explicate the views of Harre'.) My goal is to present this different philosophy of science and to suggest what implications it might have for educational research. Scientific realism is by no means free of problems, but my focus is on its introduction and explication. I shall forgo comparisons with competitive perspectives such as interpretivism, pragmatism, and critical theory, though these have been attempted (Outhwaite, 1987; Bhaskar 1989). We first must see whether scientific realism is worthy of extended critique.
The Standard View of Research
The "standard view" of science against which the critics have rebelled is frequently equated with positivism, which is itself often misunderstood. Phillips (1983, 1990) has analyzed the different meanings of positivism, and Garrison (1986) has shown how positivism differs from postpositivist philosophy. I would not call the standard view of educational research strictly positivist but it does contain elements from earlier positivist rationales. The standard view of science contains the following elements (Manicas and Secord, 1983, p. 400):
- A foundationist epistemology which sees all scientific propositions as founded on data and facts, and in which hypotheses are to be tested against the facts;
- Theories as hypothetical-deductive systems that take their meaning by being connected to observations through operational definitions;
- Research as atheoretical with predictability the critical test of theory;
- A Humean or regularity conception of causality in which causal relations are only regular contingent relations between events;
- A conception of explanation in which explanation is subsumption under natural covering laws and in which explanation and prediction are symmetrical.
Or again, "...science is an attempt to gain predictive and explanatory knowledge of the external world. To do this, one must construct theories, which consist of highly general statements, expressing the regular relationships that are found to exist in that world. These general statements, or laws, enable us both to predict and explain the phenomena that we discover by means of systematic observation and experiment. To explain something is to show that it is an instance of these regularities; and we can make predictions on the same basis....such statements must be objectively tested by means of experiment and observation, which are the only source of sure and certain empirical knowledge" (Keat and Urry, 1982, p. 4).
In the social sciences, "On the covering law model, explanation and prediction are symmetrical, and with this conception of explanation, it is necessary only that there is a constant relation between the independent and the dependent variable. Given the extraordinary limits on experimentation in the social sciences, it is no wonder, then, that regression techniques, path analyses, and so on are so attractive. Given all these assumptions, one can be a real scientist without having a theory and without ever doing a real experiment. All we need is data--and plenty of it!" (Manicas, 1987, p. 293).
This standard view has been bombarded by criticisms, especially as it applies to the social sciences, and Gage has done an admirable job summarizing these criticisms. However, the critics of the standard view have never been able to formulate an alternative conception that answers the most important questions. In my judgment the scientific realists have delivered critiques that strike at the heart of the standard view of science and have formulated an alternative vision of scientific research that deserves consideration.
Scientific Realism
Scientific realists contend that the standard view misconstrues the nature of science itself, even physical science, by misunderstanding the nature of the real world, by confusing the real world with our sense impressions of it, and by pretending that what we cannot observe doesn't exist or doesn't make any difference. The "epistemic fallacy" is that the world actually is equal to what we observe, i.e. our experiences gained through sense impressions. So the realists attack not only the epistemology of the standard view but its ontology as well. By contrast, the realist theory of science advances these views (Manicas and Secord, 1983):
- There is no incorrigible foundation for science such as sense impressions or pristine facts. Rather, knowledge is a social and historical product, facts are theory-laden, and the task of science is to invent theories to explain the real world and to test these theories by rational criteria developed within particular disciplines.
- The real world is complex and stratified so that one is always discovering more complex layers of reality to explain other levels.
- The Humean or regularity conception of causation is rejected in favor of a conception of causation in which entities act as a function of their basic structure. The task of science is to determine this structure so that one can understand how the entity acts, which must always be in terms of tendencies and probabilities, since events are the outcomes of complex causal configurations, which sometimes cancel each other out.
- Scientific explanation is not subsumption under covering laws but explanation of how structures of different kinds produce events. Explanation requires knowledge of the causal structure of the entity and a notion of the history of other interacting mechanisms. The necessity of explaining events, rather than merely documenting their sequence, is central. "Now at the core of theory is a conception or picture of a natural mechanism or structure at work. Under certain conditions some postulated mechanisms can come to be established as real. And it is in the working of such mechanisms that the objective basis of our ascriptions of natural necessity lies" (Bhaskar, 1978, p. 12).
Realism is, then, a common-sense ontology, in the sense that it takes seriously the existence of the things, structures and mechanisms revealed by the sciences at different levels of reality. There is no distinction of principle to be drawn between such assertions and claims about discrete observable 'facts'; the task of science is precisely to explain 'facts' in terms of more fundamental structures.... Realists...analyse causality in terms of the natures of things and their interactions, their causal powers (and liabilities). The guiding metaphors here are those of structures and mechanisms in reality rather than phenomena and events (Outhwaite, 1987, pp. 19, 21).
Scientific realism must also be distinguished from naive realism, which is clearly wrong. For example, a naive realist would hold that a lemon is really yellow. A scientific realist would hold that a lemon appears yellow because of the refraction of light off its surface, the particular nature of light waves, and the structure of the human eye, thus invoking the causal entities and structures that produce the phenomenon, i.e. the yellow lemon. The analysis does not stop with surface events but examines the underlying patterns and tendencies.
To base one's analysis on the constant conjunction of events is to conflate three domains, according to Bhaskar (1975):
- The empirical, which consists of experiences and sense impressions,
- The actual, which consists of events,
- The real, which consists of the entities and structures that produce events.
Bhaskar (1978, p. 13) represents the three domains--the real, the actual, and the empirical--this way. (Note that all are real but some are more inclusive):
Real Actual Empirical
Structures x
Events x x
Experiences x x x
In Bhaskar's realist ontology the entities and structures produce the events and experiences. But events can occur without being experienced, e.g. the unobserved tree falling in the forest, and causal entities can interact in such a way that they neutralize each other so that no event takes place. There is the non-event of the tree not falling. Forces in nature may be opposed to each other such that no perceptible event occurs, but the natural forces are still operating. Reality then includes the experiences we have, the events that occur, and the structures and entities that produce these experiences and events, whether or not we experience them and whether or not they cause events.
Of course, how can we possibly know about these underlying causal entities unless they do cause events and we do experience those events? We cannot know about what we can't experience, can we? It was this kind of thinking, of equating what was experienced (the empirical) with actual events (the actual) with the real (the causal entities) that led in the wrong direction. Things got turned around so that what was real was mistaken to be limited to only what we directly experienced. Reality came to be defined as equivalent to the empirical, i.e. what we experience, and anything beyond was discredited as metaphysics. This double error, of equating experience with events and events with reality, must be corrected.
The Central Notion of Causation
At the core of the standard view is the Humean or regularity theory of causation. The persuasive regularity conception of causation was formulated by David Hume, then elaborated by empiricists like John Stuart Mill and Bertrand Russell: It follows, then, that all reasonings concerning cause and effect, are founded on experience, and that all reasonings from experience are founded on supposition, and that the course of nature will continue uniformly the same. We conclude, that like causes in like circumstances will always produce like effects....We are determined by CUSTOM alone to suppose the future conformable to the past. The powers by which bodies operate are entirely unknown. We perceive only their sensible qualities: and what reason have we to think, that the same powers will always be conjoined with the same sensible qualities? (Hume, 1978 [1740] pp. 651, 653)
We have no knowledge of anything but phaenomena; and our knowledge of phaenomena is relative and not absolute. We know not the essence, nor the real mode of production, of any fact, but only its relations to other facts in the way of succession or similitude. These relations are constant; that is, always the same in the same circumstance. The constant resemblances which link phaenomena together, and the constant sequences which unite them as antecedent and consequent, are termed their laws. The laws of phaenomena are all we know respecting them. Their essential nature and their ultimate causes, either efficient or final, are unknown and inscrutable to us (Mill, 1866, p. 6).
If such objects can be verified, it must be solely through their relation to sense data; they must have some kind of correlation with sense data, and must be verifiable through their correlation alone. But how is the correlation itself ascertained? A correlation can be ascertained empirically by the correlated objects being constantly found together. But in our case, only one term of the correlation, namely the sensible term is ever found; the other term seems essentially incapable of being found. Therefore, it would seem the correlation with objects by sense, by which physics was to be verified, is itself utterly and forever unverifiable (Russell, 1957, p. 140)
In his profound skepticism Hume declared much of the world unknowable. In his analysis all knowledge is based on fallible experience; hence, all knowledge is mere supposition. We cannot know the inner powers or processes. But how can we know anything? Since nature is uniform, we can know sequences of events. Hume salvaged knowledge by proclaiming constant conjunctions of events as a basis, but at a high cost. Hume's paradigmatic example of the billiard ball, which reduced everything to sense data and discrete events, still haunts us two hundred fifty years later: Here is a billiard-ball lying on the table, and another ball moving towards it with rapidity. They strike; and the ball, which was formerly at rest, now acquires a motion. This is as perfect an instance of the relation of cause and effect as any which we know....Beyond these three circumstances of contiguity, priority, and constant conjunction, I can discover nothing in this cause (Hume, 1978 [1740], p. 649-650).
But if we stand back from the billiard table and survey the whole scene, we see what Hume has left out--the agent hitting the ball, the agent's intentions, the history of all the entities, including that of the billiard player, and the fact that we don't see what Hume tells us we see, i.e. two distinct events. The Humean analysis recognizes only the events as experienced through sense data, and thereby stipulates that causation must consist of only the succession of events, plus contiguity and temporal sequence of these events, so that the way to discern causal laws is to observe the succession of events, that is, the constant conjunction of events. The general form is, "If p, then q; p, therefore q," where p and q are discrete events.
The Humean analysis projects a flat ontology in which there is a flow of events and experiences, with patterns emerging from our observations. The scientific goal is to find regularities in the patterns. The constant conjunction of events analysis presumes that events occur in isolation from extraneous influences in a closed system. Discernment of constant conjunctions is the limit of our possible knowledge, in the Humean view, and we are exhorted to seek knowledge through strategies such as controlled experiments. Following Hume, Mill's (1843) canons of induction were based on the logic of causes and effects present and absent (constant conjunctions of events), and Mill's canons have provided a basic framework for experimental design.
By contrast, in Bhaskar's realist view constant conjunctions of events exist only under special conditions, such as carefully contrived experiments. They rarely exist in open systems in which events are being produced as a result of the interactions of many different kinds of causal entities operating at many different levels. But because no constant conjunctions of events are experienced does not mean that causal forces are not at work. In the standard empiricist view, constant conjunctions of events are both necessary and sufficient for a causal law. In the realist view constant conjunctions of events are neither necessary nor sufficient. For example, in the realist conception the absence or presence of a particular effect is neither necessary nor sufficient to determine the causal force of a program or treatment. Conceivably, a program's effect may be countered by unobserved entities or its perceived effect might be due to other entities, in spite of attempts to control for outside influences.
"Events are the conjunctures of structured processes and are always the outcome of complex causal configurations at the same and at many different levels. If this is the case, then we can also say that causal processes may have surprising effects. They need not, for example, yield the outcomes they usually do....In an open world the configurations of structures and structured processes are not predictable. Indeed for the standard view of science, the world is a determined concatenation of contingent events; for the realist, it is a contingent concatenation of real structures. And this difference is monumental" (Manicas and Secord, 1983, pp. 399, 403).
Causal laws are not dependent on empirical regularities, i.e., regular successions of events, since these are neither necessary nor sufficient to establish the laws, nor conclusively confirmed or falsified by their instances. Rather causal laws are tendencies interacting with other tendencies such that an observable event may or may not be produced (Outhwaite, 1987). The fundamental error of the Humean analysis is that the nature of reality itself is conflated with the experienced and the actual, leading to an incorrect epistemology.
Scientific Explanation
At the center of scientific realism then are different notions of causation and explanation. Scientific explanation is not subsumption under covering laws but is causal explanation which can show how some event has occurred in a particular case. Events are to be explained even when they cannot be predicted. How then does basic scientific explanation proceed, in the realist view?
Typically, then, the construction of an explanation for...some identified phenomenon will involve the building of a model, utilizing such cognitive materials and operating under the control of something like a logic of analogy and metaphor, of a mechanism, which if it were to exist and act in the postulated way would account for the phenomenon in question....The reality of the postulated explanation must then, of course, be subjected to empirical scrutiny. For, in general, more than one explanation will be consistent with the phenomenon concerned. Once this is done, the explanation must then in principle itself be explained (Bhaskar, 1979, p. 15).
Experiences are not necessarily significant. Rather one must work to produce significant experiences, and scientists sort out significant from insignificant experiences through antecedent knowledge. By prior substantive knowledge, the experimental scientist tries to exclude external influences and trigger the causal entity under study so that entity acts in relative isolation. However, some scientific realists doubt that social researchers can ever attain the degree of experimental isolation necessary, and hence see the social sciences as essentially non-experimental.
In the realist conception stated laws are not statements about events or experiences but rather statements about the ways of acting of causal entities, e.g. electrons, which produce events and experiences by acting in certain ways. The citation of a law presupposes a claim about the activity of some causal entity but not about the conditions under which that entity operates, and hence not about the outcomes of events since those are co-determined by the activities of many other structures or entities. Hence, a statement of law cannot justify a certain claim about either events or experiences (Bhaskar, 1978, p. 95).
Only when a system is sealed off from other influences can the operation of a causal entity be predicted with certainty--and this complete isolation is rarely if ever obtainable in the social sciences. Laws set constraints and limits--we do not expect a teacher to defy the laws of gravity--but do not determine events in the sense of total determination--event A is always followed by event B or if p, then q; p, then q, where p and q are events. The actual world is a world of incompletely described and incompletely known causal agents. The logic of discovery is one in which a result is identified, a plausible explanation for it is invented, and the reality of the postulated processes or entities is checked empirically (Bhaskar, 1979, p. 45).
Scientific theory then, according to Bhaskar, is not an elliptical way of referring to experience, as in positivism, but a way of referring to the hypothesized inner structures of the world which can be confirmed or falsified by certain experiences. Theory is not in a relationship of correspondence with reality and does not mirror reality. To provide an explanation is not to provide a mirror of events, a subtle but important distinction. Theory attempts to explain events, and the explanation may be adequate or inadequate. Theory must conform to standards of adequacy established within particular substantive disciplines. So the world is known only under particular descriptions and is in that limited sense epistemologically relative.
Applied science is somewhat different. The applied scientist must analyze actual situations in all their enormous complexity. In dealing with actual situations the applied scientist does not have to analyze all the causal entities at work, which would be impossible, but rather like the historian, who deals with actual events, the applied scientist looks for the causal elements that tip the balance to produce the event, without analyzing the full array of causal entities operating. "Deduction from a mere empirical generalization is very rarely explanatory, and it is only because laws usually involve more than this (as well as less) that they carry explanatory force. The fact that they commonly reflect some underlying processes, albeit imprecisely, accounts for much of the inductive reliability we ascribe to them....Their importance lies not in the precision with which they trace the characteristics of events or substances but in the fact that they provide a readily identifiable pattern" (Scriven, 1970, pp. 100-101).
Since events are the result of a multiplicity of causes, explanations usually identify a number of interacting causes that joined together to produce the event, e.g. historical explanations. An applied explanation typically consists of a set of redescriptions of the event in different terms. For example, an historical event may be explained by emphasizing geographic, political, social, or psychological factors, and a replete explanation usually invokes a mixture of radically different kinds of causes, e.g. the outcome of a battle may be explained by geography, weather, the intentions of the generals, the organization of the army, and the politics of the countries all mixed together. Since a complete causal analysis of a given event is impossible and unnecessary, which causes does one identify as being relevant? Humans themselves are complex causal agents with varied interests in changing the world, and this intentionality guides their investigations.
Realist Social Science
Are humans so different that they defy scientific investigation? The realist answer, according to Bhaskar, is that the human sciences are indeed possible but that their subject matters--humans, human relationships, and human societies--have distinct characteristics that require special attention. Namely, humans are intentional, social, and create societies and social relationships that are radically open and concept-dependent. That is, human relationships are dependent on the ideas that participants have about them. Society consists of institutions, structures, practices, and conventions that people reproduce and transform. Individual human intentions are worked out within the frameworks of these social structures, which are real entities. Like electrons, social entities are invisible and can be known only by their effects. Unlike electrons, social entities do not exist independently of the activities they govern nor of the conceptions that participants have of them, and they may be relatively unenduring. Nonetheless, these social structures, e.g. social class, exert strong influences over human activities (Bhaskar, 1979, p. 48). (Not all realists would subscribe to this particular set of substantive beliefs.)
Humans have powers of intentional action, including language and pictorial representation, which enable them to act on the world, not only to monitor their performance but to monitor the monitoring of their performance. This power of second-order monitoring--evaluation--is critical to the conduct of science itself. Humans do things intentionally to bring other things about, including the making of science (Bhaskar, 1978, p. 239). Social researchers ask, what are the underlying psychological and social structures that produce these phenomena in humans and society? Explanation often takes the form of a resolution of a complex event into its causal components, a redescription of those components, tracing the possible antecedent causes of the components, and the elimination of alternative possible causes of the components (Bhaskar, 1979, p. 164).
Furthermore, according to some realists, social relationships are so radically open that experimental closure is extremely difficult or impossible to obtain. Humans are far more responsive to their environments than are physical objects and less predictable (Secord, 1986). Hence, the human sciences cannot rely as heavily on research methods such as the experiment as a way of confirming or falsifying fundamental conceptions, as many physical sciences do. Since the social sciences are denied decisive test situations, they must be primarily explanatory rather than predictive (Bhaskar, 1979, pp. 24-25). Of course, it is possible to predict some social events on average, but these predictions lack the certainty, precision, and stability of those in the physical sciences.
Social relationships are not constant, as the Humean analysis would suggest. Laws, if they exist in open systems, are revealed as tendencies, not the certainties of constant conjunctions. To attribute a tendency to something is to say that the structure under investigation has the power to do a certain thing and that certain enabling conditions are in effect such that this power is realized or that it is frequently realized (Bhaskar, 1978, p. 229). Techniques like meta-analysis may be valuable in discerning overall tendencies that may or may not be present on given occasions. The historical and conceptual nature of the subject matter of the human sciences limits measurement possibilities and often makes precision in meaning more important than accuracy in measurement (Bhaskar, 1979, p. 58-59). Some contend that social science must be based on ordinary language (Harre' and Secord, 1973; Secord, 1986). Certainly, the agent's capacity to provide an inside account of intentional behavior seems essential to social explanation. Which research methods to employ must be decided within particular disciplines. However, having less confidence in social theories than in physical ones does not mean that one cannot discriminate between competing theories and explanations. The shape of a science depends on the nature of the subject matter, and that is determined ultimately by the nature of reality as revealed by disciplinary inquiry.
Implications for Educational Research
What difference do these considerations make to educational research? How might a scientific realist analyze educational research issues, such as the conduct of evaluation and the relevance of research to practice? (See Norris, 1983, for how realism makes a difference in test validity).
The conduct of evaluation. The standard view in evaluation has been well represented by Edward Suchman, a sociologist: "The most identifying feature of evaluative research is the presence of some goal or objective whose measure of attainment constitutes the main focus of the research problem.... Characterized this way, one may formulate an evaluation project in terms of a series of hypotheses which state that 'Activities A, B, C will produce results X, Y, and Z'" (Suchman, 1967, pp. 37-38). This version of the regularity theory of causation, in which certain events are followed by other events, is repeated in much of the evaluation literature.
In such a world of highly regular events, formal experimentation makes good sense. "The ideal evaluation study would follow the classic experimental model.... This model represents the ideal experimental design from which all adaptations must be derived....the basic logic of proof and verification will be traceable to this model" (Suchman 1967, pp. 102, 93). According to this reasoning, the best evaluation design is the pre-test, post-test control group design: "The logic of this design is foolproof. Ideally, there is no element of fallibility. Whatever differences are observed between the experimental and control groups, once the above conditions are satisfied, must be attributable to the program being evaluated" (Suchman, 1967, pp. 95-96).
One can determine which activities cause which events by employing the methods of inductive logic developed by John Stuart Mill. "...Mill's most significant contribution--for causal analysis purposes--consists of his work on the methods of agreement, differences, and concomitant variation.... The Method of Agreement states that an effect will be present when the cause is present; the Method of Difference states that the effect will be absent when the cause is absent; and the Method of Concomitant Variation implies that when both of the above relationships are observed, causal inference will be all the stronger since certain other interpretations of the covariation between the cause and effect can be ruled out" (Cook and Campbell, 1979, p. 18).
This experimental methodology was advocated in seminal works: "This chapter is committed to the experiment: as the only means for settling disputes regarding educational practice, as the only way of verifying educational improvements, and as the only way of establishing a cumulative tradition in which improvements can be introduced without the danger of a faddish discard of old wisdom in favor of inferior novelties" (Campbell and Stanley, 1963, p. 171). Whatever the intentions of these theorists, this position hardened into an orthodoxy often supported by federal policies.
Many evaluations were based on this notion of cause and effect and its derivative methodology. For example, in Follow Through it was expected that programs at different sites would produce similar results. The same programs would produce the same effects, with only minor variations. But in fact they did not. The analogy was with physical engineering in which a particular program construction would produce a certain outcome, as for example in constructing a television set (a closed system of constant conjunctions). In other words, activity A, the program, is followed by activity B, the outcome, with expectations of considerable precision. Along with this conception of causation came a corresponding notion of the program itself as consisting of engineered components for which one could specify exact outcomes. This standard conception of causation construes social reality too simply. There are too many interactions.
"Interactions are ubiquitous--that is the 'Achilles heel' of the behavioral sciences" (Cronbach, 1982, p. 150). In contrast to the standard view, and following Mackie's (1974) analysis, Cronbach formulated a causal law this way:" In S, all (ABC or DEF or JKL) are followed by P," where the letters refer to events or situations or to the absence of some objects or events (Cronbach, p. 139). Now ABC is sufficient for P to occur but not necessary because P may be preceded by DEF or JKL just as well. In other words, P may occur without ABC. On the other hand, ABC is sufficient for P to occur if all elements--A, B, C--occur together, but not if only AB or AC or BC occur alone.
Yet the situation may be even more complex. Mackie's original formulation of causal regularities is, "All F (A...B...or D... H... or ....) are P...." (Mackie, 1974, p. 66), where the ellipses indicate missing events or conditions which affect the outcome P, but which are not represented in the law and about which we know little. Such elliptical propositions represent the state of our knowledge of social causation better than statements of simple regularity, according to Mackie and Cronbach.
The problem this formulation of causation poses for the evaluator is that if event A is the program, the program is neither necessary nor sufficient for the effect P to occur. The program is only part of a larger package of events that may be followed by P. Furthermore, we are ignorant of what many of these events are, as represented by the ellipses. Hence, specifying the treatment in an experimental design may be misleading because it may lead one to believe that the program is either necessary or sufficient for the outcome to occur when it is not. In other words, the experiment cannot provide a critical test for the effects of a program. With its X and O formulation, traditional experimental design often mistakes the program for a sufficient condition, one that will produce the outcome by itself, when it is in reality an "inus" (insufficient but non-redundant part of an unnecessary but sufficient) condition. (Traditional design tries to handle this situation with a ceteris paribus clause.)
In a world in which "Activities A, B, C will produce results X, Y, and Z," the program may be a sufficient cause and the experiment a critical test of its efficacy. However, that is not the social world in general. The variability experienced in the results of the Follow Through evaluation reflects the fact that Activities A, B, C produced quite different results depending on many other factors. As insightful as the Cronbach/Mackie critique is, however, it still casts repetitions of events as the essential framework.
A realist conception of causation might see events as being produced by the interaction of a multitude of underlying causal entities operating at different levels. That is, one might construe programs themselves as events that are produced by various causal entities. The program would not be exactly the same from place to place but would differ with the multitude of factors that produce the program, e.g. different teachers and students. In other words, the program would not be seen as a fixed entity, an "x" in a design, but as itself varying from site to site wherever it is produced. Furthermore, even the same program can produce different results because of the complexity and interaction of all the structures that affect the results.
Contrary to the regularity theory, like causes do not necessarily produce like results, when causes and results are construed as events. The assumption of uniformity of nature, essential to the Humean/regularity theory and the standard view, does not hold at the level of events, and it does not hold for educational programs and their effects. Put another way, the standard view conceives educational programs and their effects as existing in a closed system, like the components of a television set, when in fact the system is open to any number of influences, in spite of attempts to close it off by experimental design, or more likely by quasi-experimental design and statistical controls. (The necessity of isolating the system from outside influences is seen from the fact that even a closed machine like a television set will not produce the same results in water).
A realist conception of causation does not mean that causes cannot be determined in specific situations. Knowledge of particulars may enable one to identify causal entities at work. Weir (1982) has argued, "...causes are identified on the basis of the details of the situation and the knowledge and understanding brought to their interpretation. The extent of this knowledge, and one's degree of familiarity with the situation, will sometimes provide a nearly conclusive justification for causal identification" (Weir, 1982, p. 177). A realist conception of causation would call for evaluation approaches that expect and track variability and irregularity of events, for somehow describing programs and their outcomes so that influences can be registered and so that causal entities and their interactions can be understood. One might draw inferences from experiments, with help of substantive knowledge, but not expect the studies to yield critical tests of the program.
Relevance of research to practice. How is it possible for research to be relevant to practice if the researcher cannot guarantee that event A will be followed by result B, the bedrock of the standard view? This issue has never been successfully resolved in the standard view of research, either in theory or practice. The usual attempt at resolution is to divide research findings into the internally valid and the externally valid. In the standard view the researcher discovers that a program has certain results in particular settings (internal validity) and then attempts to find similar settings in which the program will produce the same results (external validity). Like causes produce like effects. However, the research literature is full of cases in which programs did not have similar effects, and practitioners do not use much educational research.
As Glass has noted, "The internal validity of an experimental finding cannot be judged solely in terms of the properties of the experimental design...Internal validity of a causal claim is a complex judgment that ultimately rests on a larger number of empirical facts that necessarily extend beyond the context of a single experiment. The external validity of an experimental finding rests similarly on judgment and a network of empirical facts.... the facts are more numerous and are less well established" (Glass, 1981). One cannot depend solely on the formal characteristics of the research design to draw inferences. One must have substantive knowledge. This is true both in drawing inferences from a study and in applying the study elsewhere.
To address this problem, Cronbach (1982) has argued that external validity is more important than internal validity, contending that there are so many unexplored and unknown interactions between the program and the setting that participants in the implementing site must themselves extrapolate the research findings into their own settings by adjusting the findings to the circumstances they encounter, and only the practitioners themselves can do this. Researchers can help by conducting studies in ways that help participants extrapolate the research findings, but researchers cannot guarantee universal results for a particular program.
In a realist view although patterns of events do not repeat themselves exactly, there are transfactual causal structures that influence events and that operate in different settings, even though their interactions with other causal mechanisms may not produce the same events from site to site. The realist would expect programs not to have the same effects in different sites and circumstances. However, transfactual entities can be causally efficacious across sites, though effects might be amplified or cancelled by other factors. Hence, a goal of research is to discover entities that tend to produce effects.
For example, teachers are causal agents themselves, and it is a commonplace that particular teachers make a tremendous difference not only in the classroom but also in the implementation of programs. Parents act on this knowledge by putting their children into the classes of the good teachers. However, one can never be certain that a particular program or teacher will produce good results. The standard view has carried with it the implicit image of program participants as compliant agents who follow administrative directives and whose own views and particularities make little difference, a view refuted by implementation studies (Fullan, 1982).
In the standard view programs are conceived as discrete, reproducible activities that are sufficient to produce particular results under similar circumstances. However, programs themselves are more like events than causal entities, and the programs themselves are produced by such entities. One can evaluate programs on a particular site with considerable efficacy, even while not knowing exactly which important causal structures are at work in what interactions, but one cannot expect that program to have the same effects on other sites with great confidence. One can expect a major causal entity, e.g. a good teacher, to be effective in other settings with similar students, but even this is not certain.
There are ways of dealing with this complex social reality. If all these interactions occur in somewhat unpredictable ways so that causal laws can never be more than statements of tendencies, one might try to average across in some way to discern these tendencies, e.g. meta-analysis (Glass, McGaw, and Smith, 1981). Meta-analysis does not depend on a single critical experiment but rather summarizes across many different studies in an attempt to discern general tendencies. A program that contains strong causal structures might be expected to produce effects on average. Meta-analysis has indeed yielded results when other methods of investigation have failed, such in as discerning the effects of reducing moderately high blood pressure on heart disease.
There is a further extension of this reasoning. If teachers themselves are strong causal agents, able to dramatically affect the production of events, then their intentions and their knowledge are also important factors in good educational programs. A teacher's knowledge consists not only of subject matter but also of knowledge of concrete interactions of particular students in the classroom. The good teacher possesses knowledge of what is likely to happen with particular students when certain activities occur, and in fact the teacher may know that each student may respond in a different way to certain classroom activities. That is, the teacher possesses specific causal knowledge built on inferences made over a period of time from different sources and focused on particular students and the concrete conditions of the classroom. Being able to act effectively in this setting entails knowledge of concrete causal entities--what is likely to happen with these students, even this particular student. So the teacher can strongly influence particular students by specific activities based on knowledge of that student. Of course, the teacher may be wrong in the inferences drawn and the activities initiated. Improvement of the teacher's causal inferences themselves, based on the particulars of the teacher's students and classroom, would seem to be an important strategy for improving education. Unfortunately, in the search for general laws, not much attention has been paid to improving particular teacher concrete inferences directly. From a realist point of view the standard distinction of internal validity and external validity is inadequate. Rather one might think of the validity with which researchers draw conclusions form their studies, the validity with which practitioners draw conclusions from these studies to their own situation, and the validity with which teachers and other practitioners draw conclusions for themselves based upon their own experiences (House et al, 1989). A critical test for realism in research is that it be realistic in practice as well.
Summary
Over the past few decades the standard view of educational research has been strongly criticized. There is little doubt that the Humean or regularity theory of causation on which the standard view is based is incorrect. However, no one has succeeded in defining an alternative position that is satisfactory. Scientific realism, derived in part from studying how scientists actually conduct their research, has resulted in a new conception of science and causation that has promise as a basis for educational research.
In the scientific realist view the proper subjects of scientific investigation are the causal entities and structures that produce events. Events themselves are seen as products of the interaction of these multiplicitous causal entities and structures, which operate at different levels. In the standard view of research constant conjunctions of events are both necessary and sufficient to establish cause. In the realist view constant conjunctions are neither necessary nor sufficient. Even if one establishes such constant conjunctions, a scientific explanation is not satisfactory until one has knowledge of the underlying processes that produce them. Such an analysis explains many conundrums that the Humean conception of causation has engendered, the fundamental error being that of equating experience with events and events with reality.
Scientific realism has many implications for educational research, including how we conduct evaluations, how we validate tests, and how research relates to practice. It explains why meta-analysis sometimes succeeds where conventional experimentation fails. Such a view is also closer to educational practice. On the other hand, scientific realism is not without problems. What are the weaknesses in its notion of causation? How are research claims validated? What methods should be employed? What is the role of experiments? How does scientific realism compare to perspectives such as interpretivism, pragmatism, and critical theory? Many questions are unanswered, but the promise of this strikingly different conception of science and causation is such that they warrant further exploration.
References
Bhaskar, R. (1978). A realist theory of science. Sussex: The Harvester Press.
Bhaskar, R. (1979). The possibility of naturalism. New Jersey: Humanities Press. B
haskar, R. (1986). Scientific realism and human emancipation. London: Verso.
Bhaskar, R. (1989). Reclaiming reality. London: Verso.
Campbell,D.T.(1969).Reforms as experiments.American Psychologist,24,409-429.
Campbell, D. T. and Stanley, J. C. (1963). Experimental and quasi-experimental designs for research on teaching. In N.
L. Gage (Ed.), Handbook of research on teaching (pp. 171-246). Chicago: Rand McNally.
Cook, T. D. and Campbell, D.T. (1979). Quasi-Experimentation: design and analysis issues for field studies. Boston: Houghton Mifflin.
Cronbach, L. J. (1982). Designing evaluations of educational and social programs. San Francisco: Jossey Bass.
Fullan, M.(1982). The Meaning of Educational Change. New York: Teachers College Press.
Gage, N. L. (1989) The paradigm wars and their aftermath: A "historical" sketch of research on teaching since 1989. Educational Researcher,18,7,4-10.
Garrison, J. W. (1986). Some principles of postpositivistic philosophy of science. Educational Researcher, 12-18.
Glass, G. V. (1981). Experimental validity. In H. Mitzel (Ed.), Encyclopedia of educational research. 5th ed.
Glass, G. V, McGaw, B., and Smith, M.L. (1981). Meta-Analysis in Social Research. Beverly Hills: SAGE.
Harre', R. (1970). The principles of scientific thinking. Chicago: University of Chicago Press.
Harre', R. (1972). The philosophies of science. Oxford: Oxford University Press.
Harre', R. and Madden, E. H. (1975). Causal powers. Oxford: Basil Blackwell.
Harre', R. and Secord, P.F. (1973). The explanation of social behaviour. New Jersey: Littlefield, Adams.
House, E. R., Mathison, S., and McTaggert, R. (1989). Validity and teacher inference. Educational Researcher, 18, 7, 11-15.
Hume, D. (1978)[1740]. An abstract of a treatise of human nature. 2nd edition. Oxford: Clarendon Press, .
Keat, R. and Urry, J. (1982) Social theory as science. 2nd edition. London: Routledge and Kegan Paul.
Kuhn, T.S. (1962). The structure of scientific revolutions. Chicago: University of Chicago Press.
Mackie, J.L. (1974). The cement of the universe. Oxford: Clarendon Press.
Manicas, P.T. (1987). A history and philosophy of the social sciences. Oxford: Basil Blackwell.
Manicas, P. T. and Secord, P.F. (1983). Implications for psychology of the new philosophy of science. American Psychologist, 38, 399-413.
Manicas, P. T. and Secord, P.F. (1984). Implications for psychology: Reply to comments. American Psychologist. August, 922-926.
Mill, J. S. (1974)[1843]. A system of logic: Ratiocinative and inductive. University of Toronto Press.
Mill, J. S. (1866). August Comte and positivism. London: Trubner.
Norris, S. P. (1983). The inconsistencies at the foundation of construct validation theory. In E. R. House (Ed.),
Philosophy of Evaluation(pp. 53-74) San Francisco: Jossey-Bass.
Outhwaite, W. (1987). New philosophies of social science. London: MacMillan.
Phillips, D.C. (1983). After the wake: Postpostivistic educational thought. Educational Researcher, 13 (5), 4-12.
Phillips, D.C. (1990). Post-positivistic science: Myths and realities. In E. Guba (Ed). The paradigms dialog: options for social science inquiry. Beverly Hills, CA: SAGE Publishing.
Russell, B. (1957). Knowledge by acquaintance and knowledge by description. In Mysticism and logic. New York: Doubleday,
Scriven, M. (1956). Truisms as the grounds for historical explanation. In P. Gardiner (Ed.), Theories of history. New York: Free Press. (pp. 443-475).
Scriven, M.(1970) Explanations, predictions, and laws. In B. A. Brody (Ed). Readings in the philosophy of science (pp. 88-104). Englewood Cliffs, N. J.: Prentice Hall.
Scriven, M. (1966). Causes, connections, and conditions in history. In W.H. Dray (Ed). Philosophical analysis and history(pp. 240-254). New York: Harper and Row.
Secord, P. (1986). Explanation in the social sciences and in life situations. In D. W. Fiske and R. A. Shweder (Eds.), Metatheory in social science(pp. 197-221). Chicago: University of Chicago Press.
Suchman, E. A. (1967). Evaluative research. New York: Russell Sage.
Weir, E.E. (1982). Causal understanding in naturalistic evaluation of educational programs. Dissertation, University of Illinois, Urbana.
Thanks to Ken Howe, Bob Linn, and Dan Liston for helpful comments on this paper.
No comments:
Post a Comment