Quantitative Reviewing:
The Literature Review
as Scientific Inquiry
(research, statistical procedures, methodology)
Kenneth Ottenbacher
The l£terature review process is
conceptualized as a form of scientific inquiry that involves methodological requirements and inferences similar to those employed In
primary research. Five stages of
quantitative reviewing that parallel stages in primary investigation
are identified and briefly described. They include problem
formation, data collection, data
evaluation, analysis and interpretation, and reporting the results.
The first two stages provide
information and guidelines relevant to revzewers' employing traditional narrative procedures or
conducting reviews of qualitative
research literature. The final three
stages relate specifically to the
methodology of quantitative
revzewlng.
The argument is made that
quantitative reviewing procedures
represent a paradigm shift that can
assist researchers and clinicians in
occupational therapy to establish a
scientific data base that will serve
to guide theory development and
validate clinical practice.
umerous authorities have
commented on the need for
research in occupational therapy (I,
2). Gillelleand Kielhofner(3)argue
that the discipline's failure to meet
the challenge of increased research
may contribute to the profession's
failure to be recognized as a scientific, viable field.
Recently, Ottenbacher and Short
(4) documented an increase in particular types of empirical research
in the occupational therapy literature over the past decade. This increase in research evaluating the
theory and practice of occupational
therapy is a necessary component of
the profession's "scientific" development and expansion. The increased number of data-based papers examining the outcome of
therapeutic practice is a welcome
sign. It indicates that therapists are
adopting empirically based models
in attempts to validate therapeutic
programs. The application of the
traditional hypothesis-testing paradigm allows the investigator to establish the stability of a finding
based on probability (5). The traditional hypothesis-testing model,
originally advocated by Fisher, has
been shown to have certain limitations and weaknesses. Therefore,
alternate models have been proposed particularly for applied fields
where the strict experimental control associated with "true" experimental research is difficult to ob-
N
tain (6). Within the context of
experimental designs the hypothesis-testing model remains, however,
the most authoritative and widely
accepted in terms of "proving" or
establishing scientific fact (7).
Since the hypothesis-testing model is based on statistical probability,
there is the possibility that the
res ults of an y single study are due to
chance and therefore the possibility
of error exists (Type I and Type II
experimental error). For a particular research result to become established as a legitimate scientific
occurrence, that result must be
replicated numerous times. Fisher
originally stated, "we may say that a
phenomenon is experimentally
demonstrable when we know how
to conduct an experiment which
will rarely fail to give us statistically
significant results." (5, p 14)
Every time a particular hypothesis is tested and the results reported,
new information about the phenomenon under investigation is generated. After a large enough number
of studies investigating a particular
Kenneth Ottenbacher, Ph.D.,
OTR, is Assistant Professor,
Occupational Therapy Program,
School of Allzed Health Professions, University of Wisconsin,
Madison, 2110 Medical Sciences
Center, 1300 University Avenue,
Madison, Wisconsin 53706.
The American Journal of Occupational Therapy
Downloaded From: http://ajot.aota.org/ on 06/15/2017 Terms of Use: http://AOTA.org/terms
313
hypothesis have been conducted,
the studies containing the hypotheses tests are synthesized to determine what the aggregated investigations reveal. This building block
approach to the accumulation of
scientific knowledge has been referred to as "normal science" by
Kuhn and is a vital stage in the
development of useful knowledge
and theory (8).
Narrative reviews of the aggregated investigations have traditionally been used to achieve the goal of
synthesizing information contained
in multiple studies. These narrative
reviews have tended to be biased and
subjective in contrast to the conduct
of primary research that requires
strict control to ensure the "objectivity" of the investigation.
Pillemer and Light have defined
data synthesis or aggregation as,
"using formal procedures for combining the results from several
experiments." (9, p 117) Until rel. .
.
cently the tradltlOnal narratlve review has been the formal mechanism for data synthesis; however, as
bodies of research literature expand,
the limitations of narrative reviews
have been recognized. In discussing
the inadequacies of traditional literature reviews, Glass states that, "A
common method of integrating
several studies with inconsistent
findings is to carp on the design or
analysis deficiencies of all but a few
studies-those remaining frequently being one's own work or that of
one's students and friends-and
then advance the one or two "acceptable" studies as the truth of the matter." (\0, p 4) The subjective and
judgmental nature of traditional
literature reviews is unfortunate
because often these reviews of particular topic areas are instrumental
in establishing or refuting the empirical legitimacy of a finding.
Several authorities have recently
314
proposed the adoption of a methodology for reviewing research studies
that is based on the same standards
of objectivity and control as traditional primary research (II, 12).
The procedures are designed to treat
the review process as a unique type
of research endeavor that produces
quantitative results. The logic of
quantitative reviewing procedures
parallels that employed in primary
research investigations. The goal of
quantitative reviewing is to provide
a systematic mechanism for investigating variation in study characteristics such as sampling, design
procedures, and type and number of
dependent and independent variables and then to relate variation in
these variables to study outcomes.
As the number of empirical research reports in occupational therapy increases, therapists will begin
to rely more and more on the reviewing efforts of others. It will be
imperative that these reviews present the empirical literature as
accurately and completely as possible. The use of quantitative reviewing procedures can assist reviewers
in achieving this goal. In addition,
primary researchers may wish to use
quantitative reviewing techniques
to conduct more informative reviews of the Iiterature for a specific
study. The use of quantitative reviewing procedures may provide
the researcher with a more exact
notion of how to address a particular research question. Fi nally, quantitative reviewing procedures are
currently being used to complete
the requirements for a master's thesis or doctoral dissertation at some
institutions (13).
Investigators in the fields of psychology and education have employed quantitative reviewing procedures to provide a degree of
consensus to conflicting bodies of
literature that was not obtainable
May 1983, Volume 37, Number 5
Downloaded From: http://ajot.aota.org/ on 06/15/2017 Terms of Use: http://AOTA.org/terms
using traditional narrative reviewing techniques (14, 15). These procedures have recently been applied
to developing bodies of research
literature in occupational therapy.
For example, Ottenbacher (16) reported the results of a quantitative
review of the efficacy of sensory
integrati ve proced ures. This article
introduced the basic methodology
of quantitative reviewing and presented a practical application of
some meta-analytic techniques. The
purpose of this paper is to develop
the empirical rationale and research
paradigm associated with quantitative reviewing methodology and to
compare the stages or steps of quantitative reviewing with those of traditional primary research.
Quantitative Reviewing
At the outset it should be noted that
the quantitative reviewing methodology to be described is not appropriate for all forms of literature
review. The procedures are applicable for reviews designed to imply
generalizations about substantive
issues from a set of studies bearing
directly on these issues. Jackson (17)
refers to this type of review as an
integrative review.
The goal of integrative reviewers
is to summarize past research by
attempting to draw overall conclusions from studies believed to address a similar or identical hypothesis. Cooper (\1) argues that integrating separate research projects
involves scientific inferences as central to the validity of knowledge as
inferences involved in primary data
interpretation. Along similar lines,
Glass, McGaw, and Smith have
observed:
It is often said of experimental
research that it must be replicable to
be scientific. The true test of
whether a finding is replicable is to
replicate it; but as is observed ad
nauseum, such replications are seldom actually undertaken. Hence,
the scientific attitude is not to desire
actual replication, but such a description of a study that it could in
theory be replicated . ... Thereby,
science is guaranteed to be "intersubjective" rather than an endeavor
subject to the whims and idiosyncracies of individual researchers.
These values and standards are
mgrained in contemporary scientists' training but too often are forgotten when the context changes
slightly and the task is to integrate
numerous empirical studies instead
of to perform a single primary
study. Thus do reviews become idiosyncratic, authoritarian, subJectiveall those things that cut against the
scientific grain. (12, P 20)
Cooper (II) suggested that research reviewers should be required
to use the same rigorous methodology required of primary researchers
and that the research review process
be conceptualized as a unique type
of research endeavor complete with
formal stages. He also identified
five stages that characterize quantitative reviews. These stages are
comparable to similar stages found
in primary research. The five stages
are described below (ll):
Problem Formation. Like the
primary researcher, the reviewer employing quantitative methodology
first considers the research problem.
In its most basic form, the research
problem incl udes the definition of
variables and the rationale for relating the variables to one another.
The variables involved in any empirical inquiry must be defined in
two different ways. First, they must
be given conceptual definitions,
which include qualities that are
independent of time and space but
that distinguish events (18). Con-
ceptual definitions can differ in
abstractness, or in terms of the number of even ts to which they refer. As
Reynolds notes, "If one concept is
included in the meaning of another,
the second, or more general, concept
is considered more abstract." (19, p
50) Both primary researchers and
research reviewers must choose a
conceptual definition and a degree
of abstractness for variables contained in their problem.
Second, a variable must also be
operationally defined in order to
relate concepts to concrete events.
An operational definition is a set of
instructions describing the observable events that allow one to determine whether a concept is present
in a particular situation (19). Again,
both primary researchers and literature reviewers must specify the
operations included in their variable definitions.
Some differences between primary research a nd research reviews
can be identified at the stage of variable definition. Primary researchers
have little choice but to operationalize their concepts before the inquiry begins. Primary data collection cannot commence until variables have been given some circumscribed empirical reality. Reviewers
need not be quite so theoretically
rigorous. The literature search may
begin with only a conceptual definition and may then evaluate the
conceptual relevance of different
operations as they appear in the
li tera tu re.
Another distinction between primary research and research reviews
is that primary research usually
involves only one or two operational definitions of the same construct. In contrast, research reviews
may involve many empirical realizations of a particular concept. The
evidence retrieved by reviews thus
typically contains more method-
generated variance (i.e., multiple
operations) than evidence collected
as primary data.
Operational multiplicity, that is,
the fact that a particular concept
may be operationally defined in a
variety of ways, does more than distinguish one type of inquiry from
the other. It is also the most important source of variance in review
conclusions introduced at the problem definition stage. Operational
multiplicity can influence review
outcomes in two ways. First, the
operational definitions used in research reviews may vary. Two reviewers using an identical label for
an abstract concept may employ different operational definitions. Each
definition may contain some operations excluded by the other, or one
reviewer's definition may completely contain the other. Second, operational multiplicity can affect review
outcomes by allowing reviewers to
vary in the attention they pay to
methodological distinctions in the
literature. Some reviewers may pay
careful attention to study operations. They may decide to meticulously identify the operational and
sample distinctions among retrieved
studies. Other reviewers may feel
that method or participant-dependent relations are unlikely or they
may simply use less rigor. Because
of differences in attention to operations, two reviewers employing
identical conceptual definitions and
reviewing the same set of studies
may reach different conclusions. If
one reviewer retrieved more method
information and recognized a method-dependent relation that another
review did not test, the two conclusions could be conflicting. A similar phenomenon occurs in primary
research when, for example, one
researcher ignores individual differences related to the subjects' intellectual levels, while another re-
The Amencan Journal of Occupational TheratJy
Downloaded From: http://ajot.aota.org/ on 06/15/2017 Terms of Use: http://AOTA.org/terms
315
searcher includes intelligence measures in the design. The two investigators may uncover identical data
on the dependent variable, but the
former researcher may report no
effects, while the latter reports a
symmetric interaction between the
treatment and a trait related to cognitive abilities.
Dala Colleclion. Identifying
populations for research reviews is
complicated by the fact that reviews
involve two targets. The reviewer is
interested in reporting findings derived from all previous research in a
particular area. The method of information retrieval used allows the
reviewer a certain degree of direct
control over whether this goal is
attained. In addition, the reviewer
hopes to include studies that will
allow generalizations to the individuals that are represented in the
topic area. The reviewer's in fI uence
is limited here by the type of subjects who were sampled by the primary researchers. The primary researcher samples individuals, and
the reviewer retrieves studies. This
process is something akin to cluster
sampling (20) with clusters distinguishing people according to their
research projects. In reality, reviewers attempt to retrieve an entire
population of studies rather than to
draw representative samples of
studies from the literature. This
formidable goal is rarely achieved,
but it is more feasible in a review
than in primary research.
A reviewer can use at least five
techniques to retrieve studies on a
research problem. The most informal approach is through the "invisible college" (21). Crane notes that
"scientists working on similar problems are usually aware of each other
and in some cases attempt to systematize their contacts by exchanging reprints with one another." (22,
p 355) A second tech niq ue is the
316
ancestry approach in which one
retrieves information by "tracking"
citations from one study to another.
Most reviewers are aware of several
studies related to their problem
before they formally begin the literature search. These studies provide
bibliographies that cite earlier rela ted research.
A third retrieval technique is to
employ the descendency approach
using the Science or Social Science
Cilalion Indexes. Because citation
indexes are primaril y organized by
author (not topic), they are most
useful when particular researchers
or research papers are closel y associated with a problem area.
Reviewers can also use one or
more of the abstracting services
available in libraries, such as Index
Medicus or PsychologicalAbslrlJ.cls.
The final retrieval technique is the
on-line computer search. The computer search is a time-saving device
that can exhaustively scan several
abstracting services or citation indexes at a rapid rate. One possible
limitation to this method is that the
document indicators must be completely specified before the search
begins.
Discrepancies among review conclusions may result from the use of
different retrieval channels. The
studies available through various
sources may differ from source to
source. It is likely that two reviewers who use different techniques to
locate studies will end up with different evidence and potentially different conclusions. Diversity in information retrieval techniques is a
procedural variation introduced at
this stage that may affect review
concl usions. Therefore, reviewers
shou ld specify the retrieval methods
they employed, including descriptors and key words.
Evalualion of Data. The research
reviewer makes numerous qualita-
May 1983, Volume 37, Number 5
Downloaded From: http://ajot.aota.org/ on 06/15/2017 Terms of Use: http://AOTA.org/terms
tive judgments about each individual data point (study) after it is
retrieved. Each data point (study) is
carefully examined to determine
whether it meets certain predetermined criteria the reviewer has previously identified. These criteria are
designed to eliminate data points
(studies) not related to the problem
under investigation.
The first source of variation introduced during data examination is a
divergence in reviewers' criteria for
evaluating the quality of research.
For example, some reviewers may
decide to include only studies that
meet certain predetermined criteria
such as random assignment to treatment conditions, or blind recording
of the dependent measure.
Glass, et al. (12) suggest that all
potentially relevant studies be included in the review and'that no
prior judgments about the quality
of a study be made. The question of
"quality" can be addressed empirically by using the quantitative reviewing methodology. For example, in the quantitative review cited
earlier dealing with the efficacy of
sensory integrative therapy, studies
were coded based on the type of
design and assignment procedures
that were used. These factors were
then quantitatively compared to
study outcomes to determine
whether the results of studies using
random assignment to treatment
and control groups differed from
those studies using some form of
matching or simply using pre-existIng groups.
A second source of variance in
review conclusions that occurs during data examination is the degree
to which factors other than research
quality affect evaluative decisions.
One obvious extraneous factor is
the reviewer's expectation concerning the review outcome. As Mahoney observed, "Confirmatory ex-
periences are selecti vel y welcomed
and granted easy credibility. Disconfirmatory experiences. on the
other hand, are often ignored, discredited or treated with obvious
defensiveness." (23, pp 161-162).
Another threat to validity during
this stage, which is beyond the control of the reviewer, involves the
potential for unreliable outcomes
due to incomplete reporting by
primary researchers. Many research
reports omit discussion of some
hypotheses tested. Other reports
give only incomplete information
of the tests mentioned. If a reviewer
must estimate or omit what happened in these studies, wide confidence intervals must be placed
around review conclusions.
Analysis and Interpretation.
Glass (10) suggested that data analysis can be categorized into three
distinct activities: primary secon·
dary. and meta-analysis. Primary
anal ysis in vol ves anal yzi ng the data
from primary research for the first
time. Secondary analysis is the reanalysis of data from primary research reports. Meta-analysis occurs
when the results of independent
experiments are combined for the
purpose of statistically integrating
the findings.
Tests of statistical significance
have been the sine qua non of data
analysis in behavioral science research for the past 20 years (24). The
value of significance testing. however, has been questioned from several perspectives. It has been argued
that a null relationship rarely exists
in nature and thus the assumption
of a null hypothesis for statistical
testing is unwarranted (25), that
significance testing is strongly influenced by sample size (26), and
that significance testing establishes
the direction of a finding but not
the magnitude (25).
Significance testing. which com-
pares an observed relation to the
chance of no relation, becomes less
informative as evidence supporting
a phenomenon accumulates. The
question of importance to reviewers
employing quantitative research
methodology turns from whether
an effect exists to how much of an
effect exists.
Glass (27) and Cohen (28) popularized procedures capable of uncovering systematic variation in
study results. The procedures involve the calculation of study effect
sizes and the correlation of these
with study characteristics. Cohen
defines an effect sizes measure as
"the degree to which the null
hypothesis is false." (28, pp 9-10)
Effect size measures appropriate
for every kind of research design
and analysis have been cataloged
and presented by Cohen (28). For
instance. an effect size index referred to as the d-index has been
proposed for eval uating the sta tis tical comparison between twO groups.
The d-index is a number that tells
how far apart two group means are
in terms of their common standard
deviation. If d = .2, it indicates that
2/ IOths of a standard deviation separates the two sample means. Cohen
(28) has defined a d-index of .2 as a
"small" effect size, with .5 being
moderate, and a d of .8 or greater
considered a large effect size. This
classification system may leave
something to be desired in terms of
intuitive appeal. For this reason,
Cohen (28) also presents a "percentage of distribution overlap" measure. The overlap measure most
frequently employed in reviews of
the behavioral sciences literature is
called US. This measure tells what
percentage of the population with
the smaller mean is exceeded by 50
percent of the population with the
larger mean. The U3 value for a
d-index of .2 is 58.0. This value
indicates that the average person in
the group with the larger mean has
a "score" greater than 58 percent of
the individuals in the lower mea ned
group. A table for converting the
d-index to U3 is presented by Cohen
(28, p 22).
Friedman (29) presented a formula for computing d-index estimates from traditional inferential
statistical values such as t and F
ratios. In instances where the inferential statistical values are not reported, an estimated effect-size index may be computed from significance levels and sample size.
Finally, Glass (27) described procedures for computing effect sizes in
situations where nonparametric
statistics or some form of descriptive statistics such as percentages
are used.
Interpretation of Effect Sizes. Interpretation of effect sizes is crucial
to the understanding of quantitative reviews. An example may help
to clarify the concept. Suppose a
resea rcher cond ucted aqua n ti ta ti ve
review of studies investigating the
effect of sensory stimulation procedures on the neuromotor development of premature infants. Let us
further assume that our hypothetical researcher retrieved 50 studies
that provided statiistical data evaluating the effect of a program of
sensory stimulation versus a control
condition on the neuromotor development of premature infants. Finally, let us assume that the overall
d-index for the comparisons was
found to be 1.20.
This d-index may be interpreted
in the same manner as a standard
Z-score if the assumption of a normal distribution of outcomes IS
made. Under the assumption of
normality, the meaning of the dindex may be viewed in relation to
overlapping distributions of scores
for treatment and control groups.
The American Journal of Occupational Therapy
Downloaded From: http://ajot.aota.org/ on 06/15/2017 Terms of Use: http://AOTA.org/terms
317
Figure 1
d-index = 1.20
Treatment
Groups
Control
Groups
tion in which the reviewer would
discuss the analyses employed, including the type of effect size used,
the unit of analysis, and the actual
outcomes. Finally, a Discussion/
Concl usion section would allow the
reviewer to summarize the findings,
compare them to previous narrative
reviews and primary research, and
suggest areas in need of further
study. Several quantitative reviews
referred to previously have used this
format (14-16).
Conclusions
aa.5th
50th
Percentile of Control Groups
Figure I presents the distribution
for the treatment groups (receiving
sensory stimulation) and the control groups (not receiving stimulation). The hypothetical d-index of
1.20 indicates a superiority of slightJ y more than one standard deviation
for the treatment groups. The accompanying U3 value for ad-index
of 1.20 is 88.5. The U3 value means
that the average of the curve for the
treatment groups receiving sensory
stimulation is located above 88.5
percen t of the area under the control
group curve. This U3 also suggests
that the average subject in the treatment groups was "better off" than
88.5 percent of the subjects in the
control groups, while only 11.5 percent of the control group subjects
were improved when compared to
the average subject receiving sensory stimulation.
Reporting Results. The final
318
stage in the research review paradigm is reporting the findings.
Cooper (II) suggests that research
reviewers usi ng q uan ti tati ve methods present their findings in a format similar to that used by primary
researchers in reporting their results. In other words, the presentation should include an Introduction section in which the problem is
identified and discussed, previous
reviews of the literature are presented or summarized, and the need
for a quantitative review is established. This would be followed by a
Methods section in which the reviewer would specifically identify
the sources used to retrieve studies,
how many studies were retrieved,
and how they were coded, that is,
what information was extracted
from each report and ho"v this was
accomplished. The Methods section
would be followed by a Results sec-
May 1983, Volume 37, Numbe1· 5
Downloaded From: http://ajot.aota.org/ on 06/15/2017 Terms of Use: http://AOTA.org/terms
Cooper and Rosenthal (30) compared the use of quantitative versus
traditional procedures for summarizing research findings. The results
demonstrated that the use of quantitative techniques increased the
perceived support for, and the estimated magnitude of, the effects
being reviewed. Cooper and Rosenthal note that "conclusions based
on meta-analysis will appear to be
(and indeed they will be) more
rigorous and objective. Some of the
confusion and contradiction we
often convey about our research
may not be a function of the results
we found but of how we have chosen
to synthesize them." (30, p 449)
The rigor and objectivity of the
procedures described in previous
sections and referred to by Cooper
and Rosenthal (30) have the potential to enhance the meaningfulness
and usefulness of empirical literature produced in occupational therapy. The application of the quantitative reviewing procedures described in this paper can help synthesize a scien tific information base
by clarifying and defining review
procedures, establ ishing study-effect
sizes in areas of research interest,
and explaining how relationships
of interest are modified by study
variables and characteristics.
Obviously, quantitative review-
ing procedures are not a panacea.
The procedures contain aspects of
both art and science, as does all
research. The science is revealed in
the systematic application and definition of a research methodology
related to Ii terature reviewing, while
the art refers to the judgments that
need to be made in the application
of the procedures. Like all research
methods, quantitative research reviewing involves assumptions that
must be made explicit, and if these
assumptions are not clear to the
user or the reader, misleading results and conclusions may occur.
The ability of quantitative reviewing procedures to test or evaluate certain interactions or relationships contained within aggregated
studies does not mean thatall problems of conceptualization or methodological artifact can be resolved
in this manner. Alternate conceptualizations may rival those based
on multiple studies in the same
manner that alternate conceptualizations may account for the results
of a particular primary investigation.
Some readers may feel that the use
or abuse of quantitative reviewing
procedures may create an ill usion of
"statistical objectivity" not justified
by the data obtained from the review. In terms of specific statistical
procedures, the fact that multiple
hypotheses tests may be included in
a single study suggests that effect
sizes based on those hypotheses tests
are not independent data points.
This introduces the problem of nonindependence into the analysis and
may result in inflated or unreliable
estimates if inferential statistical
procedures are used to analyze the
data. The role of inferential statistical procedures in the data analysis
stage of quantitative reviewing is
controversial and beyond the scope
of this paper (12).
In spite of the limitations cited
above, quantitative reviewing procedures represent a significant advance over the traditional narrative
methods of integrating quantitative
research in an area of interest. They
appear particularly relevant to disciplines such as occupational therapy that are in the initial stages of
developing an empirical foundation to facilitate theory development and validate practice. The use
of quantitative reviewing procedures represents a paradigm shift in
which the literature review is conceptualized as a form of scientific
inquiry in its own right. The use of
such procedures should assist researchers in the beha vioral sciences
to establish scientifically valid data
bases to direct future investigation
and guide theory development.
REFERENCES
1. Kielhofner G, Burke JP: A model of
human occupation, Part 1. Structure
and content. Am J Occup Ther 34:572581, 1980
2. Reilly M: The educational process. Am
J Occup Ther 23: 299-307, 1969
3. Gillette N, Kielhofner G: The impact of
specialization on the professionalization and survival of occupational therapy. Am J Occup Ther 33: 20-28, 1979
4. Ottenbacher K, Short MA: Publication
trends in occupational therapy. Occup
Ther J Res 2: 80-88, 1982
5. Fisher R: The Design of Experiments
(4th Edition). Edinburgh: Olwer &
Boyd,1947
6. Cook TO, Campbell 0: Ouasi-Experimental Design and Analysis, Issues for
Field Settings, Boston: Houghton-Mifflin, 1979
7. Campbell P, Stanley J: Experimental
and Quasi-Experimental Design for
Research, Chicago: Rand McNally,
1966
8. Kuhn TS: The Structure of Scientific
Revolutions (2nd Edition). Chicago:
University of Chicago Press, 1970
9. Pille mer DB, Light RJ: SynthesiZing
outcomes: How to use research from
many studies. Harvard Educ Rev 50:
176-195,1980
10.Glass GV: Primary, secondary and
meta-analysis of research. Educ Res 5:
3-8, 1976
11. Cooper HM: Scientific guidelines for
conducting integrative research reviews. Rev Educ Res 52: 291-302,1982
12. Glass GV, McGaw B, Smith ML: MetaAnalysis in Social Research, Beverly
Hills: Sage, 1981
13. Cohen PA: A Meta-Analysis of the
Relationship between Student Ratings
of Instruction and Student Achievement, Doctoral Dissertation. Ann
Arbor: University of Michigan, 1980
14. Kavale K: The relationship between
auditory perceptual skills and reading
ability: A meta-analysis. J Learn Disabil 14: 539-546, 1981
15. Cooper H, Burger J, Good T: Gender
differences in the academic locus of
control beliefs of young children. J
Person Soc Psycho/40: 562-572, 1981
16.0ttenbacher K: Sensory integrative
therapy: Affect or effect. Am J Occup
Ther 36: 571-578,1982
17. Jackson G: Methods for integrative
reviews. Rev Educ Res 50: 438-460,
1980
18. Carlsmith J, Ellsworth P, Aronson E:
Methods of Research in Social Psychology, Reading, MA: Addison-Wesley, 1976
19. Reynolds P: A Primer in Theory Construction, Indianapolis: Bobbs-Merrill,
1971
20. Williams B: A Sampler on Sampling,
New York: Wiley, 1978
21. Price S Collaboration in an invisible
college. Am Psychol 21: 1011-1018,
1966
22. Crane 0: Social structure in a group of
scientists: A test of the "invisible college" hypothesis. Am Soc Rev 34: 335352, 1969
23. Mahoney M: Publication prejudices:
An experi mental study of confirmatory
bias in the peer review system. Cog
Ther Res 1: 161-175, 1977
24. Edington E: A new tabulation of statistical procedures used in APA journals.
Am Psycho/29: 25-26,1974
25. Bakan 0: The test of significance in
psychological research. Psychol Bull
66: 423-437, 1966
26.0'Brien TC, Shapiro B: Statistical
significance-what? Math Teach 61:
673-676, 1968
27. Glass G V: Integrating findings: The
meta-analysis of research. In Review of
Research in Education (Vol 5). Itasca,
IL: F. E. Peacock, 1977
28. Cohen J: Statistical Power Analysis for
the Behavioral Sciences (Rev Edition).
New York: Academic Press, 1977
29. Friedman H: Magnitude of experimental effect and a table for its estimation.
Psychol Bull 70: 245-251
30. Cooper HM, Rosenthal R: Statistical
versus traditional procedures for summarizing research findings. Psychol
Bull 87: 442-449, 1980
The American Journal of Occupational Therapy
Downloaded From: http://ajot.aota.org/ on 06/15/2017 Terms of Use: http://AOTA.org/terms
319