|
Clinical Epidemiology & Evidence-Based Medicine
Glossary:
Clinical Study Design and Methods Terminology
Updated August 22, 1999
Contents:
- Clinical Study Types: (In order from
strongest to weakest empirical evidence inherent to the design when properly
executed.)
- Experimental Studies:
The hallmark of the experimental study is that the allocation
or assignment of individuals is under control of investigator and thus can be randomized.
The key is that the investigator controls the assignment of the exposure or of the
treatment but otherwise symmetry of potential unknown confounders is maintained through
randomization. Properly executed experimental studies provide the strongest empirical
evidence. The randomization also provides a better foundation for statistical procedures
than do observational studies.
- Randomized Controlled Clinical Trial (RCT):
A prospective, analytical, experimental
study using primary data generated in the clinical environment. Individuals similar at the
beginning are randomly allocated to two or more treatment groups and the outcomes the
groups are compared after sufficient follow-up time. Properly executed, the RCT is the
strongest evidence of the clinical efficacy of preventive and therapeutic procedures in
the clinical setting.
- Randomized Cross-Over Clinical Trial:
A prospective, analytical, experimental study
using primary data generated in the clinical environment. Individuals with a chronic
condition are randomly allocated to one of two treatment groups, and, after a sufficient
treatment period and often a washout period, are switched to the other treatment for the
same period. This design is susceptible to bias if carry over effects from the first
treatment occur. An important variant is the "N of One" clinical trial in which
alternative treatments for a chronically affected individual are administered in a random
sequence and the individual is observed in a double blind fashion to determine which
treatment is the best.
- Randomized Controlled Laboratory Study:
A prospective, analytical, experimental
study using primary data generated in the laboratory environment. Laboratory studies are
very powerful tools for doing basic research because all extraneous factors other than
those of interest can be controlled or accounted for (e.g., age, gender, genetics,
nutrition, environment, co-morbidity, strain of infectious agent). However, this control
of other factors is also the weakness of this type of study. Animals in the clinical
environment have a wide range of all these controlled factors as well as others that are
unknown. If any interactions occur between these factors and the outcome of interest,
which is usually the case, the laboratory results are not directly applicable to the
clinical setting unless the impact of these interactions are also investigated.
- Observational Studies:
The allocation or assignment of factors is not under control
of investigator. In an observational study, the combinations are self-selected or are
"experiments of nature". For those questions where it would be unethical to
assign factors, investigators are limited to observational studies. Observational studies
provide weaker empirical evidence than do experimental studies because of the potential
for large confounding biases to be present when there is an unknown association between a
factor and an outcome. The symmetry of unknown confounders cannot be maintained. The
greatest value of these types of studies (e.g., case series, ecologic, case-control,
cohort) is that they provide preliminary evidence that can be used as the basis for
hypotheses in stronger experimental studies, such as randomized controlled trials.
- Cohort (Incidence, Longitudinal Study)
Study: A prospective, analytical,
observational study, based on data, usually primary, from a follow-up period of a group in
which some have had, have or will have the exposure of interest, to determine the
association between that exposure and an outcome. Cohort studies are susceptible to bias
by differential loss to follow-up, the lack of control over risk assignment and thus
confounder symmetry, and the potential for zero time bias when the cohort is assembled.
Because of their prospective nature, cohort studies are stronger than case-control studies
when well executed but they also are more expensive. Because of their observational
nature, cohort studies do not provide empirical evidence that is as strong as that
provided by properly executed randomized controlled clinical trials.
- Case-Control Study:
A retrospective, analytical, observational study often based on
secondary data in which the proportion of cases with a potential risk factor are compared
to the proportion of controls (individuals without the disease) with the same risk factor.
The common association measure for a case-control study is the odds ratio. These studies
are commonly used for initial, inexpensive evaluation of risk factors and are particularly
useful for rare conditions or for risk factors with long induction periods. Unfortunately,
due to the potential for many forms of bias in this study type, case control studies
provide relatively weak empirical evidence even when properly executed.
- Ecologic (Aggregate) Study:
An observational analytical study based on aggregated
secondary data. Aggregate data on risk factors and disease prevalence from different
population groups is compared to identify associations. Because all data are aggregate at
the group level, relationships at the individual level cannot be empirically determined
but are rather inferred from the group level. Thus, because of the likelihood of an
ecologic fallacy, this type of study provides weak empirical evidence.
- Cross-Sectional (Prevalence Study)
Study: A descriptive study of the
relationship between diseases and other factors at one point in time (usually) in a
defined population. Cross sectional studies lack any information on timing of exposure and
outcome relationships and include only prevalent cases.
- Case Series:
A descriptive, observational study of a series of cases, typically
describing the manifestations, clinical course, and prognosis of a condition. A case
series provides weak empirical evidence because of the lack of comparability unless the
findings are dramatically different from expectations. Case series are best used as a
source of hypotheses for investigation by stronger study designs, leading some to suggest
that the case series should be regarded as clinicians talking to researchers.
Unfortunately, the case series is the most common study type in the clinical literature.
- Case Report:
Anecdotal evidence. A description of a single case, typically
describing the manifestations, clinical course, and prognosis of that case. Due to the
wide range of natural biologic variability in these aspects, a single case report provides
little empirical evidence to the clinician. They do describe how others diagnosed and
treated the condition and what the clinical outcome was.
[Return to
Section Contents List]
[Return to Glossary Contents List]
- Validity vs. Bias:
- Validity:
Truth
- External Validity (Generalizability):
Truth beyond a study. A study is external
valid if the study conclusions represent the truth for the population to which the results
will be applied because both the study population and the reader’s population are
similar enough in important characteristics. The important characteristics are those that
would be expected to have an impact on a study’s results if they were different
(e.g., age, previous disease history, disease severity, nutritional status, co-morbidity,
...). Whether or not the study is generalizable to the population of interest to the
reader is a question only the reader can answer. External validity can occur only if the
study is first internally valid.
- Internal Validity:
Truth within a study. A study is internally valid if the study
conclusions represent the truth for the individuals studied because the results were not
likely due to the effects of chance, bias, or confounding because the study design,
execution, and analysis were correct. The statistical assessment of the effects of chance
is meaningless if sufficient bias has occurred to invalidate the study. All studies are
flawed to some degree. The crucial question that the reader must answer is whether or not
these problems were great enough that the study results are more likely due to the flaws
than the hypothesis under investigation.
- Symmetry Principle:
In a study, the principle of keeping all things between groups
similar except for the treatment of interest. This means that the same instrument is used
to measure each individual in each group, the observers know the same things about all
individuals in all groups, randomization is used to obtain a similar allocation of
individuals to each group, the groups are followed at the same time, ... .
- Confounding:
Confounding is the distortion of the effect of one risk factor by the
presence of another. Confounding occurs when another risk factor for a disease is also
associated with the risk factor being studied but acts separately. Age, breed, gender and
production levels are often confounding risk factors because animals with different values
of these are often at different risk of disease. As a result of the association between
the study and confounding risk factor, the confounder is not distributed randomly between
the group with the study risk factor and the control group without the study factor.
Confounding can be controlled by restriction, by matching on the confounding variable or
by including it in the statistical analysis.
- Bias (Systematic Error):
Any process or effect at any stage of a study from its
design to its execution to the application of information from the study, that produces
results or conclusions that differ systematically from the truth. Bias can be
reduced only by proper study design and execution and not by increasing sample size (which
only increases precision by reducing the opportunity for random chance deviation
from the truth). Almost all studies have bias, but to varying degrees. The critical
question is whether or not the results could be due in large part to bias, thus making the
conclusions invalid. Observational study designs are inherently more susceptible to bias
than are experimental study designs.
- Confounding Bias:
Systematic error due to the failure to account for the effect of
one or more variables that are related to both the causal factor being studied and the
outcome and are not distributed the same between the groups being studied. The different
distribution of these "lurking" variables between groups alters the apparent
relationship between the factor of interest and the outcome. Confounding can be accounted
for if the confounding variables are measured and are included in the statistical models
of the cause-effect relationships.
- Ecological (Aggregation) Bias (Fallacy):
Systematic error that occurs when an
association observed between variables representing group averages is mistakenly
taken to represent the actual association that exists between these variables for individuals.
This bias occurs when the nature of the association at the individual level is different
from the association observed at the group level. Data aggregated from individuals (e.g.
census averages for a region) or proxy data from other sources (e.g., the amount of
alcohol distributed in a region is a proxy for the amount of alcohol by individuals in
that region) are often easier and less expensive to acquire than are data directly from
individuals.
- Measurement Bias:
Systematic error that occurs when, because of the lack of blinding
or related reasons such as diagnostic suspicion, the measurement methods (instrument, or
observer of instrument) are consistently different between groups in the study.
- Screening Bias:
The bias that occurs when the presence of a disease is detected
earlier during its latent period by screening tests but the course of the disease is not
be changed by earlier intervention. Because the survival after screening detection is
longer than survival after detection of clinical signs, ineffective interventions appear
to be effective unless they are compared appropriately in clinical trials.
- Reader Bias:
Systematic errors of interpretation made during inference by the user
or reader of clinical information (papers, test results, ...). Such biases are due to
clinical experience, tradition, credentials, prejudice and human nature. The human
tendency is to accept information that supports pre-conceived opinions and to reject or
trivialize that which does not support preconceived opinions or that which one does not
understand. (JAMA 247:2533)
- Sampling (Selection) Biases:
Systematic error that occurs when, because of design
and execution errors in sampling, selection, or allocation methods, the study comparisons
are between groups that differ with respect to the outcome of interest for reasons other
than those under study.
- Zero Time Bias:
The bias that occurs in a prospective study when individuals are
found and enrolled in such a fashion that unintended systematic differences occur between
groups at the beginning of the study (stage of disease, confounder distribution). Cohort
studies are susceptible to zero-time bias if the cohort is not assembled properly.
- Bias Effect:
- Non-differential Bias:
Opportunities for bias are equivalent in all study groups,
which biases the outcome measure of the study toward the null of no difference between the
groups.
- Differential Bias:
Opportunities for bias are different in different study groups,
which biases the outcome measure of the study in unknown ways. Case-control studies are
highly susceptible to this form of bias between the case and control groups.
[Return to
Section Contents List]
[Return to Glossary Contents List]
- Study Objective, Direction and Timing:
- Analytic (Explanatory) Study:
The objective of an analytic study is to make causal
inferences about the nature of hypothesized relationships between risk factors and
outcomes. Statistical procedures are used to determine if a relationship was likely to
have occurred by chance alone. Analytic studies usually compare two or more groups, such
as case-control studies, cohort studies, randomized controlled clinical trials, and
laboratory studies.
- Descriptive Study:
The objective of a descriptive study is to describe the
distribution of variables in a group. Statistics serve only to describe the precision of
those measurements or to make statistical inferences about the values in the population
from which the sample was taken.
- Contemporary (Concurrent) Comparison:
Comparison is between two groups experiencing
the risk factor or the treatment at the same time. Contemporary comparison has the major
advantages that symmetry of unknown risk factors for the condition that change over time
is maintained and that measurement procedures can be performed as similarly as possible on
both groups.
- Historical (Non-concurrent) Comparison:
Comparison is of the same group or between
groups at different times that are not experiencing the risk factor or the treatment at
the same time. Historical comparison is often used to allow a group to serve as its own
historical control or is done implicitly when a group is compared to expected standards of
performance. This design provides weak evidence because symmetry isn’t assured. It is
very susceptible to bias by changes over time in uncontrollable, confounding risk factors,
such as differences in climate, management practices and nutrition. Bias due to
differences in measuring procedures over time may also account for observed differences.
- Prospective Study (Data):
Data collection and the events of interest occur after
individuals are enrolled (e.g. clinical trials and cohort studies). This prospective
collection enables the use of more solid, consistent criteria and avoids the potential
biases of retrospective recall. Prospective studies are limited to those conditions that
occur relatively frequently and to studies with relatively short follow-up periods so that
sufficient numbers of eligible individuals can be enrolled and followed within a
reasonable period.
- Retrospective Study (Data):
All events of interest have already occurred and data
are generated from historical records (secondary data) and from recall (which may result
in the presence of significant recall bias). Retrospective data is relatively inexpensive
compared to prospective studies because of the use of available information and is
typically used in case-control studies. Retrospective studies of rare conditions are much
more efficient than prospective studies because individuals experiencing the rare outcome
can be found in patient records rather than following a large number of individuals to
find a few cases.
[Return to
Section Contents List]
[Return to Glossary Contents List]
- Other Terms:
- Baseline: Health state (disease severity, confounding conditions) of individuals
at beginning of a prospective study. A difference (asymmetry) in the distributions of
baseline values between groups will bias the results.
- Blinding (Masking):
Blinding is those methods to reduce bias by preventing observers
and/or experimental subjects involved in an analytic study from knowing the hypothesis
being investigated, the case-control classification, the assignment of individuals or
groups, or the different treatments being provided. Blinding reduces bias by preserving
symmetry in the observers’ measurements and assessments. This bias is usually not due
to deliberate deception but is due to human nature and prior held beliefs about the area
of study.
- Placebo:
A placebo is the shame treatment used in a control group in place of the
actual treatment. If a drug is being evaluated, the inactive vehicle or carrier is used
alone so it is as similar as possible in appearance and in administration to the active
drug. Placebos are used to blind observers and, for human trials, the patients to which
group the patient is allocated.
- Case Definition:
The set of history, clinical signs and laboratory findings that are
used to classify an individual as a case or not for an epidemiological study. Case
definitions are needed to exclude individuals with the other conditions that occur at an
endemic background rate in a population or other characteristics that will confuse or
reduce the precision of a clinical study.
- Cohort:
A group of individuals identified on the basis of a common experience or
characteristic that is usually monitored over time from the point of assembly.
- Experimental Unit, Unit of Concern (EU):
In an experiment, the experimental unit are
the units that are randomly selected or allocated to a treatment and the unit upon which
the sample size calculations and subsequent data analysis must be based. Experimental
units are often a pen of animals or a cage of mice rather than the individuals themselves.
Analyzing data on an individual basis when groups (herds, pens) have been the basis of
random allocation is a serious error because it over-estimates precision, possibly biasing
the study toward a false-positive conclusion.
[Return to
Section Contents List]
[Return to Glossary Contents List]
- Sample Selection / Allocation
Procedures:
- Matching:
When confounding cannot be controlled by randomization, individual cases
are matched with individual controls that have similar confounding factors, such as age,
to reduce the effect of the confounding factors on the association being investigated in
analytic studies. Most commonly seen in case-control studies.
- Restriction (Specification):
Eligibility for entry into an analytic study is
restricted to individuals within a certain range of values for a confounding factor, such
as age, to reduce the effect of the confounding factor when it cannot be controlled by
randomization. Restriction limits the external validity (generalizability) to those with
the same confounder values.
- Census:
A sample that includes every individual in a population or group (e.g.,
entire herd, all known cases). A census not feasible when group is large relative to the
costs of obtaining information from individuals.
- Haphazard, Convenience, Volunteer, Judgmental Sampling:
Any sampling not involving a
truly random mechanism. A hallmark of this form of sampling is that the probability that a
given individual will be in the sample is unknown before sampling;. The theoretical basis
for statistical inference is lost and the result is inevitably biased in unknown ways.
Despite their best intentions, humans cannot choose a sample in a random fashion without a
formal randomizing mechanism.
- Consecutive (Quota) Sampling:
Sampling individuals with a given characteristic as
they are presented until enough with that characteristic are acquired. This method is okay
for descriptive studies but unfortunately not much better than haphazard sampling for
analytical observational studies.
- Random Sampling:
Each individual in the group being sampled has a known probability
of being included in the sample obtained from the group before the sampling occurs.
- Simple Random Sampling / Allocation:
Sampling conducted such that each eligible
individual in the population has the same chance of being selected or allocated
to a group. This sampling procedure is the basis of the simpler statistical analysis
procedures applied to sample data. Simple random sampling has the disadvantage of
requiring a complete list of identified individuals making up the population (the list
frame) before the sampling can be done.
- Stratified Random Sampling:
The group from which the sample is to be taken is first
stratified on the basis of a important characteristic related to the problem at hand
(e.g., age, parity, weight) into subgroups such that each individual in a subgroup has the
same probability of being included in the sample but the probabilities are different
between the subgroups or strata. Stratified random sampling assures that the different
categories of the characteristic that is the basis of the strata are sufficiently
represented in the sample but the resulting data must be analyzed using more complicated
statistical procedures (such as Mantel-Haenszel) in which the stratification is taken into
account.
- Cluster Sampling:
Staged sampling in which a random sample of natural groupings of
individuals (houses, herds, kennels, households, stables) are selected and then sampling
all the individuals within the cluster. Cluster sampling requires special statistical
methods for proper analysis of the data and is not advantageous if the individuals are
highly correlated within a group (a strong herd effect).
- Systematic Sampling:
From a random start in first n individuals, sampling every nth
animal as they are presented at the sampling site (clinic, chute, ...). Systematic
sampling will not produce a random sample if a cyclical pattern is present in the
important characteristics of the individuals as they are presented. Systematic sampling
has the advantage of requiring only knowledge of the number of animals in the population
to establish n and that anyone presenting the animals is blind to the sequence so they
cannot bias it.
[Return to
Section Contents List]
[Return to Glossary Contents List]
|