|
Introduction Randomized
clinical trials (RCT) are indispensable in phase-III-drug research in order to obtain
solid information on efficacy and safety before a new drug may be marketed. For good
reasons (e. g. safety, good observability of effects, self determination of patients)
mostly only highly selected patients are enrolled in RCTs. This fact constitutes an
essential need for phase-IV drug research. [1-4]
One element of phase-IV research is the conduct of large observational cohort studies
(OCS) which are called in Germany "Anwendungsbeobachtungen". These studies are
performed in order to observe efficacy and potential side effects of new drugs under
everyday conditions and with a minimum of intervention. In contrast to RCT, OCS typically
include patients from more extreme age groups, patients with comorbidity or other risk
factors. In order to cover a wide spectrum of patients and to observe rare events with
sufficiently high probability, OCS enroll a large number of patients, typically several
thousands. Due to the size of an OCS and due to its non-interventional character, it is
often necessary to observe the therapeutic effect by using a cheap and preferably non
invasive method which may provide less valid results than an optimal available method
("gold standard").
It is the aim of this paper to discuss an approach which may be suited to enhance the
achievable evidence from OCS. This goal can be reached by integrating a RCT into an OCS -
thus constituting a reference-controlled observational cohort study (RCOCS). The nested
RCT allows investigators to
- validate (and correct) effects observed in an OCS
- assess therapeutic effects observed in the OCS.
The validation approach may also be used for the assessment of observed effects
obtained by using surrogate markers in an OCS on the efficacy of preventive measures.
Design
and conduct of a reference-controlled observational study (RCOCS)
It is proposed to integrate a standard RCT (sample size about 100 to 200 patients per
treatment group) into a large OCS (usually several thousand patients). If the OCS is
treatment oriented, i. e., it aims to collect observations on a single drug, the nested
RCT would be placebo-controlled if this is ethically justified. If the OCS is indication
oriented, i. e., if it aims to compare observed effects of several drugs, randomizing in
the nested RCT would relate to these drugs.
It is an important feature of the RCOCS that for the OCS and for the nested RCT an
identical documentation of core information is performed. The core documentation has to be
kept as small as possible and usually includes the following items:
- Demographic characteristics
- Signs and symptoms of the disease
- Severity of the disease
- Concomitating diseases and medications
- Daily doses and application scheme of the medication
- Course of the disease during intermediate visits
- Indicators of therapeutic effects (objective measures, subjective assessment by patients
and physicians)
- Adverse events
If necessary for a validation study, additional items may be documented in the nested
RCT, e. g. results of advanced or invasive diagnostic procedures. These additional items
may be needed for the observation and assessment of outcome criteria and/or relevant
covariables (e. g. baseline variables, confounders, effect modifiers).
Dose regimens for the drugs of interest and the sequence of examinations during the
course of the study should be identical for the nested RCT and the OCS.
Medical inclusion criteria for patients enrolled into the nested RCT may be less
restrictive than in phase-III studies and should, therefore, generally be comparable to
the patients observed in the OCS. However, the spectrum of indications and relevant
covariables may still be wider in the OCS than in the nested RCT. Therefore, a careful and
identical documentation of the corresponding items is needed in order to perform an
adequate evaluation (see next section).
For reasons of observational comparability it would be desirable that the same
physicians treat patients within the OCS and the nested RCT. However, for practical
reasons (see discussion) this is presently not suggested.
Evaluation of results
Conventional separate analysis of the OCS and the nested RCT
In a first step of evaluating the results from a RCOCS, standard analyses of the OCS
and of the nested RCT are performed.
In addition, within the OCS a subgroup of patients should be identified who are - by
means of baseline values - well comparable with the patients enrolled in the nested RCT.
This identification may be achieved by a n:m matching on relevant variables. A separate
evaluation of the observations from the above defined subgroup should be compared with the
evaluation of the remainder of the OCS.
If the result of the nested RCT is negative, e. g., if a statistically significant or
medically relevant treatment efficacy cannot be demonstrated, it has to be decided whether
it is useful to carry on with further analyses. This will depend on careful comparisons
with results from the preceding successful RCT in phase III.
Validation study
If outcome measures or determination of relevant covariates are obtained by additional,
more valid techniques in the RCT, a validation study has to be performed. The basic idea
of this approach is to correct imprecise observations in the OCS by using information
which is gained from parallel measurements /observations within the nested RCT. The
corresponding statistical techniques are well known - although, unfortunately, less
frequently applied! - in the epidemiological literature. Adequate methods have been
proposed for qualitative and quantitative variables, for the correction of outcome
measures and confounders, for univariate and multivariate situations. For further details
see e. g. [5] to [23]. Therefore, there is a wide spectrum of methods which can be applied
to the clinical studies.
If an outcome measure in the OCS is effected by nondifferential misclassification, the
observed effects in the OCS will increase by the use of correction terms, however, the
corresponding confidence intervals will generally be enlarged.
The same validation techniques may also be considered for evaluating the efficacy of a
preventive measure in an OCS by means of surrogate markers if both the surrogate marker
and definite endpoints are observed in a nested RCT.
Assessment of efficacy
If a significant therapeutic effect is observed in the nested RCT, the next step is to
compare this result with the analysis in the corresponding subgroup of the OCS described
before, with or without correction by a preceeding validation as described in the previous
paragraph. The comparison may be performed in descriptive terms, e. g. by describing mean
values or rates of observed effects and their corresponding confidence intervals. The
comparison of observed effects in the RCT and the OCS can also be performed by using
formal statistical tests on therapeutic equivalence [24]. However, the corresponding test
results cannot be used with the same strength as within an ordinary RCT.
If the observed effect in the OCS is smaller than in the RCT one may conclude that
under everyday conditions the drug is not as effective as within the setting of the
controlled clinical trial and one may look for possible reasons for such differences (e.
g. differences in compliance). If the observed effect in the OCS is more pronounced than
in the RCT, this may be attributable to a placebo effect (since all patients in the OCS
are informed that they receive an effective drug) or to a too "optimistic"
assessment by the treating physician. When the interpretation of the observed effect in
the subcohort of the OCS is not clear, it will also be difficult to interpret the results
of the complete OCS. If the effects in the RCT and the OCS are equivalent, it may be
concluded that observation of effects in both studies is well comparable and one may
proceed with the next step.
In the next step the comparison of observed effects between OCS and the nested RCT can
be extended to the remainder of the OCS. The result of this comparison may be judged in
relation to the comparison within the OCS mentioned before. Once again, this comparison
may be performed by using descriptive or analytical statistical techniques.
If the observed effects in the remainder of the OCS are both equivalent with the ones
observed in the subcohort and in the RCT, one may conclude that the results from the RCT
may be generalized to a wider population similar in structure to the OCS. If the observed
effects differ between the different subcohorts of the OCS this may lead to additional,
confirmatory trials and/or to specific recommendations regarding the use of the drug (e.
g. if it appears to be less effective in elderly patients).
Discussion
At the time of marketing approval of a new drug, the clinical information is still
limited. Therefore, additional research is needed in the post marketing phase (phase IV).
There is both a need for additional randomized trials, sometimes for "mega
trials", and also for observational studies.
In order to enhance the achievable evidence from an observational study, combining it
with an experimental study is proposed. In order to obtain valid information from this new
type of study, it is important to provide good observational comparability of the
different study populations. This can be achieved by a study design which has identical
treatment regimens and identical case documentation.
In order to have simple protocols for the physicians and not to complicate the
recruitment of patients for the OCS and the nested RCT, it is preferable that both
elements of the RCOCS are performed by different groups of physicians. In order to obtain
comparable observations it is important to standardize the instructions for performing the
clinical observations and the related documentation. Due to the non interventional
character of the OCS this part of the RCOCS has to determine the procedures for the nested
RCT.
In general, it cannot be ruled out that the fundamental difference between an
experimental study (one has only to think of the need to obtain informed consent) and a
non-intervening observation may be associated with a difference in observable effects.
However, the non experimental drug treatment is in essence the situation for which we need
valid information. Therefore, it is necessary to direct scientific research to this
situation.
The identical core documentation of relevant variables allows investigators to check
whether systematic differences between the baseline variables of patients in the OCS and
the nested RCT have occurred. The proposed selection of a comparable subset of patients
from the OCS should in any case provide a basis for valid comparisons. However, it has to
be judged carefully, whether and how the observations from the remainder of the OCS may be
generalised. This will depend on the consistency of the observed effects, sometimes also
on plausible inconsistencies, e. g. different effects in different age groups.
If the nested RCT is placebo-controlled, the size of the placebo effect may be smaller
than in the OCS since the patients in the RCT are informed that they may receive a placebo
whereas all patients in the OCS are informed that they will be treated with an effective
drug.
The proposed equivalence tests for comparing the observed effects in the OCS and the
nested RCT may be used for descriptive purposes only. However, this kind of application
which has less strength than would be obtainable from comparisons of randomized groups,
may lead to useful and valid information.
The RCOCS may be compared with other techniques which have been proposed for similar
purposes. Olschewski et al [25,26] have described the "comprehensive cohort
study" (CCS). This was a proposal to follow up patients who refused participation in
a randomized trial but were treated with the same drugs as used in the corresponding
trial. Although the statistical evaluation techniques described for the CCS may also be
used for the RCOCS, there is a fundamental difference between the two approaches: Whereas
the nested RCT in the RCOCS is defined by design, the observational part of the CCS
results from the decision of the patients and thus may be influenced by more kinds of
potential biases. In order to maintain this advantage of the RCOCS patients who refuse to
take part in the RCT should not be enrolled for the OCS.
Other proposals for phase IV research relate to the use of routine data from
doctor’s office computers or to the use of available databases, e.g. from an
insurance company or HMO. However, this kind of data seems to be less valid than data from
a carefully planned and designed RCOCS, since data from doctors’ offices lack
systematic, coherent documentation.
Conclusion
The reference controlled observational cohort study is a new tool for phase IV drug
research. It may combine positive elements of experimental and observational studies in
the postmarketing phase of a drug and thus may be specially suited to enhance the
achievable evidence from observational cohort studies.
References
- [1]
- Michaelis, J. (1992): Stellenwert und Methodik der Anwendungsbeobachtung.
In: Habs M et al. (eds.) Die Anwendungsbeobachtung beim niedergelassenen Arzt,
Universitätsverlag Jena
- [2]
- Victor, N., Schäfer, H., Novak, H. (eds.) (1991): Arzneimittelforschung
nach der Zulassung Springer Verlag, Berlin-Heidelberg
- [3]
- Schäfer, H., Victor, N., Michaelis, J. (1993): Erforschung von Nutzen
und Risiken zugelassener Arzneimittel, DÄ 90, 1179-89
- [4]
- Bethge, H., Kiep, G. (eds) (1992): Post marketing surveillance as a tool
for evaluation of drug experience, Demeter, Gräfelfing
- [5]
- Armstrong, B.G., Whittemore, A.S., Howe, G.R. (1989): Analysis of
case-control data with covariate measurement error: application to diet and colon cancer,
Statistics in Medicine, 8, 1151-1163
- [6]
- Blettner, M. und Becher, H. (1990): Adjusting for misselection using
subsampling in epidemiological studies, Statistics in Medicine, 9, 1339-1352
- [7]
- Brenner, H. und Blettner M. (1996): Misclassification bias arising from
random error in exposure measurement strategies, American Journal of Epidemiology, 138,
453-461
- [8]
- Barron, B.A. (1977): The effects of missclassification on the estimation
of relative risk, Biometrics 33, 414-418
- [9]
- Carroll, R.J., Ruppert, D., Stefanski, L.A. (1995): Measurement error in
nonlinear models, Chapman & Hall London
- [10]
- Chen, T.T. (1989): A review of methods for misclassified categorical data
in epidemiology, Statistics in Medicine, 8, 1095-1106
- [11]
- Duffy, S. W., Rohan, T.E., Day. N. E. (1989): Misclassification in more
than one factor in a case-control study: a combination of Mantel-Haenszel and maximum
likelihood approaches, Statistics in Medicine, 8, 1529-1536
- [12]
- Duffy, S.W., Maximovitch, D.M., Day, N.E. (1992): External validation,
repeat determination, and precision of risk estimation in misclassified exposure data in
epidemiology, Journal of Epidemiology and Community Health, 46, 620-624
- [13]
- Fuller, W.A. (1987): Measurement error models, John Wiley & Sons
- [14]
- Greenland, S. (1988): Statistical uncertainty due to misclassification:
implications for validation substudies, Journal of Clinical Epidemiology, 41, 1167-1174
- [15]
- Greenland, S. (1989): Variance estimation for epidemiologic effect
estimates under misclassification, Statistics in Medicine, 8, 825-829
- [16]
- Holford, R.H., Stack, C. (1995): Study design for epidemiologic studies
with measurement error, Statistical Methods in Medical Research, 4, 339-358
- [17]
- Kaaks, R., Riboli, E., Esteve, J. (1994): Estimating the accuracy of
dietary questionnaire assessments: validation in terms of structural equation models,
Statistics in Medicine, 13, 127-142
- [18]
- Kaldor, J. und Clayton, D. (1985): Latent class analysis in chronic
disease epidemiology, Statistics in Medicine,4, 327-335
- [19]
- Marshall, J.R. und Hastrup, J.L. (1990): Mismeasurement and the resonance
of strong confounders: uncorrelated errors, American Journal of Epidemiology, 143,
1069-1078
- [20]
- Rosner. B., Spiegelman, D., Willett, W.C. (1990): Correction of logistic
regression relative risk estimates and confidence intervals for measurement error: the
case of multiple covariates measured with error, American Journal of Epidemiology, 132,
734-745
- [21]
- Rosner,B. (1996): Measurement error models for ordinal exposure variables
measured with errors, Statistics in Medicine, Vol.15, 293-303
- [22]
- Wacholder S. et al. (1993): Validation studies using an alloyed gold
standard, American Journal of Epidemiology, 11, 1251-1258
- [23]
- Walker, A.M. und Lanes, S.F. (1991): Misclassification of covariates,
Statistics in Medicine, 10, 1181-1196
- [24]
- Wellek, S. (1994): Statistische Methoden zum Nachweis der Äquivalenz, G.
Fischer Verlag, Stuttgart
- [25]
- Olschewski, M., Scheurlen, H. (1985): Comprehensive cohort study: an
alternative to randomized consent design in a breast preservation trial, Methods Inf.
Medicine, 24, 131-134
- [26]
- Olschewski, M., Schumacher, M., Davis, K.B. (1992): Analysis of
randomized and non-randomized patients in clinical trials using the comprehensive cohort
follow-up study design, Controlled Clinical Trials, 13, 226 - 239
|
