Nonrandomized Comparative Clinical Studies - Proceedings of the International Conference on Nonrandomized Comparative Clinical Studies in Heidelberg, April 10 -11,1997

The Paired Availability Design: An Update

S. G. Baker

• Abstract
• Requirements and assumptions
• Computations
• References

Abstract

Baker and Lindeman [ 3] introduced the paired availability design for strengthening inference when using historical controls. We review the design in the context of the following updates. First, we make the notation similar to that in the recent literature on all-or-none compliance in randomized trials. See the review in Baker [ 2] and Angrist et al. [ 1] . Second, in addition to excess risk, we consider the relative risk as a possible test statistic. Cuzick et al [ 4] independently made similar calculations in the context of a randomized trial with all-or-none compliance. Third, we recommend using the inverse of the variance rather than the inverse of the standard error when weighting estimates from multiple pairs. This was also independently suggested by Cuzick et al. [ 4] in the context of randomized trials. Fourth, to improve the sample size calculation we suggest a method for using exogenous data to estimate the variation due to random time changes. Fifth, we propose an adjustment for one type of systematic change over time.

Requirements and assumptions

The requirements for the paired availability design are as follows:

(i) The intervention is the availability, not the receipt, of new treatment.

(ii) There are multiple pairs of hospitals for which there was a period of time when the newtreatment was less available followed immediately by a period of time when the new treatment was more available.

(iii) Eligible subjects arise from a stable population with little in- or out- migration, e.g. nulliparous women in labor from a region serving the only hospital in a geographic area [ 3] .

(iv) There are no systematic changes over time, except for one special type discussed later.

The analysis involves a comparison of binary outcomes, success or failure, before and after the change in availability of treatment among multiple hospitals. Without loss of generality suppose the time periods in (ii) are last year and this year. We define control subjects as all eligible subjects last year when the new treatment was less available; we define study group subjects as all eligible subjects this year when the new treatment is more available. Requirements (i) and (iii) avoid the type of selection bias arising in the standard use of historical controls in which patients are referred to the new treatment [ 5] . The use of multiple hospitals in (ii) reduces extra variability from random changes over time which affect all subjects at a particular hospital, such as a change in support staff. It may be possible to remove the effect of in-migration in (iii) by dropping from the analysis any subject from outside the stable population who receives treatment. When designing an observational study, it is helpful to think about the corresponding randomized design. The idea of using the availability of treatment is related to the randomized consent design [ 10] . The idea of a paired analysis with multiple hospitals is related to the paired clustered randomization [ 7] .

To analyze data from the paired availability design, we estimate efficacy, the effect of receipt of treatment, as opposed to effectiveness, the effect of the availability of treatment. Not only is efficacy easier to interpret than effectiveness, it is preferable for combining results over hospitals because it does not depend on the fraction who receive the new treatment. To estimate efficacy, we consider the following thought experiment. In the first scenario all subjects are eligible last year, as if they were all in the control group. In a second scenario all subjects are eligible this year, as if they were all in the study group. We make the following assumptions.

Assumption 1: There are three types of subjects defined in terms of the thought experiment. Type A subjects always receive the new treatment regardless of scenario. Type C subjects receive the new treatment conditional on being in the second scenario. Type N subjects never receive the new treatment, regardless of scenario.

Assumption 2: We assume the same distribution of subject types in the control and study groups. This would always hold in a randomized study; it holds here if requirements (iii) and (iv) are satisfied. The probabilities of types A, C, and N, are denoted g _A, g _C, and g _N, respectively.

Assumption 3: Given treatment and subject type, failure does not depend on group. As a consequence, we can write pr (fail ï A), = pr (fail ï C, old treatment), = pr (fail ï C, new treatment), and = pr (fail ï N). Our goal is to compare with .

Computations

For estimation, it is helpful to use the following table. Let p_z denote the observed fraction receiving the new treatment in group z, z = 0,1 corresponding to the control and the study group, respectively. Let denote the observed fraction in group z who receive the old (new) treatment and fail.

		fraction treated		fraction treated and failed
group	treatment	observed	expected	observed	expected
control	old new	1-p0 p0	g N+g C g A
study	old new	1-p1 p1	g N g C+g A

Let denote the observed fraction in group z who fail. The difference in the effect of receipt of treatment is

,

which is the difference in the effect of availability divided by the difference in availability. A similar formulation appears in the literature on all-or-none compliance [ 2] . Let n_z denote the number of subjects in group z. The variance is approximately

.

Another possible measure of effect is the relative risk for the effect of receipt of treatment,

with asymptotic variance of the logarithm, computed by the delta method,

.

The above estimate and variance reduce to that in Sommer and Zeger [ 9] when there are no type A subjects.

In the remainder of the paper, we consider as the test statistic although one could also use log(RR). We wish to test H₀: d =0 versus H_A d =d ₀. For an equivalence trial, as in Baker and Lindeman [ 3] , one could test H₀: d =d ₀ versus H_A :d =0. The design involves m hospitals, indexed by i. Let , v_i and n_zi, denote the estimated difference, its variance, and the number of subjects in group z, respectively. The overall test statistic, t, is a weighted average of over all hospitals:

where and .

To minimize the variance of t, the weights are proportional to the inverse of the sampling variance, v_i [ 8] . Similar weights are used in fixed-effect meta-analyses.

To avoid assuming a distribution for , we use a permutation test (e.g. [ 6] ). There are 2^m possible permutations of the form . The p-value is the fraction of permutations in which t is greater than or equal to the observed value from the data. To obtain a 1-a confidence interval, one selects d ₀ to reject the null hypothesis at levels a /2 and 1-a /2.

For computing sample size, we assume the sampling distribution is ~ , and the distribution of random time changes is q _i ~ . Although the model appears similar to a random effects model, it differs in the use of d _i rather than d in the latter formula. We use d _i instead of d so the true difference can vary among hospitals. The quantity s ² represents extra variability due to random time changes. The model induces the following variance on , var(). To reduce the number of quantities to specify, we use the following upper bound for var(),

.

where n_min is the smallest number of control and study subjects among all the hospitals and D _min is the smallest change in availability. Applying the usual sample size formula with a correction [ 7] gives

m= number of hospitals =.

We compute m for various values of n_min, D _min, s ². If there are exogenous data on the effect of another intervention with the same variation due to random time changes, we can estimate s ² for use in the sample size calculation. Let denote the effect of the other intervention on hospital i with sampling variance , i= 1,2,3,...,m. Because var() , we can compute the variance of mean effect in two ways,

,

,

which we can set equal and solve for s ².

The basic design reduces selection bias and the variance from random time changes. In order to reduce bias from systematic time changes, we need data from an earlier control group with the same availability of treatment as in the control group. We can adjust for a systematic time change which is same from the earlier control to the control group as from the control to the study group. It is helpful to consult the following table:

		fraction treated		fraction treated and failed
group	treatment	observed	expected	observed	expected
earlier control	old new	1-pn p0	g N+g C g A
control	old new	1-p0 p0	g N+g C g A	f0
study	old new	1-p1 p1	g A g C+g A	f1

Let d denote the effect of treatment, as before, and let e denote the effect of time. Without an earlier control group, the effect of treatment is confounded by the time effect: . However, it can be unconfounded if . Therefore the estimated time effect is , and the estimated treatment effect, adjusting for this systematic time change, is .

In conclusion, the paired availability design reduces selection bias by using all eligible patients from a stable population over a period of time, rather than selective referrals as with standard historical controls. It reduces the variance from random time changes by averaging efficacy over multiple hospitals. By using an earlier control group, it is possible to adjust for a certain type of systematic time change.

References

[1]

Angrist, J.D., Imbens, G.W., Rubin, D.R. (1996): Identification of causal effects using instrumental variables. Journal of the American Statistical Association 91, 444-455.

[2]

Baker, S.G. (1997): All-or-none compliance. The Encyclopedia of Statistical Sciences S. Kotz, C. Read, D. Banks (eds.). New York: John Wiley and Sons, 134-138.

[3]

Baker, S.G., Lindeman, K.S. (1994): The paired availability design: a proposal for evaluating epidural analgesia during labor. Statistics in Medicine 13, 2269-2278.

[4]

Cuzick, J, Edward, R., Segnan, N. (1997): Adjusting for non-compliance and contamination in randomized clinical trials. Statistics in Medicine 16, 1017-1029.

[5]

Doll, R., Peto, R. (1980): Randomized controlled trials and retrospective controls. British Medical Journal i, 44.

[6]

Feinstein, A.R. (1993): Permutation tests and statistical significance. M.D. Computing, 10 28-41.

[7]

Gail, M.H., Byar, D.P., Pechacek, T.F. Corle, D.K. (1992): Aspects of statistical design for the community intervention trial for smoking cessation (COMMIT). Controlled Clinical Trials 12, 6-21.

[8]

Kendall, M.G., Stuart, A. (1961): The Advanced Theory of Statistics, Volume 2: Inference and Relationship, 3rd ed. London: Charles Griffin, p33, Ex 17.21.

[9]

Sommer, A., Zeger, S.L. (1991): On estimating efficacy from clinical trials. Statistics in Medicine 10, 45-52; with Correction, Statistics in Medicine, 13, 1897 (1994).

[10]

Zelen, M. (1979): A new design for randomized clinical trials. New England Journal of Medicine 300, 1242-1245.