Deficiencies of Case Control Studies
(September 30, 2010)
[This page remains a sketch. It was hurriedly created mainly to provide a reference for a planned comment on Sedgwick P. Statistical Question: Confounding in case control studies. BMJ 2010; 341:c5136.]
The case control study is provides a seemingly efficient means of determining strengths of association from a small sample. The following table is based on data from a 1993 American Journal of Public Health article addressing the association between hair dry and cancer.
|
Date from Case Control Study of Effects of Hair Dies on Risk of Cancer
|
|
Factor
|
Nature
|
Cases
|
Controls
|
|
N
|
Dye
|
159
|
619
|
|
Y
|
Dye
|
14
|
31
|
The table indicates that from 14 of 173 persons with cancer (cases) used hair dye compared with 31 of 650 persons without cancer (controls). From this one can determine either (a) that the ratio of the odds that a person with cancer used dye to the odds that a person without cancer used dye or (b) that the ratio of the odds that a person using dye had cancer to the odds that a person not using dye had cancer is 1.76.
In cases where the adverse outcome is rather rare (i.e., cases are rare), (b) would approximate the relative risk of cancer for those using dye compared with those not using dye (which we could illustrate by multiplying both the dyers and non-dyers among the controls by 1000). The more common the outcome, the greater will be the odds ratio compared with the relative risk. For example, if the persons in the study reflected the entire universe (which would mean a rather high cancer rate), the relative risk would be only 1.52.
There is a larger problem with case control study than the failure of the odds ratio to approximate the relative risk. Such problem lies in the fact that the relative risk, being influenced by overall prevalence, is not a useful indicator of the strength of association. In order to determine the strengths of association one must have the actual outcome rates, as discussed in the Subgroup Effects page of the Scanlan’s Rule page. See also recent references 1 and 2. But in a case control study, while it may be possible to know the relative risk (or something close to it), the actual rates are not known.
The problem is similar to that addressed in the Representational Disparities sub-page of the Scanlan’s Rule page, an illustration of which is found on the Gender differences in DODT sub-page of the Vignettes.
References:
1. Problems in identifying interaction where groups have different base rates. BMJ Sept. 21, 2010 (responding to Altman DG, Bland JM. Interaction revisited: the difference between two estimates. BMJ 2003;326:219): http://www.bmj.com/content/326/7382/219/reply#bmj_el_241943
2. Rethinking the premises of subgroup analyses. BMJ June 7, 2010 (responding to Sun X, Briel M. Walter SD, and Guyatt GH. Is as subgroup effect believable? Updating criteria to evaluated the credibility of subgroup analyses. BMJ 2010;340:850-854): http://www.bmj.com/cgi/eletters/340/mar30_3/c117
|