The references listed in Section A to D of this page take for granted that the views they express as to the correlations between prevalence of an outcome and binary measures of difference are correct. I expect that there will eventually be universal acceptance, if not of the precise views I have been expressing since 1987, at least of the fact that the standard measures of differences between outcome rates are in some manner affected by overall prevalence and that, hence, none of those measures can provide useful information regarding the comparative situation of two groups without taking overall prevalence into account. But even in the countries where issues of the relationships between overall prevalence of an outcome and various measures of experiencing or avoiding it have lately been given a good deal more attention than in the United States, it may be some time before there will be widespread, much less universal, understanding of these issues.
(a) National Center for Health Statistics: As discussed in Section E.4, the NCHS responded to reference A10 (Race and Mortality, Society 2000) and A5 (Divining Difference, Chance 1994) by recommending that all disparities be measured in terms of relative differences in adverse outcomes. See a1 and a2 below. In doing so, however, NCHS failed to acknowledge that those references did not merely show situations where relative differences in experiencing an outcome and relative differences in avoiding the outcome changed in opposite directions as the prevalence of an outcome changed, but maintained that the described patter of the two relative differences was systematic, being driven by the properties of the underlying risk distributions. See, e.g., D6 (Comment on Keppel and Pearcy). Keppel and Pearcy’s reply to this comment (item a3 below) is somewhat cryptic (though it does reflect some appreciation that the crucial issue involves what is occurring with the underlying distributions, something that, as discussed below, so far only Carr-Hill/Chalmers-Dixon, Gallestey, and possibly Gansky, seem to have recognized). On the panel at the 7th International Conference on Health Policy Statistics at which I presented reference B13 (ICHPS 2008), Kenneth G. Keppel, the principal author of the NCHS position (in presenting item a4 below), expressed the view that the points I have expressed in B13 and elsewhere, while correct with respect to cross-sectional data, are not correct with respect to longitudinal data, stating that patterns observed over time are not the same as those observed when a cutoff is raised or lowered. But, while patterns observed over time will rarely if ever be precisely those illustrated by the lowering or raising of a cutoff, they will almost invariably exhibit similar tendencies. And it makes no sense to make comparisons over time without consideration of such tendencies.
Keppel and Pearcy recently addressed this matter further in a 2009 Chance article, simply observing:
James Scanlan, in his 1994 CHANCE article "Divining Difference," pointed out that the relative difference in rates of survival between black and white infants decreases as the relative difference in rates of infant mortality increases. When disparities are measured in HP2010, indicators are usually expressed in terms of adverse events so meaningful comparisons can be made across indicators.
But they offer no justification for the approach and ignore entirely the criticisms I have repeatedly made of the NCHS approach, including in the 2006 Chance editorial Can We Actually Measure Health Disparities. If NCHS is going to provide useful guidance in this area, it must directly address these issues. Specifically, it must address whether relative differences in adverse and favorable outcomes (as well as other measures) tend to be affected by the overall prevalence of an outcome and, if so, whether those measures can provide useful information about whether disparities are changing over time or are otherwise larger in one setting than another if prevalence is not in some manner taken into account. And the longer it takes for NCHS to address the issues, the greater will be the amount of flawed research for which NCHS will be in part responsible.
(b) Carr-Hill and Chalmers-Dixon. In the 2005 Southeast Public Health Observatory Handbook of Health Inequalities Measurement, relying on my 2001 presentation on these issues in Oslo (B1), Carr-Hill and Chalmers-Dixon (item b below) (at 171-72), explicitly accepted the reasoning of that presentation with regard to relative differences, including a recognition of the way the statistical patterns are functions of the properties of the underlying distributions. They noted:
[I]f inequalities in health were to be measured in terms of the numbers and proportion who survive rather than the numbers and rates of death, the picture is very different (Part (b) of Table 11.4). The point is that as a negative outcome becomes more rare, it is more and more likely to occur disproportionately among the less advantaged groups. Conversely, as a valued outcome becomes relatively rarer, it is likely to be concentrated among the elite. This is a simple consequence of the statistical distributions [citing B1], rather than another example of inequalities.
But the lengthy document, which discusses a variety of health disparities issues and measurement techniques, seems not to recognize the implications of such acceptance as to other measures it discusses. In my view, the acceptance calls into question much of the reasoning in the remainder of the document. For the patterns described here affect each of the measures discussed in the Carr-Hill/Chalmers-Dixon document.
(c) Houweling et al. A 2007 article by Houweling et al. (item c below) is the article mentioned in Section E.2 as one co-authored by two of the authors of the 1997 Lancet article that calls the conclusions of the Lancet article into question. See also B51a (Comment on Mackenbach Lancet 1997). The Houweling article is in part a response to Race and Mortality (A10) and questions Race and Mortality for overstating the force of the tendencies it describes. The Houweling article ignores entirely Race and Mortality’s discussion as to why certain patterns will tend to be observed and why in some cases they will tend not to be observed – in particular the explanation that differences between rates will be a function of (a) the prevalence of the outcome and (b) the size of the difference between the underlying distributions. But the Houweling article nevertheless finds systematic correlations between the prevalence of an outcome and relative differences in experiencing it and avoiding it that are the same as those described in Race and Mortality. Houweling et al. were unaware of the 2006 Chance editorial, Can We Actually Measure Health Disparities? (A12) and other treatments of absolute differences between 2005 and 2007 (see Section E.3 of MHD), but their study found systematic correlations between absolute differences and the prevalence of an outcome according to the same reverse U-shaped pattern illustrated or described in the references listed in Section E.3. However, the Houweling article suggested that the odds ratio would avoid the problems arising from the correlations it describes. In my view, as discussed in the Section E.3 references, the odds ratio does not avoid such problems, because differences measured in odds ratios tend also to be correlated with the prevalence of an outcome (according to a pattern that is the opposite of that exhibited by absolute differences). Also, in my view, the explanations Houweling et al. offer for the observed patterns are less sound than those described in Race and Mortality and many other places listed on this page. Nevertheless, as with my own treatments of these issues, the Houweling article raises questions as to the validity of the overwhelming majority of health disparities research to date.
Because the Houweling authors included two of the most prominent authorities on health inequalities measurement (Johann P. Mackenbach and Anton E. Kunst), the article is potentially of great significance. But those authors have gone on to do subsequent work in ways that, according to their own reasoning in Houweling et al., is flawed for failure to consider the implications of overall prevalence. See items D70 (Comment on Mackenbach NEJM 2008) and D113 (Comment on Mackenbach BMJ 2011).
(d) Eikemo et al. Whereas the Houweling article responded to Race and Mortality without the authors’ evidencing an awareness of the 2006 Chance editorial (A12), a 2009 article by Eikemo et al. (item c2 below) in the same journal responded specifically to the Chance editorial, while also discussing the Houweling article. Like Houweling et al., Eikemo et al. found correlations between relative differences and prevalence of an outcome and concluded that overall prevalence must be taken into account in interpreting relative differences. Like the Houweling article, however, the Eikemo article offered little guidance on how to do that. Also like the Houweling article, the Eikemo article failed to consider the reasons for the observed patterns that had been explained at some length in Race and Mortality and varied other places.
Item D72 is a comment on Eikemo et al. and somewhat on Houweling et al. It was submitted to the International Journal for Equity in Health at the beginning of October 2009. The journal originally was to publish it upon receiving comments from Eikemo et al. and Houweling et al. but decided it could only publish a much shorter version (and is awaiting my submission of the shorter version). The comment stresses the critical point that, as noted above, observed patterns are functions of (a) the prevalence of the outcome and (b) the differences in the underlying distributions, as had been made clear enough in Race and Mortality. Yet, in contrast to Carr-Hill and Chalmers-Dixon and Gallestey, neither Houweling et al. nor Eikemo et al. seem to recognize these forces. See also D79 (Comment on Huijts and Eikemo EJPH 2009), which discusses an article by Huijts and Eikemo (item c3 below) that alludes to the patterns I have described as a potential explanation for variations in health inequalities across countries.
(e) Bauld et al. A 2008 article by Bauld et al. (item d below) devotes several paragraphs to discussion of my treatments of the pattern whereby the rarer an outcome, the greater tends to be the relative difference in experiencing it and the smaller tends to be the relative difference in rates of avoiding it. They caution that, assuming the correctness of my reasoning, “[i]f governments fail to take account of ‘Scanlan’s rule,’ they run the risk of guaranteeing failure, largely for conceptual and methodological reasons rather than social welfare reasons.” But the authors do not make clear whether they wholly accept the reasoning or not, and it is not clear whether they think that my discussions of these patterns are based solely on observations without regard to the identification of the factors underlying the patterns. Further, they find among “real-world examples that resonate with Scanlan’s arguments” a study by Gisselmann finding that part of the reason for increasing social inequalities in birth outcomes in Sweden “’is likely to be found in the decline in the proportion of women with low education.’” But the pattern identified by Gisselmann raises a compositional issue akin to that addressed in introductory section of D43 (Comment on Boström). (Other issues raised in the Gisselmann article are discussed at pages 13-14 of B7 (BSPS 2006).) The compositional issue addressed by Gisselmann involves a matter quite different from the implications of what Bauld et al. term “Scanlan’s rule,” which implications are present without regard to compositional changes. Finally, the authors conclude by recommending that governmental entities in the United Kingdom continue to establish and monitor health inequalities reductions goals. But if one accepts my reasoning, there is no purpose in creating or monitoring goals until one has effectively addressed the measurement issue.
(f) David Mechanic. In several articles since 2002 (items e1 to e3 below), David Mechanic has discussed or cited A3 (Public Interest 1991) or A12 (Chance 2006). Referencing illustrations of large relative differences where outcomes are rare, and increasing relative differences corresponding with declining absolute differences, he has observed that the former have been overemphasized and the latter provide more valuable information as to the public importance of a disparity. On the basis of what he has written, however, I cannot say whether he fully agrees with my reasoning regarding the failure of either measure to provide useful information as to whether a disparity is increasing or decreasing in a meaningful sense.
(g) Jorge Bacallao Gallestey. In two 2007 articles (items f1, f2) Jorge Bacallao Gallestey cited A12 (Chance 2006) and, like Carr-Hill and Chalmers-Dixon, recognized the distributional basis for the patterns whereby relative differences in one outcome and its opposite would change systematically in opposite directions as the outcome changed in prevalence. See especially Table 2 in item f1. But, as was also the case with Carr-Hill and Chalmers-Dixon, other parts of these articles seem not to reflect recognition of the implications of that recognition as to other measures the articles describe.
(h) Stuart Gansky. In a 2009 presentation at the Joint Statistical Meetings (JSM) (item g below) Stuart Gansky cited A12 (Chance 2006) for the proposition that a useful measure of health disparities must not change when there occurs a simple overall change in prevalence. He then presented results of study of how health disparities indexes “are affected by underlying factors conditional on prevalence difference (i.e. probit model),” stating (in the abstract)
[Health Disparities Indexes] were estimated for the California Oral Health Needs Assessment 2004, a complex survey, to assess associations with untreated caries. Absolute measure, slope index of inequality, relative index of inequality (RII-mean), and absolute concentration index (ACI) were prevalence invariant with constant prevalence difference in a probit simulation model; relative measures depended on prevalence.
I am unfamiliar with the detail of the analysis. But inasmuch as a probit model appears to achieve the same results as the approach described on the Solutions sub-page of the Measuring Health Disparities (see introductory note to the Solutions sub-page and January 9, 2010 Follow-up Comment on Morita) it may well satisfy the criterion of remaining invariant as overall prevalence changes. Gansky and his colleagues may soon publish a paper based on the JSM presentation and that should shed further light on his technique.
(i) Cristina Masseria. In a 2009 article (item h below) on health disparities measurement issues, Christina Masseria relied on A12 (Chance 2006) for the proposition that “caution is needed when interpreting the results of relative inequalities since they increase (decrease) as a consequence of a decrease (increase) in the overall level of mortality (survival).” That seems almost to capture the point, though failing to recognize that the pattern is a tendency that must be interpreted with a regard for the factors underlying it. In any case, she goes on to state: “Houweling et al. [item c1 here] have shown that low levels of mortality can be achieved together with low levels of health disparities. Indeed, a recent US study shows that there is no clear relationship between level of health and relative inequalities.” Thus, in my view, Masseria, like Houweling et al. and Eikemo et al. (and Huijts and Eikemo and Remes et al. (see below)), fails to recognize the forces that are invariably at work, as discussed above with regard to the Houweling and Eikemo articles, and that the function of health disparities research is to identify patterns of health disparities that are not functions of overall prevalence.
(j) Remes et al. In a 2009 article (item i below), Remes et al. (item h), citing Race and Mortality, state:
Scanlan [] argues that all measures of differences between binary outcomes are affected by the overall prevalence of an outcome and changes in it. For example, declining mortality tends to be accompanied by increasing relative differentials. Departures from the expected patterns of change are still possible, as demonstrated by Houweling et al [item c1] in a cross-national comparison study.
While not necessarily quarreling with Race and Mortality, this article is similar to several of the other above-discussed items in failing to reflect a recognition of the forces underlying the observed patterns that Race and Mortality described, as reflected by the discussion throughout the remainder of the article. See D13 (Comment on Edwards), D2 (Comment on Shaw), D39 (Comment on Singh) responding, respectively, to references 12, 13, and 14 of Remes et al.
Items by authors referenced in Section E.7 are:
a.1. Keppel K., Pamuk E., Lynch J., et al. 2005. Methodological issues in measuring health disparities. Vital Health Stat 2005;2 (141): http://www.cdc.gov/nchs/data/series/sr_02/sr02_141.pdf
a.2. Keppel KG, Pearcy JN. Measuring relative disparities in terms of adverse events. J Public Health Manag Pract 2005;11(6):479–483
a.3. Keppel K.G., Pearcy J.N. 2006. Response to Scanlan concerning: measuring health disparities in terms of adverse events. J Public Health Manag Pract 12(3):295:
http://www.nursingcenter.com/library/JournalArticle.asp?Article_ID=641472
a.4. Keppel KG. Measuring Disparities in Health People 2010, presented at the 7th International presented at the 7th International Conference on Health Policy Statistics, Philadelphia, PA, Jan. 17-18, 2008 (invited session).
a.5. Keppel KG, Pearcy JN. Healthy People 2010: Measuring Disparities in Health. Chance 2009;22(1)_____.
b. Carr-Hill R, Chalmers-Dixon P. The Public Health Observatory Handbook of Health Inequalities Measurement. Oxford: SEPHO; 2005: http://www.sepho.org.uk/extras/rch_handbook.aspx
c.1. Houweling TAJ, Kunst AE, Huisman M, Mackenbach JP. Using relative and absolute measures for monitoring health inequalities: experiences from cross-national analyses on maternal and child health. International Journal for Equity in Health 2007;6:15: http://www.equityhealthj.com/content/6/1/15
c.2. Eikemo TA, Skalicka V, Avendano M. Variations in health inequalities: are they a mathematical artifact? International Journal for Equity in Health 2009;8:32: http://www.equityhealthj.com/content/pdf/1475-9276-8-32.pdf
c.3. Huijts T, Eikemo TA. Causality, social selectivity or artefacts? Why socioeconomic inequalities in health are not smallest in the Nordic countries. Eur J Pub Health 2009;19:452-53
d. Bauld L, Day P, Judge K. Off target: A critical review of setting goals for reducing health inequalities in the United Kingdom. Int J Health Serv 2008;38(3):439-454: http://baywood.metapress.com/app/home/contribution.asp?referrer=parent&backto=issue,4,11;journal,7,157;linkingpublicationresults,1:300313,1; http://www.britannica.com/bps/additionalcontent/18/33140632/Off-Target-A-Critical-Review-of-Setting-Goals-for-Reducing-Health-Inequalities-in-the-United-Kingdom
e.1. Mechanic D. Disadvantage, inequality and social policy. Health Affairs 2002;21(2):48-59;
e.2 Mechanic D. Who shall lead: Is there a future for population health? J Health Politics, Policy and Law 2003;28(2):421-442.
e.3 Mechanic D. Population health challenges for science and society. Milbank Quarterly 2007;85(3):553-559.
f.1. Gallestey JB. Two controversial topics in the context of health inequality measurement. Rev. Cuban Public Health 2007;33(3): http://bvs.sld.cu/revistas/spu/vol33_3_07/spu18207.htm
f.2. Gallestey JB. Indicadores basados en la noción de entropía para la medición de las desigualdades sociales en salud Indicators based on the notion of entropy for the measurement of social inequalities in health. Rev. Cuban Public Health 2007; 33 (4):http://bvs.sld.cu/revistas/spu/vol33_4_07/spu07407.html
g. Gansky S. A Simulation Study of Health Disparity Indexes: How Do They Depend on Prevalence? presented at 2009 Joint Statistical Meetings of the American Statistical Association, International Biometric Society, Institute for Mathematical Statistics, and Canadian Statistical Society, Washington, DC, Aug. 1-6, 2009: http://www.amstat.org/meetings/jsm/2009/onlineprogram/index.cfm?fuseaction=abstract_details&abstractid=304602
h. Masseria C. Health inequality: Why is it important and can we actually measure it? Eurohealth 2009;15(3):4-6: http://www.euro.who.int/Document/Obs/Eurohealth15_3.pdf
i. Remes H, Martikainen P, Valkonen T. Mortality inequalities by parental education among children and young adults in Finland 1990e2004. J Epidemiol Community Health 2010;64:136-141.
|