Appraising Representational Disparities
(May 7, 2010)
This item addresses the impossibility of soundly appraising the size of a difference between the proportion a group comprises of persons who might experience some outcome and the proportion the group comprises of persons who actually experiencing the outcome.
Most of the disparities measurement issues addressed on this page involve comparisons of the rates at which two or more groups experience some favorable or adverse outcome. There are, however, a variety of settings where, often for want of better information, a disparities issue is framed in terms of the difference between the proportion a group comprises of persons who might experience some outcome and the proportion the group comprises of those who experience the outcome. Examples of the latter include:
o Comparison between the proportion a group comprises of drivers and the proportion it comprises of persons stopped by law enforcement officials.
o Comparison between the proportion a group comprises of the total population and the proportion it comprises of persons in poverty, persons selected for jury duty, etc.
o Comparison between the proportion a group comprises of applicants and the proportion it comprises of hires.
o Comparison between the proportion a group comprises of employees and the proportion it comprises of terminated employees.
In such cases, a disparity is often measured in terms of the difference between the actual and the expected proportions the group comprises of those experiencing the outcome and it may be measured in terms similar to the measures typically employed in evaluating differences in outcome rates. For example, if Group A comprises 20% of applicants and 10% of hires, the disparity might be discussed in terms to the effect that Group A comprised 50% fewer hires than expected or to the effect that the proportion it comprised of hires was 10 percentage points less than expected.
It is also possible to determine the relative difference between the selection rates of Group A and persons not in Group A, which we can term Group B. In the example just mentioned we would first derive for each group the ratio of the proportion it comprises of selections to the proportion it comprises of those eligible to be selected – thus, for Group A, 10/20, or .5, and for Group B, 90/80, or 1.125. Then, depending on which of those two ratios we used as the numerator in another ratio, we could say either (a) that the ratio of Group B’s selection rate to Group A’s selection rate is 2.25, which means that members of Group B are 125% more likely to be selected than members of Group A, or (b) that the ratio of Group A’s selection rate to Group B’s selection rate is .222, which means that members of Group A are 77.8% less likely to be selected than members of Group B. Notice, however, that while we can derive the relative differences between the rates, we cannot derive the rates themselves.
As discussed on the Measuring Health Disparities (MHD) and the Scanlan’s Rule pages of this site, most efforts to appraise the differences between even known outcome rates as they bear on some issue in the law or the social and medical sciences are flawed for failing to consider the way standard measures of differences are affected by the overall prevalence of an outcome. But at least when we know the actual rates, we can frequently derive a sound measure of differences between rates by means of the method described on the Solutions sub-page of MHD.
But to do that we must know the actual outcome rates, not just the relative difference between the outcome rates. Since we cannot determine the actual outcome rates based solely on the proportions a group comprises of those eligible to experience some outcome and those experiencing the outcome, there exists no sound method for appraising the size of the difference between the proportion a group comprises of those eligible and the proportion of those selected. That is, for example, we cannot determine whether a situation where a group comprises 20% of applicants and 10% of hire involves a larger or smaller disparity than one where it comprises 15% of applicants and 5% of hires even though we know that the relative difference is larger in the latter case. See Case Study sub-page of Scanlan’s Rule page.