James P. Scanlan, Attorney at Law

Drawing Inferences

(Sept. 19, 2021)

This is one of the subpages to the Criminal Justice Disparities page of jpscanlan.com.  That page and it subpages principally address the mistaken belief, promoted by the U.S. government and countless organizations that purport to have expertise in the analysis of data on demographic differences, that generally reducing adverse criminal justice outcomes tend to reduce (a) relative racial differences in rates of experiencing the outcome and (b) the proportion Blacks make up of persons experiencing the outcomes.).  In fact, as explained more fully on the Criminal Justice Disparities page, the opposite is the case.

That is, as I have explained in scores of places with respect to any favorable or adverse outcome since 1987, when two groups differ in their susceptibility to an outcome, generally reducing the outcome, while tending to reduce relative differences in rates of avoiding the outcome (i.e., experiencing the opposite outcome), tends to increase relative difference in rates of experiencing the outcome itself.  Correspondingly, reducing the outcome, while tending to increase the proportion the more susceptible group makes up of persons avoiding the outcome (thus reducing all measures of difference between the proportion the group makes up of the population and the proportion it makes up of persons avoiding the outcome), tends also to increase the proportion the group makes up of persons experiencing the outcome itself (thus increasing all measures of difference between the proportion the group makes up of the population and the proportion it makes up of persons experiencing the outcome). 

This page, however, does not involve the mistaken understanding of the effects of policies on measures of racial but rather another manifestation of the failure to understand how measures tend to be affected by the prevalence of an outcome.

One of the many remarkable examples of innumeracy among persons analyzing demographic differences involves the way that for many decades observers have been drawing inferences about processes based on the comparative of relative difference or relative effects (or indicators that are functions of relative differences or relative effects[i]) in whichever side of a favorable or adverse outcome dichotomy the observers happen to be examining.  Invariably, they have done so while unaware that it is even possible for the comparative size of relative difference in the opposite outcome to support an opposite inference, much less that such will be the usual case.   They have also failed to reflect an understanding of the reasons to expect the observed pattern regardless of the factor about which an inference is drawn. 

A fair summary of this situation may be found in the section titled “Illogical Expectations and Unfounded Inferences” (at 339-341) of “Race and Mortality Revisited,” Society (July/Aug. 2014).  But see also discussion of various issues in “The Perils of Provocative Statistics,” Public Interest (Winter 1991), and “Race and Mortality,” Society (Jan./Feb. 2000). 

In the criminal justice disparities context, two manifestation of the failure to understand this issue may be found in observers’ finding evidence that racial bias influences traffic stop decisions based on the fact that relative racial difference in traffic stops (a) are smaller when made on the basis of radar than when made on the basis of visual observation of driver speed (see the Department of Justice’s March 4, 2015 report “Investigation of the Ferguson Police Department” (at 69-70)) or (b) are smaller when made during hours when driver race is less visible than when driver race is more visible (see discussion of “veil of darkness” hypothesis in Pierson et al., A large scale analysis of racial disparities in police stops across the United States, Nature Human Behavior (May 4, 2020) (at 742-743)).

The former inference must be appraised with recognition of the following.  If one group is more likely to speed than another, then the relative difference between rates of exceeding the speed limit by a substantial margin will typically be greater than the relative difference between rates of exceeding the speed limit at all.  And there apparently is evidence that the relative difference between Black and white speeding rates if greater for substantially exceeding the speed limit than for exceeding the speed limit at all.  See Kocieniewski, “Study Suggests Racial Gap in Speeding in New Jersey,” New York Times (Mar. 21, 2021).  Thus, in the event that officers are more circumspect about stopping drivers for speeding when they must make visual determinations of driver speed than when they have radar evidence of drive speed – something that seems likely – that would tend to result in larger relative differences for stopping drivers for speeding in the former case than the latter case.  Compare discussion of determinations of test failure by teachers on test graded objectively and tests graded subjectively (which would also apply to retention decisions) at pages 5-6 of Response of James P. Scanlan to Office of Management and Budget Request for Information “Methods and Leading Practices for Advancing Equity and Support for Underserved Communities Through Government” (FR Doc No: 2021-09109) (July 6, 2021).

In the case of the “veil of darkness” hypothesis, it would also be necessary to know whether there is more circumspection in stopping drivers after dark than during daylight hours, including how the level of traffic at various times of day influences the frequency of decisions to stop drivers for speeding. 

In any case, as with all other inferences based on the comparative size of some measure of difference involving outcomes, it is impossible to draw sound inference about the mental process underlying traffic stope decisions without a complete understanding of how the measure employed tends to be affected by the prevalence of an outcome.