James P. Scanlan, Attorney at Law

Credit Score Illustrations

(Jan. 12, 2012; rev. Nov. 17, 2014)

This page was originally created simply to show the patterns by which measures tend to change as the prevalence of an outcome changes within a population for who credit score data was available.  It was expanded in November 2014 to illustrate shortcomings of the method for appraising the strength of the forces causing outcome rates to differ discussed in the Solutions subpage of Measuring Health Disparities page of jpscanlan.com and recently explained in Race and Mortality Revisited (Society 2014). 

As discussed in many places, the pattern whereby the rarer an outcome the greater tends to be the relative difference in experiencing it and the smaller tends to be the relative difference in avoiding it can be illustrated with virtually any data set that allows one to examine various points on a continuum of factors associated with experiencing an outcome.  The pattern (and related patterns discussed in the introduction to the Scanlan’s Rule page (SR)) are illustrated with hypothetical test score data in Table 1 of the BSPS 2006 presentation (and the tables and figures of many other presentations found here); with NHANES data on blood pressure and folate level in the figures of  the ICHPS 2008 presentation and the table on the NHANES Illustrations sub-page of SR;), life table information in the tables of the Life Tables Illustrations sub-page of SR; information from the Framingham Studies on the Framingham Illustrations sub-page of SR; and data on rates at which blacks and white fall below various percentages of the poverty line in the Income Illustrations sub-page of SR and the tables and figures of  Can We Actually Measure Health Disparities? (Chance 2006).

This subpage page illustrates these patterns with data on black and white credit scores among mortgage applicants at Wells Fargo Mortgage, as presented in the Table 4 of the report of plaintiffs’ expert Howell E. Jackson submitted in support of class certification in In re Wells Fargo Mortgage Litigation, No. 8-CV-01930-MMC (JL) (M.D. Cal.).[i]  These data  differ from other data in the above illustration  in that, inasmuch as the data are limited to persons who had credit scores of 300 or above and thus the populations covered in the table are  truncated portions of the larger population.  Such fact has implications akin to those discussed in ICHPS 2008, the Truncation Issues subpage of MHD and the Life Tables Illustrations subpage of SR. 

The populations covered in the table are also limited to the persons in each credit score range who secured mortgages.  That the table reflects only a partial picture likely raises some of the issues raised in Illusions of Job Segregation (Public Interest, Fall 1988), with potential bearing on the probative value of the data in the case in which they were used and utility of the data for illustration the patterns described on the main SR page.  But it may be some time before I think that matter through and it may be impossible to fully explore those implications without data on the rejected applications (though it has been treated somewhat in my Fair Lending Studies Paint Incomplete Picture (American Banker, April 24, 2013)).  I believe, however, the absence of data on rejected applications will not materially detract from the illustrative utility of the data (save with respect to the truncation issue noted with regard to odds ratios and with respect to the illustration in Table 2). 

Table 1 below presents data with regard to falling below and above certain credit scores akin to that presented on falling above and below various ratios of the poverty line in Table 1 of Can We Actually Measure Health Disparities? (Chance 2006) and falling above and below certain test cutoffs in Table 1 of the BSPS 2006 presentation.  The patterns of relative differences between rates of falling above or below each credit level, absolute differences between rates, and differences measured by odds ratios are then illustrated in Figures 1 through 3 (which may found here).

The reader should bear in mind that the groups falling below a score of 540 are actually comprised of persons with scores between 300 and 499 (and that the totals on which the percentages in each row are calculated similarly include only persons scoring 300 or above).  With respect to the purpose of this page, the implications of such fact are limited to the odds ratio patterns shown in the final column of Table 1 and in Figure 3.  That is, whereas in normal data the differences measured by the odds ratio tends to track the pattern of changes of the larger relative difference, because the data are from a truncated portion of a larger distribution, the difference measured by the odds consistently decreases as the adverse outcome becomes more common (a matter treated in a number of the figures in the ICHPS 2008 presentation.) 

Table 1.  Proportions of Blacks and Whites Falling Below and Above Each Credit Score Level and Standard Measures of Difference [Ref b2711 a 8]
Point	Score	% of B Bel	% of B Abv	%of White Bel	% of White Abv	B/W Bel Ratio	W/B Abv Ratio	AD	OR
1	540	3.48%	96.52%	0.63%	99.37%	5.52	1.03	0.03	5.69
2	560	6.33%	93.67%	1.28%	98.72%	4.95	1.05	0.05	5.21
3	580	10.82%	89.18%	2.39%	97.61%	4.52	1.09	0.08	4.95
4	600	16.83%	83.17%	4.11%	95.89%	4.10	1.15	0.13	4.72
5	620	24.33%	75.67%	6.72%	93.28%	3.62	1.23	0.18	4.46
6	640	34.20%	65.80%	10.76%	89.24%	3.18	1.36	0.23	4.31
7	660	44.30%	55.70%	16.10%	83.90%	2.75	1.51	0.28	4.15
8	680	54.58%	45.42%	23.27%	76.73%	2.35	1.69	0.31	3.96
9	700	64.33%	35.67%	32.17%	67.83%	2.00	1.90	0.32	3.80
10	720	72.99%	27.01%	42.23%	57.77%	1.73	2.14	0.31	3.70
11	740	80.50%	19.50%	53.20%	46.80%	1.51	2.40	0.27	3.63
12	760	87.50%	12.50%	66.03%	33.97%	1.33	2.72	0.21	3.60
13	780	93.68%	6.32%	81.10%	18.90%	1.16	2.99	0.13	3.45
14	800	98.35%	1.65%	94.85%	5.15%	1.04	3.12	0.03	3.23

 

Figure 1 shows (a) the black-white ratio of falling below each indicated credit score and (b) the white-black ratio of falling above each indicated credit score.  It shows the common pattern whereby the lower the credit level, the larger the relative difference between rates of falling below it and the smaller the relative difference between rates of falling above it. 

[Figure 1 may be found here.]

Figure 2 shows the absolute difference between rates of falling above each credit level.  The pattern comports closely with the pattern described on the introduction to the Scanlan’s Rule page whereby the absolute difference tends to changes in the same direction as the smaller relative difference as the prevalence of an outcome changes (except in the range where one group’s rate of experiencing one outcome is above 50% and the other group’s rate is below 50%). 

[Figure 2 may be found here.]

Figure 3 shows the ratio of the black odds of falling below the credit score to the white odds falling below the score.  As was also shown in Figure 10 of ICHPS 2008, while in truncated data the other measures (the two relative differences and the absolute difference) tend to behave in the same way as in data on the entire population, the difference measured by the odds ratio does not.  The odds ratio for falling below the level decreases as the prevalence of falling below the level becomes more common.[ii] 

[Figure 3 may be found here.]

Table 2 uses the same outcome rates as Table 1 to illustrate some shortcomings of the Solutions approach (using the EES for estimated effect size) to measuring the forces causing outcome rates to differ.  The value of the measure is that, when the underlying distributions are normal, it does not change when there occurs a simple change in the prevalence of an outcome akin to that effected by the lowering of a test cutoff.  Thus, consider a situation where the underlying distributions of the advantaged and disadvantaged groups (with respect to factors associated with a favorable outcome) are the those reflected in the credit score data and one wants to determine the strength of the forces causing the outcome rates to differ (for example, to try and determine whether it is more likely that the difference in outcome rates resulted from biased decisions or differences in characteristics of the groups, or to determine whether one set of procedures has less of disparate impact than another).  For simplicity, I show only the favorable outcome rates for the advantaged and disadvantaged groups (in this case whites and blacks) along with the EES.   

The table shows that when favorable outcome rates are extremely low, as when there exists a very high standard for receiving the favorable outcome within this population, the disparity  (as reflected by the EES) would be perceived as comparatively small.  As standards are lowered, the EES increases for a time, then decreases.  Thus, at different levels of prevalence the EES would provide different interpretations as to the strength of the forces causing the outcome rates to differ.  The fact that the EES does not remain constant as the standard is lowered is presumably a function of the population is truncated by including only persons with a credit score above 300 (though it might be fairer to regard it as truncated at a score of 500 given the small number of persons between 300 and 500) (the point addressed in the Truncation Issues subpage) and possibly a function of the fact that the population does not include persons who declined loan offers because they found the terms unsatisfactory.   But the changes in the EES could also be partly be a function of the absence of normality in the overall income distributions of either blacks or whites as well as the fact that two income distributions would have different standard deviations.

Table 2.  Proportions of Blacks and Whites Falling Below Each Credit Score Level and Estimated Effect Size at Each Cut Point [Ref b5916 a]
Cutoff	Black Fav Rate	White Fav Rate	EES
800	1.65%	5.15%	0.50
780	6.32%	18.90%	0.65
760	12.50%	33.97%	0.74
740	19.50%	46.80%	0.78
720	27.01%	57.77%	0.81
700	35.67%	67.83%	0.83
680	45.42%	76.73%	0.85
660	55.70%	83.90%	0.85
640	65.80%	89.24%	0.83
620	75.67%	93.28%	0.80
600	83.17%	95.89%	0.78
580	89.18%	97.61%	0.74
560	93.67%	98.72%	0.72
540	96.52%	99.37%	0.69