Measuring Healthcare Disparities
Third North American Congress of Epidemiology
Montreal, Quebec, June 21-24, 2011
James P. Scanlan
Attorney at Law
Washington, DC
jps@jpscanlan.com
Key Points
• Standard measures of differences between outcome rates
(proportions) are problematic for measuring health and healthcare
disparities because each is affected by the overall prevalence of an
outcome.
• Healthcare disparities research is in disarray because of observers’
reliance on various measures without recognition of the way each
measure is affected by the overall prevalence of an outcome.
• There exists only one answer to whether a disparity has increased
or decreased over time or is otherwise larger in one setting than
another.
• Fourth, that answer can be divined, albeit imperfectly, by deriving
from each pair of outcome rates the difference between means of
the underlying risk distributions.
References
• Measuring Health Disparities page (MHD) of jpscanlan.com (especially
the Pay for Performance , Solutions sub-pages and Section E.7)
• Scanlan’s Rule page of jpscanlan.com and its twenty sub-pages
(especially the Immunization Disparities sub-page)
• Mortality and Survival page of jpscanlan.com
• Measurement Problems in the National Healthcare Disparities Report
(APHA 2007)
• “Can We actually Measure Health Disparities?,” Chance 2006
• “Race and Mortality,” Society 2000
• “Divining Difference,” Chance 1994
• “The Perils of Provocative Statistics,” Public Interest 1991
Patterns of Distributionally-Driven Changes in Standard
Measures of Differences Between Rates as an Outcome
Increases in Overall Prevalence
• Relative differences in experiencing the outcome tend to decrease.
• Relative differences in failing to experience the outcome tend to
increase.
• Absolute differences between rates tend to increase to the point
where the first group’s rate reaches 50%; behave inconsistently
until the second group’s rate reaches 50%; then decline. Absolute
differences tend also to move in the same direction of the smaller
relative difference. See Introduction to Scanlan’s Rule page for
nuances.
• Differences measured by odds ratios tend to change in the opposite
direction of absolute differences (hence to track the larger relative
difference).
Ratios (Abs Perc Pnt Diff)
Fig 1: Ratios of (1) Advantaged Group (AG) Success Rate to
Disadvantaged Group (DG) Success Rate, (2) DG Fail Rate to AG Fail
Rate, and (3) DG Fail Odds to AG Fails Odds; and (4) Absolute
Difference Between Rates
5
4
(1) AG Succ Rate/DG Succ Rate
3
(2) Ratio DG Fail Rate/AG Fail Rate
(3) DG Fail Odds/AG Fail Odds
(4) Absolute Diff Betw Rates
2
1
0
1
3
5 10 20 30 40 50 60 70 80 90 95 97 99
Cutoffs Defined by AG Success Rate
Fig 2: Absolute Difference Between Success (or Failure)
Rates of AG and DG at Various Cutoffs
Percentage Points
50
40
30
Absolute Diff Betw Rates
20
10
0
1
3
5 10 20 30 40 50 60 70 80 90 95 97 99
Cutoffs Defined by AG Success Rate
Patterns of Distributionally-Driven Changes in the
Concentration Index as an Outcome Increases in Overall
Prevalence
• Concentration index value adverse to the disadvantaged group
for failing to experience the outcome tends to decrease (i.e.,
failure to experience the outcome becomes more concentrated
in the disadvantaged group).
• Concentration index value adverse to the disadvantaged group
for experiencing the outcome tends to decrease (i.e., outcome
becomes less concentrated in the advantaged group).
• See Concentration Index sub-page of MHD and Table 1 of
Chance 2006. Latter shows how decreasing poverty increases
proportion blacks make up of the poor and of the non-poor.
Fig 3: Concentration Index Values Adverse to
Disadvantage Group for Failure and Success at Various
Cutoffs
Cutoffs Defined by AG Success Rate
0
Concentration Index
1
-0.1
3
5 10 20 30 40 50 60 70 80 90 95 97 99
Fail Concentration Index
Success Concentration
Index
-0.2
-0.3
Other Illustrative Data
•
•
•
•
•
Income data (Chance 2006)
NHANES Illustrations
Framingham Illustrations
Life Table Illustration
Other types of data: test scores of any sort,
foot race results, propensity score data,
mortgage eligibility ratings, etc.
Reminder One
It does not matter that one observes
departures from the described prevalencerelated (distributionally-driven) patterns.
Actual patterns are functions of both (a) the
prevalence-related forces and (b) the
differences between the underlying
distributions in the settings being compared.
Reminder Two
That the prevalence-related forces may depart
from those I describe (e.g., distributions may
be irregular) may indeed complicate efforts to
appraise the size of disparities. But such
possibility cannot justify reliance on standard
measures of differences between outcome
rates without consideration of the prevalencerelated forces.
Key Government Approaches to Disparities
Measurement
• National Center for Health Statistics (Health People 2010,
2020 etc)
– relative differences in adverse outcomes
• Agency for Healthcare Research and Quality HRQ (National
Healthcare Disparities Report)
– whichever relative difference (favorable or adverse) is
larger
• Centers for Disease Control and Prevention (Jan. 2011 Health
Disparities and Inequalities Report)
– absolute differences between rates
Table 1: Illustration Based on Morita et. al. (Pediatrics 2008) Data on Black
and White Hepatitis Vaccination Rates Pre and Post School-Entry Vaccination
Requirement (see Comment on Morita)
Period
Grade
Year
White
Rate
Black
Rate
Fav
Ratio
Adv
Ratio
AbsDf
EES
PreRq
5
1996
8%
3%
2.67
1.05
0.05
0.47
Post 1
5
1997
46%
33%
1.39
1.24
0.13
0.34
Post 2
5
1998
50%
39%
1.28
1.22
0.11
0.29
PreRq
9
1996
46%
32%
1.44
1.26
0.14
0.37
Post 1
9
1997
89%
84%
1.06
1.45
0.05
0.24
Post 2
9
1998
93%
89%
1.04
1.57
0.04
0.26
Table 2: Illustration of Appraisals of the Comparative Degree of Employer Bias Using
Different Measures of Disparities in Selection/Rejection
Employer AG Sel Rate DG Sel Rate RR Selection RR Rejection
AbsDf
OR
A
20.0%
9.0%
2.22 (1)
1.14 (4)
0.11 (4)
2.53 (1)
B
40.1%
22.7%
1.77 (2)
1.29 (3)
0.17(2)
2.29 (3)
C
59.9%
40.5%
1.48 (3)
1.48 (2)
0.19 (1)
2.19 (4)
D
90.0%
78.2%
1.15 (4)
2.18 (1)
0.12 (3)
2.50 (2)
• parenthetical numbers reflect the rankings of most to least discriminatory
employer using the particular measure.
Larger Implications
• Pay-for-Performance
• Subgroup Effects
• Meta-Analysis
• Case Control Studies