Beyond Legal Mandates:
Using Disparity Study Results to
Improve DBE Programs
2017 WASHTO
Civil Rights Training Symposium
Colette Holt
Attorney at Law
Disparity Studies: Why They Matter
• Provide litigation defense
– Programs without solid studies will be struck down
– Studies aren’t challenged; programs are challenged
• Meet regulatory requirements
– Set overall, annual M/W/DBE goals
– Develop M/W/DBE contract goals
• Make administrative improvements
– Obtain confidential customer feedback
– Create focus on data collection & monitoring
– Supportive administrative accountability
Colette Holt & Associates
2
Disparity Study Challenges
• Inadequate funding
• Incomplete understanding of methodologies
• Missing contract data
– Prime information, including payments
– Subcontractor information
• Unrealistic timelines
• Political pressures
Colette Holt & Associates
3
Disparity Studies Gone Bad:
Agency A
• Weak legal analysis
• No economy-wide evidence of discrimination
• Weak anecdotal evidence; experiences of nonM/W/DBEs not explored
• No analysis of contracts over $500K
• Overly broad industry categories (i.e., no NAICS
codes)
• Incorrect market area definition
Colette Holt & Associates
4
Disparity Studies Gone Bad:
Agency A, cont’d.
• No info on prime contractor survey, response
rates or non-response testing
• Majority of construction contracts & dollars not
included
• Incomplete availability measures
• Unnecessary & confused “willingness” test for
primes
• Unnecessary & indefensible “capacity” analysis
• Result: weak program with low goals
Colette Holt & Associates
5
Disparity Studies Gone Bad:
Agency B
• Improper availability analysis
– Excluded minority & women firms that “might” not be
eligible to be certified as DBEs
– Excluded firms that had not bid
– Excluded discouraged firms
– Excluded firms that did not participate in the survey,
even if they were working for the agency
– No non-response testing for survey
– Original D & B universe of 50,000 firms reduced to
3,400
Colette Holt & Associates
6
Disparity Studies Gone Bad:
Agency B, cont’d.
• No Census SBO or ACS analysis
• Unnecessary & indefensible “capacity” analysis
• Result: some minority groups were dropped
from the race-conscious program & a low overall
goal was adopted
Colette Holt & Associates
7
Disparity Study Elements
• Determine utilization of M/W/DBEs as % of total
dollars in the agency’s geographic & product
marketplaces
– Use highest level of detail (NAICS not “construction”)
– Do not set a ceiling (e.g., $500K); set a floor (e.g.,
informal threshold)
– Fill in missing non-M/W/DBE subcontractor data
– Obtain large majority of contracts & contract dollars
(e.g., 85%)
Colette Holt & Associates
8
Disparity Study Elements
• Determine M/W/DBE availability
– Create database of relevant agency projects
– Identify the geographic & product markets empirically
– Count all businesses in relevant markets; cast a “wide
net”; do not limit pool to certified firms, bidders lists,
etc.
– Identify all M/W/DBEs in those markets
– Provide dollar-weighted estimates to set annual goals
– Provide dollar-weighted estimates by 6 digit NAICS
code to set DBE contract goals
Colette Holt & Associates
9
Disparity Study Elements
• Do not limit availability by surveys
• Do not conduct a “capacity” analysis
–
–
–
–
–
No common definition
Ignores the elasticity of supply, especially in construction
What about subcontracts?
Disparities persist even when variables are controlled for
Variables (revenues, years in business, bonding limits,
etc.) are impacted by discrimination
– Ignores the DBE program’s remedial nature by locking in
the results of past discrimination
– “Capacity” argument rejected by courts when explained
by expert testimony
Colette Holt & Associates
10
Disparity Study Elements
• Conduct an agency contracts disparity analysis
for local programs & DBE programs in the 9th
Circuit
– Necessary but not sufficient for current programs
because of the effect of remedial market intervention
– A finding of no disparity isn’t the end of the analysis;
consider
• Effects of the existing program
• Continuing impact of discrimination
Colette Holt & Associates
13
Disparity Study Elements
• Conduct an economy-wide disparity
analysis
– Look outside agency’s own contracting activities
– M/W/DBEs’ vs. non-M/W/DBEs’ business
formation rates & earnings & firm receipts from
Census data sources
– Human capital discrimination review
– Critical element of legal defense for existing
programs
Colette Holt & Associates
14
Disparity Study Elements
• Collect anecdotal evidence
– Necessary but not sufficient
– Explore current effects of past biases & exclusion
– Examine denials of full & fair access to government
contracts & subcontracts
– Evaluate existing programs, including race-neutral
measures, for effectiveness in remedying
discrimination & providing opportunities
– Critical element for M/W/DBEs’participation
Colette Holt & Associates
15
Disparity Study Elements
• Conduct a program review
– Interview M/W/DBEs, primes & staff
– Evaluate the effectiveness of contract goals
– Evaluate the effectiveness of race-neutral measures
• Utilization on no-goals contracts
• Small business elements
– Size standards & personal net worth criteria
– Setasides
– Contract goals
• Supportive services efforts
• Business Development Program
Colette Holt & Associates
14
Disparity Study Data Requirements
• Internally review your data before issuing a RFP
• Minimum data requirements
– Contract description/type of work
– Unique contract identification number
– Prime vendor payment data
– Prime vendor contact information
Colette Holt & Associates
15
Disparity Study Data Requirements
• Beneficial data
– Prime vendor contact information
– 6-digit NAICS or other detailed industry codes
– Race & gender of the majority owner
– Subcontractor information
• Subcontractor payment data
• Subcontractor contact information
• 6-digit NAICS or other detailed industry codes
• Race & gender of the majority owner
Colette Holt & Associates
16
Disparity Study
RFP Design & Process Tips
• Allow at least one year for study completion
• Analyze 3-5 years of data
• Evaluate cost factors
• Include legal counsel at all steps
• Use a general rather than a detailed scope of
work
• Clearly & in detail describe the state of the agency’s
prime & subcontract data
Colette Holt & Associates
17
Disparity Study Results
• What if a group’s utilization does not show large
statistical disparities?
– Role of large contracts?
– Role of large awards to a small number of firms?
– Cause vs. effect? Utilization on no goals contracts?
– Anecdotal data?
– Utilization on other local agencies contracts?
– Disparities in economy-wide utilization?
Colette Holt & Associates
18
Disparity Study Results
• Using study data for program implementation
– Using the study results to develop the overall triennial
DBE goal
– Developing & integrating study data into the contract
goal setting methodology
– Ensuring future adequate data collection through
electronic data collection & monitoring
Colette Holt & Associates
19
Disparity Study Results
• Issues identified in the program review
– Contracting policies & procedures
– Experience & bonding requirements
– Certification issues
– Contract goal setting methodology
– Good faith efforts reviews
– Commercially useful function issues
– Payment
Colette Holt & Associates
20
Study Methodology Matters
• Does the approach meet legal & social science
standards?
• Does the agency want a strong remedial program?
– Remedying discrimination vs. “race-neutrality”
• Do the results from prior studies comport with
reality?
• Do not fall in love with the model; has any agency
“fixed” discrimination?
Colette Holt & Associates
23
Study Methodology Matters
• Limits of data & statistical analysis
– Statistics can only support inferences; they don’t “prove”
anything
– Data & methodology are not value neutral
• Lens brought to bear affects the research paradigm
• Unintended & implicit biases
– Amazon refused to deliver to certain zip codes
– Google searches showed women lower paying jobs
Colette Holt & Associates
23
Colette Holt
3350 Brunell Drive • Oakland, CA 94619 • 773.255.6844
[email protected] •Twitter: @mwbelaw
24