Government agencies are embracing machine learning to support a variety of resource allocation decisions. The U.S. Environmental Protection Agency (EPA), for example, has engaged academic research labs to test the use of machine learning in support of an important national initiative to reduce Clean Water Act violations. We evaluate prototypical risk prediction models that can support compliance interventions and demonstrate how critical algorithmic design choices can generate or mitigate disparate impact in environmental enforcement. First, we show that the definition of which facilities to focus on through this national compliance initiative hinges on arbitrary differences in state-level permitting schemes, causing a shift in environmental protection away from areas with more minority populations. Second, the policy objective to reduce the noncompliance rate is encoded in a classification model, which does not account for the extent of pollution beyond the permitted limit. We hence compare allocation schemes between regression and classification, and show that the latter directs attention towards facilities in more rural and white areas. Overall, our study illustrates that as machine learning enters government, algorithmic design can both embed and elucidate sources of administrative policy discretion with discernable distributional consequences.
Regulation, Evaluation, and Governance Lab