Recent years have witnessed an increase in the interest in corpus linguistics – the quantitative analysis of large volumes of text, sometimes aided with machine learning – to inform legal meaning. Researchers have claimed that corpus linguistics enables robust, rigorous, and transparent discovery of the original public meaning of constitutional provisions and the meaning of statutory text. We contribute to this debate from the perspective of researchers in computational text analysis. We document tensions between such empirical semantic meaning approaches and judicial interpretation, where the use of corpus linguistics may sub silentio clash with express jurisprudential commitments. First, corpus linguistics may rely on foreign law to interpret U.S. provisions in a way that some judges would disparage. Second, corpus linguistics may offer legislative and ratification history that contradicts textualist commitments in statutory interpretation and raises questions for originalist methodology. Third, corpus linguistics may represent elite, not ordinary public meaning. We illustrate the sensitivity of these approaches to modeling choices and argue that these tensions are only likely to be exacerbated as corpus linguistics moves further into machine learning and artificial intelligence, where claims about meaning can be quite model sensitive. We conclude with proposals for improving evidentiary gatekeeping and adversarial testing of corpus linguistics and language modeling in law.
Regulation, Evaluation, and Governance Lab