Stanford University's Regulation, Evaluation, and Governance Lab (RegLab) has partnered with Santa Clara County to harness the power of artificial intelligence (AI) in addressing a monumental task: identifying and redacting racially restrictive covenants from over 5 million property deed records.
The project, led by Stanford's RegLab and including affiliates from Stanford's Human-Centered Artificial Intelligence institute (HAI) and Princeton University, aims to comply with California's Assembly Bill 1466 (AB 1466). Enacted in 2021, AB 1466 requires all 58 counties in the state to create programmes to identify and redact deed records containing racial covenants.
Racial covenants, once common in property deeds across the United States, prohibited the sale or occupancy of properties based on race, ethnicity, or religion. Although deemed legally unenforceable since 1948, these discriminatory clauses persist in many historical property records.
The scale of the task is daunting. Santa Clara County alone possesses 24 million deed documents, totaling 84 million pages, dating back to 1850. Manual review of these documents would be prohibitively time-consuming and expensive.
Louis Chiaramonte, Assistant County Clerk-Recorder, told the Stanford HAI news, "Prior to this collaboration, our team manually read close to 100,000 pages over weeks to identify racial covenants. It was a challenging undertaking."
The Stanford team's solution leverages a state-of-the-art open language model, fine-tuned to detect racial covenants with near-perfect accuracy. This AI system is estimated to save 86,500 person-hours and cost less than 2% of what comparable proprietary models would.
Mirac Suzgun, a JD/PhD student in computer science and co-lead author of the study, stated to the Stanford HAI news, "We believe this is a compelling illustration of an academic-government collaboration to make this kind of legislative mandate much easier to achieve and to shine a light on historical patterns of housing discrimination."
The researchers focused on 5.2 million deed records from 1902-1980, the period most relevant to racial covenants. Their findings revealed that one in four properties in Santa Clara County were subject to racial covenants as of 1950.
Notably, the team discovered that only 10 developers were responsible for a third of the identified covenants.The study also uncovered a striking instance of a San Jose-owned cemetery that included burial deeds exclusively for "Caucasians".
Greta Hansen, Chief Operating Officer of Santa Clara County, expressed gratitude for the partnership in the Stanford HAI news article: "We're grateful for our collaboration with Stanford, which has helped the county substantially expedite this process, saving tax-payer dollars and staff time."
The team has released their paper and is making the model available to enable all jurisdictions faced with similar tasks to identify, redact, and develop historical registers of racial covenants more effectively.
This innovative use of AI not only accelerates compliance with AB 1466 but also provides valuable insights into the historical patterns of housing discrimination in Santa Clara County. As other jurisdictions grapple with similar challenges, this model could prove instrumental in addressing the lingering effects of past discriminatory practices in US property records.