Predictive policing can carry old bias into new systems, research finds

Predictive-policing systems can reproduce problems contained in the police records used to build or guide them, research shows.

Sarah Brayne’s study of data practices inside the Los Angeles Police Department found that large-scale data systems added numerical risk scores to officers’ discretionary judgments. Police information was also being used to anticipate potential incidents, rather than only to respond to events that had already occurred. The study concluded that these analytics could extend earlier surveillance practices.

A separate examination looked at 13 U.S. jurisdictions where predictive-policing tools were used alongside documented instances of unlawful or biased policing. In nine of those jurisdictions, records from those periods were available to train or inform predictive systems. Chicago, New Orleans and Maricopa County were among the cases examined.

The researchers said that relying on compromised records could lead to unlawful predictions and create cycles that intensify the original harm. If a system directs more attention toward places or groups already overrepresented in police records, the resulting data can continue to reflect that focus.

The same concern appears in healthcare algorithms, although the mechanism is different. A 2019 study examined a system that predicted healthcare costs and used spending as an indicator of medical need. It found that Black patients who were sicker than White patients could receive similar risk scores.

The study attributed the racial disparity to the choice of spending as a stand-in for illness. Replacing that cost-based indicator with a measure tied to health would have raised Black patients’ representation among people selected for additional assistance from 17.7% to 46.5%.

The research points to several ways to examine these systems more carefully. Researchers discussed independent review, efforts to correct problematic data and human oversight as safeguards. Those measures address both the records used by predictive tools and the decisions made after a system produces a score.

Together, the studies show why a system can appear data-driven while still reproducing patterns from earlier institutional decisions. The central issue is not only how an algorithm calculates a result, but also which records and measurements shape that calculation.