When police came, all Taki Allen was holding was an empty Dorito bag. But because his school’s AI weapons detector mistook the bag for a gun, armed cops rushed to the scene, ordered him to his knees, then cuffed and searched him. Obviously, there was no gun.
Baltimore County Public Schools’ superintendent responded that the system worked just the way it was supposed to—and that's the problem. While the criminal legal system’s reliance on AI technology has dangerous consequences for Black people, human choice also poses a danger.
The ACLU faults a combination of human choice—the school, the tech’s vendor, the security staff who called police, and the cops who arrived based on a photograph. Escalating to police surrounding an unarmed student, guns drawn, was avoidable.
The same goes for blatantly wrongful arrests based on faulty AI facial recognition tech, error-ridden police reports and legal documents, and more. But the system’s violence is only evolving. Think school resource officers inventing and exacerbating crises, or arbitrarily administering suspension policies.
We should dismantle anti-Black systems and technologies—and hold individuals accountable for the choices they make. Advocating for a world where youth like Taki Allen won't be targets for eating chips will require more than changing the process or removing AI gun detection from schools.