In 2015, the president of a small Catholic university in Maryland, Mount St. Mary's, wanted to improve what schools call the "retention rate," meaning the share of incoming students who stay. He had freshmen fill out a survey to find the ones who were struggling. Sweet, right? Find them so you can help them. Not quite. The plan, according to what professors said in the meeting and what was later published, was to nudge the most fragile students out in the first few weeks, before the date when they officially count as "enrolled." That way, the statistic would look gorgeous. When professors objected, the president told them they thought of the students as cuddly bunnies, and that you just had to drown the bunnies. His words. He resigned in February 2016.

Now, that president was not using artificial intelligence. But think about what happens when you hand somebody like that a tool that says "I predict who will fail." He does not have to say it in a faculty meeting, where somebody can leak it to the student paper. A screen does it, quietly, with a number, and nobody knows why. And the screen never resigns.

Today we are going to talk about that character. The machine that hands down a sentence without explaining, without an appeal, and without blushing. Five ways predictive AI fails. Five. With real cases, real dates, and a bit of good news at the end.

A number with no explanation is a sentence with no trial.