The
debate over predictive analytics in child welfare will continue right
after this important message:
A bunch of data analysts,
presumably working for a firm that sells sporting goods, are spying on a
woman’s recreational habits. They have amassed so much data and their
algorithms are so wonderful that it’s like having a camera watching her 24/7.
Not only do they know her preferences, they know exactly why she prefers one
sport over another and exactly what she’ll do next.
In other words, they’re stalking her.
But this is not presented as a warning of the dangers of
predictive analytics. On the contrary, virtual stalking is what they’re
selling.
That’s because the commercial is not aimed at consumers – such
as the woman being stalked. The target audience is potential stalkers; in this
case people who want to sell her stuff.
The maker
of the stalking – er, analytics – software, and maker of the commercial, is SAP
– described as one of the
“market leaders” in predictive analytics and a potential competitor in the
child welfare market.
[UPDATE: MARCH 14, 2018: When I originally posted this, I embedded the actual commercial. But SAP seems to have removed it from YouTube.]
Unlike the bland reassurances given when people raise concerns
about predictive analytics, the commercial reveals the real mindset of some of
the human beings pushing big data.
Apparently, no one at SAP was creeped out by the ad’s Orwellian
overtones. The slogan might as well have been “Big Data Is Watching You.” That
alone ought to be enough to make anyone think twice about turning these
companies loose in the child welfare field.
And it ought to make anyone think twice about giving this kind
of power to secretive, unaccountable child welfare bureaucracies that have
almost unlimited power to take away people’s children.
But in case that’s not enough, there’s also:
§ ProPublica’s expose of profound racial bias in a
field far closer to child welfare than sporting goods: criminal justice.
§ The findings from New Zealand about bias in predictive
analytics in child welfare itself.
§ The fact
that when the highly touted predictive analytics program being tested in Los
Angeles predicted a child would be the victim of a “critical incident,” it
apparently was wrong 95 percent of the time.
[Links to all of this can be found in our publication Big Data is Watching You]
There’s little harm in mistakenly telling a sporting goods
salesperson to promote golf clubs to someone who’s more interested in soccer.
There’s far more harm in telling a child protective services caseworker that a
child is in grave danger when he’s not.
