If the Orwellian predictive analytics model in Allegheny County is really so great, why does the county’s human services director have to obfuscate about what it is and how it works?
The sentiment is understandable. Cherna has held the job since 1997. I served on a search committee that
unanimously recommended him. For a long
time I was proud of that. Cherna built an impressive record as a reformer,
curbing the number of children in foster care and becoming a leader in the use
of kinship foster care, among other accomplishments. That built him a reservoir of goodwill.
But there are two principal problems with the “trust Marc”
argument. First, Marc won’t be there forever.
And second, the way he is selling the program undermines trust.
It’s not the embrace of predictive analytics that raises
questions about trust. There are other good, honorable child welfare system
leaders who support it (though it appears that most of the leading proponents
also are the most
extreme supporters of a take-the-child-and-run approach to child
welfare). The trust questions arise
because of how the county has sought to sell the first predictive analytics
program, the Allegheny
Family Screening Tool (AFST), which is used to help decide when to
investigate reports alleging child neglect, and the even
worse “Hello Baby” program – the one that seeks to slap that risk score on
every newborn in the county.
The problems with the sales job are apparent in an
article Cherna wrote for the Department of Health and Human Services
publication Children’s Bureau Express.
Compare the rhetoric to reality
Cherna writes that the program is “supported by the use of
integrated data and a predictive risk model…”
That’s it; there’s no further explanation. In fact, the “predictive risk model” is the
lynchpin of Hello Baby – the whole thing collapses without it.
Under that model, vast amounts of data are collected, most of it from systems
that enmesh primarily poor people. Then the system coughs up a “risk score” for
the newborn based on how their parents have interacted with those systems. Cherna promises that the risk score will be
used only to target prevention programs. Those with the highest risk scores
will get the most intensive “help.” But
there is nothing to stop Cherna, or his successor, from using the data in any
other way they choose. We just have to trust them.
Cherna writes that “Hello Baby is a program of DHS's Office
of Community Services and is not connected with child protective services.” He neglects two key points:
● The helpers – who are being sent into a home precisely
because an algorithm says the baby is at high risk of child abuse -- also are
mandated reporters of child abuse.
● The person in charge of both the Office of Community
Services and child protective services is – Marc Cherna. He, or a successor or
a successor’s successor, are free to tear down that wall of separation.
Cherna writes that the program is “voluntary.” Technically, yes. But when most people think of voluntary, they
think of something that is offered to them and they then choose whether to take
part. Hello Baby doesn’t work that
way. Hello Baby assumes that you have agreed to surrender your data and have your
baby labeled with a risk score at birth unless
you affirmatively opt-out. And it is
unclear how much chance you have to do that.
So ask yourself: If Hello Baby is so great, why do they have
to, in effect, sneak it past the very people its proponents say are most likely
to benefit, instead of being open and aboveboard about exactly what it is, and
letting people opt in if they really want it?
If it’s as wonderful as Cherna says, people should be lining up to take
part.
Cherna goes on to tout the “ethical analyses” done of Hello
Baby. But he doesn’t mention how the
county has stacked the deck for such analyses from the start.
The
first “ethics review” done for the AFST program, was a once-over-lightly
review of a few papers, all of which were written by one of the designers of
AFST or by one of the “reviewers” himself.
That reviewer is a faculty
colleague of one of the AFST designers – in fact, they’ve co-authored papers
together.
Even that review said AFST was ethical, in part, because it
was triggered only if someone called the child abuse hotline and alleged
maltreatment. They wrote:
[The issue of informed
consent] is one of a number of points at which we think that it is ethically
significant that the AFST will provide risk assessment in response to a call to
the call center, rather than at the
birth of every child. [Emphasis added.]
So, now that Cherna is rolling out a use of analytics that
does just that, what to do? First, call it “voluntary” by, as noted above,
stretching the meaning of that word almost to the breaking point. Then, commission more ethics reviews! But make sure they’re from people strongly
predisposed to support Hello Baby or apparently unfamiliar with child
welfare.
All of which leaves one question: Should we be comfortable
giving so much data and so much power to use that data to an agency that sells
the program this way?
