Evaluating alternatives to foster care: The advocate in scholar’s clothing

The lead story in the current issue of Youth Today is about how to measure the success or failure of programs serving youth in an "evidence based" world. Should programs be funded only if they can prove their success based on the same kind of "Randomized Controlled Trials" (RCTs) used to test medicines? Or are there so many more variables in human services programs than in pills that a lot of very successful programs, and those who benefit from them, would lose out?

It's an important debate, and one which probably calls for a "middle ground" answer, in which there is a reasonable standard of proof of effectiveness short of RCTs. But that's not the main topic here today. Today's primary topics are double standards, hypocrisy and failure to disclose at least the appearance of conflict-of-interest.

The issues arise in a letter cited in a sidebar at the end of the Youth Today story. The letter, available here, was sent to President Obama by four "scholars." The issue is funding for "home visiting" programs, in which someone from a government or private agency visits new mothers to offer help with child rearing. These are, in fact, useful programs, though exactly how useful is open to debate. And they are purely voluntary – we really are talking about help here, not investigations. (Though as I'll discuss in a future post, at least one "scholar" may want to change that.)

Because these programs widen the net of intervention into families – in some cases, potentially, to every new mother in the country - and do absolutely nothing to threaten the child welfare status quo, they are enormously popular in America's child welfare establishment. Since almost no one is opposed, I call them "goody-two-shoes prevention."

But not all of these programs have been proven equally effective. The strongest evidence, by far, supports one particular model, the Nurse Family Partnership program in which, as the name implies, the home visitors are nurses.

There is less evidence behind other programs, including what is probably the best known, "Healthy Families America," a project of a group called Prevent Child Abuse America.

President Obama originally proposed a big increase in funding for home visiting – but only for the Nurse Home Visitor model because it has the strongest evidence base.

The four scholars urged him to fund programs like Healthy Families America as well. Their position has some merit (and the Obama Administration ultimately came up with what appears to be a reasonable compromise). It is, in fact, far harder to apply the Randomized Controlled Trial model to evaluating social programs, and relying only on that model can be a serious mistake, something that Lisbeth Schorr, senior Fellow at the Center for the Study of Social Policy and Lecturer in Social Medicine at Harvard, discusses in several articles on her website. Her absence from the Youth Today story is an unfortunate omission. (About 14 years ago, I assisted Prof. Schorr with the editing of one of her books, and was paid for the work.)


In addition, though there are many excellent scholars studying child welfare, the field is so loaded with bias in favor of taking away children, and so poorly self-policed, that a lot of good programs probably would be strangled if a pure medical model were applied now.

PCAA'S RECORD OF EXTREMISM

The letter from the four "scholars" proves the point – though not in the way they had intended.

One of the "scholars" is one Dr. Deborah Daro, a "research fellow" at the Chapin Hall Center for Children at the University of Chicago.

But before she was Deborah Daro, scholar, she was Deborah Daro, advocate – serving as "research director" of the group now called Prevent Child Abuse America (PCAA) at a time when the group took a variety of extremist positions. These included everything from fanning the flames of the 1980s "mass molestation" and satanic ritual abuse witchhunts (relying in part on a psychiatrist who later would have his license suspended for two years on grounds of "dishonorable, unethical and unprofessional conduct") to publishing a special Spider Man comic book – still in print - that can't help but leave young children with the impression that they should turn in their parents if they get a spanking.

At Daro's PCAA, poverty had nothing to do with neglect – ever. A PCAA pamphlet declared that "the heart of the problem always is an emotional lacking in the parents" for which the solution is "re-education." [Emphasis added.]

Indeed, Daro's mentor, PCAA's first executive director, Ann Cohn Donnelly, seemingly relished rubbing salt in the wounds of parents who lost their children to foster care because they couldn't afford adequate housing or had to leave their children home alone to keep their jobs.

Said Donnelly: "There are a tremendously large number of people in this country who have little or no money who do not neglect their children. When parents neglect their children and are of low income, it is not sufficient to say they are excused because they have no money."

The sheer cruelty of the comment aside, it also is the equivalent of saying that, because there are a tremendously large number of smokers who don't get lung cancer, smoking has nothing to do with lung cancer.

Daro's own work at this time included a 1988 letter she wrote to The Wall Street Journal hyping the numbers in a federal report on child abuse issued at the time, the second "National Incidence Study" known as NIS-2.

She cited a part of the study which found that, in Daro's words: "Nine percent of the cases determined to be unfounded by Child Protective Service workers did indeed involve mistreatment that involved significant harm to the child."

But Daro left out the fact that, using the same criterion, the same study found that from 27 percent to 61 percent of the cases CPS workers "substantiated" should have been unfounded. In other words, caseworkers were two to six times more likely to mistakenly "substantiate" a case than to mistakenly label one "unfounded."

That information is in the same document, on the same page, in the same table as the information Daro chose to include in her letter – yet she left that part out.

And she didn't do it just once. She took the same information out of context in the same way in a report issued in 1992.

In short, the PCAA of Donnelly and Daro was the epitome of 19^th Century "child saving" 100 years after it should have been extinct.

And Daro is still at it. When the fourth National Incidence Study (NIS-4) was released earlier this year and showed a significant decline in child abuse, Daro tried as hard to minimize the findings as she tried to hype NIS-2 more than two decades ago. (Our own analysis of NIS-4 is available here; see especially how our analysis of the study's findings on "neglect" differs from Daro's.).

WHAT THE "SCHOLAR" DIDN'T TELL THE PRESIDENT

But not everything PCCA did was harmful. They came up with a promising strategy for helping new mothers, thereby reducing child abuse. They called it "Healthy Families America." In the years since, HFA has become a huge part of PCCA's agenda. One could argue that PCCA, and those who created the program, have staked their reputations on it.

One of those people was Deborah Daro. In fact, when interviewed for a book, Daro declared that "In cooperation with my colleagues at Prevent Child Abuse America, we developed Healthy Families America."

But that little detail isn't in the letter to President Obama. Nowhere in the letter is there any disclosure of the fact that what Deborah Daro really is saying boils down to: Mr. President, please make an exception and lower the standards of evidence for the program I helped to invent.

This doesn't make the request invalid. Nor was it wrong for Daro to make the request. The problem is the absence of the kind of basic disclosure expected in other fields. For example, leading medical journals expect that if a study purporting to prove the effectiveness of a drug was paid for by the company that invented the drug, this fact will be disclosed.

The other problem with Daro's position is that she appears to be selective about which programs deserve this kind of consideration.

Daro led a Chapin Hall team that evaluated a promising child welfare innovation called Community Partnerships for Child Protection. This innovation long has been listed among NCCPR's Ways to do Child Welfare Right.
(The evaluation was published in 2005. Five years earlier, NCCPR received general operating funds from the Edna McConnell Clark Foundation, which created the Community Partnerships initiative.)

Did I say Chapin Hall evaluated the initiative? Savaged is more like it. The evaluation set an impossibly high standard for what the program sites had to do to prove success.

The Chapin Hall team essentially wrote off Community Partnerships despite the fact that, at three of four sites, there was significantly greater improvement in reducing child abuse at the partnership sites than in the surrounding communities, using the measure chosen by Chapin Hall itself. But these data were buried in charts in an appendix. The overall message was "community partnerships don't work" and that's the message picked up, understandably, by reporters writing about the study.

At one of the Community Partnership sites, St. Louis, there also were very large reductions in entries into foster care. But for Community Partnerships, Daro decided one site was not enough. In contrast, in her letter, Daro cites evidence of healthier births from just one of scores of Healthy Families America sites as sufficient reason for a big federal investment in the program.

In short, the evidence for the success of Community Partnerships is as least as strong as the evidence for Healthy Families America. And given that the whole approach of Community Partnerships – including working to cushion the blows of poverty and extending a helping hand instead of a wagging finger - is the opposite of that advocated by PCAA when Daro was their research director, it's fair to ask if Daro should have recused herself from any involvement in the Community Partnership evaluation.

THE RECORD OF CHAPIN HALL

This kind of issue is nothing new for Chapin Hall. They similarly stacked the deck against Intensive Family Preservation Services (IFPS) – but in that instance, other researchers did more careful studies proving its effectiveness. Indeed, in its comprehensive survey of what passes the "evidence-based" test and what doesn't, the Washington State Institute for Public Policy rates IFPS programs that follow the model of the original such program, Homebuilders, among the relatively few interventions that is truly an evidence-based success. Their detailed evaluation is available here. But Chapin Hall's hatchet job set back the field for years. (For details see NCCPR Issue Paper #11).

And it was a leading "scholar" at Chapin Hall who led an effort to punish a researcher for disclosing the very fact that child welfare scholarship is riddled with bias in favor of "child saving" interventions, something discussed in detail in this previous post to the Blog.

Now, don't get me wrong. I can't wait for the field to be fully ready for evidence-based evaluation, and for funders to act on the results. We already know that, when that day comes the government will cut off all funding for residential treatment centers (RTCs) and parking place "shelters," the CASA program will disappear, and there will be vastly less foster care, since none of these programs has real evidence of success, and they've all got significant evidence of failure.

But that's the whole problem with the debate over evidence-based funding. What evidence-based really means is: If you want to challenge the child welfare status-quo you'd better be able to dot every i and cross every t on a huge pile of randomized controlled trials. If, on the other hand, you just want to conduct business as usual and shovel children into substitute care – no evidence is required.

Or, to put it another way, the findings from the RCTs are no match for the clout of the RTCs.

LATER THIS WEEK: Does Chapin Hall want YOU (investigated for child abuse)?