ANY AGENCY THIS SLOPPY WITH DATA IS BEING SLOPPY WITH CHILDREN’S LIVES
Sections of long reports with headings like “accessing and utilizing data” tend to get overlooked.
But with the latest report from the monitor for the Michigan child welfare consent decree, that would be a mistake. A lot of the stuff that should make jaws drop is in that section – pages 51 to 55.
It’s clear that this is among the main reasons the monitoring team wants the DHS leadership fired. There are constant references to leadership failure.
The monitoring team was appalled by the sheer sloppiness of DHS leaders when it comes to their handling of data. For good reason. Any agency this sloppy with data is being sloppy with children’s lives. According to the report:
It is the assessment of the monitoring team that the data quality issues uncovered in the course of the verification work should have been identified by DHS leadership prior to submission to the monitoring team. It is deeply troubling that DHS leadership submitted incomplete information to the court.
When I first read about this, in the summary at the start of the report, I drafted a memo to reporters in Michigan. I said, in my usual, undiplomatic way that DHS had made “the kind of math error that most children can spot by Junior High School.” But I would up replacing that with a direct quote from the report itself, which is nearly as blunt:
For Period Three, DHS produced the four [data] cohorts to the monitoring team, and the monitoring team immediately noticed a problem using basic addition and subtraction: beginning with the number of children in care at the end of Period Two, adding the number of all children entering care during Period Three, and subtracting all children who left care during Period Three, should have equaled the number of children in care at the end of Period Three. It did not. [Emphasis added].
And it’s worth reading this relatively long excerpt to get a sense of just how sloppy and lackadaisical the DHS leadership has been about this, and how furious the monitoring team is about it:
Finally, despite flagging this issue to DHS repeatedly, DHS leadership has not established a high-level verification and review process to test each report and compare reports for consistency from different units of DHS prior to forwarding those reports to the monitoring team. The monitoring team has consistently found important differences between one report and another.
For example, the monitoring team has found a more than 25 percent difference between the number of adolescents in care reported from one source versus another. Training data is not reconciled with caseload staffing data. BCAL data is not reconciled with data produced by the DHS Children’s Service Administration. Spreadsheets in the same report do not add to the reported totals – and sometimes are not added at all.
Much of the data produced consistently arrives without any analysis, including any critical caveats about the quality of the data. When the monitoring team calls these issues to the attention of individual staff, they are cooperative and respond to the best of their ability. And staff have developed a helpful standard reporting format used for a number of reports that includes the parameters of the report, data definitions, summary information, and the detailed reporting. But these are errors and omissions that leadership should be raising long before that information is sent to the monitoring team. The failure to do so suggests the leadership team is not reviewing the data, which leaves open the question of what information the leadership team is utilizing to manage the reform. [Emphasis added.]
I don’t think it’s setting the bar too high to suggest that, when Michigan’s new governor, Rick Snyder, reviews candidates for leadership jobs at the Michigan Department of Human Services, he includes among the qualifications the ability to add.
