by Watson Scott Swail, Ed.D., President, Educational Policy Institute

For decades, the Educational Policy Institute and SwailLandis has worked with schools, colleges, and universities to help these organizations increase student persistence and graduation rates. We’d like to say that we have been very successful in helping these places, but it is difficult to state because the actual change tends to happen long after our consulting and evaluation contracts have expired. As dastardly as it sounds, my sense is that most institutions do not make positive change for students because it is too hard, takes too long, and there isn’t the institutional or political conviction to follow through on their ambition.

Institutions say a whole lot about the great work they do for students. It is in many ways both truthful and expected marketing. I have visited hundreds of institutions that do marvelous things for students. But sometimes the hyperbole outweighs the actions.

Two recent stories about institutional fraud—for lack of better or more accurate words—are germane to this discussion. The first was reported on NPR’s All Things Considered on November 28, 2017, and the second yesterday in Crain’s Chicago Business.

A few years ago, we conducted a large study in the District of Columbia in the most economically-challenged areas of the city—Wards 8 and 9. The study was conducted for a large philanthropy to verify the impact of their funding. We visited a number of schools and enjoyed talking with principals, teachers, and others related to the program, including then-DCPS Chancellor (e.g., Superintendent) Kaya Henderson. While some great things were happening, we left with reservations about the study and about the program in general and reported as such.

And then I read this NPR report yesterday.

NPR reported in late November that the graduate rates at Ballou High School in DCPS did not represent reality.  Ballou was one of our target schools a few years back. The conclusion of the NPR study found that the majority of Ballou High School’s 100 percent graduating class had missed more than six weeks of school, but graduated regardless.

A teacher was quoted in the NPR piece: “You saw kids walking across the stage, who, they’re nice young people, but they don’t deserve to be walking across the stage.” And if teachers raised an issue about passing students along, they were “painted as ‘haters’ who don’t care about students.” Some teachers said that teachers were let go if they held students back.

Students in Ballou High School’s 2017 graduating class, grouped by number of unexcused absences. Each dot represents one student.

Rewind two years previous when we visited Ballou High School. I had previously visited Ballou about 20 years ago before its recent $124 million renovation. To put it succinctly, no one would have wanted to send their children to the old Ballou. The conditions were atrocious. Water fountains and bathrooms were inoperable. Students reportedly smoked and had sex in bathrooms and closets. And then there was the rat infestation. We had one teacher tell us that he saw a rat run across his desk one day.

When we visited two years ago during the inaugural year of the new building, we were simply astonished. It was, arguably, the nicest school building we have ever seen let alone visited. With a state of the art theater, indoor pool, and even a triangular quad (I know, but what would you call it?) that, interestingly enough, they didn’t allow students to go in to.

But while the “new” Ballou looked wonderful, what was going on in the school was problematic, as now evident through the reporting by WAMU and NPR. The report stated that half of the graduates missed more than three months of unexcused absences in their senior year.

Less than two weeks ago, the principal at Ballou High School was removed from his position.

Former DCPS Chancellor Kaya Henderson, a protégée of lightening rod Michelle Rhee, often spoke about the progress that her district had made, just as Rhee did before her. Even the Washington Post wrote gleamingly about the progress during her five-year tenure, it is difficult to trust the data, especially after the NPR report. For those of us on the outside, we wonder whether DCPS really did improve rather than play the numbers. The Post said that the four-year graduation rate in the district improved from 53 to 64 percent from 2011 to 2015. They also said that the proficiency of fourth-grade students in reading increased from 14 to 30 percent from 2007 to 2015.

Given what was learned in the NPR study and report, and given what we learned in our study a few years ago in the District, I have trouble buying their progress. Quite simply, what we saw on the ground buttresses what NPR found. When we were looking for data from DCPS, we couldn’t get it. They wouldn’t give it to us even though we were working on a very important study for a philanthropic organization that pushed over a $100 million into the district. Hurdle after hurdle was put in front of us in our quest to access student data. In the end, the data we did get came from a third-party not-for-profit organization not affiliated directly with DCPS.

This lack of transparency in the district screams about the district mindset. And while I do not wish to lump everyone into the same category, because it is difficult to compare schools in northwest versus schools in southeast, DCPS has been and remains a troubled school district.

In a separate article from yesterday, we learn about the 2015 selection of Kennedy-King College in Chicago by the Aspen Institute for tripling its graduation rate in recent years. The College received national recognition and a $100,000 prize from Aspen. The problem comes from the fact that Chicago City Colleges had omitted previous pastry school graduates from the denominator, thereby making their graduation rate look better (the Pastry school has much higher graduation rates than other colleges and programs, so by not including them in the denominator and including graduates in the numerator, the college was able to show an incredible increase from 8.9 percent graduation rate in 2009 to 25.9 percent in 2013).

Here is the takeaway for readers: when one comes across a really large statistical increase, it probably isn’t true. And for the Aspen Institute, a very worthwhile and important organization, they should have known better. I’ve been around this carousel too many times. After peeling back the proverbial onion a little more, one finds that the denominator or the numerator has changed definition.

The same goes for DCPS and other schools. They like to play with the numerators and denominators to show their best face. We see it in pre-college outreach programs all the time. They post that 100 percent of their high school students graduate from the program and go to college. But they only count students who are in their program at the start of the 12^th grade instead of those who start at the beginning of the program in 9th or 10ths grade (or earlier). GEAR UP programs consistently miscount their graduates, as does Upward Bound. They use augmented data to demonstrate progress and shade inconsistencies. Let us be clear: many of these programs do really significant work. But the administrators  should all take another math course. Or ethics. Probably both.

These two stories tell us to be wary of the gains suggested from educational institutions. Hopefully, most of them are accurate. But school districts and colleges paint the best picture of themselves in public reporting, just like almost any other organization or company does. Quarterly forecasts by public companies often paint a very rosy picture for stock purposes, but can also be burned if they flat out misrepresent data and can be fined accordingly by the Securities and Exchange Commission (SEC) and other organizations. Schools of any level rarely get their hands slapped for similar but different situations.

In the end, we can only help schools when we have an accurate picture of what is happening to their students.