New Haven Independent

Peter Behuniak (pictured), a University of Connecticut educational psychology professor, came to Monday night’s Board meeting to target the accuracy of ConnCAN’s statistics and methodology in its “report cards” on the state’s schools — including New Haven’s.

Behuniak is the author of a study commissioned by the Connecticut Educational Association that reanalyzed student assessment results and demographic data based on which ConnCAN came up with its top ten lists in its report “The State of Connecticut Public Education: A 2006 Report Card for Elementary & Middle Schools.”

In that report charter schools, as a group, came out looking good compared to traditional public schools. But the mayor and BOE have long contended ConnCAN’s data did not take into account the city’s highly mobile student population, its growing arrivals of non-English speaking immigrant students, who must take CMTs when their English is still sub-par, and other results-obscuring factors.

The BOE invited Behuniak to share the main points of his recently published critique. Among these were, first, that lists dealing with achievement gaps with low income, African-American and Hispanic students are inaccurate and not justified because the data used was only from 30 percent of the schools in the state. Second, lists dealing with year-to-year improvement in the report are also inaccurate because the data used no matched sets. That is, because of student mobility and other factors, nowhere were, for example, the same 80 third-graders measured against those very same 80 in fourth grade. The groups in schools were instead changing in un-accounted ways and thus inaccurate.

Therefore, Behuniak, concluded, “There is insufficient evidence presented in the report to support the observations or to imply the superiority of one school type over another.”

“I teach educational research,” BOE chair Dr. Brian Perkins, said, “and this work wouldn’t pass my course. Is it an honest error or a deliberate attempt to deceive the public?” He concluded it was the latter and said, “I don’t think it’s worth the paper it’s written on.”

The mayor also said he felt the report was “cooking the books” and an enterprise engaged in not by dispassionate researchers but by an interested party, whose work ultimately is not in the service of the kids of the state. Why? Because it has contributed, he went on, to the augmenting of the funding of charters at the expense of traditional schools. “We’ve never been in this situation before.” The urgency, he suggested, was in part the reason Behuniak had been invited to make the criticisms public.

Marc Porter Magee, the director of communication and research for ConnCAN, agreed and disagreed. To the criticism that only 30 percent of state schools were used in the study, he said, “That’s right. We use all the data provided by the state’s Department of Education, but many schools do not report, and legally don’t have to. There are 169 districts and 600,000 kids. If a school, for example, has only two African-Americans, they don’t report on that. That’s for privacy reasons. But many schools don’t have to report if they have 19 African Americans.”

Magee agreed that this threshold was too low, and that the state should require schools with, say, 15 African-American or Hispanic kids, to report, so the picture can be more accurate.

However, he defended the report cards’ findings. “When we do these lists, we do capture, for example, the highest African-American performances in schools with large populations of these kids, and no one’s done that before. It’s valuable for parents to know, for example, that Elm City Prep (the charter school) in New Haven had 76 percent of kids at goal. This is significant.”

Certain facts, he said, were also incontestable and also had to be highlighted: For example, the achievement gap between African-American students and whites in the state being the 49th out of 50th in the country, at the bottom next only to Alabama. That, he said, was based not on ConnCAN data but work done by NAEP, the National Association of Educational Programs.

To the charge that year-to-year performance gains, or declines, are inaccurate because of social mobility and others factors not being isolated, he said that was partially true. “That’s why we have advocated for and helped to secure $6 million for a matched sample study,” of the kind Behuniak said was lacking. The State Department of Education is going to be doing this. Magee said Connecticut is only one of eight states in the nation that currently doesn’t have this accurate data.

Nevertheless, he contended the picture was not rendered all that inaccurate, in part because 93 percent, he said, of kids in the state, return to the same school.

“We’re eager to get these new samples and integrate them into our research in the future.”

He insisted the purpose of the report cards was not to flog traditional schools at the expense of the charters but to report data as best as was possible given the data out there, so that people understand the problems and the challenges. Of many examples he gave, he cited the Church Street School in Hamden, a traditional elementary whose overall performance gain, 22 percent, was attributable, he said, to consistent, rigorous focus on achievement.

Nevertheless, Dr. M. Ann Levett (foreground), the newest BOE member, and an educator with the Yale Child Study Center whose expertise is on the African-American achievement gap, called ConnCAN’s report “tragic.” She said experts could parse the statistics. But when such skewed findings are presented to parents looking for a school — especially when presented in ConnCAN’s attractive and accessible report formats — a certain trust is shattered with parents who assume the truth is being presented to them, she argued.

She and other BOE members called for dissemination of the Behuniak report. The mayor said the fuller picture needs to be told — urgently.

It may, however, turn out to be either a continuation of the slipperiness of statistics and a he- said-she said dilemma, or even more confusion for the public. Magee threw the gauntlet right to Behuniak himself who, he said, told the Waterbury Republican-American newspaper on Jan. 25 that the numbers in ConnCAN’s report were accurate, and that for middle schools, for example, “overall charters had the largest mean performance gains and improvements — 8.4 percent for charters, 1.1 percent for traditional schools, and 0 percent for magnets.”

Bring on the next witness. Perhaps it should be Magee or a spokesperson for ConnCAN itself to address the BOE? He said he was disappointed ConnCAN was not invited.

In a related development, the BOE Monday night also voted to appoint Mayor DeStefano to sit on Amistad Academy’s board, Dr. Levett to sit on Elm City Prep’s, and Michael Nast to Common Ground’s. Under a state statue passed in July last year, charter schools are now required to have representatives from the local school board on their boards.

For other stories on this fractious dialogue, click here and here.