Profs Tapped To Fix School-Choice Lottery

Newly arrived in the city, parents with a 4-year-old plan to enter the lottery for kindergarten spots at one of New Haven’s public schools next year. They know they’re rolling the dice in a game many consider rigged or broken.

They don’t like their neighborhood school. They keep hearing good reviews of the city’s magnet and charter offerings.

On the lottery application, they rank Davis Street, Mauro-Sheridan, Barnard, and Elm City Montessori as their top four picks.

In the process, they’re playing the longest possible odds.

Their kid could begin the school year seated in one of the most highly coveted spots — or stuck in a less desirable school.

Would knowing the exact odds make any difference in the parents’s choices? If they’d been warned about their slim chances, would they be more likely to swap in a backup? If they still went ahead with a big gamble, should the lottery administrators take that as a signal of how deeply they care about their picks?

The school district has tapped three academics to study those very questions in an effort to make the lottery process more transparent, simple and equitable this year.

At its Monday night meeting on Meadow Street, the Board of Education’s Finance & Operations Committee recommended approving a non-financial agreement with three assistant professors —  Princeton’s Adam Kapor and Christopher Neilson and University of Chicago’s Seth Zimmerman —  to recommend how the district could do a better job marketing and communicating about a revamped enrollment process.

Christopher Peak PhotoAs part of the agreement, New Haven will change the algorithm that it uses to assign students in 2019-20. Modeled on applications in New York, Chicago and Boston, the new model will ideally reward students for listing the schools that they’d like to attend rather than for gaming the system.

“We would be digging into this year’s data to make recommendations for the following year,” said Michele Bonanno, the magnet-school coordinator who’s temporarily overseeing the Choice & Enrollment Office after Sherri Davis-Googe’s resignation. “That could potentially include changes to the algorithm, changes to the way that we collect applications and the choices that we allow families, the way we communicate that, how we collect feedback. All of that will be in a set of recommendations to the board to set policy.”

For the following year, in 2020-21, the academics will design online tools, like chat bots, that can walk families through the process, introducing them to more schools and showing them how to maximize their odds of a school they like. The contract states that the school district will approve each “smart marketing” and “application assistance” tool before it goes live on the website.

The researchers have been studying New Haven’s school system for close to a decade, starting as graduate students in economics while at Yale. In an early study, they looked at the effects of New Haven’s $1.4 billion school construction project on enrollment counts, test scores and housing prices. More recently, they turned their attention to the district’s school-choice lottery.

“They’ve been helping us think about our school-choice strategies: the way we deal with the algorithm of the lottery, to help run simulations, to do smarter marketing and to do more with online chat-bots,” Bonanno said. “They’ll be helping us communicate through the process, not just advertising before.”

How It Works, Or Doesn’t

New Haven Public SchoolsIn years past, New Haven has asked students to rank their top four choices.

Once all the applications are returned in April, the lottery algorithm begins by clustering every student with the same first-choice school.

Then it splits them up into groups based on whether they qualify for preferential treatment, either because they live in the neighborhood or have a sibling already in the school.

From there, the algorithm randomly assigns an order within each group and starts seating them, in order of priority, until every seat is taken. Theoretically, a class could be filled up entirely with students who have neighborhood and sibling preferences without room for anyone else.

The system then regroups any students who don’t have a spot yet by their second-choice school, on down through all four rankings.

Students who don’t get into any of their choices are assigned to the school closest to their home with available seats, while they see if they can get off the waitlist.

Faulty Assumptions

In 2015 and 2017, Kapor, Neilson and Zimmerman sent out rounds of surveys to 417 households with eighth-graders about this process. They were testing how familiar families were with the sorting mechanism and gauging how students thought about the tradeoffs between their preferences and their chances in getting in.

After matching the questionnaires against the actual lottery submissions, the researchers found that students were indeed strategizing about which schools to prioritize, yet their gaming was often based on faulty assumptions.

Students’ understanding of the algorithm was just as bad as if they’d randomly guessed at answers.

The most common pitfall families run into is how to play their preferential status. Often, they depend on it as a backup to guarantee their lower picks. But if it’s a highly competitive school, all the slots might be taken up before they can even use their special priority.

“In school choice settings, where a group of mostly lower-income households submit application portfolios at most a handful of times in their lives, belief errors are relatively large,” Kapor, Neilson and Zimmerman wrote.

Parents have long been clamoring for statistics around the lottery process, saying for years that they were going into the lottery without knowing their chances of being accepted. In 2013, the district first released full data on the number of students who applied and were accepted to each school, the number of students who had neighborhood or sibling preference for each school, and the number of students accepted to each school that labeled it their first, second, third and fourth choice.

Then, after worrying that parents were picking schools simply because they had low acceptance rates, the district reversed course and stopped releasing full stats. Last year, the magnet-school guide had no information about the chances of getting in to a given school.

In the paper, which was published in the American Economic Review in 2016, Kapor, Neilson and Zimmerman said that there’s currently more uninformed parents making mistakes than sophisticated gamblers making savvy bets. They concluded that “offering some means to learn about admissions probabilities for different portfolios” would likely be “welfare-improving.”

Equity

Figuring out exactly what that will look like is the subject of Kapor, Neilson and Zimmerman’s next experiment.

As part of the memorandum of understanding that the Finance & Operations Committee reviewed on Monday, the academics agreed to provide a data analyst who will assist the Choice & Enrollment Office in-house at no charge for the next two years. With additional surveys and data analysis, they’ll review how the revamped lottery changes placement outcomes, family satisfaction and student achievement.

“Throughout my transition report, we heard, ‘What are we doing as it relates to equity in all areas?’ We’re going to look at that in how we place students,” said Superintendent Carol Birks. “Right now, not all parents feel like they have a real choice in the decision. That’s why we’re looking at this early and having a data analyst to see what policies might be.”

Tags: , , , , ,

Post a Comment

Commenting has closed for this entry

Comments

posted by: Wooster Squared on January 8, 2019  4:31pm

I’m glad to see that there’s serious interest in fixing such a broken system. I do wish that they would focus on simpler more obvious solutions, however, such as guaranteeing students spots in their neighborhood school, which is something the overwhelming majority towns in the state do.

From personal experience, I can tell you the school lottery was THE reason I decided to move my family to the suburbs. Many of my neighbors are also New Haven transplants for the same reason.

New Haven has made so much progress in the last 25 years, but it’s always going to struggle to hold on to middle class families if it insists on sticking to a system that gives parents so little say in where their children go to school.

posted by: Bill Saunders on January 8, 2019  6:22pm

Notions of ‘School Choice’ have created an untenable octopus of a ‘system’ that does not serve the students at large.  There is nothing ‘progressive’ going on here—it is just another layer of bureaucracy building.

A good cleaning of the Meadow Street Chicken Coop is in order.

God, everything looked so promising back when the State was throwing scads of money around to build ‘new buildings’...

Remember when ‘school choice’ was going to mitigate a severely broken educational system

Remember when the only thing ‘holding our students back’ was decrepit old schools….

1.5 Billion Dollars later (coupled with the toking of ‘time’) , and it turns out that the only thing decrepit was the BOE itself, and the grandiose vision of it’s ‘former City Executive’.

But we knew that already…......

posted by: Jill_the_Pill on January 8, 2019  7:55pm

Is the district paid by the charter companies to market and run the lottery for their programs?

posted by: Thomas Alfred Paine on January 8, 2019  8:39pm

No matter how you slice it, dice it and dress it up, “School Choice” is choice for the few and not for the majority!
In the United States of America, there should be absolutely no such thing as public schools classified, judged or rumored to be the “most highly coveted” schools or “less desirable schools.”
Every New Haven public school student should have equal access to a quality education with the same type of educational, human and material resources.
These lottery games ARE games of chance! It is a grand farce perpetrated upon the public!
If every school were equal, there would be no need for school choice lotteries or new and improved school choice lotteries!
We do not need a group of professors who do not live in this city and have no children in our schools to improve something that in the final analysis does confront the cruel reality we now face: the EDUCATIONAL INEQUITIES WITHIN OUR OWN NEW HAVEN PUBLIC SCHOOLS!
There are only a certain number of slots available in our “CHOICE’’ schools. Some parents resort to lying, cheating, and political influence or intervention to get their kids in one of them, while the rest of our kids are “STUCK” in schools that many parents consider “inferior, problem, troubled or failing schools! No school should ever be called failing.
Such things ought not be! But because “it be,” it is a blatant admission that the system is failing a majority of those kids and their parents who CANNOT and WILL NOT ever get into one of those “CHOICE” schools!
The way to solve this problem is not to look to three professors to resolve a dilemma that can only be addressed and resolved by making sure that every New Haven kid has the same quality of instruction, educators, educational resources and other materials IN EVERY SINGLE NEW HAVEN PUBLIC SCHOOL, WITH THE EXACT SAME PER PUPIL EXPENDITURE IN EVERY SCHOOL!
When every school is created equal, CHOICE will cease to matter.
That should be the primary focus and goal of our Board of Education!

posted by: WestHavenBIcyclist on January 9, 2019  6:56am

The numbers for Mauro-Sheridan are artificially high because only a few slots open up for K. Most of the class fills up with the Pre-K classes in prior years. Not sure about Montessori.

I don’t disagree with the fairness and equity issues raised by the commenters. There are also great things about the magnet system, which has enabled our sons to come in from “the suburbs” and ttend very high quality, ethnically and culturally diverse schools.  The other district magnet schools tangibly counteract some of the negative, divisive, and economically damaging effects of suburban flight.
The school has been great for our children and our family. I’d like to see the conversation more about bringing the excellence of some of the magnet schools to all schools in the district rather than the “end school choice” argument, which would dismantle one very positive thing New Haven schools.

posted by: repmd on January 9, 2019  7:04am

Remember how the best and brightest brought us to victory in the Vietnam War. These ivy league experts are charged to solve a problem but we do not understand the problem. The current system was set up to address the issue of racial disparity in the public school system. Currently there not enough white children in the system to have meaningful integration. As the magnet system was developed to attract students from the suburbs, it would be interesting to know the racial make up of the children who enter the NHPS from the suburbs. Returning to the issue of the lottery system, we could open any open seat in a magnet school to neighborhood children on first come first serve bases. This would also reduce the need to spend 25 million dollars on busing students.

posted by: Molly W on January 9, 2019  10:01am

Thanks for this article- it’s offers a clear explanation of a process that’s very confusing for people. I’d like to dig into the accepted narrative that parents can move to a city with a kid that’s never attended a K and up school and yet already know that they don’t like their neighborhood school. I’m not taking issue with the author describing this imaginary family. I want to take issue with the described family, who we all could know well. Why doesn’t the family like the school? Who have they spoken to? What bias/racism are they accepting in the given reviews of the school? Are they consulting the bogus resource of Great Schools (https://medium.com/s/story/the-problem-with-great-schools-69b4ef4f5079)? Were they brought here by Yale, who will offer “run for the hills or shoreline” or private only advice when it comes to schools? (Cuz you know, their grab on Hooker ran out a few years ago.)

Finally, the idea of a chat bot is an interesting one. But it also seems like a potential mess of IA confusion in a process that includes many people whose first language is not English.

posted by: Conscience on January 9, 2019  10:44am

This is a small measure that is admirable in many ways but tantamount to making a fairer system for getting a small number of children into choice schools. The case can be made that the prime beneficiaries are suburban parents who want their children in diverse urban schools that may have more to offer than many suburban schools,  as well as middle class Black professionals who want to choose the schools for their children. The rich get richer and the poor are assigned to the low income schools in their neighborhoods that have limited resources, higher teacher turnover, and too few parent activists making demands on the system for equity.
New Haven needs an aggressive, research-based strategy for teaching poor children that includes year round education and enrichment. This approach has the greatest chance for helping these kids compete in a race where they are already behind.

posted by: David Backeberg on January 9, 2019  11:04am

>The numbers for Mauro-Sheridan are artificially high because only a few slots open up for K. Most of the class fills up with the Pre-K classes in prior years. Not sure about Montessori.

Yes WestHavenBIcyclist is right. This is true for Kindergarten classes at any schools that have an attached pre-K, including the Montessori. Although some schools have more Kindergarten seats than pre-K seats, such that even if all pre-K students are filled into K slots for the next year there will still be slots open.

The first choice for slots go to the students who are raising up a grade and remaining at the school. For some reason, this article fails to mention those students, many or most of which don’t want to make any changes each time the lottery rolls around, and as such, if the class size is constant from grade to grade, very few seats ever open up in those grades at those schools.

Note also that at Magnet schools with interdistrict enrollment, the funding model is based on enrolling students NOT from New Haven as well as from New Haven. As such, your odds of getting in can actually be improved if you leave New Haven and then apply from a suburb that is eligible for the interdistrict magnet lottery. Behold, the double-edged sword of the interdistrict magnet funding model. School funding in this state is an amazing thing to witness. Maybe somebody can write an article about the history of ACES, how it works, how it’s funded, and who can apply. Those schools are still a mystery to me.

posted by: Teachergal on January 10, 2019  7:29pm

Can’t not comment on this. Worked in a well know magnet for 20of my 33 year career in New Haven. First, pre-k for many is free daycare and the parents move their kids out after a year. Secondly, we received many white kids from the suburbs with IEPs or behavior problems which NH has enough of already. Thirdly, most magnets are like any other NH schools, big test taking factories. Our school fought to get any of our magnet philosophy followed. New Haven is a mess and it’s too bad. With creative leaders we might have a chance but there are very few of those.

posted by: wendy1 on January 15, 2019  11:21am

I want good public schools.  If parents choose private or religious schools or homeschooling instead, fine with me.  I hope they can afford it.  I went to a private HS but public school K-8.  Our public schools are my focus and I am meeting with some teachers Thursday.  I recommend that all of you read the NH Teacher Wellness Survey Summary 2016-2017.