I've written before about school choice in San Francisco, and about how Muriel Niederle and Clayton Featherstone led the effort by a group of our colleagues to design a strategy-proof choice algorithm (explained here at a Board meeting in 2010), based on transfer cycles ("top trading cycles" to game theorists...). The school board adopted the plan, but then the staff of the school district decided to implement it themselves, without making the details public. Fast forward to 2012, when the first children have been assigned by the new plan.
Rachel Norton's blog has a post about it here: They're out! School Assignment Letter 2012, and an earlier one here, with a link to a March 5, 2012 SFUSD report on Student Assignment. As in the previous SFUSD reports, this does not describe the choice algorithm, it only describes the "tie breakers" that are used whenever the algorithm would otherwise try to assign more students to a school than it has room for.
This outraged Stan Goldberg (who reports about SF schools as "Senior Dad"), and he posted a video about the lack of transparency called Assignment System Fraud?
He must be an influential guy, because this prompted SFUSD to post some new information, including this "fact sheet" dated March 23, called How does the student assignment computer program work? It still doesn't come close to explaining the actual algorithm they use, but it does include a diagram of "transfer cycles."
Which raises a question. If they in fact implemented the plan we proposed and the Board adopted, you would think they would want to make this clear. The benefits of a strategy-proof assignment procedure can only be realized if parents know that they can safely list their true preferences.
On the other hand, if the algorithm isn't correctly implemented, or if some other assignment algorithm is implemented (whether or not it includes some use of transfer cycles) then it would most likely not be strategy-proof, that is, it might not be safe for parents to reveal their true preferences, and it might be in the interest of some to "game the system" in some way. That might account for a desire to keep the algorithm secret. (So might a desire to avoid revealing any inadvertent mistakes in implementation...)
I should say that SFUSD's brief description of their algorithm doesn't look to me like it describes one that is strategy-proof...:(On the contrary, it looks like it might be patched together from something like Boston's old immediate acceptance algorithm followed by some trading...but then again, it isn't a complete enough description to make me confident that it is a description of whatever they are in fact doing...)
Anyway, one point of this post is to say that, unlike the case of the systems in New York and Boston and the work that IIPSC is doing around the country, my colleagues and I don't know what algorithm SFUSD is using, even though we know what we proposed and the Board adopted. So...this post is a bit like the ads that sometimes appeared in the financial sections of newspapers when I was young, which, following a divorce, would announce that Mr John Doe was henceforth no longer responsible for any debts incurred by the former Mrs John Doe...
Sunday, March 25, 2012
Subscribe to:
Post Comments (Atom)
Posts
11 comments:
The new SFUSD system appears to give undue citywide advantage to families who live in the attendance areas of popular schools. If they rank that school #1, they have a greater chance of getting it than someone outside the attendance area, barring other tie-breakers such as low-income housing census tracts. That part seems fair enough. People were clambering for neighborhood schools and this is what they get.
But, they also appear to have an advantage to be assigned to schools outside their own attendance area through the swap system. The algorithm has not been released, but what appears to be happening is that families are assigned to their attendance area school through the initial tie-breaker system, and then once they are assigned to that school, they are eligible to be swapped to one higher up their list either in the first or subsequent rounds.
This is all well and good for these families.
The families who the system did not work for were the families who got none of their choices in the first round. In subsequent rounds, the system first swapped families who already got one of their choices, and there were never any openings for those who got nothing in the first round.
Middle school assignment did not have attendance areas last year, but the system worked the same other than that. What I observed was families who had already gotten a "good school" for their child, although not their first choice school, were able to appeal into subsequent rounds and trade up.
But other friends who did not get any of their requested schools never got a school on their list at all, even into October of the fall semester. All these families had made reasonable lists of schools, including some more popular and some less popular schools. But they still got nothing because they didn't have a school anyone wanted to swap with. They had no chips to play the game with!
Maybe this makes sense to an economist, but it certainly made no sense on the playground as Suzy's parents are ecstatic because they traded from #2 to #1 school, and Billy's parents are still left with nothing at all. Three friends left the public school system and ended up at parochial school when it became clear they never would get a school under the new system.
Is this really the way the system was designed to work?
@Anne,
If a family places their coveted attendance area school third they could use that if they get it to trade up to schools in which they had no tiebreaker. they would have an unfair advantage (IMO) over a family who lived in an unpopular attendance area.
Is my statement correct?
@Stan
That's what I'm seeing. Just wondering if that's how the system is supposed to work under market design theory.
In an ideal world, everyone would have a some currency to start with and then swaps would rationalize it. But the way it's working is that some people end up after the first round with no school (currency) and can't participate in the swaps. By the time all the swaps happen in the second and subsequent rounds, there are no vacant spaces for those who came up short in the first round.
@Anne and Stan: While I'm happy to explain and defend systems that my colleagues and I design, one point of my post was to make clear that I don't pretend to understand what the current SFUSD system does. It is of course the job of the school district to explain and defend the school choice system it uses.
That being said, in a well designed choice system,trades aren't a crazy idea, even though they benefit people who already were tentatively assigned a place to trade. In a well run system, this doesn't hurt those who weren't assigned, as the places being traded would have been assigned to others in any event. The point is to design a system that provides benefits to everyone. Eventually that involves making sure there are adequate numbers of places in good schools--no assignment system can fix a shortage, it can just make it more bearable by assigning available places as well as possible to families that find them desirable.
So if there is a serious problem with the current SFUSD choice system, as there may be, it likely lies with the way places are assigned in general, and not simply with the fact that trades may be involved. (And if there aren't enough places in good schools, that's a problem that the choice system by itself can't fix, although a good system can reduce the pain.)
Good luck on straightening things out, it's work worth doing.
It's not an undue advantage if a family wants to go to their local school. We're such a family - all we want is to go to the school around the corner. We put it as #1. And yet, that wasn't enough. We know of several families from outside the attendance area getting it despite not having any tiebreaker advantage. There are some real flaws in the system.
A. -- agreed. My understanding is that families were given an attendance area tiebreaker so that their child could attend their local school. I am not sure that the district intended to give these families an advantage over other families citywide through the swap system. But that appears to be how it's working.
Thank you for clarifying the swapping system, Al. It sounds like the second and subsequent rounds worked as they were supposed to, as painful as that may be for families who had no swappable school.
The question remains how the district made the initial allocations. Did the school district intend to give a citywide advantage to families with attractive attendance area schools, if they actually ranked a non--AA school higher than their AA school? Maybe they did, and maybe this is legitimate. But it's a question that needs to be answered.
Without the algorithm I don't know that we can fully understand the system.
I'm glad to see that people are finally looking at this issue. I don't think TTC itself is unfair, but the Jim Crow-ish neighborhood preference sets up an unfair advantage that is magnified by TTC.
Empirically, it looks as if the TTC mechanism had the expected effect -- families near unpopular schools overwhelmingly chose out-of-neighborhood schools and received disproportionately few of their choices. I take that as evidence that TTC is being applied as stated, although it's not conclusive.
"Empirically, it looks as if the TTC mechanism had the expected effect -- families near unpopular schools overwhelmingly chose out-of-neighborhood schools and received disproportionately few of their choices. I take that as evidence that TTC is being applied as stated, although it's not conclusive."
In fairness, some of this is mitigated by the preference given for those in the low-test score census tracts.
Please note that today I asked Rachel Norton for enrollment statistics in and out of CTIP1 zones. Additionally, I asked her to publish statistics that reflect those applying for the first time for their oldest child.
The inclusion of siblings presents a much too optimistic statistic.
In spite of the fact that parents have been asking for these numbers for weeks, this has not been published.
The lack of transparency of the assignment algorithm and the hiding of the number of families who do not get any of their school choices, does lead one to believe that the SFUSD Board has something to hide.
There is at least some evidence that SFUSD is not implementing the algorithm as designed. Once parents received their initial assignment last spring, they a certain number of weeks to register at their designated school or lose their child's spot. It turns out that at least this year, a rogue SFUSD staffer decided not to release unclaimed spots after the first round of the lottery. The spots were only released after a couple rounds when a parent asked Commissioner Norton to look into the matter. This kind of manipulation of the system by SFUSD makes it even more imperative that the district release their algorithm lest parents lose trust in the system.
Post a Comment