Followup on school choice in San Francisco

My previous post on how school choice is faring in San Francisco was called  School choice in San Francisco: a promise of transparency.  That promise still hasn't been fulfilled.

The idea was that, after the adoption by the school board of a New school choice system in San Francisco, SFUSD decided to implement the new, strategy-proof  "assignment with transfers" choice system itself (San Francisco school choice goes in-house).

School Board member Rachel Norton wrote in a November 9, 2010 blog post that
"Staff did pledge to make the documentation of the algorithm requirements and process flows public by February; I will continue to push to make the assignment algorithm itself open source."

While SFUSD has prepared a number of documents since then, none of them seem to contain a description of the SF school choice algorithm as actually implemented by the staff. All I can find are descriptions of the priorities used for tie-breaking if more children than can be accommodated by a school would otherwise have been assigned there, but no description of the process by which they would have been assigned before tie breaking has to be invoked.

The latest document of that sort, via Rachel Norton's June 1 blog post, is here: Board of Education Policy.
On page 7, under the heading "Method of Allocating Seats," the document states "The SFUSD will replace the diversity index lottery system with an assignment with transfers algorithm that uses school requests from families and the preferences outlined in this student assignment policy."
However the document doesn't describe the assignment with transfers algorithm at all, just the tie breaking priorities.

So...I'm still in the dark about whether SFUSD has actually implemented the choice system the Board adopted, and I bet SF parents and board members are too.
**************

Read more »

San Francisco school choice goes in-house

Those of you who have been following school choice developments here know that, for the past year,  Atila Abdulkadiroglu, Clayton Featherstone, Muriel Niederle, Parag Pathak and I have been helping the San Francisco Unified School District design a new school choice system, which was adopted by the SF School Board last March.

The original plan was that we would continue to offer our services free of charge to implement the software, and then help monitor the effects of the new choice system.

Last week we heard from SFUSD staff that, because of concerns about sharing confidential data for monitoring the effects of the new system, they have decided to do further development in-house, and so will develop software to implement the new design on their own.

The SFUSD staff  have  been left with a sufficiently detailed description of the "assignment with transfers" design the Board  approved to move ahead with it if they wish. But it will take a good deal of care in implementing the new algorithm in software if its desirable properties--strategic simplicity and non wastefulness--are to be realized. (Both of these features were lacking in the old SFUSD assignment system, the one to be replaced.)

Below are links to some of the key developments before last week.

Here is a post with a link to the video of Muriel Niederle presenting the new design that the Board ultimately voted to adopt: SF School Board Meeting, Feb 17: new choice system.
And here is a link to the slides she presented, giving a description (with examples) of the new choice algorithm: Assignment in the SFUSD, and discussions of the features that make it strategically simple, non wasteful, and flexible.

In March 2010 the San Francisco Board of Education unanimously approved the new system. In their March 2010 press release (now here), the SFUSD reported (emphasis added):

"The choice algorithm was designed with the help of a volunteer team of market design experts who have previously been involved in designing choice algorithms for school choice in Boston and New York City. Volunteers from four prominent universities contributed to the effort, including Clayton Featherstone and Muriel Niederle of Stanford University, Atila Abdulkadiroglu of Duke University, Parag Pathak of MIT, and Alvin Roth of Harvard.
“We are pleased that the district has decided to adopt a choice architecture that makes it safe for parents to concentrate their effort on determining which schools they prefer, with confidence that they won’t hurt their chances by listing their preferences truthfully,” said Niederle and Featherstone, the Stanford research team."

A very simple description of the basic assignment with transfer algorithm is given on page 370 of this paper (where it is called "top trading cycles"). Much of the SFUSD school Board debate has focused on what priorities to implement. What is described on page 370 is a simple version of the underlying choice engine into which the priorities go (with a somewhat different description than in Muriel's slides). The priorities can depend on any characteristics of the students (e.g. previous schools, or siblings, or home zip code) or of the school (e.g. neighborhood or historical student composition). But to keep the process strategically simple--to make it safe for families to rank schools according to their true preferences--the priority of a student at a school cannot depend on how that student ranked that school. (If you happen to be the programmer asked to implement this, drop me a line if you run into trouble:)

Some related developments can be followed on the blog of SF Board of Education member Rachel Norton, including this September 15 post on delay in the implementation of the middle school assignment plan: Recap: Assignment committee recommends delay

General background on the theory and practice of designing school choice algorithms can be found here, and my earlier posts on San Francisco schools are here.

Update, 9/30/10: many comments followed the link to this post at The SF K Files
Update 10/2/10: Rachel Norton, the SF Board member/blogger gets emails asking if the new system will be strategy proof, and she says it will be: Reader mail: questions on student assignment

Congestion in SF public school choice

One thing that computerized school choice is supposed to do is reduce congestion that sometimes stops school districts from matching students to schools in a timely way. San Francisco has a computerized system, but they are nevertheless running into congestion this year. SFGate has the story:
High anxiety as SF public school assignments run late, By Nanette Asimov

"A school district glitch has parents biting their nails in San Francisco this week.
Thousands of dollars are on the line for families that are prepared to lay out hefty deposits for private schools by this week’s deadlines — but hope they won’t have to if they can get into a public school of their choice.
The trouble is, the San Francisco Unified School District may not be able to tell them about their public school options, from elementary through high school, before private-school down payments are due Wednesday through Friday. The district missed its March 17 deadline for sending out school-assignment letters because of “unforeseen staffing emergencies,” said spokeswoman Gentle Blythe.
“We have people who haven’t slept in days” trying to make sure that 83,000 school options for 14,000 students are all correct, Blythe said, adding that she can’t reveal more about the problem because of employee confidentiality.
...
"The deadline for private high school deposits is Wednesday at noon for parents applying for financial aid and Friday at noon for those paying full price. Private elementary and middle schools have a Thursday deadline. And although most private schools coordinated their deposit due dates with the public school district this year, the district’s glitch has thrown the careful planning into disarray."
****************

Update: SF school-assignment letters to be mailed out Monday night  By Nanette Asimov Updated 4:19 pm, Monday, March 20, 2017

"The San Francisco district sends out public-school assignments by U.S. mail because “the letters provide the documentation families need to register at school sites and serves to further verify their address,” spokeswoman Gentle Blythe said.
However, parents facing an imminent private-school deadline who haven’t gotten a letter by Tuesday can email enrollinschool@sfusd.edu.
“We will do what we can to help you after March 21,” says a notice on the district’s website."

Gail Cornwall responds to the recent NY Times story on SF schools

Gail Cornwall, who follows San Francisco schools, replies to a recent article in the NY Times:

A cautionary tale about linking school choice and segregation

"Late last month, New York Times’ national education reporter Dana Goldstein wrote about public school choice and segregated schools in San Francisco. Headlined San Francisco Had an Ambitious Plan to Tackle School Segregation. It Made It Worse, the story hits several nails squarely on the head.

...

"But there are several important weaknesses in Goldstein’s article that could mislead parents, readers, and policymakers.

"The piece lays blame for segregation at the feet of San Francisco’s citywide public school choice system. It oversimplifies the views and priorities of lower-income non-white families. And, though Goldstein told me it wasn’t meant to, the article seems to endorse a controversial return to a restriction of choice in favor of a form of neighborhood attendance zones."

**********
Here's my earlier post on the NY Times article:

Tuesday, May 7, 2019

School choice in San Francisco--update in the NYT

I've blogged about other articles by Ms. Cornwall.

School choice in San Francisco--update in the NYT

Here's the NY Times story: San Francisco Had an Ambitious Plan to Tackle School Segregation. It Made It Worse.

“Our current system is broken,” said Stevon Cook, president of the district Board of Education, which, late last year, passed a resolution to overhaul the process. “We’ve inadvertently made the schools more segregated.”
...
"About a quarter of the city’s children are enrolled in private school, a higher percentage than in some other major cities, like New York, where it is around 20 percent. The lottery system is thought to be a major reason wealthy parents here opt out of public schools, further worsening segregation."
**********

The San Francisco Unified School District interacted with market designers some years ago, but ultimately turned down their (our) help and decided to deal with the existing problems in-house.  Here are some old blog posts...

Thursday, September 23, 2010

San Francisco school choice goes in-house

And

Thursday, June 2, 2011

Followup on school choice in San Francisco

School districts face a multitude of problems

My colleagues and I mostly interact with municipal school districts regarding questions of school choice. But of course school boards and districts have to deal with a whole multitude of problems, some of which sometimes impact their school choice decisions (either directly, or just because some problems are even more pressing).

In NY, Class Sizes Grew in City Despite ’07 Deal to Cut Them.
And here's a story, that in discussing opposition to Mayor Bloomberg's newly nominated schools Chancellor, also provides a good summary of the size of the changes he and Chancellor Joel Klein tackled: Frustrations With Mayor Are Backdrop to Nominee Uproar
     "Mr. Klein took office soon after the State Legislature handed the mayor control of the school system — no one yet knew how it would play out, and few would have imagined the scope of the overhaul that the mayor has since engineered. Mr. Bloomberg had yet to dissolve the 32 school districts that were a bedrock of the system and that, while chronically corrupt in some places, were also a crucial lifeline for parents with questions and concerns.
     "Sure, people who follow such things knew that the Board of Education, once a hotbed of political squabbling, would be replaced by a panel whose majority would be appointed by the mayor. But no one foresaw that if the mayor’s appointees disagreed with his policies they would be fired, as two were on the eve of a controversial panel vote in 2004, rendering the panel toothless. Now that it is clear that mayoral control really means mayoral control, the question of who would execute the mayor’s wishes has taken on more importance. "

Boston also has serious budget problems, which directly affect school choice: School officials in Hub urge closings
"The scenarios following Superintendent Carol R. Johnson’s recommendations last month to close several schools represent the stark realities confronting the school system as it prepares for its fourth consecutive year of budget cutting. The district is projecting a $63 million shortfall for the school year beginning September 2011. Last night, officials said the 2012-13 school year could be even worse, with an anticipated $91 million shortfall...
"The 40-minute financial presentation last night did little to quell the protests from the schools recommended for closure: the East Zone Learning Center, Emerson Elementary, Clap Elementary, Social Justice Academy, and the Engineering School in Hyde Park....
"In January, the school district is expected to raise once again the divisive issue of changing the way the district assigns students to schools after a similar attempt failed two years ago amid public uproar. The effort is intended to reduce busing costs by shrinking the geographic regions from which families can choose schools. If the district comes up with a palatable plan, any savings are not expected to be realized for at least two years."

and
More schools targeted to close in Hub
"The School Committee is slated to vote Dec. 15 on Johnson’s proposal. The committee needs to act quickly because next month parents start submitting their choices of where they want to send their children to school next fall.
...
"But some of those buildings might not sit empty for long. In a separate announcement late yesterday afternoon, Johnson and Mayor Thomas M. Menino said some buildings may be leased to charter schools, which are planning an aggressive expansion in the city."

In San Francisco, there are problems on the school district staff: San Francisco School Administrators Schemed to Take Money, Documents Say
"A group of San Francisco Unified School District administrators, including an associate superintendent, engaged in a long-running scheme to funnel district money into their personal bank accounts via nonprofit community organizations, according to internal documents. "
In short, market design takes place, when it does, amidst a noisy, difficult background, particularly in tough financial times.

School choice in San Francisco faces ongoing problems (and not just school choice)

The SF Chronicle has the story:

Why San Francisco needs a full-time school board
By Gail Cornwall, May 17, 2017  

"Ever wonder why the pace of change in public education falls somewhere between inching and crawling in arguably the most progressive, innovative city in the world? San Francisco Unified School District’s red tape and lack of resources are to blame, but there’s also a story of unpaid workers, organizational mutiny and missed opportunity.

Here’s an example: In 2009 the school board set out to redesign its method for assigning students to schools. Though the topic sounds dry, matching thousands of children to seats at more than a hundred programs — while taking into consideration parental preference, geography, diversity and more — involves the sexiest corner of economics: game theory.

Luckily, the board had the assistance of a group representing Harvard, Stanford, Duke and MIT. Nobel Prize winner Alvin Roth, Muriel Niederle, Clayton Featherstone and others proposed helping to create, monitor and adjust a cutting-edge algorithm for free. In March 2010, the board voted unanimously to take the offer.

But that September, district staff sent Roth’s team an email amounting to “Thank you, goodbye.” District officials had decided to instead “develop software to implement the new design on their own,” Roth reported.

Today, the state of the district’s homegrown assignment algorithm, known to parents as “the lottery,” is described by board member Mark Sanchez as “broken” and “untenable,” and by board member Rachel Norton as “probably the biggest policy issue that our community engages with us on.”

Neil Dorosin directs the Institute for Innovation in Public School Choice, a nonprofit Roth and his team formed. When I recently asked Dorosin what kind of personnel would be needed to create an effective school assignment algorithm, he said, “Either a mathematician or an economist who knows about algorithms, and … a software engineer who could operationalize it. I would be stunned if they have that.”

Those who shared these concerns back in 2010 called on district staff to explain themselves. Despite making a pledge to the board to disclose the algorithm developed in-house, by March 2012 staff still hadn’t issued “a complete enough description to [know] … if they in fact implemented the plan … the board adopted,” said Roth.

Lack of compliance with board directives sounds crazy, but Sanchez, who served on the board from 2001 to 2009 and won re-election in November, said it happens all the time. “There are so many examples,” he said.

How could that be possible? Because board members each receive “about $6K a year — and everyone has a full-time job doing something else — they’re just too busy to check in and cajole, Sanchez said. The only thing the board really can do, he said, is fire the superintendent when “a lot of that piles up.”

That’s why Sanchez and San Francisco Supervisor Jane Kim have discussed putting forward a ballot measure to increase school board member compensation. Sanchez said it would give board members “the average beginning pay for a teacher in the Bay Area ... probably ending up at around $45,000” (drawn from the city’s budget, rather than the school district’s). Following the model adopted by Los Angeles in 2006, the full amount would be available only to those who forsake other employment, he said.

Meanwhile, parents fret over the lottery. The long, complex application process — where paperwork is submitted in person in January and decisions are issued in March, then again in May, through three more rounds of supplication and the first two weeks of school — fails low-income families who lack the time or bureaucratic savvy to effectively engage. Those who do manage to navigate the process, one emotional parent told the school board committee Monday night, often find the experience “time-consuming, frustrating and stressful.” Raman Khanna, a member of the Ulloa Elementary School PTA, referred to another outcome: professional “flight.” Because of the lottery, he said, “a lot of the colleagues that I talk to … leave the city or they go to private school.”

Roth’s and Dorosin’s organization has worked with cities across the country to use data and technology to improve school assignment. Dorosin said the nonprofit’s modest fees are often covered by outside grants and other funding. This March they invited SFUSD board members and district officials to reach out again.

The response? The board committee announced Monday it would “not be taking action,” and district staff proposed two timelines for reform: one would give the board two years to articulate a new direction for the assignment system; the other, labeled “if policy development moves quickly,” would still give them a full year to do so and then another 18 months for district staff to implement it. Tommy Williams, a parent who works for the U.S. Army Corps of Engineers, said: “The fact that we’re talking about the 202[0] or 2021 school year is very frustrating.”

Board members could start meeting with national experts with just days’ notice to hash out a broad-strokes plan, but that won’t happen until the fall. “It’s clearly an urgent issue,” Sanchez said, “but it’s one of many, many things that we have to deal with. ... We had to hire a superintendent, and now we’re involved in negotiations for the contract so we wanted to focus on that.”

Rionda Batiste, co-chair of the district’s African American Parent Advisory Council, won many approving head nods at Monday’s meeting when she said: “I don’t understand why this is something that cannot happen simultaneously.”

“If we really wanted to speed things along,” Sanchez told me, “we’d have more meetings. Intuitively, we all know [paying board members] would make things move faster.”


Nationally, school board compensation is all over the board. Connecticut pays nothing, while Florida’s lowest paying county, as of 2014 reportedly offered $24,290 a year. According to the National School Boards Association, approximately 75 percent of small-district school board members serve as volunteers while around 40 percent of large-district ones receive a “modest salary.”

This divide makes sense, because there’s much more work to be done in a district with 50,000 students than one with only a few hundred. When being an effective board member requires a full-time commitment from someone who must already work full-time elsewhere, “it’s a structural problem,” Sanchez said.

A second one resurfaced at Monday’s meeting. Orla O’Keefe, the district’s chief of policy and operations, told the board: “We need a larger number of staff with the technical skills and knowledge needed to complete assignment runs,” including to “[e]xplore leveraging district ... online registration functionality with a potential online application pilot.” In other words, while the board takes its time deciding what major changes to make, district staff propose once again building in-house. Meanwhile, a Nobel Prize-winning economist — and the tools his team has honed — wait in the wings.

Maybe instituting board salaries can buy our elected representatives the time they need to pursue public-private partnerships that bring expertise and manpower to the task of matching students with schools. Hopefully, this time it will be with the support of district staff, such as newly anointed Superintendent Vincent Matthews who, calling the meeting “democracy in action,” said Monday he’s “looking forward to moving this forward.”

Until then, Sanchez said, “It’s in a holding pattern.”

School choice in San Francisco, Chicago, and Cambridge MA

Changes in school assignment procedures are being discussed in San Francisco, Chicago, and even Cambridge, MA.

In San Francisco, the redesign is well under way, and the school board has a link to the redesign process: What Have We Done So Far.

An impresssive parent-organized blog outlines the San Francisco Student Assignment system, and the current process intended to reform it, initiated by the San Francisco Unified School District. One of the early posts assembles some of the relevant materials, including the SFUSD page about the Student Assignment Redesign (including some history and legal context), and the
SFUSD technical description of the current Student Assignment Process (which can now be found here).

Chicago has begun to rethink its school choice system; see New Proposed Admission Policy Information


In Cambridge, Parag Pathak proposed that a strategy-proof mechanism replace the old Boston-style mechanism (no longer used in Boston); see School Assigning Process Criticized--MIT professor presents a possible solution to the problem

New school choice system in San Francisco

Board Approves New Student Assignment System for San Francisco Schools (now here)

Most of the last minute discussion was about what priorities different kinds of students will have at different kinds of schools. That is something that is likely to be adjusted from year to year. But the nice thing is that the underlying choice architecture will make it safe for parents to state their true preferences however the priorities are adjusted.

From the press release: "The choice algorithm was designed with the help of a volunteer team of market design experts who have previously been involved in designing choice algorithms for school choice in Boston and New York City. Volunteers from four prominent universities contributed to the effort, including Clayton Featherstone and Muriel Niederle of Stanford University, Atila Abdulkadiroglu of Duke University, Parag Pathak of MIT, and Alvin Roth of Harvard.
“We are pleased that the district has decided to adopt a choice architecture that makes it safe for parents to concentrate their effort on determining which schools they prefer, with confidence that they won’t hurt their chances by listing their preferences truthfully,” said Niederle and Featherstone, the Stanford research team."

Here are Rachel Norton's comments (she's a school board member with a blog), and here's the story from the SF Chronicle. Here are some of my recent posts on school choice; many of the recent ones tell the SF story as it unfolded.

Now, on to implementation.

Interview concerning San Francisco school choice

The San Francisco Briefing Room carries an audio interview by Stan Goldberg with Attila Abdulkadiroglu and me (about 30 minutes): Assignment System at Risk.
Here's his blurb: "With the deadline for submitting applications for school assignment in the San Francisco Unified School District rapidly approaching the school district has advised its independent advisors who were scheduled to program the assignment system for free that their services were not needed. Does this action imperil the implementation of the system on time? Has school district transparency moved back to the dark ages? Here’s the story from the design team’s perspective."


(Here are my previous posts on San Francisco school choice.)

San Francisco may re-emphasize neighborhood schools in their school choice plan

From the SF Chronicle: Plan's goal: Get S.F. families into neighborhood school

"Unlike families who live in the suburbs, San Francisco residents don't automatically get assigned to the school near their homes.

"The system is built on decades of desegregation efforts and the idea of equal access to all schools.

"It requires families to submit a list of schools that they want their children to attend. If a school has enough spots for the families who want in, there are no issues. But if a school has fewer seats than families who listed it, a complicated tie-breaker system kicks in.

"Siblings of students get the first available seats. Then, families living in census tracts where students post the lowest test scores - which the district calls CTIP, for Census Tract Integration Preference - get second priority. Those in the school's attendance area are third, followed by everyone else.

"Schools like Clarendon, with high test scores, low student poverty and experienced teachers, fill up with siblings and CTIP families, leaving few or no seats for students who live in the neighborhood.

"Norton and Fewer want to flip things so attendance area comes before CTIP, giving higher priority to families who live nearby than to those living in presumably disadvantaged neighborhoods.

"The CTIP tie-breaker, introduced three years ago, was supposed to help diversify schools without specifically using race.

"It didn't work.

"All the board members appreciate diversity and want to eliminate racial isolation in our schools," Fewer said. "We just don't know if the CTIP preference is doing this. It's time to revisit it."


"Data from the past three years show 28 schools - a quarter of all campuses - are still racially isolated, meaning that 60 percent of enrollment is a single ethnic group.

"Instead of creating a big melting pot in schools, a CTIP address has become a golden ticket for families who wanted to attend the city's most popular schools, Norton said. It has also created demand for housing in CTIP areas, with real estate agents promoting those neighborhoods and people lying about their address to get an advantage.

"And even if a family living in a CTIP area were wealthy, it would not matter - they would still get the same high-priority status as someone living in poverty.

"There was no means test, said McCarthy, whose children were assigned to Sanchez Elementary, more than 2 miles from home.

"Someone that makes half a million a year that just bought a $2 million home in the Mission" has a better chance at Clarendon, she said. "That's unfair that they can kind of trump us."

"The CTIP wasn't intended to give an unfair advantage or attract people to buy or rent in certain neighborhoods, Norton said.

"People are making really big life decisions so they can be in a CTIP zone," she added. "That makes me very uneasy."

"In addition, African American and Latino families are less likely to participate in the first round of the school assignment lottery than white or Asian families, which again raises the question of whom the CTIP preference is serving."
*********

Background material: from SFUSD (San Francisco Public Schools).  Student Assignment and Enrollment Reports

School choice in San Francisco, reports on first year

I've written before about school choice in San Francisco, and about how Muriel Niederle and Clayton Featherstone led the effort by a group of our colleagues to design a strategy-proof choice algorithm (explained here at a Board meeting in 2010), based on transfer cycles ("top trading cycles" to game theorists...). The school board adopted the plan, but then the staff of the school district decided to implement it themselves, without making the details public. Fast forward to 2012, when the first children have been assigned by the new plan.

Rachel Norton's blog has a post about it here: They're out! School Assignment Letter 2012, and an earlier one here, with a link to a March 5, 2012 SFUSD report on Student Assignment. As in the previous SFUSD reports, this does not describe the choice algorithm, it only describes the "tie breakers" that are used whenever the algorithm would otherwise try to assign more students to a school than it has room for.

This outraged Stan Goldberg (who reports about SF schools as "Senior Dad"), and he posted a video about the lack of transparency called Assignment System Fraud?

He must be an influential guy, because this prompted SFUSD to post some new information, including this "fact sheet" dated March 23, called How does the student assignment computer program work?  It still doesn't come close to explaining the actual algorithm they use, but it does include a diagram of "transfer cycles."

Which raises a question. If they in fact implemented the plan we proposed and the Board adopted, you would think they would want to make this clear. The benefits of a strategy-proof assignment procedure can only be realized if parents know that they can safely list their true preferences.

On the other hand, if the algorithm isn't correctly implemented, or if some other assignment algorithm is implemented (whether or not it includes some use of transfer cycles) then it would most likely not be strategy-proof, that is, it might not be safe for parents to reveal their true preferences, and it might be in the interest of some to "game the system" in some way. That might account for a desire to keep the algorithm secret. (So might a desire to avoid revealing any inadvertent mistakes in implementation...)

I should say that SFUSD's brief description of their algorithm doesn't look to me like it describes one that is strategy-proof...:(On the contrary, it looks like it might be patched together from something like Boston's old immediate acceptance algorithm followed by some trading...but then again, it isn't a complete enough description to make me confident that it is a description of whatever they are in fact doing...)

Anyway, one point of this post is to say that, unlike the case of the systems in New York and Boston and the work that IIPSC is doing around the country, my colleagues and I don't know what algorithm SFUSD is using, even though we know what we proposed and the Board adopted. So...this post is a bit like the ads that sometimes appeared in the financial sections of newspapers when I was young, which, following a divorce, would announce that Mr John Doe was henceforth no longer responsible for any debts incurred by the former Mrs John Doe...

Further followup on school choice in San Francisco

My Thursday post, Followup on school choice in San Francisco, has generated some followup on its own, in the form of an audio interview yesterday of School Board member Rachel Norton by Stan Goldberg who follows the SF school system under the name Senior Dad.  He summarizes the interview as "Straight answers from Commissioner Norton “because people have a right to know”."

The issue is whether the algorithm adopted by the board last year was in fact implemented correctly by the district staff. It's an important question because the correctly implemented algorithm would be strategy proof, and if parents had confidence in this it would vastly simplify the school choice system from parents' point of view.

Here is my very incomplete and possibly imperfect transcript to give the flavor of the last 5 minutes of the interview (starting just after minute 39) in which Stan Goldberg (SG) raises this issue, and Rachel Norton (RN) replies. It's worth listening to.

SG “The school district was supposed to release the algorithm they were assigning students on, and so far they have not released that algorithm.”

RN “you’ve been reading Al Roth’s blog” ...“I’ve advocated for that, and will continue to advocate for that. I don’t think the staff right now wants to do that. [laughter] But short of 5 votes, 4 votes, they don’t have to.

…

SG ‘why should the public trust the school district?  “I’ve had the deputy Superintendent say ‘you guys shouldn’t trust us, we haven’t been reliable’. He said that; I believed him.”

RN “I don’t know what to tell you Sam, I think we should release the algorithm, and I’ve said that to staff, I’ve said that to the Superintendent”…short of 3 other board members joining with me and demanding that it be released the superintendent can do what he thinks is best,  unless he’s ordered by the board to do something else…”

SG “not releasing the algorithm makes everybody think something funny is going on…”

RN” well, not everyone…”

SF School Board Meeting, Feb 17: new choice system

At the latest public meeting of the San Francisco Board of Education (last night, Wednesday, Feb. 17), the commissioners and the public were engaged in a detailed discussion of the algorithms and priorities being proposed for the new school choice system.

Muriel Niederle explains and answers questions about the new Assignment with Transfers school choice plans being proposed (with variations for elementary school, middle school, and high schools). She comes on just after minute 36 of this video of the 3 hour meeting, and her presentation, interspersed with questions and answers, continues for a little over an hour (to minute 1:39), although she's back answering questions at the end again. Also presenting the general plan and answering questions is Orla O'Keefe, the SFUSD official leading the effort to design the new school choice system.

There's something very encouraging about seeing the public policy discussion focusing on choice systems that are non-wasteful (Pareto efficient, for you economists), strategically simple for parents (so that truthful preference revelation is a dominant strategy), and flexible (so that the school board can tweak the system in years to come without harming the first two properties). The 'political' issues are the priorities that different children have at different schools.

Another attractive aspect of the proposal (discussed by Ms. O'Keefe following Muriel's presentation) is that data would be collected each year for continual monitoring of how the choice and assignment system is working.

The discussion touches on a number of interesting questions. (Even if the algorithm makes truthful preference revelation the best strategy, there are still issues of checking e.g. addresses in any system in which priorities at schools depend on home address...). But it looks like SF is well launched on adopting a sensible, workable, well thought out and flexible framework.

School choice in SF moves forward

Yesterday Muriel Niederle and Clayton Featherstone were among the presenters to the San Francisco Board of Education, speaking about possible designs for a new school choice system there. It seems that they are well on the way to a good outcome.

One of the Board members, Rachel Norton, has a blog on which she posted before and after accounts of the meeting:
Tonight’s student assignment meeting should be interesting!
Recap: Closing in on a student assignment policy

Here is a video of the whole meeting (but you can navigate a bit so you don't have to watch the full 3 hours: Muriel's testimony, from her slide presentation through answering of questions from the board is from 1:09 to 2:09 on the video).

For the technically inclined, papers about our prior work on school choice systems in NYC and Boston are here.

It has been mentioned in the SF discussions that our team of market designers has worked on a number of problems aside from school choice, so here are background links on some of them for SF readers who are interested:
National Resident Matching Program and related medical labor markets
Gastroenterologists, Orthopaedic surgeons
Kidney Exchange
AEA market for new economists

School choice in San Francisco: a promise of transparency

San Francisco school board member Rachel Norton blogs about the most recent school board meeting this week, concerning the new SF school choice algorithm, which they are now implementing in house (see this earlier post, and a followup interview).

The latest news sounds good regarding plans for transparency. Norton writes:

"Staff did pledge to make the documentation of the algorithm requirements and process flows public by February; I will continue to push to make the assignment algorithm itself open source."

Joel Klein on school choice

Joel Klein, who was Chancellor of NYC schools when school choice was introduced in New York City high schools, writes in the Daily News: Harness the power of school choice: Competition works in education, too

Of course, details matter: see yesterday's post on an effort that didn't quite work out as planned, in San Francisco. Yesterday's post also has links to some school choice efforts that seem quite promising, however.

ASSA meetings in San Diego--Market design on Friday

The ASSA meetings are a cornucopia.  Here are some sessions related to market design that caught my eye in the preliminary program for the first day of conferencing, Friday January 3. No one can go to all of them, aside from interviewing junior market candidates, some of these sessions conflict with each other...:-(

Frontiers in Market Design
Paper Session
 Friday, Jan. 3, 2020   8:00 AM - 10:00 AM
 Marriott Marquis San Diego, Catalina
Hosted By: ECONOMETRIC SOCIETY
Chair: Eric Budish, University of Chicago
Targeting In-Kind Transfers through Market Design: A Revealed Preference Analysis of Public Housing Allocation
Daniel Waldinger, New York University

Approximating the Equilibrium Effects of Informed School Choice
Claudia Allende, Columbia University and Princeton University
Francisco Gallego, Pontifical Catholic University of Chile
Christopher Neilson, Princeton University

The Efficiency of A Dynamic Decentralized Two-Sided Matching Market
Tracy Liu, Tsinghua University
Zhixi Wan, Didi Chuxing
Chenyu Yang, University of Rochester

Will the Market Fix the Market? A Theory of Stock Exchange Competition and Innovation
Eric Budish, University of Chicago
Robin Lee, Harvard University
John Shim, University of Chicago

When Do Cardinal Mechanisms Outperform Ordinal Mechanisms?: Operationalizing Pseudomarkets
Hulya Eraslan, Rice University
Jeremy Fox, Rice University
Yinghua He, Rice University
Yakym Pirozhenko, Rice University
*********
Search and Matching in Education Markets
Paper Session
 Friday, Jan. 3, 2020   10:15 AM - 12:15 PM (PST)
 Marriott Marquis San Diego, Rancho Santa Fe 2
Hosted By: AMERICAN ECONOMIC ASSOCIATION
Chair: Eric Budish, University of Chicago

Simultaneous Search: Beyond Independent Successes
Ran Shorrer, Pennsylvania State University

Search Costs, Biased Beliefs and School Choice under Endogenous Consideration Sets
Christopher Neilson, Princeton University
Claudia Allende, Columbia University
Patrick Agte, Princeton University
Adam Kapor, Princeton University

Facilitating Student Information Acquisition in Matching Markets
Nicole Immorlica, Microsoft Research
Jacob Leshno, University of Chicago
Irene Lo, Stanford University
Brendan Lucier, Microsoft Research

Why Are Schools Segregated? Evidence from the Secondary-School Match in Amsterdam
Hessel Oosterbeek, University of Amsterdam
Sandor Sovago, University of Groningen
Bas van der Klaauw, VU University Amsterdam

***********
Market Design
Paper Session
 Friday, Jan. 3, 2020   10:15 AM - 12:15 PM
 Marriott Marquis San Diego, Del Mar
Hosted By: ECONOMETRIC SOCIETY
Chair: Sergei Severinov, University of British Columbia

Market Design and Walrasian Equilibrium
Faruk Gul, Princeton University
Wolfgang Pesendorfer, Princeton University
Mu Zhang, Princeton University

Repeat Applications in College Admissions
Yeon-Koo Che, Columbia University
Jinwoo Kim, Seoul National University
Youngwoo Koh, Hanyang University

Entry-Proofness and Market Breakdown under Adverse Selection
Thomas Mariotti, Toulouse School of Economics

Who Wants to Be an Auctioneer?
Sergei Severinov, University of British Columbia
Gabor Virag, University of Toronto
**********
Transportation Economics
Paper Session
 Friday, Jan. 3, 2020   10:15 AM - 12:15 PM (PST)
 Marriott Marquis San Diego, La Costa
Hosted By: ECONOMETRIC SOCIETY
Chair: Tobias Salz, Massachusetts Institute of Technology

The Selection of Prices and Commissions in a Spatial Model of Ride-Hailing
Cemil Selcuk, Cardiff University

The Welfare Effect of Road Congestion Pricing: Experimental Evidence and Equilibrium Implications
Gabriel Kreindler, University of Chicago

Customer Preference and Station Network in the London Bike Share System
Elena Belavina, Cornell University
Karan Girotra, Cornell University
Pu He, Columbia University
Fanyin Zheng, Columbia University

Platform Design in Ride Hail: An Empirical Investigation
Nicholas Buchholz, Princeton University
Laura Doval, California Institute of Technology
Jakub Kastl, Princeton University
Filip Matejka, Charles University and Academy of Science
Tobias Salz, Massachusetts Institute of Technology
**********

Information (Design), Black Markets, and Congestion
Paper Session
 Friday, Jan. 3, 2020   2:30 PM - 4:30 PM
 Manchester Grand Hyatt San Diego, Torrey Hills AB
Hosted By: ECONOMIC SCIENCE ASSOCIATION
Chair: Dorothea Kuebler, WZB Berlin Social Science Center
An Experimental Study of Matching Markets with Incomplete Information
Marina Agranov, California Institute of Technology
Ahrash Dianat, University of Essex
Larry Samuelson, Yale University
Leeat Yariv, Princeton University

Information Design in Dynamic Contests: An Experimental Study
Yan Chen, University of Michigan
Mohamed Mostagir, University of Michigan
Iman Yeckehzaare, University of Michigan

How to Avoid Black Markets for Appointments with Online Booking Systems
Rustamdjan Hakimov, University of Lausanne
C.-Philipp Heller, NERA Economic Consulting
Dorothea Kuebler, WZB Berlin Social Science Center
Morimitsu Kurino, Keio University

Application Costs and Congestion in Matching Markets
Yinghua He, Rice University
Thierry Magnac, Toulouse School of Economics

Discussant(s)
Christian Basteck, ECARES Brussels
Lionel Page, University of Technology Sydney
Robert Hammond, University of Alabama
Ahrash Dianat, University of Essex
*******

Tech Economics
Paper Session
 Friday, Jan. 3, 2020   2:30 PM - 4:30 PM
 Marriott Marquis San Diego, San Diego Ballroom A
Hosted By: NATIONAL ASSOCIATION FOR BUSINESS ECONOMICS
Chair: Michael Luca, Harvard Business School

GDPR and the Home Bias of Venture Investment
Jian Jia, Illinois Institute of Technology
Ginger Jin, University of Maryland
Liad Wagman, Illinois Institute of Technology

New Goods, Productivity and the Measurement of Inflation: Using Machine Learning to Improve Quality Adjustments
Victor Chernozhukov, Massachusetts Institute of Technology
Patrick Bajari, Amazon

Double Randomized Online Experiments
Guido Imbens, Stanford University
Patrick Bajari, Amazon

School choice in Darebin City, Victoria, Australia

It sounds like at least one school district in Australia has less bureaucracy than my colleagues and I have encountered in helping American cities reorganize their school assignment systems. Kwanghui Lim at CoRE Economics reports: Game Theory in Action: Sven Feldmann on Kindergarten Matching.

More on school choice here, and here. (And here is a video of Muriel Niederle presenting a new school choice algorithm to the San Francisco school board meeting that gave the go ahead for a redesign there.)

Matching and Market Design at INFORMS in San Francisco, Sunday November 9

There's a lot of market design at the INFORMS annual meeting, Nov 9-12.

On Sunday I'll start the day off with a talk from  10-10:50 called
"Market Design and the Economist as Engineer."

That will be followed by a cluster of talks organized by Itai Ashlagi called (embarrassingly)
Matching and Market Design (in honor of Al Roth), consisting of the following sessions

Cluster :	Matching and Market Design (in honor of Al Roth)
Session Information		: Sunday Nov 09, 11:00 - 12:30
Title:	Empirical Market Design
Chair:	Ramesh Johari,Stanford University,
Abstract Details
Title:	Quality Externalities and the Limits of Reputation in Two-Sided Markets
	Presenting Author:	Steve Tadelis,Professor, UC Berkeley, Haas School of Business, 2220 Piedmont Ave, Berkeley, United States of America, stadelis@haas.berkeley.edu
	Co-Author:	Chris Nosko,Booth School of Business, University Of Chicago, Chicago, United States of America, cnosko@chicagobooth.edu
Abstract:		Using data from eBay, we argue that platforms can mitigate externalities by actively screening sellers and promoting the prominence of better quality sellers. Exploiting the bias in feedback, we create a measure of seller quality and demonstrate the benefits of our approach through a controlled experiment that prioritizes better quality sellers to a random subset of buyers. .
Title:	On the Near Impossibility of Measuring the Returns to Advertising
	Presenting Author:	Randall Lewis,Economic Research Scientist, Google Inc., 1600 Amphitheatre Parkway, Mountain View Ca 94043, United States of America, randall@econinformatics.com
	Co-Author:	Justin Rao,Economic Research Scientist, Microsoft Research, New York City NY, United States of America, Justin.Rao@microsoft.com
Abstract:		Firms have a hard time measuring the causal impact of advertising expenditures on profit. In twenty-five online field experiments, individual-level sales are volatile relative to the per capita cost of a campaign--a small impact on a noisy dependent variable can generate positive returns. Experiments can need more than ten million person-weeks. Further, small selection biases can severely bias observational estimates. Weak informational feedback and technological advances shape ad marketplaces.
Title:	Corporate Prediction Markets: Evidence from Google, Ford, and Firm X
	Presenting Author:	Bo Cowgill,UC Berkeley, 1931 Diamond St Apt 3, SAN FRANCISCO Ca 94131, United States of America, bo.cowgill@gmail.com
	Co-Author:	Eric Zitezwitz,Dartmouth College, 6106 Rockefeller Hall, Hanover NH, United States of America, zitzewitz@dartmouth.edu
Abstract:		We examine data from prediction markets run by Google, Ford and Firm X (a large private materials company). Despite theoretically adverse conditions, we find these markets are relatively efficient, and improve upon the forecasts of experts at all three firms by as much as a 25% reduction in MSE. The most notable inefficiency is an optimism bias in the markets at Google and Ford. The inefficiencies that do exist become smaller over time for reasons we document.
Title:	At What Quality and What Price? Inducing Separating Equilibria as a Market Design Problem
	Presenting Author:	John Horton,Professor, NYU Stern School of Business, Kaufman Management Center, 44 West Fourth St, 8-81, New York NY 10012, United States of America, john.joseph.horton@gmail.com
	Co-Author:	Ramesh Johari,Stanford University, Huang 311, Stanford, United States of America, ramesh.johari@stanford.edu
Abstract:		A tool to promote revelation of buyers' price/quality preferences was experimentally introduced into an online labor market. In the treatment cells of the experiment, upon posting a job, buyers chose what price/quality level they were seeking from sellers. We find that buyers readily reveal their preferences and that this revelation---which itself was experimentally manipulated---strongly induced seller-side sorting.

Title:	******** Matching and Market Design
Chair:	Jacob Leshno,Columbia University,
Abstract Details
Title:	Matching in Networks
	Presenting Author:	Michael Ostrovsky,Associate Professor of Economics, Stanford Graduate School of Business, 655 Knight Way, Stanford CA 94305, United States of America, ostrovsky@stanford.edu
Abstract:		In this talk, I will present results on the existence and properties of stable outcomes in trading networks.
Title:	Matching with Peers in School Choice
	Presenting Author:	Atila Abdulkadiroglu,Professor, Duke University, 213 Social Sciences Building, Durham NC 27708, United States of America, atila.abdulkadiroglu@duke.edu
Abstract:		We develop a theory for matching of students to schools with peers and study various matching mechanisms with field data.
Title:	Endogenous preferences and the role of the mechanism in school choice
	Presenting Author:	Estelle Cantillon,Senior Research Fellow, Université Libre de Bruxelles (ECARES), 50, av FD Roosevelt, CP 114, Brussels 1050, Belgium, Estelle.Cantillon@ulb.ac.be
Abstract:		We consider a school choice model where preferences over schools are endogenous because students care about the quality of their peers. In such a setting, the mechanism affects the degree of preference polarization. We show how mechanisms can be designed to reduce polarization and improve the distribution of ranks of assigned schools in students’ preferences. A policy change in the city of Ghent (Belgium) provides a test for the predictions of the theory.
Title:	Evidence of Strategic Behavior in Hospital Claims Reporting
	Presenting Author:	Hamsa Bastani,Stanford University, Stanford, Stanford, United States of America, hsridhar@stanford.edu
	Co-Author:	Mohsen Bayati,Stanford Graduate School of Business, Stanford CA 94305, United States of America, bayati@gsb.stanford.edu
		Joel Goh,joelgoh@stanford.edu
		Stefanos Zenios,Charles A. Holloway Professor of Operations, Information, and Technology and Professor of Health Care Management, Stanford Graduate School of Business, 655 Knight Way, Stanford CA 94305, United States of America, stefzen@GSB.Stanford.Edu
Abstract:		We provide evidence from Medicare claims data that hospitals engage in upcoding behavior when reporting hospital-acquired infections that are no longer reimbursed by Medicare. In particular, we show that hospitals sometimes mark a hospital-acquired infection as present-on-admission, presumably in order to collect greater reimbursement.

*******

Title:	Matching Markets
Chair:	Yash Kanoria,Columbia Business School,
Abstract Details
Title:	Stable Matching in Large Economies
	Presenting Author:	Fuhito Kojima,Stanford University, 579 Serra Mall, Stanford CA 943055007, United States of America, fkojima@stanford.edu
Abstract:		Complementarities of preferences have been known to jeopardize stability of two-sided matching markets, yet they are a pervasive feature in many matching markets. In large markets, we demonstrate that if each firm's choice changes continuously as the set of available workers changes, then there exists a stable matching even if firm preferences exhibit complementarity. Building on this result, we show that there exists an approximately stable matching in any large finite economy.
Title:	The Prior-Independence Approach
	Presenting Author:	Inbal Talgam-Cohen,PhD Candidate, Stanford University, 86 Hulme Ct, Apt 108, Stanford CA 94305, United States of America, italgam@stanford.edu
	Co-Author:	Tim Roughgarden,Stanford, 353 Serra Street, Stanford, United States of America, tim@cs.stanford.edu
Abstract:		The matching literature has recently begun to consider priors on agents’ utilities. One of the barriers to adopting this potentially very fruitful approach is that priors add significant informational assumptions to the model. We survey a successful alternative approach from mechanism design called prior independence, which alleviates such assumptions while still reaping most benefits. We discuss both sampling-based methods and methods based on ensuring sufficient competition in the market.
Title:	The structure of the core in assignment markets
	Presenting Author:	Yash Kanoria,Columbia Business School, 404 Uris Hall, New York NY 10027, United States of America, ykanoria@columbia.edu
	Co-Author:	Daniela Saban,Columbia University, Uris Hall, 4I, New York, United States of America, dhs2131@columbia.edu
		Jay Sethuraman,Columbia University, IEOR Department, New York, United States of America, jay@ieor.columbia.edu
Abstract:		Assignment markets (Shapley & Shubik 1971) involve matching with transfers, as in labor markets and housing markets. We consider a two-sided assignment market with agent types and stochastic structure similar to models used in empirical studies. Each agent has a randomly drawn "productivity" associated with each type on the other side. We characterize how the structure of the core, i.e., the set of stable outcomes, is determined by market characteristics.