Wednesday, November 20, 2024

School choice, school quality, and student performance

 School choice, which allows children to move away from neighborhood schools, doesn't by itself improve school quality, although it may allow under-performing schools to become smaller, which may make them easier to fix, or to close.

Here are two  recent assessments of two different transportation options available to Boston public school students. Both conclude that school quality matters.

The first is an op-ed in the Boston Globe saying that school choice among Boston public schools has led to too much transportation and not enough innovation. (One of the authors, Parag Pathak, played a critical role in designing  Boston's current school choice system.)

Boston needs to reexamine school assignment system
Rather than investing in high-cost travel to send students to schools across the city, Boston should consider redirecting those funds toward improving schools close to home.
By Joshua Angrist, Parag Pathak and Amanda Schmidt 

"Boston’s school assignment system has changed considerably since the 1970s. Busing today is voluntary: Students can choose to attend schools far from where they live as well as a range of neighborhood schools. This choice allows historically disadvantaged students to attend schools with more peers of different backgrounds, an option that many choose. Roughly three-quarters of students opted to enroll in non-neighborhood schools in the 2000s and 2010s. A recent study by our organization, MIT Blueprint Labs, shows that today’s assignment system works in the sense of facilitating integration.

However, the costs of the current system are high. Among the 100 US school districts with the highest enrollment, Boston maintains the greatest per-student transportation costs in the country. As of 2021, the city spent over $2,000 per student on travel, equivalent to 8 percent of per-pupil school spending.

Furthermore, the educational gains afforded by district-wide choice are less clear than the integration gains. Our research, which uses credible, randomized methods designed by Blueprint Labs to gauge the causal effect of enrollment at different types of schools, paints a nuanced picture of the benefits of travel to non-neighborhood schools. Black and Hispanic students who travel to a non-neighborhood school have more white and Asian peers than they otherwise would. But travel does not impact learning as measured by MCAS scores, high school graduation rates, or college enrollment. We argue that this is because in the current BPS choice system — unlike the separate and unequal system of 1974 — the schools students travel to are no better than those nearby.

...

"The vast sums that now go to cross-neighborhood transportation might be better spent. The city might instead invest in programs with proven educational benefits. Saga Education’s effective high-dosage tutoring program, for example, cost just $1,800 per student in 2023. This spending may do more to close racial achievement gaps than non-neighborhood assignment.

"Some might counter that choice is intrinsically valuable and that neighborhood schools are likely to be more segregated than the schools that many historically disadvantaged families choose today. These undeniable benefits must be weighed, however, against alternative uses of the money that flows to busing. Boston schools have improved greatly since 1974: Dropout rates for all students have declined, and gaps by race, while still present, have narrowed. School assignment plans originating in 1974 may therefore be less useful today. It’s time to consider changing transportation policy in light of these changes in the city’s education landscape."

####### 

Here's a paper, about a different transportation program available to some* Boston students, which takes them out of Boston to schools in neighboring towns and cities in the metropolitan area.  Moving to those suburban schools apparently improves student performance more than moving from one Boston school to another.

 Busing to Opportunity? The Impacts of the METCO Voluntary School Desegregation Program on Urban Students of Color  by Elizabeth Setren, NBER Working Paper 32864, DOI 10.3386/w32864, August 2024

Abstract: School assignment policies are a key lever to increase access to high performing schools and to promote racial and socioeconomic integration. For over 50 years, the Metropolitan Council for Educational Opportunity (METCO) has bussed students of color from Boston, Massachusetts to relatively wealthier and predominantly White suburbs. Using a combination of digitized historical records and administrative data, I analyze the short and long run effects of attending a high-performing suburban school for applicants to the METCO program. I compare those with and without offers to enroll in suburban schools. I use a two-stage least squares approach that utilizes the waitlist assignment priorities and controls for a rich set of characteristics from birth records and application data. Attending a suburban school boosts 10th grade Math and English test scores by 0.13 and 0.21 standard deviations respectively. The program reduces dropout rates by 75 percent and increases on-time high school graduation by 13 percentage points. The suburban schools increase four-year college aspirations by 17 percentage points and enrollment by 21 percentage points. Participation results in a 12 percentage point increase in four-year college graduation rates. Enrollment increases average earnings at age 35 by $16,250. Evidence of tracking to lower performing classes in the suburban schools suggests these effects could be larger with access to more advanced coursework. Effects are strongest for students whose parents did not graduate college."

*"The program is very popular: 50 percent of Black youth in Boston applied and 20 percent of Latinx youth in the past 20 years"

...

"After demonstrating the comparability of students with and without offers, I estimate the impact of receiving an offer to the program and the impact of participating in the program. Offers to enroll in suburban districts serve as instrumental variables and all models control for approximate waitlist position using age at the time of application, gender, and race controls. Therefore the estimates compare the outcomes of those who enroll in METCO to applicants with similar demographics, who applied at similar times, but did not enroll because they were not selected from the waitlist."

Tuesday, November 19, 2024

AI assisted job applications (and application screening)

 NBC news has the story:

AI is supposed to make applying to jobs easier — but it might be creating another problem
Companies have touted new AI technology that allows users to apply to thousands of jobs per day, flooding job openings with resumes.
  By Sophia Pargas

"With the help of new AI software, applicants can now revise their resumes and cover letters, receive live interview scripts and launch chatbots to submit thousands of applications almost instantly.

...

"LazyApply, Simplify and AI Hawk are all AI services that launch assistants to help collect applicant information and submit automated job applications through sites like LinkedIn and Indeed. Using the tools, job seekers can save hundreds of hours — and apply to up to thousands of jobs a day, according to the services.

"AI Apply, which claims its users are “80% more likely to get hired,” offers tools like a cover letter and resume builder, an auto-apply feature, an interview practice generator and a specialized interview buddy. A premium membership starts at $38 a month, according to the website — a cost AI Apply notes is much lower and more accessible than the cost of a career counselor.

...

"According to the company’s respective websites, OfferGoose has helped applicants land over 17,000 job offers, and AI Apply has over 340,000 “freelancers and job seekers” using the service.

"Nevertheless, many companies have enlisted additional safeguards to detect the use of AI. Some employers now require e-confirmation codes to submit applications and prevent automated submissions. Others have added prompts asking AI bots to use specific words like “banana” in responses to catch chat-generated application answers, according to longtime job recruiter and tech agency director of talent Mike Peditto.

"But job seekers are not the only party using AI in the application process, according to a recent University of Washington study. The researchers estimated that “99% of Fortune 500 companies are already using some sort of AI assistance when making hiring decisions,” and found that resume-screening tools have an incredible bias toward white, male applicants — with the large language models favoring white-associated males roughly 85% of the time and disadvantaging Black males in “up to 100% of cases.”

"The use of AI by both applicants and employers has created a distinct dynamic, in which applicants are using AI, but are also reticent about employers screening them with AI."

Monday, November 18, 2024

What preferences are revealed by market designs? by Oğuzhan Çelebi (who is on the job market)

Oğuzhan Çelebi is on the job market this year (listed on the job market websites of both MIT and Stanford where he's finishing a two year postdoc). He has a new paper that takes a really novel approach to market design.  It looks at the implicit preference relation (if one exists) that a mechanism reveals when a revealed preference analysis is done of the choices it makes from different possible menus of alternatives. The specific focus is on preferences for diversity revealed by mechanisms used in connection with affirmative action.

Diversity Preferences and Affirmative Action, by Oğuzhan Çelebi 

Abstract: "In various contexts, institutions allocate resources using rules that determine selections given the set of candidates. Many of these rules feature affirmative action, accounting for both identity and (match) quality of individuals. This paper studies the relationship between these rules and the preferences underlying them. I map the standard setting of market design to the revealed preference framework, interpreting choice rules as observed choices made across different situations. I provide a condition that characterizes when a rule can be rationalized by preferences based on identities and qualities. I apply tests based on this condition to evaluate real-world mechanisms, including India’s main affirmative action policy for allocating government jobs, and find that it cannot be rationalized. When identities are multidimensional, I show that non-intersectional views of diversity can be exploited by dominant groups to increase their representation and cause the choice rules to violate the substitutes condition, a key requirement for the use of stable matching mechanisms. I also characterize rules that can be rationalized by preferences separable in diversity and quality, demonstrating that they lead to a unique selection within the broader set of policies that reserve places based on individuals’ identities."

########

He's a target of opportunity for any department interested in market design.

Sunday, November 17, 2024

Opt-out defaults do not increase organ donation rates, in Public Health

 No one said market design was going to be easy...(well, some people did, but it turns out it isn't.)

Dallacker, M., L. Appelius, A. M. Brandmaier, A. S. Morais, and R. Hertwig. "Opt-out defaults do not increase organ donation rates." Public Health 236 (2024): 436-440. 

"Objectives: To increase organ donation rates, many countries have switched from an opt-in (‘explicit consent’) default for organ donation to an opt-out (‘presumed consent’) default. This study sought to determine the extent to which this change in default has led to an increase in the number of deceased individuals who become organ donors.
 

"Study design: Longitudinal retrospective analysis.
 

"Methods: We conducted a retrospective analysis of within-country longitudinal data to assess the effect of changing the organ donation default policy from opt-in to opt-out. Our analysis focused on the longitudinal deceased donor rates in five countries (Argentina, Chile, Sweden, Uruguay, Wales) that had adopted this change. Using a Bayesian aggregated binomial regression model, we estimated the odds of organ donation within each country over time, as well as the effect of the policy switch.
 

"Results: Switching from an opt-in to an opt-out default did not result in an increase in donation rates when averaged across countries. Moreover, the opt-out default did not lead to even a gradual increase in donations: there was no discernible difference in the linear rate of change of donations after the change in default. Finally, the COVID-19 pandemic was associated with a reduction in the odds of donation across all five countries.
 

Conclusions: Our longitudinal analysis suggests that changing to an opt-out default does not increase organ donation rates. Unless flanked by investments in healthcare, public awareness campaigns, and efforts to address the concerns of the deceased's relatives, a shift to an opt-out default is unlikely to increase organ donations."

 


...

"Family objections, often a significant barrier to deceased organ donation, should also be addressed. In many countries—including Chile, Sweden, and Wales—the consent of next of kin is necessary for organ donation. The veto power given to families has also been cited as a reason why the opt-out default does not significantly improve donation rates over the opt-in system.27,28 Considering expressed preferences, whether of the deceased or their relatives, overrides the default. Ultimately, the implications for transplantation outcomes between opt-in and opt-out defaults only differ in the rare cases when no explicit statements of preference were made by either the deceased or their relatives. A previous cross-country scenario analysis has shown that, when family preferences are honoured, shifting from an opt-in to an opt-out default alone would only increase organ recovery by 0%–5%.29 

######

 HT: Frank McCormick


Related:

Monday, July 22, 2024 Don't take "No" for an answer in deceased organ donor registration (a paper forthcoming after ten+ years)