Transition from medical school to residency: defending the parts that work well (namely the NRMP Resident Match)

This post is about a recently published paper concerning the design of the market for new doctors in the U.S.  But it will require some background for most readers of this blog.   The short summary is that the market is experiencing problems related to congestion, and one of the proposals to address these problems was deeply flawed, and would have reduced market thickness and caused substantial direct harm to participants if implemented, and created instabilities that would likely have caused indirect harms to the match process in subsequent years. But this needed to be explained in the medical community, since that proposal was being  very actively advocated.

For those of you already steeped in the background, you can go straight to the paper, here.

Itai Ashlagi, Ephy Love, Jason I. Reminick, Alvin E. Roth; Early vs Single Match in the Transition to Residency: Analysis Using NRMP Data From 2014 to 2021. J Grad Med Educ 1 April 2023; 15 (2): 219–227. doi: https://doi.org/10.4300/JGME-D-22-00177.1

If the title doesn't remind you of the vigorous advocacy for an early match for select positions, here is some of the relevant back story.

The market for new doctors--i.e. the transition from medical school to residency--is experiencing growing pains as the number of applications and interviews has grown, which imposes costs on both applicants and residency programs.  

Below is a schematic of that process, which begins with applicants submitting applications electronically, which makes it easy to submit many.  This is followed by residency programs inviting some of their applicants to interview. The movement to Zoom interviews has made it easier to have many interviews also (although interviews were multiplying even before they moved to Zoom).  

After interviews, programs and applicants participate in the famous centralized clearinghouse called The Match, run by the NRMP. Programs and applicants each submit rank order lists (ROLs) ranking those with whom they interviewed, and a deferred acceptance algorithm (the Roth-Peranson algorithm) produces a stable matching, which is publicly announced on Match Day. (Unmatched people and positions are invited into a now computer-mediated scramble, called SOAP, and these matches too are announced on Match  Day.)

The Match had its origins as a way to control the "unraveling" of the market into inefficient bilateral contracts, in which employment contracts were made long before employment would commence, via exploding offers that left most applicants with very little ability to compare options.  This kind of market failure afflicted not only the market for new physicians (residents), but also the market for later specialization (as fellows). Consequently, over the years, many specialties have turned to matching for their fellowship positions as well.

  The boxes in brown in the schematic are those that constitute "The Match:" the formulation and submission of the ROLs, and the processing of these into a stable matching of programs to residents.  Congestion is bedeviling the parts in blue.

The boxes colored brown are 'The Match' in which participants formulate and submit rank order lists (ROLs), after which a deferred acceptance algorithm produces a stable matching of applicants to programs, which is accepted by programs and applicants on Match Day. The boxes in blue, the applications and interviews that precede the Match, are presently suffering from some congestion.  Some specialties have been experimenting with signals (loosely modeled on those in the market for new Economics PhDs, but implemented differently by different medical specialties).

The proposal in question was to divide the match into two matches, run sequentially, with the first match only allowing half of the available positions to be filled.  The particular proposal was to do this first for the OB-GYN specialty, thus separating that from the other specialties in an early match, with only half of the OB-GYN positions available early.

This proposal came out of a study funded by the American Medical Association, and it was claimed, without any evidence being offered, that it would solve the current problems facing the transition to residency.  Our paper was written to provide some evidence of the likely effects, by simulating the proposed process using the preferences (ROLs) submitted in previous years.  

The results show that the proposal would largely harm OB-GYN applicants by giving them less preferred positions than they could get in a traditional single match, and that it would create instabilities that would encourage strategic behavior that would likely undermine the successful operation of the match in subsequent years.

Itai Ashlagi, Ephy Love, Jason I. Reminick, Alvin E. Roth; Early vs Single Match in the Transition to Residency: Analysis Using NRMP Data From 2014 to 2021. J Grad Med Educ 1 April 2023; 15 (2): 219–227. doi: https://doi.org/10.4300/JGME-D-22-00177.1

Abstract:

"Background--An Early Result Acceptance Program (ERAP) has been proposed for obstetrics and gynecology (OB/GYN) to address challenges in the transition to residency. However, there are no available data-driven analyses on the effects of ERAP on the residency transition.

"Objective--We used National Resident Matching Program (NRMP) data to simulate the outcomes of ERAP and compare those to what occurred in the Match historically.

"Methods--We simulated ERAP outcomes in OB/GYN, using the de-identified applicant and program rank order lists from 2014 to 2021, and compared them to the actual NRMP Match outcomes. We report outcomes and sensitivity analyses and consider likely behavioral adaptations.

"Results--Fourteen percent of applicants receive a less preferred match under ERAP, while only 8% of applicants receive a more preferred match. Less preferred matches disproportionately affect DOs and international medical graduates (IMGs) compared to US MD seniors. Forty-one percent of programs fill with more preferred sets of applicants, while 24% fill with less preferred sets of applicants. Twelve percent of applicants and 52% of programs are in mutually dissatisfied applicant-program pairs (a pair in which both prefer each other to the match each received). Seventy percent of applicants who receive less preferred matches are part of a mutually dissatisfied pair. In 75% of programs with more preferred outcomes, at least one assigned applicant is part of a mutually dissatisfied pair.

"Conclusions--In this simulation, ERAP fills most OB/GYN positions, but many applicants and programs receive less preferred matches, and disparities increase for DOs and IMGs. ERAP creates mutually dissatisfied applicant-program pairs and problems for mixed-specialty couples, which provides incentives for gamesmanship."

************

I'm hopeful this paper will effectively contribute to the ongoing discussion of how, and how not, to modify the design of the whole process of transition to residency with an aim to fixing the parts that need fixing, without damaging the parts that work well, i.e. while doing no harm. 

(Signaling will likely continue to play a role in this.)