Many preference signals as a soft cap on number of applications in medical residency matching

 Here's a review article on matching for medical residents,  with particular attention to neurosurgery, in the Cureus Journal of Medical Science.  In specialties that (like neurosurgery) allow applicants to send many signals, many applicants signal to and match with programs with which they have some prior connection.

Ozair, Ahmad, Jacob T. Hanson, Donald K. Detchou, Matthew P. Blackwell, Abigail Jenkins, Marianne I. Tissot, Umaru Barrie et al. "Program Signaling and Geographic Preferences in the United States Residency Match for Neurosurgery." Cureus 16, no. 9 (2024).

Abstract: Postgraduate residency training has long been the cornerstone of academic medicine in the United States. The Electronic Residency Application Service (ERAS), managed by the Association of American Medical Colleges (AAMC), is the central residency application platform in the United States for most clinical specialties, with the National Residency Matching Program (NRMP) being the algorithm for matching residency programs with applicants. However, the determination of the best fit between ERAS applicants and programs has been increasingly challenged by the rising number of applicants per residency spot. This application overburdening across competitive specialties led to several adverse downstream effects, which affected all stakeholders. While several changes and proposals were made to rectify the issue of application overburdening, the 2020-2021 ERAS Match Cycle finally saw several competitive specialties, including otolaryngology and urology, utilize a new system of supplemental residency application based on preference signals/tokens. These tokens permit applicants to electronically signal a select number of programs in a specialty of choice, with the program reviewing the application now cognizant that they have been signaled, i.e., the applicant has chosen to use up a limited set of signals for their program. Initial results from otolaryngology and urology, as described in this article, indicated the value of this new system to both applicants and educators. Given the favorable outcomes and broader uptake of the system among other specialties, the field of neurosurgery adopted the utilization of the ERAS-based program signaling and geographic preference for the first time for the 2022-2023 Residency Application Cycle and later opted to continue them for the 2023-2024 and 2024-2025 cycles. For the 2024-2025 Match Cycle, neurosurgery applicants have 25 signals, i.e., a "high-signal" approach, where non-signaled programs have a low interview conversion rate. This literature review discusses the rationale behind the change, the outcomes of other competitive specialties from prior cycles, the evolving nature of the change, and the potential impact on applicants and programs. As we describe in this review, signaling may potentially represent a surrogate form of an application cap. Other considerations relate to cost savings for both applicants and programs from a high-signal approach in neurosurgery. These modifications represent a foundational attempt to alleviate the application overburdening and non-holistic review in the residency application process, including for neurosurgery. While these changes have been a welcomed addition for all stakeholders in residency match cycles so far, further prospectively directed surveys along with qualitative research studies are warranted to better delineate the downstream impact of these changes and guide further optimization of the application system.

The Path to a Match for Interventional Cardiology Fellowships

The Society for Cardiovascular Angiography & Interventions has started a fellowship match, and here's an article describing the familiar marketplace failure that led to that decision, involving unraveling of application, interview and appointment dates, with the resulting congestion and exploding offers, and the process of reaching sufficient consensus to move to a centralized match ( to be run by the NRMP).

The Path to a Match for Interventional Cardiology Fellowship: A Major SCAI Initiative  by Douglas E. Drachman MD, FSCAI (Chair) a, Tayo Addo MD b, Robert J. Applegate MD, MSCAI c, Robert C. Bartel MSc, CAE d, Anna E. Bortnick MD, PhD, MSc, FSCAI e, Francesca M. Dea d, Tarek Helmy MD, MSCAI f, Timothy D. Henry MD, MSCAI g, Adnan Khalif MD, FSCAI h, Ajay J. Kirtane MD, SM, FSCAI i, Michael Levy MD, MPH, FSCAI j, Michael J. Lim MD, MSCAI k, Ehtisham Mahmud MD, MSCAI l, Nino Mihatov MD, FSCAI m, Sahil A. Parikh MD, FSCAI i, Laura Porter CMP d, Abhiram Prasad MD n, Sunil V. Rao MD, FSCAI o, Louai Razzouk MD, MPH, FSCAI o, Samit Shah MD, PhD, FSCAI p, Adhir Shroff MD, MPH, FSCAI q, Jacqueline E. Tamis-Holland MD, FSCAI r, Poonam Velagapudi MD, FSCAI s, Fredrick G. Welt MD, FSCAI t, J. Dawn Abbott MD, FSCAI (Co-Chair), Journal of the Society for Cardiovascular Angiography & Interventions, in press.

"Abstract: The field of interventional cardiology (IC) has evolved dramatically over the past 40 years. Training and certification in IC have kept pace, with the development of accredited IC fellowship training programs, training statements, and subspecialty board certification. The application process, however, remained fragmented with lack of a universal process or time frame. In recent years, growing competition among training programs for the strongest candidates resulted in time-limited offers and high-pressure situations that disadvantaged candidates. A grassroots effort was recently undertaken by a Society for Cardiovascular Angiography & Interventions task force, to create equity in the system by establishing a national Match for IC fellowship. This manuscript explores the rationale, process, and implications of this endeavor."

"over the past several years program directors and candidates found that the process has devolved, with wide variation in application timelines and on-the-spot offers, which disadvantage candidates and programs looking to interview a range of applicants.

"The pressures and unfair features of the existing system were further fueled by the transition to virtual interviews related to the COVID-19 pandemic. With logistics of travel no longer a consideration, programs could commence interviews nearly immediately after the applications became available. This led to more candidates being interviewed in rapid succession, and a system evolved in which programs quickly assessed candidates, offered positions, and applied pressure for candidates to accept offers or be passed over for other candidates.

"In response to the shortcomings of the current system, members of Society for Cardiovascular Angiography & Interventions (SCAI) were inspired to lead a grassroots educational campaign to organize IC program directors and the broader interventional community to commit to a regulated “Match” process under the established National Resident Match Program (NRMP). This manuscript provides an account of how this process unfolded and how a Match for IC fellowship was ultimately created.

...

"From the applicant’s perspective, the lack of a structured timeline for the application process required candidates to make career decisions early in the first year of cardiovascular disease training and to compose their application materials 2 years in advance of starting IC training. With ERAS open to application submission in the fall of the second year for the December release to programs, fellows had limited time on clinical rotations to determine their interest and aptitude for IC. Additionally, letters of recommendation, written at this early stage, risked not being fully reflective of each candidate’s capacity to improve and develop the technical skills and clinical knowledge important for success in the field. There were other disadvantages to candidates in the existing system. Fellows at programs with an IC fellowship had an advantage of securing an internal spot but were often pressured to limit their exploration of the opportunities at other programs, potentially disadvantaging them in the long term.

"Another problem with the existing system was that the pressure to recruit candidates on a tight timeline limited the opportunity to interview applicants from a wide variety and diversity of programs, potentially reducing the ability to recruit underrepresented candidates from varied programs. Despite an overall increase in the diversity of physicians entering the workforce,11 there has been little change in the applicant pool for IC over the years, with fewer than 5% of applicants self-reporting as Black race or Hispanic ethnicity and only 10% identifying as women.12

"Competition among the programs, each vying for the seemingly strongest candidates, degenerated into a system that favored quick decision-making on the part of programs to offer positions as early as possible. The influence of the COVID-19 pandemic in 2020 and 2021 negatively impacted an already high-pressure application process, compounding its many weaknesses.13 Fellowship interviews were hosted virtually rather than in person, which enabled candidates to interview at a greater number of programs without the need to travel. In addition, the virtual format accelerated the tempo of an application process that was already felt to be too fast, resulting in an increase in so-called “exploding offers”—offers that required the accepted candidate to respond within a very short timeframe or risk losing the offer. This practice placed significant pressure on candidates to make quick decisions, often forcing them to determine whether to accept the offer from 1 institution before having the opportunity to participate in interviews with—let alone see and evaluate—other programs or fully understand the ramifications of accepting an offer on their personal lives. At the same time, the accelerated timetable left many programs scrambling to identify applicants, as the number of available candidates diminished rapidly due to applicants accepting time-sensitive, exploding offers.

...

"As with other national efforts of this magnitude, the path to develop consensus in favor of a Match was not without challenges. There were several program directors around the country who strongly opposed the institution of a Match. These were well-regarded academicians and clinician educators who expressed very sincere concerns about the impact on fellows in their programs. The members of the SCAI Match Task Force addressed as many concerns as possible, providing the information necessary for each program director to make the best decision for their institution. A minority of program directors remained opposed to the initiative or did not engage with Task Force members despite multiple attempts to be contacted.

"The Match campaign proved highly effective, and by November 2022, the 75% threshold of programs and positions to implement the Match was met

...

"As the sponsor of the Match, SCAI considered the pros and cons of the “All In Policy,” where registered programs must attempt to fill all ACGME positions at the program through the Match.15,16 SCAI opted out of the “All In Policy” to allow programs to have flexibility for unique situations that require commitment to a candidate outside of the Match. 

...

"As a result of the successful implementation of the Match in IC, the first Match cycle for incoming IC fellows will open in the summer of 2024. Individuals eligible to apply include cardiovascular disease fellows in their third or final year of training and graduates who have completed fellowship and are in clinical practice. This class will start IC training in July 2025"

NRMP Board of Directors (2020-2024), and a 40th anniversary

 This bit of glass marks the end of my term on the National Resident Matching Program (NRMP) Board of Directors.

One of the issues that consumed a lot of attention during my term is discussed in this post:

Friday, April 21, 2023

Transition from medical school to residency: defending the parts that work well (namely the NRMP Resident Match)

And here are all my posts about residents and fellows, going back to the beginning of this blog in 2008. (It's been interesting watching medical specialties begin to develop signaling in ways  reminiscent of signaling in the Economics job market, to deal with congestion of interviews and applications.*)

My first paper dealing explicitly with The Match suddenly seems to have been published 40 years ago:

Roth, A.E. "The Evolution of the Labor Market for Medical Interns and Residents: A Case Study in Game Theory", Journal of Political Economy, Vol. 92, 1984, 991‑1016. http://web.stanford.edu/~alroth/papers/evolut.pdf 

And the main report (with Elliott Peranson) of our redesign of The Match is now a quarter of a century old:

Roth, A.E. and E. Peranson, "The Redesign of the Matching Market for American Physicians: Some Engineering Aspects of Economic Design,” American Economic Review, 89, 4, September, 1999, 748-780. https://www.aeaweb.org/articles?id=10.1257/aer.89.4.748

*See yesterday's post for some discussion of market design interventions in job markets.

Match Day for new doctors

 The 2024 Match for new American doctors was announced yesterday by the NRMP, the National Resident Matching Program. Congratulations to all!

Here are some links:

Advance Data Tables
Match By the Numbers
Press Release

Signaling for residency programs in dermatology, general surgery, and internal medicine

We're starting to see some data from signaling for residency applications.  This paper observes that programs are more likely to interview candidates who send them a signal. (Economists will worry that this reflects which programs are signaled and not just the effect of a signal...)  These three specialties have relatively few signals, more like economics than like Orthopedic Surgery (which has 30 signals).  And the table indicates that more interviews are offered than signals received, so that's another difference from Ortho...)

Rosenblatt, Adena E., Jennifer LaFemina, Lonika Sood, Jennifer Choi, Jennifer Serfin, Bobby Naemi, and Dana Dunleavy. "Impact of Preference Signals on Interview Selection Across Multiple Residency Specialties and Programs." Journal of Graduate Medical Education 15, no. 6 (2023): 702.

"Abstract

"Background Program signaling is an innovation that allows applicants to express interest in specific programs while providing programs the opportunity to review genuinely interested applicants during the interview selection process.

"Objective To examine the influence of program signaling on “selected to interview” status across specialties in the 2022 Electronic Residency Application Service (ERAS) application cycle.

"Methods Dermatology, general surgery-categorical (GS), and internal medicine-categorical (IM-C) programs that participated in the signaling section of the 2022 supplemental ERAS application (SuppApp) were included. Applicant signal data was collected from SuppApp, applicant self-reported characteristics collected from the MyERAS Application for Residency Applicants, and 2020 program characteristics collected from the 2020 GME Track Survey. Applicant probability of being selected for interview was analyzed using logistic regression, determined by the selected to interview status in the ERAS Program Director’s WorkStation.

"Results Dermatology had a 62% participation rate (73 of 117 programs), GS a 75% participation rate (174 of 232 programs), and IM-C an 86% participation rate (309 of 361 programs). In all 3 specialties examined, on average, signaling increased the likelihood of being selected to interview compared to applicants who did not signal. This finding held across gender and underrepresented in medicine (UIM) groups in all 3 specialties, across applicant types (MDs, DOs, international medical graduates) for GS and IM-C, and after controlling for United States Medical Licensing Examination Step 1 scores.

"Conclusions Although there was variability by program, signaling increased likelihood of being selected for interview without negatively affecting any specific gender or UIM group."

Data from future years will be needed to determine how signaling is influencing the distribution of residents to programs.

Transition to residency conference: Oct 5-7

I'll be a panelist at the  the NRMP conference Transition to Residency,  in Boston, Oct 5-7

"The National Resident Matching Program® (NRMP®) will convene its stakeholder conference in Boston this year. The meeting is intended to provide a forum for robust conversation among members of the undergraduate and graduate medical education communities about issues relevant to the transition to residency."

Here's the list of plenary speakers.

Friday, Oct 6, 8:30 – 9:45 AM Plenary I

The Future of the Transition to Residency: Assessing the Impact of Proposed Change

Panelists:

John Combes, MD

Alvin Roth, PhD

Charles (Tom) Thomas, MA, MPhil

Mathematics and Computer Science of Market and Mechanism Design: SLMath introductory workshop (videos)

Last week I gave the opening talk of the week long  Introductory Workshop at SLMath, on Mathematics and Computer Science of Market and Mechanism Design.  Some of the video lectures are now online here (consisting mostly of slides and voice).

My talk introduces the general themes of market design by recounting the history and challenges facing the market for new doctors from 1900 through this year.

Berkeley's Simons Laufer Mathematical Sciences Institute (SLMath), formerly known as the Mathematical Sciences Research Institute (MSRI) has a commanding view of the SF Bay.

Signaling for Orthopaedic surgery residencies

 The Journal of Bone and Joint Surgery reports a small survey about signaling behavior among applicants for orthopaedic surgery residencies.

Deckey, David G., Eugenia Lin, Coltin RB Gerhart, Joseph C. Brinkman, Karan A. Patel, and Joshua S. Bingham. "Decoding the Signals: An Analysis of Preference Signaling in the 2023 Orthopaedic Surgery Residency Match." JBJS Open Access 8, no. 3 (2023).

"While previously used in other specialties, the preference signaling program (PSP) was implemented in the 2022 to 2023 orthopaedic surgery residency application process for the first time. The PSP allowed for 30 signaling tokens to be sent by applicants to programs of their choice to indicate particular interest in a program.

...

"An anonymous electronic survey was emailed to all orthopaedic surgery residency applicants who applied to the authors' institution during the 2022 to 2023 application cycle. The survey was sent after match lists were submitted and closed before the release of match results. 

...

"The survey was completed by 101 applicants. Applicants applied to a mean of 90 programs (range: 10-197) and received an average of 12 interview invitations (range: 0-39). Applicants almost uniformly used all 30 signals, with nearly two-thirds signaling their home programs (65%, 49/76), and nearly all applicants sending signals to programs at which they performed away rotations (95.7%, 88/92). Applicants received a mean of 9 invitations from programs they signaled, compared with 2 invitations from programs they did not signal."

#######

Applicants report sending signals to all the programs that would have been expected to automatically give them interviews even in the absence of a signaling mechanism--namely their home programs and those which they have spent time visiting in 'away rotations.'

In the Economics job market, which may have been the first to introduce signaling, we limited applicants to 2 signals, and advised them not to signal jobs in which they already had well established mutual interests, i.e. not to signal jobs which they felt would interview them without signals. One of the ideas behind the Econ signaling mechanism is that there are many ways applicants can send signals of interest within a network to which they are well connected. Since some applicants are better connected than others, we were offering some signals that could be sent out of network.

It will be interesting to understand if signals to Ortho residencies are helping promote out of network interviews, or are largely strengthening the network connections already established by medical schools (home programs) and away rotations.

OB-GYN doctors will use a new application system to apply to residency programs (but will continue to go through the NRMP resident match)

 Before new doctors can participate in the resident match (by engaging with the NRMP), they first have to apply to residency programs, and arrange interviews.  This process has been experiencing congestion, and the specialty of Obstetrics and Gynecology has now decided to switch application services. 

However, participation in the NRMP will not change: the Association of Professors of Gynecology and Obstetrics (APGO) FAQ states "Obstetrics and gynecology applicants will use the National Residency Match Program (NRMP) for the Match. This new application does not change how the applicant or programs interact with the NRMP Match system."

Medpage Today has the story:

Ob/Gyn Switching to Independent System for Residency Applications— This is the last year ob/gyn will use ERAS, despite helping to pilot the program  by Rachael Robertson, Enterprise & Investigative Writer, MedPage Today August 25, 2023

"Beginning next year, ob/gyn programs will start using an independent system for processing residency applications, rather than the Electronic Residency Application Service (ERAS).

"The joint decision to switch to the new system was made by the American College of Obstetricians and Gynecologists (ACOG), the Association of Professors of Gynecology and Obstetrics (APGO), and the Council on Resident Education in Obstetrics and Gynecology (CREOG). The new system will be managed by Liaison International, which uses "Centralized Application Service (CAS) technology," according to the company's website.

...

"A joint statement on the APGO website opens in a new tab or window

said that the new system "will be user friendly and efficient, less expensive for applicants, and will directly decrease the burdens faced by program directors, program managers, and applicants alike," and "will incorporate the entirety of interview season functions, from application submission, review, interview offers and interviews, to rank list submission."

ACOG explained that the decision to pull the ERAS stemmed from the Right Resident, Right Program, Ready Day One initiativeopens in a new tab or window, noting that the new system is mobile-friendly and "will include immediate fee reduction," as detailed on their FAQ pageopens in a new tab or window.

In response, the Association of American Medical Colleges (AAMC), which runs ERAS, issued a statementopens in a new tab or window attributed to President and CEO David J. Skorton, MD, and Alison J. Whelan, MD, the chief academic officer, saying they were "surprised and dismayed" by the decision. 

...

"Bryan Carmody, MD, of Eastern Virginia Medical School in Norfolk, shared information  opens in a new tab or window

about the change on social media, writing on his blogopens in a new tab or window that ob/gyn program directors helped to pilot ERAS when it was first rolled out in the mid-90s.

Carmody told MedPage Today that he anticipates the biggest downsides will fall on applicants, such as those who want to apply to another specialty in addition to ob/gyn.

"Those applicants will have to use one system to apply to ob/gyn and another to their other specialty," he explained. "The same thing applies to applicants who fail to match. They'll have to use ERAS to apply to another specialty during SOAP [Supplemental Offer and Acceptance Program] since few, if any, ob/gyn positions are typically available."

*************

OB-GYN will continue to employ signaling  (very loosely modeled on the signaling used in the Econ PhD job market, but asking applicants to submit 3 "gold" signals and 15 "silver" signals): 

Program Signaling for OBGYN Residency Application Background and FAQs

**************

Earlier:

Friday, April 21, 2023

Transition from medical school to residency: defending the parts that work well (namely the NRMP Resident Match)

Resident match video from the NRMP

 Different labor markets are organized differently. One difference between the market for new doctors and the markets for new Ph.D.s is that doctors use a centralized clearinghouse, so a lot of things happen at the same time.  Here's a video from the National Resident Matching Program that gives some idea about that.

Transition from medical school to residency: defending the parts that work well (namely the NRMP Resident Match)

This post is about a recently published paper concerning the design of the market for new doctors in the U.S.  But it will require some background for most readers of this blog.   The short summary is that the market is experiencing problems related to congestion, and one of the proposals to address these problems was deeply flawed, and would have reduced market thickness and caused substantial direct harm to participants if implemented, and created instabilities that would likely have caused indirect harms to the match process in subsequent years. But this needed to be explained in the medical community, since that proposal was being  very actively advocated.

For those of you already steeped in the background, you can go straight to the paper, here.

Itai Ashlagi, Ephy Love, Jason I. Reminick, Alvin E. Roth; Early vs Single Match in the Transition to Residency: Analysis Using NRMP Data From 2014 to 2021. J Grad Med Educ 1 April 2023; 15 (2): 219–227. doi: https://doi.org/10.4300/JGME-D-22-00177.1

If the title doesn't remind you of the vigorous advocacy for an early match for select positions, here is some of the relevant back story.

The market for new doctors--i.e. the transition from medical school to residency--is experiencing growing pains as the number of applications and interviews has grown, which imposes costs on both applicants and residency programs.  

Below is a schematic of that process, which begins with applicants submitting applications electronically, which makes it easy to submit many.  This is followed by residency programs inviting some of their applicants to interview. The movement to Zoom interviews has made it easier to have many interviews also (although interviews were multiplying even before they moved to Zoom).  

After interviews, programs and applicants participate in the famous centralized clearinghouse called The Match, run by the NRMP. Programs and applicants each submit rank order lists (ROLs) ranking those with whom they interviewed, and a deferred acceptance algorithm (the Roth-Peranson algorithm) produces a stable matching, which is publicly announced on Match Day. (Unmatched people and positions are invited into a now computer-mediated scramble, called SOAP, and these matches too are announced on Match  Day.)

The Match had its origins as a way to control the "unraveling" of the market into inefficient bilateral contracts, in which employment contracts were made long before employment would commence, via exploding offers that left most applicants with very little ability to compare options.  This kind of market failure afflicted not only the market for new physicians (residents), but also the market for later specialization (as fellows). Consequently, over the years, many specialties have turned to matching for their fellowship positions as well.

  The boxes in brown in the schematic are those that constitute "The Match:" the formulation and submission of the ROLs, and the processing of these into a stable matching of programs to residents.  Congestion is bedeviling the parts in blue.

The boxes colored brown are 'The Match' in which participants formulate and submit rank order lists (ROLs), after which a deferred acceptance algorithm produces a stable matching of applicants to programs, which is accepted by programs and applicants on Match Day. The boxes in blue, the applications and interviews that precede the Match, are presently suffering from some congestion.  Some specialties have been experimenting with signals (loosely modeled on those in the market for new Economics PhDs, but implemented differently by different medical specialties).

The proposal in question was to divide the match into two matches, run sequentially, with the first match only allowing half of the available positions to be filled.  The particular proposal was to do this first for the OB-GYN specialty, thus separating that from the other specialties in an early match, with only half of the OB-GYN positions available early.

This proposal came out of a study funded by the American Medical Association, and it was claimed, without any evidence being offered, that it would solve the current problems facing the transition to residency.  Our paper was written to provide some evidence of the likely effects, by simulating the proposed process using the preferences (ROLs) submitted in previous years.  

The results show that the proposal would largely harm OB-GYN applicants by giving them less preferred positions than they could get in a traditional single match, and that it would create instabilities that would encourage strategic behavior that would likely undermine the successful operation of the match in subsequent years.

Itai Ashlagi, Ephy Love, Jason I. Reminick, Alvin E. Roth; Early vs Single Match in the Transition to Residency: Analysis Using NRMP Data From 2014 to 2021. J Grad Med Educ 1 April 2023; 15 (2): 219–227. doi: https://doi.org/10.4300/JGME-D-22-00177.1

Abstract:

"Background--An Early Result Acceptance Program (ERAP) has been proposed for obstetrics and gynecology (OB/GYN) to address challenges in the transition to residency. However, there are no available data-driven analyses on the effects of ERAP on the residency transition.

"Objective--We used National Resident Matching Program (NRMP) data to simulate the outcomes of ERAP and compare those to what occurred in the Match historically.

"Methods--We simulated ERAP outcomes in OB/GYN, using the de-identified applicant and program rank order lists from 2014 to 2021, and compared them to the actual NRMP Match outcomes. We report outcomes and sensitivity analyses and consider likely behavioral adaptations.

"Results--Fourteen percent of applicants receive a less preferred match under ERAP, while only 8% of applicants receive a more preferred match. Less preferred matches disproportionately affect DOs and international medical graduates (IMGs) compared to US MD seniors. Forty-one percent of programs fill with more preferred sets of applicants, while 24% fill with less preferred sets of applicants. Twelve percent of applicants and 52% of programs are in mutually dissatisfied applicant-program pairs (a pair in which both prefer each other to the match each received). Seventy percent of applicants who receive less preferred matches are part of a mutually dissatisfied pair. In 75% of programs with more preferred outcomes, at least one assigned applicant is part of a mutually dissatisfied pair.

"Conclusions--In this simulation, ERAP fills most OB/GYN positions, but many applicants and programs receive less preferred matches, and disparities increase for DOs and IMGs. ERAP creates mutually dissatisfied applicant-program pairs and problems for mixed-specialty couples, which provides incentives for gamesmanship."

************

I'm hopeful this paper will effectively contribute to the ongoing discussion of how, and how not, to modify the design of the whole process of transition to residency with an aim to fixing the parts that need fixing, without damaging the parts that work well, i.e. while doing no harm. 

(Signaling will likely continue to play a role in this.)

Improving the transition to (surgical) residency

The transition from medical school to residency is presently troubled by congestion involving (too) many applications and interviews.  It's a subject of considerable discussion in the medical community, sometimes hampered between the parts of the process that proceed the Match, and the Match itself (which is the clearinghouse run by the NRMP that, after all applications and interviews have been processed, solicits rank order lists and turns them into a matching of doctors to residency programs)..  Here's a paper that focuses sensibly on the runup to the Match, even though its title follows the (unfortunately common) practice of calling the whole process the Match.

Designing the “match of the future”: challenges and proposed solutions in the interview and match phase of the UME–GME transition by Sophia K. McKinley, Maria S. Altieri, Olabisi Sheppard, Kimberly Hendershot, Keneeshia Williams, Brigitte K. Smith on behalf of the ASE Graduate Surgical Education Committee, Global Surgical Education - Journal of the Association for Surgical Education : 17 November

Table 1 Challenges and proposed solutions in the surgical resident selection process (click to embiggen)

The past and future of the transition from medical school to residency, in the Journal of Graduate Medical Education, by Williamson, Soane, and Carmody

 The October issue of the Journal of Graduate Medical Education considers the past and future of the transition to residency.

The US Residency Match at 70: What Was, What Is, and What Could Be  by Edwin Williamson, MD; Caroline Soane, BA; J. Bryan Carmody, MD, MPH, J Grad Med Educ (2022) 14 (5): 519–521., https://doi.org/10.4300/JGME-D-22-00248.1

"But while early offers are long gone, the residency selection process now faces a new set of challenges related to the increasing number of applications submitted by contemporary applicants. In 2020 the average US medical school graduate submitted 70 residency applications.9  The average for some specialties is even higher. For instance, in 2022, the average osteopathic medical school applicant in obstetrics and gynecology submitted 85 applications, while US MD applicants in orthopedic surgery submitted 96 applications, and international medical graduates submitted 100 applications each to internal medicine programs.10  This overapplication increases costs for applicants and programs, leads to reliance on convenient screening metrics in applicant evaluation, and does not ultimately improve Match rates.9 "

The importance of couples matching for medical residents (it's missed where it's missing)

 One of the successes of the design of the NRMP Match for medical residents is that it accommodates couples, using the Roth-Peranson algorithm*.  Here's an article reflecting on the fact that the Ophthalmology and Urology matches are done outside of the NRMP, and don't have a couples match.

Massenzio, Samantha S., Tara A. Uhler, Erik M. Massenzio, Emily Sun, Divya Srikumaran, Marisa M. Clifton, Laura K. Green, Grace Sun, Jiangxia Wang, and Fasika A. Woreta. "Navigating the Ophthalmology & Urology Match with a Significant Other." Journal of Surgical Education (2022).

"• There is an increasing number of couples applying for residency

• Ophthalmology and urology applicants cannot utilize the NRMP Couples Match system

• A Couples Match is highly desired by applicants to these two specialties

• The lack of a Couples Match is a deterrent to these specialties for some applicants

• Systems to aid applicants to these specialties with significant others are needed"

...

"The Couples Match is currently not offered to applicants to ophthalmology or urology as these specialties utilize separate match systems outside of NRMP - the San Francisco (SF) Match1 and Urology Residency Matching Program,2 respectively. Historically, the NRMP implemented the match starting in 1952 for internship programs (postgraduate year 1) only. Individual specialties later established their own systems for matching to advanced training beyond internship; for ophthalmology this occurred in 1979, and urology in 1985. While other specialties have since merged their match processes with the NRMP, ophthalmology and urology continue to facilitate their own match.3, 4, 5

"Ophthalmology and urology also have an “early match,” with match results released in January or February versus in mid-March for applicants using the NRMP. This was historically advantageous because of the preliminary internship year required for both ophthalmology and urology, allowing applicants to rank their preference for internship year in the NRMP based on the outcome of their specialty match. However, as of 2021 and 2019, ophthalmology6 and urology7 respectively transitioned to an integrated or joint internship model, meaning that an internship position is secured at the same time as the specialty match. Given these changes, it is currently timely to evaluate applicant viewpoints on the Couples Match.

...

"Survey Findings

"107 respondents reported having a significant other in medicine (72 ophthalmology, 35 urology), making up 31% of all respondents. 68 (64%) significant others applied in the same cycle as the survey respondent, 11 (10%) applied before, and 28 (26%) will apply after. If the Couples Match had been available, 78% of respondents with a significant other who applied in the same cycle reported that they would have participated.

...

"The lack of a Couples Match is a deterrent to ophthalmology and urology for over one-fifth of applicants with a significant other. Of applicants’ partners who considered ophthalmology or urology, over one-third reported to have been deterred. These findings suggest that for many students who want to be in the same location as their partner in medicine, ophthalmology and urology are specialty choices that may be less desirable toward this end. It is unclear how this impacts efforts to foster more diversity (in terms of gender, race, and other factors) in these specialties.

"The inability for ophthalmology and urology applicants to use the Couples Match adds significant stress to an already difficult application season with survey respondents commenting on the negative mental health effects. In addition, there is currently no official avenue for ophthalmology and urology applicants to communicate a desire to match in proximity to a significant other. Although applicants found methods to overcome this barrier, such as by mentioning their significant other during interviews, a majority indicated that there were times that they were hesitant to discuss their significant other out of concern that it would negatively affect their chances of matching, with female applicants disproportionately affected.

"Regarding the early match, most applicants liked receiving match designations sooner; however, there are mixed responses as to whether or not this timeline is helpful for individuals with a significant other in medicine. The authors are aware that a benefit of the early match for ophthalmology/urology applicants is that a significant other applying through the regular NRMP match has the opportunity to selectively contact programs in the vicinity of the applicant's matched program to express heightened interest. Disadvantages to the early match include that once the ophthalmology/urology applicant matches, their significant other has a limited number of programs in the vicinity to choose from - this is in contrast to a Couples Match where any pair of programs in any location may be ranked. This may also be exacerbated by recent efforts to limit the number of applications submitted or interviews accepted by each student.16, 17, 18 Further, the early match does not help individuals whose significant other is also applying to ophthalmology or urology as they would not be able to take advantage of the difference in timeline between partners. It is presently very challenging for a couple who both want to apply in the same year to ophthalmology or urology to be able to coordinate their match outcome to the same location."

##########

*Roth, A. E. and Elliott Peranson, "The Redesign of the Matching Market for American Physicians: Some Engineering Aspects of Economic Design," American Economic Review, 89, 4, September, 1999, 748-780

An Interview Match for medical residents and fellows--a preliminary proposal

 There is a lot of concern in the graduate medical education community that too much time and treasure is being spent on too many unproductive interviews prior to the submission of rank order lists for the Match.  Here's discussion of a proposal for an interview match, to precede the interview stage before the actual NRMP Match.

Explaining a Potential Interview Match for Graduate Medical Education, by Irene Wapnir, MD; Itai Ashlagi, PhD; Alvin E. Roth, PhD; Erling Skancke, MS; Akhil Vohra, PhD; Irene Lo, PhD; Marc L. Melcher, MD, PhD, J Grad Med Educ (2021) 13 (6): 764–767.  https://doi.org/10.4300/JGME-D-20-01422.1

"Residency and fellowship candidates are applying to more programs to enhance their chances of securing interviews and matching favorably. The COVID-19 pandemic has shifted interviews to video formats, which lowers interview-associated costs for applicants but may further increase application numbers.1  While a candidate's application to a training program communicates some interest in the program, the relative amount of interest is obscured when candidates apply to large numbers of programs. We suspect that, as a result, programs host large numbers of low-yield interviews.

"The number of interviews is steadily increasing, and there is widespread agreement on the need to ease congestion in the pre-Match evaluation process.2  Proposals to reduce this burden include signaling (organized, centrally-controlled protocol for limited communication of interest),3–5  capping the number of applications or the number of interviews,6,7  and an early acceptance matching program as in college admissions.8,9 

"We propose another solution, an “interview match” to address the expanding number of interviews.10  An interview match enables candidates and programs to express preferences privately by ranking their interview choices individually or in tiers. This may ease congestion in the “marketplace,” reduce costs for candidates, favor interviews that are more likely to lead to a match in the final Match, and avoid interviews unlikely to convert to a match. An interview match algorithm would match based on the same “deferred-acceptance” algorithm currently used by the National Resident Matching Program but adapted to a “many-to-many” setting where candidates and programs receive multiple interviews."

NRMP Statement On Interviewing

 The National Resident Matching Program (NRMP) has a statement on interviewing, which precedes the NRMP match for new American doctors, and that reflects concerns that the interviewing process has become congested.

NRMP Statement On Interviewing

"The National Resident Matching Program® (NRMP®) has heard the concerns of learners and programs in the medical education community about the interview process and wants to encourage equitable practices among applicants and programs. As such we have developed the following recommendations. Although especially true during the enduring times of the pandemic, the recommendations align with the NRMP’s long-standing commitment to maintaining a fair, efficient, reliable, and transparent process for all. Recommendations also foster well-being among all parties.

"NRMP Recommendations for Programs:

1. Programs should conduct all interviews virtually for the 2021-2022 cycle.

2. Programs should extend interview offers that equal, not exceed, the total number of available interview slots.

3. Applicants should be given a minimum of 48 hours to respond to an interview offer.

"NRMP Recommendations for Applicants:

1. Applicants should make timely decisions about interview offers and promptly notify programs in which they are no longer interested, freeing up interview slots.

2. Applicants should give ample, adequate (e.g., one week) notice to programs in the event they change their minds and decline accepted interviews."

Some history of the National Resident Matching Program

 Here's a short history of the resident match, including the recent merger of MD and DO student applicants, and some thoughts about current issues.

Acad Med. 2021 Aug; 96(8): 1116–1119.

The Single Match: Reflections on the National Resident Matching Program’s Sustained Partnership With Learners  by Zaid I. Almarzooq, MBBCh, Heather A Lillemoe, MD, Ebony White-Manigault, MPH, Thomas Wickham, DO, MPH, and Laurie S. Curtin, PhD

Here's the concluding paragraph:

"The NRMP has come a long way, but we recognize that the residency selection process still is fraught with stress and uncertainty, albeit for reasons different from those that prompted creation of the Match. Application inflation, debt, and a disproportionate reliance on licensure exam scores have contributed to a climate that makes the transition to residency perhaps as stressful as when the Match was created nearly 70 years ago. 18,19 However, as the NRMP moves beyond achievement of the Single Match milestone and we reflect on the organization’s history of responding to the needs of its constituents, we believe the NRMP will continue to evolve and identify innovative and meaningful ways to address learner needs. We hope learners of all kinds value that commitment and stand ready to support the NRMP’s efforts to continually improve the transition to residency."

An interview in (not on) Clinical Orthopaedics and Related Research, largely about resident matching

 For those  readers of this blog who may have missed the May issue of Clinical Orthopaedics and Related Research, here's an interview by the editor

A Conversation with ... Alvin E. Roth PhD, Economist, Game Theorist, and Nobel Laureate Who Improved the Modern Residency Match  by Leopold, Seth S. MD, Clinical Orthopaedics and Related Research, Volume 479(5), May 2021, p 863-866   doi: 10.1097/CORR.0000000000001758

The interview includes this long answer to a short question about the resident Match:

Seth S. Leopold MD: Many readers will dispute the idea that the Match is resistant to strategic manipulation (“gaming the system”); why do you believe it is, and why do you think this perception persists?

Alvin E. Roth PhD: That question requires a somewhat complicated answer. The Match is built around an idea of how to organize a simple labor market, and that idea had to be adapted to the complex structure of the modern medical labor force. A simple labor market would be one in which graduating medical students each seek a single position, positions are well described in advance, and applicants and residency programs can each rank-order all of their possible matches; that is, applicants can rank programs and programs can rank applicants. That simple market can be modeled mathematically, and it can be shown that a deferred acceptance algorithm with applicants proposing makes it a dominant strategy for all applicants to submit rank-order lists corresponding to their true preferences. (A dominant strategy is one in which regardless of what rank-order lists others submit, no applicant can do better than to rank residency programs in order of his or her true preferences. For instance, your chance of getting your second-choice program if your first choice rejects you is exactly the same as if you had listed your second choice first.)

"That’s a theoretical answer about a market that is quite a bit simpler than the modern market for residencies. The deferred acceptance algorithm for that simple market was studied by Gale and Shapley [8], for which Shapley shared the 2012 Nobel Prize in Economics. (I had earlier shown that in a simple market, applicants can’t profitably manipulate their rank-order lists [16].)

"The actual modern market for residencies differs from that simple market in several ways. For one thing, not all applicants are seeking a single position. This can happen for several reasons, the most important of which is that couples can enter the Match looking for pairs of jobs; in 2020, for example, more than one thousand couples submitted rank-order lists consisting of pairs of jobs. There are also many more residency programs than an applicant can submit on a rank-order list, and many more applicants than programs can interview, so decisions have to be made beforehand that are more complicated than how to order the rank-order list. These complications may also add to confusion about the Match and about how the Match algorithm works.

"Computational studies of the Match nevertheless confirm that once interviews are over and an applicant has decided what programs to apply to, it is perfectly safe to submit a rank-order list that corresponds to the applicant’s true preferences [18]. To put it another way, there is no advantage to submitting a rank-order list that differs from an applicant’s preferences (and there is a danger in submitting a different rank-order list, because the Match will use the submitted list to make matches, in order).

"This fact doesn’t seem to have yet penetrated to everyone who participates in the Match [13]. For this reason, all those who advise medical students entering the Match should increase their advising efforts around this point.

"Note that the Match is only the final part of the transition to residency (or to fellowships). That transition starts with applications and interviews and includes various kinds of signals, like exam scores and transcripts and letters of reference. While the dominant-strategy property of the Match makes that part of the process strategically simple (that is, we can confidently advise students to submit rank-order lists in order of their true preferences), the other parts of the process (what rotations to take before applying, where to apply, how to conduct yourself at interviews) are not simple at all."

NRMP Position Statement On The (In)Feasibility Of An Early Match

 There has been some suggestion that dividing the resident match into early and late matches might be a way to address the congestion in applications and interviews that has bedeviled the transition from medical school to residency in recent years.  The NRMP now has a statement pointing out that there are serious problems with that idea.

NRMP Position Statement On The Feasibility Of An Early Match

"For the past eighteen months the National Resident Matching Program® (NRMP®) has been working closely with other national medical education organizations to examine the current state of the transition to residency. Conversations have focused on mitigating burdens for both applicants and programs in the selection and recruitment process and addressing uncertainty in the future of the interview cycle.

...

"Among the proposed solutions to current challenges in the transition to residency are calls for an early match. Specifically, NRMP has been asked to implement the Early Result and Acceptance Program (ERAP) pilot program proposed for Obstetrics and Gynecology, created through American Medical Association’s Reimagining Residency Grant, “Transforming the UME to GME Transition: Right Resident, Right Program, Ready Day One”. The stated goals of the ERAP pilot are to allow applicants to engage in strategic decision-making, reduce burden on programs while hypothesizing that the change will result in holistic review, and reduce necessary applications and interviews. ERAP calls for an early match to begin in September 2022 for the 2023 Match cycle. ERAP permits applicants to apply to a maximum of three programs in the early match with programs including up to 50% of their positions if they choose to participate. This statement outlines NRMP’s concerns about the structure of the ERAP pilot program, the lack of evidence supporting the proposed changes to the Match, the implications of an early match for the matching process, and preliminary findings of modeling an early match being conducted by experts in market design and the matching algorithm.

"The NRMP has reviewed the ERAP pilot program with consideration for whether changes to the matching process have the potential to inadvertently disadvantage Match participants. It is through that lens NRMP remains concerned with the following aspects of the ERAP pilot:

"Although voluntary, applicants may feel pressured to participate in an early match where up to half the available positions in a specialty may fill before the Main Residency Match® opens.

"There exists no mechanism for demonstrating how an early match will make visible less competitive applicants and those underrepresented in medicine, which is hypothesized in the project document.

"The proposed limit of three applications per applicant could force applicants to make compromises not present in the Match today. ...While the ERAP investigation team hypothesized that the application limit will increase holistic review by programs, there are no mandates to ensure that programs conduct holistic review nor are there restrictions on the number of applications programs may accept, interviews they may offer, or applicants they may rank. With no objective evidence to support the hypothesis, we cannot conclude that the proposed application limit would increase holistic review of applications.

"There exists no mechanism for safeguarding an applicant’s failure to match in the early match from programs as they enter the Main Residency Match, which could result in the applicant being viewed as less competitive.

"In addition to concerns about disadvantaging applicants, NRMP is mindful of possible behavior changes resulting from changes to the Match process that could affect Match outcomes for all Match participants.

• "The structure of an early match does not allow for mixed-specialty couples ranking or multispecialty individual ranking, which may cause applicants to reconsider their specialty choices, fundamentally changing their career path.
• "Programs may have insufficient information (e.g., clinical evaluations, MSPE, LORs) to evaluate applicants fully and fairly in the early match.
• Programs may see a surge in non-traditional applicants as the early match provides three opportunities to enter training through either the early match, the Main Residency Match, or SOAP®. This may result in an increased number of applications or applicants who may otherwise not select the specialty.
• Not matching in the early match is likely to increase the number of applications per individual in the Main Residency Match, as applicants enter a matching cycle with only half of the positions remaining available. This may increase stress, cost, and could adversely affect the wellness of applicants.

...

"it is important to first outline the core concepts of the match as a stable “market”. The Match was established in 1952, to solve a “congestion” problem in medical residencies involving applications, offers, and acceptances. In a May 2021 pre-submission working paper, Itai Ashlagi, Ph.D. and Alvin Roth, Ph.D. describe the consequences of congestion as “unraveling” where programs initially responded to congestion by making “exploding offers” that prevented applicants from considering many programs because they were pressed to accept an early offer, before knowing whether an offer from a more preferred program might be forthcoming if they waited. The authors note that NRMP’s matching process, in its current form, has four distinct properties that are relevant to managing the problems of congestion and unraveling and maintaining a stable matching market. Specifically, the NRMP matching process

"1. Is Uncongested: participants make all decisions (on Rank Order Lists) in advance, so there is no delay in processing offers, rejections, and acceptances, which is done by the computerized Roth-Peranson algorithm.

"2. Defers acceptances: preferences of applicants and programs are not finalized until all preferences have been considered, thereby producing stable matching: i.e., matching in which there are no “blocking pairs” of applicants and programs not matched to one another but who both would prefer to be.

"3. Promotes true preferences: it is safe for participants to state their true preferences when they submit their Rank Order Lists (ROLs).

"4. Establishes a “thick” market: most residency programs in most specialties participate in the NRMP Match, which also allows for multi-specialty applications and couple matching (including for mixed-specialty couples).

"The authors opine that an early match such as the proposed ERAP pilot followed by the Main Residency Match would not share three of the four important properties of the Match:

"1. An early match would dilute the thick market: not all positions would be available at the same time (and further, it would not allow applicants to express multi-specialty preferences, nor would it accommodate mixed-specialty couples).

"2. early match would introduce complicated strategic decisions into the formulation of ROLs: it would no longer be safe for participants to submit ROLs straightforwardly corresponding to their preferences.

"3. An early match would not produce a stable matching: there would be mutually disappointed blocking pairs of mismatched applicants and programs. This would also make it less safe to report ROLs that straightforwardly corresponded to preferences."

Match Day 2021 for medical residents

 Yesterday was Match Day, during Covid Year, and the aggregate data are reassuring that virtual (instead of in-person) interviews left the Match, in aggregate, much as before. The NRMP reports on match results, and Thalamus reports on interviews among the largest specialties.

Here's the NRMP press release:

Press Release: NRMP Delivers Strong Residency Match During Uncertain Times

"The 2021 Main Residency Match was the largest in NRMP history. There were 38,106 total positions offered, the most ever, and 35,194 first-year (PGY-1) positions offered, an increase of 928 (2.7%) over 2020. The growth in positions was supported by continued growth in the number of Match-participating programs. A record-high 5,915 programs were part of the Match, 88 more than 2020. In five years, the number of Match-participating programs has increased by 845 (16.7%), spurred in part by the completion of the transition to the single accreditation system for allopathic and osteopathic programs.

"Rather than faltering in these uncertain times, program fill rates increased across the board. Of the 38,106 total positions available, 36,179 filled, representing a 2.6 percent increase of filled positions over 2020. Of the 35,194 first-year positions available, 33,535 filled, representing a 2.9 percent increase of first-year filled positions. Those fill rates drove the percent of all positions filled from 94.6 to 94.9 percent and the percent of PGY-1 positions filled from 94.6 to 94.8 percent in 2021. There were 1,927 unfilled positions after the matching algorithm was processed, a decline of 71 (3.6%) compared to 2020.

...

"Percent of Applicants Matched to PGY-1 Positions Declines Slightly for Seniors; Rates Remain High. With all applicant groups demonstrating increases in the number of applicants submitting rank ordered lists of programs and ultimately matching to first-year PGY-1 positions, the overall percent matched declined modestly for some groups. Specifically, the percent of U.S. MD seniors matched to PGY-1 positions declined from 93.7 to 92.8, and the percent of U.S. DO seniors matched to PGY-1 positions declined from 90.7 to 89.1 percent. Non-U.S. citizen IMGs saw the largest decline, from 61.1 percent in 2020 to 54.8 percent in 2021. The unavailability of medical licensure examinations in the early stages of the pandemic coupled with permanent changes to the scoring and administration of those examinations by the end of 2020 created significant challenges for IMGs this year and likely contributed to the decline. Additionally, changes in clinical rotations may have affected match rates. The overall percent of applicants matched to PGY-1 positions declined from 80.8 to 78.5 percent."

***********

And here's a post from Thalamus, the interview managing service whose motto is "connecting the docs."

Explaining COVID’s Impact on the 2020-2021 Virtual Recruitment Season and NRMP Match Outcomes  March 19, 2021 by Team Thalamus

Here are their concluding remarks:

"1. The number of interview invitations stayed the same.

"2. The number of interviews completed by both applicants and programs went up. 

"3. The rate of interview cancellations decreased. 

"4. And while the candidates receiving the top 20% of interview offers completed more interviews than other candidates, overall applicants and programs both completed more interviews, thereby lengthening rank lists and providing each greater opportunity to match.

"While the slight decrease in match rates is due to a disproportionally larger number of applicants entering the match in comparison to the growth rates of the number of available residency positions, more candidates matched than ever before, because significantly more unique applicants received opportunities to interview.

"And therefore, overall, the match rate held steady as it has had in recent years, driven by its Nobel Prize winning application of the stable marriage algorithm.  

"Of course, there are several factors at play here including where applicants and programs enter preferences of where or whom they would like to match, respectively.  Similarly, given visa restrictions "IMGs were particularly disadvantaged this year more than usual, which lead to their larger resultant drop in their match rate.  There are continued challenges here including increasing the number of positions of available training positions to match a continued acceleration and growth of the applicant pool, and data can help expand upon this work. "

"Overall, the challenges of over-interviewing in GME was not greatly affected by the virtual interview process, and the greatly hypothesized “match crisis” appears to have been avoided.  Yet, this process shed significantly light on a continued systemic problem: Due to a lack of transparency, applicants continue to overapply, and residency programs continue to over-interview, creating a costly and anxiety-ridden process for all.  Data and technology can help change this for the future as COVID leaves its recognizable mark on the medical residency recruitment and match process."