Congestion in interviews for pediatric surgery fellowships

 In recent years, pediatric surgery has been a very popular subspecialty among Stanford surgical residents (upon completion of their 5 year general surgery residency).  A lot of time and treasure is spent interviewing for these relatively few fellowship positions: except in 2020 when interviews were remote, fellowship applicants pay for their own travel, etc.  And Stanford hospitals pull back on elective surgeries while the surgical residents are on the road interviewing.  Is so much interviewing inefficient?  Many think so, and here are some data.

Analysis of the pediatric surgery fellowship application process using the Thalamus™ database, by  Saunders Lin, Jason Reminick, Ephy Love, Benedict Nwomeh, Sanjay Krishnaswami, Journal of Pediatric Surgery, Volume 56, Issue 6, June 2021, Pages 1095-1100

"Background: The pediatric surgery fellowship interview process is costly and time intensive. We hypothesized that the increasing number of interviews completed by applicants and programs have become inefficient over time.

...

"Results: Our dataset included 34, 41, and 45 programs, which represented 81%, 91%, and 97% of all programs in 2018, 2019, and 2020, respectively. The median number of interviews completed per program remained constant, while the median number of interviews per applicant increased from 9.0 in 2018 to 13.0 in 2020. For 75% of programs, a program required only 4 or less candidates to fill their position. On average, 96% of program interviews do not result in a matched candidate.

"Conclusions: Programs offer interviews out of proportion to the number of positions available, and most applicants attend all interviews offered. We recommend an initial program goal of 20 interviews, which may be achieved by increased use of virtual interviews and the creation of program-level data on ideal applicant profiles.

...

"1. Introduction: With the advent of computer scheduling software and electronic interview platforms, data collection regarding the pediatric surgery fellowship interview process on a national level is now possible. One such platform is Thalamus, a scheduling software currently used for pediatric surgery fellowship interviews [1].

"The pediatric surgery match remains one of the most competitive fellowship application processes, with a total of 43 available positions for 78 applicants in the 2020 match cycle [2]. Published data show that extensive time and monetary resources are used every applicant cycle, with the average candidate spending around 14% of pretax salary and using up to three full weeks of residency days to complete interviews [3]. Despite these costs, however, programs continue to place considerable value on in-person interviews.

...

"2.1. Data source and methods: Thalamus is a comprehensive online and mobile Graduate Medical Education (GME) scheduling and communication software currently used in the pediatric surgery interview process. For applicants, features include a real-time scheduling system with online and mobile compatibility that allows applicants to self-schedule and instantly confirm their interview dates. From a program perspective, Thalamus is able to handle all interview confirmations, cancellations and rescheduling, and allows for comprehensive collection of applicant and program data both on the aggregate and individual levels.

"Thalamus was founded in 2013 and has been used in pediatric surgery since December 2016. The software is also currently used by more than 2200 residency and fellowship programs at more than 200 hospital systems across more than 100 specialties. It segments each institution by institutional ID and each program within each institution by program + ACGME ID (or a similar number for non-ACGME accredited programs). This is a cloud hosted database on the Microsoft Azure/SQL Server. Thalamus maintains several IRB approved/exempt research relationships with various specialties and other leadership organizations in Graduate Medical Education. This data is not shared between programs nor any other organization outside of Thalamus.

"We performed a retrospective investigation using Thalamus to identify population-level parameters regarding the pediatric surgery match between 2018 and 2020. This study was deemed exempt from approval by the Oregon Health and Sciences University Institutional Review Board as it did not contain patient data and applicant data was de-identified.

"3.2. Individual program and applicant data: With regards to individual program and applicant match data, the mean number of interviews offered and completed per program were similar in all three years (Table 2). The highest number of interviews a program completed was 44 in 2020. The number of interviews offered and completed per program have remained constant during the time-period. In contrast, both the mean and median number of interviews received and completed by applicants have increased. The median number of interviews completed per applicant increased 33.3% between 2018 and 2019 and an additional 12.5% between 2019 and 2020. Furthermore, the number of applicants who complete three or less interviews have been decreasing in the past three years: 25% in 2018, 20.6% in 2019, and 11.4% in 2020. Conversely, the number of applicants who completed more than 20 interviews has also been increasing in the past three years.

Signals and interviews in the transition from medical school to residency

Late last year I was interviewed by Dr. Seth Leopold, who is a Professor in the Department of Orthopaedics and Sports Medicine at the University of Washington School of Medicine, and Editor-in-Chief of the journal Clinical Orthopaedics and Related Research.   That interview has just appeared ahead of print on the journal's website: 

A Conversation with … Alvin E. Roth PhD, Economist, Game Theorist, and Nobel Laureate Who Improved the Modern Residency Match  by Leopold, Seth S. MD, Clinical Orthopaedics and Related Research: April 7, 2021 - Publish Ahead of Print - doi: 10.1097/CORR.0000000000001758

Here's one part of our Q&A:

Dr. Leopold:You once commented in a Not the Last Word column in CORR® that the Match might be improved if a bit more room could be made for candidates to send “signals” to programs that indicate particular interest[5]; if you could make one change to the Match right now to make it fairer all around, what would that change be?

Dr. Roth: I don’t yet know enough about the whole pre-Match process of applications and interviews to answer that confidently. I’m hoping to gain access to data that will illuminate more clearly how applications lead to interviews, and how interviews interact with other kinds of information to influence what rank-order lists are submitted by applicants and programs. Some of that process is surely in flux, between the pandemic causing interviews to be conducted remotely and the United States Medical Licensing Examination Step 1 going pass/fail. Signaling is a way to address miscoordination in interviewing (such as whether too many interviews are concentrating on too few candidates), but there are other ways the interview process might be broken that might better be addressed by other tweaks in how interviews are organized.

Dr. Leopold:I believe the study you’re proposing here would find a very attentive audience, both in medical schools and residency programs across the country, especially competitive ones like orthopaedic surgery. Based on other kinds of markets you’ve evaluated—I recognize I’m asking you to speculate—what do you think you might find here?

Dr. Roth: Presently, in at least some specialties, many interviews are conducted for each residency and fellowship position. It could be that interviews play a critical role in allowing programs and applicants to assess each other, regardless of the other information they may have. But it could also be that at least some interviews are being conducted “defensively,” because all the interviews that others are participating in make it hard for each program or applicant to predict how likely any interview will lead to a position being offered and accepted in the Match. So, it is possible that there is “too much” interviewing, in the sense that in perhaps predictable ways, some programs are interviewing some candidates they can virtually never hire, and some candidates they would never want to hire. Conversely, applicants are interviewing for some jobs they have hardly any chance of being offered, and some they sensibly think they won’t need to take. Of course, some things can be predictable even if they can’t be predicted by individual applicants and programs with the information they now have available. It might therefore be possible to suggest institutional reforms that would help reduce the uncertainty in deciding which interviews to offer. That might also reduce the number (and costs) of interviews. (In just such a way, the Match helped solve the problem of uncertainty involved in offers and acceptances, back when offers were exploding.) And there’s a possibility that fewer interviews could make everyone better off in terms of expectations, particularly if participants on both sides of the market will feel a reduced need to do so many interviews if everyone else reduces the number they do. But as you say, until we can look into this carefully, I’m just speculating.

Match Day 2021 for medical residents

 Yesterday was Match Day, during Covid Year, and the aggregate data are reassuring that virtual (instead of in-person) interviews left the Match, in aggregate, much as before. The NRMP reports on match results, and Thalamus reports on interviews among the largest specialties.

Here's the NRMP press release:

Press Release: NRMP Delivers Strong Residency Match During Uncertain Times

"The 2021 Main Residency Match was the largest in NRMP history. There were 38,106 total positions offered, the most ever, and 35,194 first-year (PGY-1) positions offered, an increase of 928 (2.7%) over 2020. The growth in positions was supported by continued growth in the number of Match-participating programs. A record-high 5,915 programs were part of the Match, 88 more than 2020. In five years, the number of Match-participating programs has increased by 845 (16.7%), spurred in part by the completion of the transition to the single accreditation system for allopathic and osteopathic programs.

"Rather than faltering in these uncertain times, program fill rates increased across the board. Of the 38,106 total positions available, 36,179 filled, representing a 2.6 percent increase of filled positions over 2020. Of the 35,194 first-year positions available, 33,535 filled, representing a 2.9 percent increase of first-year filled positions. Those fill rates drove the percent of all positions filled from 94.6 to 94.9 percent and the percent of PGY-1 positions filled from 94.6 to 94.8 percent in 2021. There were 1,927 unfilled positions after the matching algorithm was processed, a decline of 71 (3.6%) compared to 2020.

...

"Percent of Applicants Matched to PGY-1 Positions Declines Slightly for Seniors; Rates Remain High. With all applicant groups demonstrating increases in the number of applicants submitting rank ordered lists of programs and ultimately matching to first-year PGY-1 positions, the overall percent matched declined modestly for some groups. Specifically, the percent of U.S. MD seniors matched to PGY-1 positions declined from 93.7 to 92.8, and the percent of U.S. DO seniors matched to PGY-1 positions declined from 90.7 to 89.1 percent. Non-U.S. citizen IMGs saw the largest decline, from 61.1 percent in 2020 to 54.8 percent in 2021. The unavailability of medical licensure examinations in the early stages of the pandemic coupled with permanent changes to the scoring and administration of those examinations by the end of 2020 created significant challenges for IMGs this year and likely contributed to the decline. Additionally, changes in clinical rotations may have affected match rates. The overall percent of applicants matched to PGY-1 positions declined from 80.8 to 78.5 percent."

***********

And here's a post from Thalamus, the interview managing service whose motto is "connecting the docs."

Explaining COVID’s Impact on the 2020-2021 Virtual Recruitment Season and NRMP Match Outcomes  March 19, 2021 by Team Thalamus

Here are their concluding remarks:

"1. The number of interview invitations stayed the same.

"2. The number of interviews completed by both applicants and programs went up. 

"3. The rate of interview cancellations decreased. 

"4. And while the candidates receiving the top 20% of interview offers completed more interviews than other candidates, overall applicants and programs both completed more interviews, thereby lengthening rank lists and providing each greater opportunity to match.

"While the slight decrease in match rates is due to a disproportionally larger number of applicants entering the match in comparison to the growth rates of the number of available residency positions, more candidates matched than ever before, because significantly more unique applicants received opportunities to interview.

"And therefore, overall, the match rate held steady as it has had in recent years, driven by its Nobel Prize winning application of the stable marriage algorithm.  

"Of course, there are several factors at play here including where applicants and programs enter preferences of where or whom they would like to match, respectively.  Similarly, given visa restrictions "IMGs were particularly disadvantaged this year more than usual, which lead to their larger resultant drop in their match rate.  There are continued challenges here including increasing the number of positions of available training positions to match a continued acceleration and growth of the applicant pool, and data can help expand upon this work. "

"Overall, the challenges of over-interviewing in GME was not greatly affected by the virtual interview process, and the greatly hypothesized “match crisis” appears to have been avoided.  Yet, this process shed significantly light on a continued systemic problem: Due to a lack of transparency, applicants continue to overapply, and residency programs continue to over-interview, creating a costly and anxiety-ridden process for all.  Data and technology can help change this for the future as COVID leaves its recognizable mark on the medical residency recruitment and match process."

A call for capping residency interviews

 One clear symptom that the marketplace for medical residents is in crisis is the persistent drumbeat of suggestions for how to modify it.  The transition from medical school to residency has become congested, with many applications and interviews preceding the centralized clearinghouse known as the Match (which will yield its results on March 19).  

One way to treat a disease is to treat some of its most obvious symptoms. Here's the latest such proposal, to put a cap on the number of interviews. (Readers of this blog will wonder how those will be coordinated, and a number of proposals have been made including signaling, or a centralized interview match.)  

I'm hoping that data will become available to allow these proposals to be better evaluated, and perhaps to allow a market design that will deal with causes as well as symptoms.

Here's the latest, from Medscape.

Fixing the Match Crisis Starts With Capping Interviews  by Helen K. Morgan, MD

"Concern over the so-called "Match crisis" increases every cycle. This year, pandemic-related changes have shined a spotlight on the skewed distribution of interviews. Thanks to the shift from in-person to virtual interviews, applicants were no longer limited by travel and financial concerns. According to some experts, this has resulted in "top" candidates taking additional slots and subsequently reducing opportunities for others.

"Worry about residency interview distribution has surged, with letters of concern posted by the Association of American Medical Colleges and the American College of Surgeons. Before the start of this season, my colleagues and I modeled the potentially dire consequences of ob/gyn applicants "hoarding" too many interviews in an article published in the Journal of Surgical Education.

The residency application problems exposed by the pandemic aren't going anywhere without action. Establishing a cap on interviews is now clearly necessary...."

HT: Mike Rees

How do Zoom interviews change labor markets? Interview Hoarding by Manjunath and Morrill

 Suppose there were a pandemic that caused widespread lockdowns.  How might this influence the outcome of a labor market that was forced to switch from in-person, on-site interviews to remote interviews via Zoom or its equivalents?

Vikram Manjunath and Thayer Morrill take up the challenge, motivated by the case of the National Resident Matching Program, which matches new doctors to hospital residency programs. (Match Day is March 19 this year, so we may know some relevant things about how the pandemic influenced the Match not too long after.)

Interview Hoarding  by Vikram Manjunath and Thayer Morrill, February 22, 2021

Abstract: Many centralized matching markets are preceded by interviews between the participants. We study the impact on the final match of an increase to the number of interviews one side of the market can participate in. Our motivation is the match between residents and hospitals where, due to the COVID-19 pandemic, interviews for the 2020-21 season of the NRMP match have switched to a virtual format. This has drastically reduced the cost to applicants of accepting interview offers. However, the reduction in cost is not symmetric since applicants, not programs, bore most of the costs of in-person interviews. We show that if doctors are willing to accept more interviews but the hospitals do not increase the number of interviews they offer, no doctor will be better off and potentially many doctors will be harmed. This adverse consequence results from a mechanism we describe as interview hoarding. We prove this analytically and characterize optimal mitigation strategies for special cases. We use simulations to extend the insights from our analytical results to more general settings.

***********

Update: Manjunath, Vikram, and Thayer Morrill. "Interview hoarding." Theoretical Economics 18, no. 2 (2023): 503-527.

Preference Signaling for the Otolaryngology Interview Market

 From the journal The Laryngoscope, a thoughtful description of the growing interest in signaling for medical residency interviewing (which I believe will be implemented for Otolaryngology residency positions in the coming year).

Preference Signaling for the Otolaryngology Interview Market

C.W. David Chang MD  Steven D. Pletcher MD  Marc C. Thorne MD, MPH  Sonya Malekzadeh MD

First published: 06 October 2020 https://doi.org/10.1002/lary.29151

"The impact of the coronavirus disease 2019 pandemic extends beyond patient care and into graduate medical education (GME). The pandemic has created disarray in the residency application process. Visiting rotations and residency interviews—two cornerstones of the application cycle—are gone.

"Just as the pandemic has exposed healthcare disparities in medical care, it also shines a light on inequalities with GME. Even before the pandemic, many residency specialties observed a meteoric rise in the number of applications submitted by each applicant. In 2019, otolaryngology applicants submitted an average of 72 applications, an 80% rise over 15 years.1 This increase drives a cycle of programs receiving more applications and students feeling the need to apply more broadly to maintain competitiveness. Students with monetary resources are better able to mitigate match risk through prolific residency application and by traveling for away rotations to cultivate faculty advocates. Financially disadvantaged applicants may find it more difficult to amass influential social capital.

"With the deluge of applications, applicants are unable to distinguish themselves from the crowd. Such dilution impairs the applicants' abilities to credibly convey interests to programs. Similarly, the program director has a hard time selecting candidates from a pool of excellent applicants for interview.

"The interview is a limited resource. Selection committees often react to this scarcity by declining to interview qualified candidates they think (but do not really know) are unlikely to choose their program and instead interview candidates who they think (but do not really know) are more likely to accept an offer. This approach is inefficient.

"Preference signaling is an intriguing solution. Since 2006, the American Economic Association has operated a signaling service to facilitate job interviews for graduate students. This applicant‐initiated concept aligns goals of interested applicants with programs. Students send signals to up to two employers to indicate their interest in receiving an interview. In reviewing their outcomes, signals were found to increase probability of interview, especially for niche scenarios (nongraduate applicants, applications to liberal arts colleges, and small city locations).2 Signaling has received interest among medical residency specialties as well.3, 4

...

"The Otolaryngology Program Directors Organization (OPDO) Council has worked diligently with stakeholders to incorporate their input throughout the development process. We thank members and leaders of the academic otolaryngology community, including the Association of Academic Departments in Otolaryngology (AADO), the Society of University Otolaryngologists (SUO), and the greater community of program directors for their support. We appreciate the guidance and valuable insight from the Association of American Medical Colleges (AAMC)—specifically the Group on Student Affairs (GSA), the Committee on Student Affairs (COSA), and the Electronic Residency Application Service (ERAS)—along with the National Resident Matching Program (NRMP). We are hopeful that signaling will improve the residency interview selection process by facilitating the successful pairing of applicants with programs."

Pandemic disruptions in the market for medical residents

 The coronavirus pandemic and associated lockdowns and limitations have stressed a number of labor markets, including the one for new physicians.  Here's an article from the Journal of Surgical Education that suggests that, in a world of online interviewing, the number of interviews might usefully be capped. They also recommend signalling...

The Case for Capping Residency Interviews

Helen Kang Morgan, MD,*,1 Abigail F. Winkel, MD,† Taylor Standiford, BS,‡ Rodrigo Muñoz, MD,§ Eric A. Strand, MD,║ David A. Marzano, MD,* Tony Ogburn, MD,¶ Carol A. Major, MD,# Susan Cox, MD,⁎⁎ and Maya M. Hammoud, MD, MBA

J Surg Educ. 2020 Sep 14, doi: 10.1016/j.jsurg.2020.08.033 [Epub ahead of print] PMCID: PMC7489264

"As a result of the COVID-19 pandemic, residency programs will make an abrupt shift to virtual interviews in the 2021 residency application cycle.1 ... Medical students, medical schools, and residency programs have needed to react to sudden developments such as cancelled clinical electives, delayed or cancelled United States Medical Licensing Exams (USMLE),2 significant limitations on visiting student elective and sub-internship rotations,3 and changes in Electronic Residency Application Service (ERAS) timelines.4 Given this context, applicants may opt to increase their total number of residency applications as well as interviews accepted and completed, especially since they will no longer be limited by travel and cost deterrents.5, 6, 7 Likewise, residency programs are no longer logistically restrained to configure an applicant's interview schedule on a single day, and will have the ability to schedule interviews throughout multiple days and during non-business hours. In-person interactions provided by traditional interview day experiences have historically weighed heavily in determining mutual compatibility8 , 9; thus, both stakeholder groups will be looking to raise their chances of finding a match, including potentially increasing the number of interviews.

This is particularly troublesome given the current state of residency application processes, "fraught with misaligned stakeholder incentives.10, 11, 12, 13, 14, 15 Although the ratio of positions per applicant is higher now than ever before, the number of applications per applicant have risen.16 , 17 These numbers have increased rapidly in certain specialties, with the mean number of applications per applicant in obstetrics and gynecology (OBGYN) rising from 28 in 2010 to 66 in 2019.18 The consequences of application inflation are numerous and include decreased abilities for residency programs to perform holistic review of applicants with increased reliance on metrics such as USMLE scores. Residency programs also need to devote significant faculty and administrative time for the interview processes.19 The consequence of application inflation that will be of crucial importance this application cycle is the growing awareness that a small percentage of applicants has been receiving a disproportionate percentage of interview offers.20 , 21 In the era of virtual interviews, if these applicants choose to schedule all of their interview offers, there is a real potential for detrimental downstream effects to other applicants. This may also lead to a greater number of unfilled residency spots, with a larger number of programs and applicants required to enter into the Supplemental Offer and Acceptance Program. Given the paucity of data to inform best practices, there is a pressing need to model the potential effects of current application processes and applicant strategies in this disrupted application cycle."

...

"All stakeholders urgently need equitable solutions that address both individual and systems-level problems for this coming application cycle and beyond.30 Capping the number of interviews that an applicant can schedule could remedy 1 pressing flaw in current application interview processes. Implementing caps at the interview scheduling stage is preferable to capping at the application stage given the multiple complexities that must be considered such as DO and IMG status, and overall competitiveness. In addition, exceptions may need to be made for individuals participating in the Couples Match. The potential legal implications of mandatory interview caps are in the infancy of exploration. Capping interviews would likely not violate anti-trust laws given that applicants would still have the choice of where they would like to interview, however these issues would need to be further investigated. New measures such as preference signaling mechanisms30, 31, 32 need to be urgently considered in order for programs to be able to prioritize whom to offer interviews. The use of “tokens” would enable applicants to be able to convey interest to a set number of programs; this has been well-described in graduate PhD economics education literature.33 It will be imperative for “fit” to not become a proxy for decisions guided by unconscious bias,34 but instead, for principles of equity and inclusion to guide change during this time of accelerated change."

HT: Marc Melcher

More applications, from fewer applicants per position in Vascular Surgery residencies

Trends in the 10-year history of the vascular integrated residency match: More work, higher cost, same result
Katherine K.McMackin MD, Francis J.Caputo MD, Nicholas G.Hoell MD, JoseTrani MD, Jeffrey P.Carpenter MD, Joseph V.Lombardi MD
Journal of Vascular Surgery, Volume 72, Issue 1, July 2020, Pages 298-303

"During the last 10 years, the number of vascular surgery integrated residency spots rose from 9 to 60 per year. Most programs offer one spot per year; none offer more than two. The average number of applications received by programs rose from 17 applications in 2008 to 63.8 in 2017. The average rank list depth needed by programs to fill the spots has not increased (range, 2.5-5.1; standard deviation, 0.73). The proportional depth of the applicant pool decreased from 4.6 U.S. and Canadian applicants for every one residency spot in 2008 to 1.7 applicants for every one residency spot in 2017. Applicant quality metrics were available for 2 years (2014 and 2016). Step 1 scores (237/239), Step 2 scores (250/250), research experiences (3.7/4.2), and volunteer experiences (5.9/5.5) remained nearly unchanged. The number of contiguous ranks for matched applicants remained stable (12.3/12.8).

"Conclusions:
The current system promotes multiple inefficiencies, resulting in application glut. Fewer applicants are flooding programs with an increasing number of applications. More money is being spent on Electronic Residency Application Service applications without changes in the number needed to rank by applicants or programs to achieve a match. There is no improvement in the quality of the applicant. Should these trends continue, they represent an unsustainable model for resident selection."

Curbside interviewing / hiring joins curbside pickup in a social-distancing economy

The WSJ has the story:

Job Recruitment Adopts Social Distancing as Coronavirus Alters Practices
Employers rethink how they hire, trying remote onboarding and curbside job fairs to reduce risks
By James T. Areddy, June 22, 2020

"During the parking-lot screening, the couple and other job seekers sat in their cars and provided recruiters, wearing face masks, with basic information for possible callbacks on a range of jobs in light industry and those requiring skilled labor.

"It is an example of how social distancing, needed to stem the coronavirus pandemic, is altering job recruitment."

I join the NRMP board of directors

In the 1990's I worked closely with the NRMP on the resident match, and since then I've been following it from a distance.  Now I'll join their Board.  I imagine that during my term there may be an opportunity to look at how the overall system of applications, interviews and matching have co-evolved and adapted to each other.

Here's the NRMP press release:

 NRMP Board Of Directors Welcomes Outstanding Cohort Of New Members
Nobel Prize Recipient Dr. Alvin E. Roth Among Talented Slate of Elected Individuals
Washington, D.C., June 22, 2020 –

At its June meeting, the National Resident Matching Program® (NRMP®) Board of Directors elected five individuals for terms that begin July 1, 2020. The NRMP’s 19-member Board includes medical school deans, institutional officials, clinical program directors, resident/fellow physicians, medical students, and one public member. The appointment of Dr. Alvin E. Roth, the 2012 co-recipient of the Sveriges Riksbank Prize in Economic Sciences in Memory of Alfred Nobel, highlights the quality of the slate of new members.

“The individuals recently elected to the NRMP Board of Directors will bring an impressive mix of professional experiences and fresh perspectives to the governance of the organization,” said NRMP Board Chair Dr. Steven J. Scheinman. “They were selected from a deep pool of accomplished applicants, and we look forward to working with them on an array of initiatives.”

The term for directors is four years, with a maximum of two terms. The term for resident/fellow and student directors is two years. Listed alphabetically, the new members include:

Ricardo J. Boccardo Bello, M.D., General Surgery resident at the University of Massachusetts Medical School. A graduate of the Universidad Central de Venezuela in Caracas, he earned a Master of Public Health from the London School of Hygiene and Tropical Medicine and completed his PGY-1 General Surgery residency and postdoctoral research fellowship in Microsurgery Outcomes at Johns Hopkins University. While at Johns Hopkins, he received the Core Surgery Clerkship Outstanding Junior Resident Teaching Award.

Sydney Miller, Michigan State University College of Osteopathic Medicine student. Elected President of her class, Ms. Miller serves alongside faculty leaders as a member of the Dean’s Executive Board, which deliberates the college’s undergraduate and graduate medical education programs and other strategic initiatives. She earned her undergraduate degree in Human Biology at Michigan State University. Her interests include increasing awareness and appreciation of osteopathic medicine.

Alvin E. Roth, Ph.D., Senior Fellow at the Stanford Institute for Economic Policy Research and a Professor, by courtesy, for the school of Management Science and Engineering. He is also the George Gund Professor of Economics and Business Administration Emeritus at Harvard University. Dr. Roth’s academic expertise is in game theory, experimental economics, and market design, and he was co-recipient of the 2012 Nobel Prize in Economics for work which focused in part on the NRMP’s matching algorithm. He earned a doctoral degree in Operations Research from Stanford University.

Morgan Swanson, M.D./Ph.D. student at the University of Iowa Carver College of Medicine and College of Public Health. She has served with the Association of American Medical Colleges’ Organization of Student Representatives (OSR), most recently as one of 12 students selected to its Administrative Board. Ms. Swanson graduated from Iowa State University and serves as a member of the Editorial Board of the U.S. Centers for Disease Control and Prevention’s Morbidity and Mortality Weekly Report.

Christopher B. Traner, M.D., recent graduate of Neurology residency at Yale-New Haven Hospital and incoming Epilepsy/Neurophysiology fellow at Yale. He received the Yale Neurology Department’s Lewis Levy Award, presented to the PGY-2 resident who best exemplifies clinical excellence. He advised undergraduates as a Kaplan MCAT Advantage Plus Mentor while attending the University of Toledo College of Medicine. As a resident at Yale, Dr. Traner was a member of his program’s Interview Committee.

The newly elected Board members replace those whose terms conclude on June 30: Dr. Zaid Almarzooq, Cardiology fellow, Brigham and Women’s Hospital; Dr. Jessica Fried, chief resident, Diagnostic Radiology, Hospital of the University of Pennsylvania; Dr. Eriny Hanna, 2020 graduate of Vanderbilt University School of Medicine transitioning to an Emergency Medicine residency at Vanderbilt; Father Daniel Morrissey, O.P.; and Dr. Thomas Wickham, 2020 graduate of the University of New England College of Osteopathic Medicine transitioning to a Family Medicine residency at UMass Memorial Medical Center.

About NRMP
The National Resident Matching Program® (NRMP®) is a private, non-profit organization established in 1952 at the request of medical students to provide an orderly and fair mechanism for matching the preferences of applicants for U.S. residency positions with the preferences of residency program directors. In addition to the annual Main Residency Match® for almost 44,000 registrants, the NRMP conducts Fellowship Matches for more than 60 subspecialties through its Specialties Matching Service® (SMS®).

How will the pandemic affect the medical Match?

Some thoughts on the medical match in JAMA:

Potential Implications of COVID-19 for the 2020-2021 Residency Application Cycle
Maya M. Hammoud, MD, MBA1; Taylor Standiford, BS1; J. Bryan Carmody, MD, MPH2
JAMA. Published online June 03, 2020. doi:10.1001/jama.2020.8911

"Even before COVID-19, calls to reform the residency selection process were becoming more frequent.1,2 Many issues are related to the increasing number of programs to which applicants apply. In 2019-2020, applicants from US medical schools applied to an average of 65 programs, and international medical graduates (IMGs) applied to an average of 137 programs.3 This number of applications likely does not improve match rates and imposes a substantial cost on applicants and a potentially unmanageable load on program directors.

"It is possible that the disruptions caused by COVID-19 may result in an increase in the number of applications and further stress this already challenged system. Due to testing center closures, many applicants have been unable to take portions of the United States Medical Licensing Examination (USMLE). This is especially critical for IMGs, who must pass the Step 2 Clinical Skills Examination to obtain certification from the Educational Commission for Foreign Medical Graduates and apply to US residency programs. Additionally, medical schools have shortened clerkships, shifted to virtual rotations, and canceled away electives, all of which may reduce student opportunities to obtain meaningful faculty evaluations, letters of recommendation, and signal their interest to programs. Students will encounter significant uncertainty regarding how their applications will be evaluated and may respond by applying to even more programs.

Program directors may have difficulty identifying applicants to interview without use of traditional screening metrics. Yet, challenges will persist even after interviews are offered; if travel disruptions and social distancing persist into the interview season, programs may be unable to offer in-person interviews. Temporary solutions, such as conducting virtual interviews or waiving requirements for USMLE scores and letters of recommendation, will be necessary for the selection process to function. But these stopgap solutions may exacerbate existing problems with residency selection and lead to undesirable consequences. For instance, the use of virtual interviews could result in applicants participating in more interviews. Currently, the number of interviews an applicant attends is limited by time and travel expense, but these constraints will be less relevant with virtual interviews. Yet because many programs rely on the same screening metrics, many programs already overinvite the same pool of highly-qualified applicants, with just 7% to 21% of the applicant pool filling half of all interview slots in some specialties.4 The result of those applicants accepting more interview invitations could be an increase in both the number of unmatched applicants and unfilled programs.

Interview congestion in the Ophthalmology Residency Match

An ophthalmology residency program surveyed all its applicants on their experience in the match:

Current Applicant Perceptions of the Ophthalmology Residency Match
Michael J. Venincasa, MD; Louis Z. Cai, MD; Steven J. Gedde, MD; Tara Uhler, MD; Jayanth Sridhar, MD

JAMA Ophthalmology May 2020 Volume 138, Number 5 

"Hundreds of individuals apply for ophthalmology residency positions each year using the Centralized Application Services (CAS), administered by San Francisco Residency and Fellowship Matching Services (SF Match). Although the match rate remains relatively stable at approximately 75%, the mean number of applications submitted has risen from 48 in 2008 to 75 in 2019.1,2 In 2010, highly qualified applicants were advised to apply to between 10 and 20 residency programs,3 but more recent studies suggested a target of 45 applications for these applicants and more than 80 for applicants with less competitive qualification.2 The application process represents a considerable financial burden for applicants; in 2018-2019, the CAS application alone cost $685 to apply to 45 programs, which increased to $1910 for 80 programs. These high costs are not unique to ophthalmology. In emergency medicine, the cost of securing a residency position was estimated at $8312 in 2016.4

These trends also come with increasing administrative burden for residency programs tasked with reviewing rising numbers of applications. As a result, many programs have increasingly emphasized quantifiable cognitive measures, such as clinical grades and the US Medical Licensing Examination (USMLE) board scores.3 The USMLE Step 1 scores and Alpha Omega Alpha Honors Medical Society membership are factors with statistically significant associations with matching into an ophthalmology residency.5

...

"Respondents applied to a mean (SD) of 76.4 (23.5) ophthalmology residency programs, received 14.0 (9.0) invitations to interview (Figure 1), and attended 10.3 (4.4) interviews. 

...

"When respondents received an interview invitation without the involvement of a wait-list, they most commonly reported receiving the invitation between 3 and 4 weeks prior to the interview date (n = 87 [47.8%]). When instead receiving their invitation from a wait-list (n = 92 [49.7%]), the most common lead time was 1 to 2 weeks prior to the interview date (n = 43 [46.2%]), with 20 (21.5%) invitations arriving less than 1 week prior

...

"Many applicants struggled with scheduling conflicts with other residency programs, where interview dates overlapped or the desired date was filled at the time of invitation response. Certain dates were especially popular for residency programs, with 23 of 116 programs (19.8%) holding interviews on a single day during the 2018-2019 interview cycle.

***********

Note that the Ophtalmology residency match is run by SFMatch, not the larger NRMP, but the growing number of applications and interviews are common to both matching platforms.

Choosing who to interview in the sports medicine fellowship match

Little is known about what goes on between application to fellowship (or residency) programs, and rank-ordering for the relevant medical Match.  Here's an attempt at looking into the black box of choosing who to interview, and combining interview information with other information to determine rank order lists:

Factors Used by Program Directors in the Orthopedic Sports Medicine Fellowship Match
Travis Menge, Ashley Nord, Kendall Hamilton, Monica LaPointe and Peter J.L. Jebson
Journal of Surgery [Jurnalul de chirurgie], Volume 16:2, 2020

Abstract
Background: Obtaining an orthopedic sports medicine fellowship position is becoming more difficult, as the number of residents seeking post-graduate training continues to increase.
Objective: To identify factors that orthopedic surgery sports medicine fellowship program directors deem valuable in selecting applicants.
Methods: A web-based questionnaire was sent to all ACGME accredited sports medicine fellowship program directors in the United States in 2016. The questionnaire was designed to identify the most important criteria in selecting applicants for an interview, and ranking candidates to match into their program.
Results: Thirty-five of ninety-one program directors responded. The criteria for offering an applicant an interview were quality of recommendation letter, technical competence, and residency program reputation. Letters of recommendation that held the highest value were from the chief of sports medicine and another sports medicine surgeon in the department. The most important features of the interview were the applicant’s ability to articulate thoughts, the maturity of the applicant, and the ability of the applicant to listen well. The attributes deemed most important in high ranking a candidate included the applicant’s commitment to hard work, quality of the interview, and quality of letters of recommendation.

Applications and interviews prior to matching in Orthopaedic Surgery

It takes many applications and interviews to match for orthopaedic surgery residencies...

Matching in Orthopaedic Surgery
Chen, Antonia F. MD, MBA; Secrist, Eric S. MD; Scannell, Brian P. MD; Patt, Joshua C. MD, MPH
Journal of the American Academy of Orthopaedic Surgeons: February 15, 2020 - Volume 28 - Issue 4 - p 135-144
doi: 10.5435/JAAOS-D-19-00313

Abstract
In 2016, 1,137 fourth year medical students submitted applications for orthopedic surgery residency positions. Students applied to an average of 79 programs, resulting in in a total of 89,846 applications being submitted for 727 first year residency positions. This ratio of 124 applications per position is two SDs above the mean relative to other medical specialties. The average applicant for orthopaedic surgery residency attends 2.4 away rotations, as attending 2 away rotations increases an applicant's odds of matching, and submits 83 applications. This excessive number of applications overburdens programs, subjects applicants to considerable costs, and diminishes the quality of fit between interviewees and programs. Eighty-three percent of program directors use step 1 United States Medical Licensing Examination scores as a screening tool to decrease the number of applications necessary for review. The average matched applicant attended 11.5 interviews, and Step 1 scores, research productivity, and Alpha Omega Alpha (AOA) status can be used to predict the number of applications necessary to obtain 12 interviews. AOA membership has the strongest influence on interview yield. Applicants report spending an average of approximately $7,000 on the interview process, and 72% borrow money to cover these costs. Post-interview contact, although forbidden by the National Resident Matching Program , has been reported by 60% to 64% of applicants.

Interviews and matching--Echenique, González, Wilson, and Yariv

Interviews are an important feature of many matching markets. Here's a recent paper about that:

Top of the Batch: Interviews and the Match
Federico Echenique* Ruy González† Alistair Wilson‡ Leeat Yariv§ ¶
February 14, 2020

Abstract
Most doctors in the NRMP are matched to one of their most-preferred internship programs. Since various surveys indicate similarities across doctors’ preferences, this suggests a puzzle. How can nearly everyone get a position in a highly-desirable program when positions in each program are scarce? We provide one possible explanation for this puzzle. We show that the patterns observed in the NRMP data may be an artifact of the interview process that precedes the match. Our analysis highlights the importance of interactions occurring outside of a matching clearinghouse for resulting outcomes, and casts doubts on analysis of clearinghouses that take reported preferences at face value.

More proposals for reducing applications and interviews before medical resident matching

If the number of proposals for reform is an indicator of a brewing problem (and I think it is), it's time to think about the various application and interview processes that precede the NRMP resident match.  Here are three more...

J. Bryan Carmody (2020) Applying Smarter: A Critique of the AAMC Apply Smart Tools. Journal of Graduate Medical Education: February 2020, Vol. 12, No. 1, pp. 10-13.
https://doi.org/10.4300/JGME-D-19-00495.1

"Medical students today submit more residency applications than ever before. This trend is costly for students and imposes a substantial burden on program directors. Yet, despite a steady increase in the number of applications submitted per applicant, overall match rates have not improved.1 Put another way, applicants could collectively apply to fewer programs than they do now—and enjoy essentially the same overall match rate.

To assist students in determining the optimal number of residency programs to which they should apply, the Association of American Medical Colleges (AAMC) unveiled Apply Smart in 2016. The website notes that “there is a point where the relationship between the number of applications submitted and the likelihood of entry into a residency changes,” and suggests that students should consider limiting their applications at this point of diminishing returns.2 Responses to Apply Smart have been positive, with deans and program directors praising the tools and encouraging their use in counseling medical students.3–6

At first glance, the Apply Smart analyses seem highly informative. Yet, closer inspection reveals methodologic issues that introduce bias and suggest the need for improvements.
...
"Overapplication is costly, for applicants and programs alike. Given the incentives for medical students to overapply, it is unclear whether informational strategies alone can curtail overapplication. Until graduate medical education leaders are willing to support application caps or a fundamental restructuring of the Match to better allow signaling between applicants and programs,13 it is imperative that informational strategies present unbiased data that can aid students in applying to an appropriate number of programs. We cannot be satisfied for students to Apply Smart—we need to help them apply smarter."

************

Joseph G. Monir (2020) Reforming the Match: A Proposal for a New 3-Phase System. Journal of Graduate Medical Education: February 2020, Vol. 12, No. 1, pp. 7-9.
https://doi.org/10.4300/JGME-D-19-00425.1

"The National Resident Matching Program (NRMP) was originally devised in 1952 to bring order to the chaotic residency application process.1 It currently has 2 phases: The Match and the Supplemental Offer and Acceptance Program (SOAP). Applications are submitted through the Electronic Residency Application Service (ERAS). While this is a significantly superior system to its predecessor, it is not flawless. As both residency applicants and positions have become more competitive, the flaws of the current system are becoming increasingly problematic for all involved parties.

"Despite evidence that applying to a greater number of programs does not improve Match rates,2 applicants feel pressured to apply to more and more programs to avoid going unmatched. In 2018, each applicant submitted a mean of 90.6 applications (mean of 60.3 for US graduates and 136.4 for international graduates).3 Applicants feel forced to accept interviews at programs which they have minimal interest in attending, to the detriment of other genuinely interested students. This, in turn, pressures other applicants to do the same lest they be disadvantaged. Whipple et al confirmed this vicious cycle using their computer model for competitive residencies, where they found that applying to the maximum number of programs “led to a poor result for the majority of students when all applicants undertook the strategy.”4

"As a result of this vicious cycle, residency programs are inundated with an average of 996 applications per program5 and have few reliable methods of identifying which of those applicants would seriously consider training there. Interviewing applicants who have no desire to attend that program is a costly endeavor. Gardner and colleagues found that, when accounting for both material and personnel costs, programs spent on average $18,648 ± $13,383 per open position and $1,221 ± $894 per interviewee.6

"Applying so broadly is costly for applicants as well. Fogel et al7 found that 84% of applicants believed residency interviews were too expensive, with 64% of applicants spending at least $2,500, and those entering competitive fields spending considerably more. In a study of orthopedic surgery applicants, for example, interview costs averaged $7,119 (range $2,500–$15,000).8 These figures do not include the cost of the applications themselves or externships, making the total residency application cost significantly higher.

"It is clear that the current system is fraught with inefficiency and that all involved parties would be better served by a system where applicants only apply to the programs they would seriously consider attending.

"The 3-Phase System
Transitioning the Match to a 3-phase process as outlined below could address the aforementioned problems.

"Phase 1
This phase would run from September through December and would function the same as the Match is currently. However, applicants would be limited in the number of positions for which they could apply. The results of Phase 1 would be e-mailed to applicants in the beginning of January. Most interviews already occur within this time frame, so most programs could continue using the systems they have in place. The majority of positions would be anticipated to fill in Phase 1.

"Phase 2
This phase would run from January through March. Those who do not match in Phase 1 would proceed to Phase 2. This would again function like the current Match, with applicants applying to open programs, attending in-person interviews, and submitting a rank list. There would be no cap on the number of Phase 2 applications, allowing applicants to apply broadly. The results of Phase 2 would be e-mailed to applicants in March. The majority of unfilled positions from Phase 1 would be expected to fill in Phase 2.

Phase 3
This phase would be the current SOAP week, with daily rounds of interviews and offers. The number of applicants and positions entering the SOAP would likely decrease significantly."
**********

Plast Reconstr Surg. 2019 Feb;143(2):634-639. doi: 10.1097/PRS.0000000000005254.
Solving Congestion in the Plastic Surgery Match: A Game Theory Analysis.
Molina Burbano F1, Yao A, Burish N, Ingargiola M, Freeman M, Stock J, Taub PJ.

Abstract
Plastic and reconstructive surgery is among the most competitive specialties in the residency match. Applicants seeking to maximize their chances of a successful match often submit numerous applications to the National Residency Matching Program. It is not uncommon for those applying to plastic and reconstructive surgery to apply to every program. The high application volume imparts significant time and financial burden for applicants and programs alike. Furthermore, it makes distinguishing between applicants with a genuine interest in a specific program and those who are merely hoping to improve their chances vastly more difficult. The authors sought to characterize trends in the match rate, as the number of integrated plastic and reconstructive surgery programs continues to increase. Furthermore, they reviewed the literature on game theory for possible solutions to residency application congestion. The authors propose the use of the game theory model to explain the observed results and show why an application limit is the most reasonable approach to address this issue.

Early admissions for medical residencies? An anguished response to the increasing numbers of applications and interviews.

Here's a proposal to introduce something like the early admissions programs that have become common in college admissions (where they cause new problems while partially addressing the issue of too many applications...)

Improving the Residency Application and Selection Process
An Optional Early Result Acceptance Program
Maya M. Hammoud, MD, MBA1; John Andrews, MD2; Susan E. Skochelak, MD, MPH2
JAMA. Published online January 23, 2020. doi:10.1001/jama.2019.21212

"from 2011 to 2019, applications per applicant increased from 15.2 to 34.8 for family medicine, from 30.5 to 61.3 for obstetrics and gynecology, and from 21.6 to 51.9 for psychiatry.1 Similarly, the number of applications received by each program also has increased across all specialties, some by more than 200%. For example, from 2011 to 2019, the mean number of applications received by family medicine programs increased from 76 to 251 and received by psychiatry programs increased from 115 to 446.1

"A cycle involving increased numbers of applications and increased reliance on standardized testing has resulted in behavioral changes in both applicants and residency programs. Currently, senior medical students spend large amounts of time and money during the last year of medical school applying to an increasing number of programs and meeting the demands of the residency application process.
...
"Meanwhile, to process the high volume of applications received, programs are likely relying more on quantitative metrics, such as United States Medical Licensing Examination (USMLE) Step 1 scores, for screening.
...
"A new approach to help decrease the number of applications by giving students the option of an early application and expeditious result match program may be helpful. One possible approach might be an early result acceptance program (ERAP), in which students would be permitted to apply to a maximum of 5 programs, and programs would be limited to filling half of all their available spots."

Interviews in the Medical Physics residency match (too many, and what to do about it..)

Medical physics has a residency match, and like other residency matches it is suffering from (apparently) too many interviews.

Here's a signed editorial in the Journal of Applied Clinical Medical Physics (one correlate of it being an editorial rather than a paper is this:
"Received: 4 September 2019 | Accepted: 5 September 2019")

Some considerations in optimizing the Medical Physics Match
by Richard V. Butler1, John H. Huston1, George Starkschall2
1Department of Economics, Trinity University, San Antonio, TX,
2Department of Radiation Physics, The University of Texas MDAnderson Cancer Center, Houston, TX

"In 2018, 79% of graduates of CAMPEP‐accredited graduate programs were accepted into residency programs.4 Consequently, to ensure a match, candidates interview at many programs. There is also a harmful feedback mechanism here. As applicants apply to more programs, the acceptance rate at each program declines. Consequently, applicants may apply to even more programs to increase their perceived probability of acceptance into a program. This is costly for the candidates in terms of travel expenses, and costly for the interviewing faculty in terms of time away from research, clinic, and teaching.
...
"Because the problem of optimal applications is an economics problem, there has been a search for solutions and a developing literature on the subject. Balter et al.5 show that limiting the number of applications candidates can submit is superior to limiting the number of applications a program can evaluate. Entering an application limit into the Gale/Shapley algorithm that underlies the matching process, the authors conclude that "the optimal limit in the number of applications balances the tradeoff between being unmatched and gaining a better match in the aggregate, and the benefit can be considerable if the graduates'preferences over the positions are not very correlated.
...
"Another approach to a solution is "signaling." A program would be permitted to notify a small number (somewhere between three and five) of applicants prior to interviews that it is seriously interested in them. This gives the applicant useful information about his/her chances at that particular program and so makes the benefit function a bit less fuzzy. Because the problem in medical physics seems to be more at the interview stage than the initial application stage, some form of signaling by institutions offering residencies might help reduce uncertainty so that at least some applicants could focus on the places where they have good chance and pass on visits to some of their more marginal options."
***********

Here are earlier posts on the medical physics residency match.

History job market conference interviews are history

Inside Higher Ed has the story on the history job market (which they conflate with the AEA's recent decision to try to eliminate interviews in hotel rooms):

Killing the Conference Interview
American Historical Association ends annual meeting interviews and American Economic Association ends single hotel room interviews.
By Colleen Flaherty

"It's official: the American Historical Association will stop supporting first-round job interviews at its annual meeting.

"The group floated the idea this spring, citing a decline in registered departmental searches -- from 270 for the 2005 conference to 20 this year -- and a desire to take the meeting in new directions.

"After hearing overwhelming positive feedback from members, the AHA Council voted to end the 70-year-old tradition."

*************

I'm not intimately familiar with the History job market, but for economists, I think the tradition of interviewing at the January meetings has had a good effect on the job market, helping to coordinate timings, reduce costs, and provide a thick early part of the market.  I hope that we won't be starting on the road to moving interviews elsewhere and (particularly) at earlier and more diffuse times.

More chaos in the medical resident interviewing process

Not only do graduating medical students go on (too) many interviews for residency positions, but the process by which interviews are offered and accepted is chaotic. (I'm reminded of the process of offering and accepting actual positions in the 1940's, before a matching clearinghouse was first developed...)

Here's a paper from the August 2019 issue of the Journal of Graduate Medical Education, by three concerned doctors at Northwestern University, who describe the situation and then offer their own suggestions. (I admire the description, and am skeptical that the suggestions are radical enough to change the incentives responsible for the current congestion...)

A Challenge to Disrupt the Disruptive Process of Residency Interview Invitations
Matthew R. Klein, MD, MPH
Sandra M. Sanguino, MD, MPH
David H. Salzman, MD, MEd

"Residency programs may also contribute to the sense of urgency to secure an interview by offering
more invitations than available interview slots. The rise in the number of applications per
applicant, while certainly multifactorial, is facilitated in part by the ease of online interview scheduling...
...
"In the current environment, applicants who receive an interview invitation while they are in a situation where interruption is impossible—such as taking a clerkship examination, assisting in an operating room, performing a bedside procedure, participating in a difficult conversation with a patient or family, rounding on the wards, or simply being asleep as a result of time zone differences—are effectively penalized. A delay in response may result in the inability to schedule a favorable date or an increase in the cost to travel to an interview at a less convenient time, or it may preclude an interview altogether if all interview slots have been taken.
...
"We propose a 2-step process that we believe balances the need for residency programs to recruit
applicants and schedule interviews with the goal of preserving an environment that allows students to
maintain their commitment to their educational and patient care responsibilities. We recommend the
following process
Step 1: A program informs an applicant of an invitation for an interview. In that communication,
the program shares the interview dates and indicates the date and time when online interview
scheduling will become available. This allows medical students to plan in advance when they
need to be briefly excused from clinical or educational responsibilities to attend to interview
scheduling
Step 2: At the time indicated in the initial communication, and no earlier than the day after
that communication, the online interview scheduling system opens to applicants
...
"In an effort to further decrease pressure to reply immediately to an interview invitation, programs
should also ensure that the number of initial invitations does not exceed the number of interview
slots available. Additional interview offers from a wait-list could occur as needed based on unclaimed
interview slots or subsequent cancellations."