Jobs and spouses in Denmark

 Matching is both consequential and difficult: it is how we sort into jobs and careers, and marriages and families.  Here's a paper that looks at the relationship between those two matching markets, taking advantage of the fact that Danish medical grads get random priorities, which determine their early-career job matches.

Causal Effects of Early Career Sorting on Labor and Marriage Market Choices: A Foundation for Gender Disparities and Norms  by Itzik Fadlon, Frederik Plesner Lyngse & Torben Heien Nielsen, NBER WORKING PAPER 28245, DOI 10.3386/w28245 ISSUE DATE December 2020, REVISION DATE July 2022

Abstract: "We study whether and how early labor market choices determine longer-run career versus family outcomes differentially for male and female professionals. We analyze the physician labor market by exploiting a randomized lottery that determines the sorting of Danish physicians into internships across local labor markets. Using administrative data spanning ten years after physicians’ graduations, we find causal effects of early-career sorting on a range of life cycle outcomes that cascade from labor market choices, including human capital accumulation and occupational choice, to marriage market choices, including matching and fertility. The persistent effects are entirely concentrated among women, whereas men experience only temporary career disruptions. The evidence points to differential family-career tradeoffs and the mentorship employers provide as channels underlying this gender divergence. Our findings have implications for policies aimed at gender equality in outcomes, as they reveal how persistent gaps can arise even in institutionally gender-neutral settings with early-stage equality of opportunity."

"placement into medical internships—i.e., physicians’ first jobs—is governed in Denmark by a purely randomized lottery ... As we verify, students with the best lottery ranks,who are the ones that choose  first,are  effectively  unrestricted in their  choices  and  are assigned  their  highest priorities,whereas students with the worst lottery ranks,who are the ones that choose last and well after their choice sets have narrowed, are assigned their lowest priorities.

...

"we exploit a novel dataset that combines the formal lottery data we have digitized with a range of administrative datasets on all medical doctors in Denmark. ... we can link households using spousal and parent-child linkages to investigate family formation and fertility. Together, the data allow us to study a wide range of lifecycle choices, in both the labor market and the marriage market, which provides us with the unique advantage of conducting a comprehensive analysis on the broad potential causal effects of early careerson work versus family tradeoffs. The data allow us to track our sample over a long period of up to ten years after the treatment.

...

"We show that the women who have more children due to the treatment also invest less in human capital, and that their location decisions reflect family considerations as they show increased propensity to live near grandparents. This is consistent with women crowding out long-run career goals for more family-oriented choices as  a  result  of  unfavorable early-career placements.  In  comparison,  men engage in career-oriented actions in response to unfavorable placements,which help them fend off potential adverse effects. .... the data are strongly inconsistent with differential preferences  over  entry-level  positions as  a  channel. Males  and  females  reveal  very  similar  aggregate preferences in their choices over entry-level markets and positions.

Medical resident sleep time reduces harmful errors

 Here's a new study of the effect on patient safety of the limitation on resident work hours to no more than 16 hour shifts, which was in effect in the US from 2011 to 2017.

National improvements in resident physician-reported patient safety after limiting first-year resident physicians’ extended duration work shifts: a pooled analysis of prospective cohort studies  BMJ Quality & Safety Published Online First: 10 May 2022. doi: 10.1136/bmjqs-2021-014375by Matthew D   Weaver1,2, Christopher P Landrigan1,3,4, Jason P Sullivan1, Conor S O'Brien1, Salim Qadri1, Natalie Viyaran1, Charles A Czeisler1,2, Laura K Barger1,2

Abstract: Background The Accreditation Council for Graduate Medical Education (ACGME) enacted a policy in 2011 that restricted first-year resident physicians in the USA to work no more than 16 consecutive hours. This was rescinded in 2017.

Methods "We conducted a nationwide prospective cohort study of resident physicians for 5 academic years (2002–2007) before and for 3 academic years (2014–2017) after implementation of the 16 hours 2011 ACGME work-hour limit. Our analyses compare trends in resident physician-reported medical errors between the two cohorts to evaluate the impact of this policy change.

"Results 14 796 residents provided data describing 78 101 months of direct patient care. After adjustment for potential confounders, the work-hour policy was associated with a 32% reduced risk of resident physician-reported significant medical errors (rate ratio (RR) 0.68; 95% CI 0.64 to 0.72), a 34% reduced risk of reported preventable adverse events (RR 0.66; 95% CI 0.59 to 0.74) and a 63% reduced risk of reported medical errors resulting in patient death (RR 0.37; 95% CI 0.28 to 0.49).

"Conclusions These findings have broad relevance for those who work in and receive care from academic hospitals in the USA. The decision to lift this work hour policy in 2017 may expose patients to preventable harm."

***

"From 2003 to 2011, the Accreditation Council for Graduate Medical Education (ACGME) limited residents in their first postgraduate year to a maximum of 30 consecutive work hours, including 6 hours for continuity of care and educational activities (30 hours 2003 ACGME work-hour limit).2 Subsequent evaluations found that shifts of 24 or more hours were associated with increased odds of fatigue-related medical errors and preventable adverse events (PAEs),3 percutaneous injuries4 and motor vehicle crashes.5 A randomised controlled trial found that limiting first-year resident physicians to 16 consecutive work hours significantly improved resident alertness and patient safety.6 7 Altogether, a body of evidence accumulated suggesting that reducing or eliminating shifts longer than 16 hours did not negatively impact resident education and likely improved patient safety and resident quality of life.8 Subsequently, the Institute of Medicine of the National Academies (IOM) reviewed the available evidence and concluded that it was unsafe for any resident physician to provide clinical care for >16 consecutive hours without sleep.9 10 In response, the ACGME issued new work-hour regulations on 1 July 2011, limiting first-year resident physicians to a maximum of 16 consecutive work hours and emphasising a commitment to patient safety and mitigation of fatigue-related risks (16 hours 2011 ACGME work-hour limit).11

"The response within the medical community to the 16 hours 2011 ACGME work-hour limit was mixed.12 Many stakeholders expected the changes to diminish the educational experience.13 The increased frequency of patient handoffs raised concerns, as physician-to-physician handoffs have historically been non-standardised and prone to error.14 In addition, the work-hour limitations were not accompanied by an increased number of residency slots, leading to work compression and a shift in some responsibilities to other clinical providers,15 as well as concerns about resident physician understaffing. Several studies of the 16 hours 2011 ACGME work-hour limit found that it had no impact on hospital-level mortality or mortality following surgical procedures.16–18 In light of these studies and opposition to the work-hour limit from within the medical community, the ACGME lifted the 16-hour limit as of 1 July 2017, again allowing first-year resident physicians to be scheduled for 24 hours of continuous work, plus up to 4 hours for care transitions (28 hours 2017 ACGME work-hour limit)."

Medical residents organize to bargain collectively

Medpage has the story:

A Tale of Two Hospitals: Resident Mistreatment and Exploitation— The pandemic has ignited a wave of resident unionization by Jessie Ge, MD, and Rayyan Kamal, MD 

Stanford Health Care

"In December 2020, Stanford Health Care rolled out a COVID-19 vaccination plan that excluded nearly all 1,400-plus resident and fellow physicians from eligibility. For the residents and fellows who had been working tirelessly at the frontlines of the pandemic, this was the last straw. In our view, this didn't appear to be a one-time mistake of some algorithm, but a continued pattern of employer neglect and exploitation of our labor. That's when we knew we had to start organizing for real power.

"A little over a year later, on February 22, 2022, we had gathered supermajority support from our co-workers to unionize with the Committee of Interns and Residents (CIR). However, our request for voluntary recognition from Stanford was denied. In the face of potential anti-union tactics from Stanford, we communicated through our own website and tweetorials, re-centering our mission for a meaningful voice and focusing on widespread support, not only from our own residents, but also from our local elected officials and other labor unions.

"The Stanford nurses were some of our biggest supporters -- we shared a need for better working conditions to deliver better patient care. They have been unionized for over 50 years, and their collective strength protected them from being stretched even thinner during the pandemic. ... Despite a tough employer campaign to defeat our union, we won our election 835 to 214 this week.

...

Greater Lawrence Family Health Center

"On March 15, 2022, we officially won our union. While the hospital's efforts did manage to turn a few who had initially supported the union, 72% of our unit voted in favor of CIR in an election where every single resident participated. A 100% turnout rate is unprecedented even in a small program. It speaks volumes to how connected we've become throughout this process. We are in solidarity with house staff everywhere going public, filing for unionization, and taking the next step toward social progress and justice for academic medicine.

"While we don't know where this new movement in resident unionization will ultimately lead, we know we're headed in the right direction. Doctors are people too and we must rehumanize medicine for everyone -- both our patients and the people who make hospitals run.

"Jessie Ge, MD, is a fourth-year urology resident at Stanford Health Care. Rayyan Kamal, MD, is a second-year family medicine resident at Greater Lawrence Family Health Center.

****************

Related:

Wednesday, December 22, 2021

Will unionization at universities change the United Auto Workers?

Signaling for Otolaryngology residency programs

 Here's a report on the use of signaling for residency positions, from the Oto match.

Pletcher, Steven, Chang, C.W., Thorne, Marc, MD, MPH & Malekzadeh, Sonya. (2022). The Otolaryngology Residency Program Preference Signaling Experience. Academic Medicine, 97, 664-668. https://doi.org/10.1097/ACM.0000000000004441

"The average applicant to Otolaryngology-Head and Neck Surgery in the 2021 residency application cycle applied to more than 50% of otolaryngology programs nationwide, submitting 77 applications, 1 a 34% increase over 5 years. 2 This surge has made it difficult for residency programs to holistically review applications and has limited opportunities for applicants to stand out to programs of particular interest.

...

"A formal preference signaling process provides all applicants with access to a known and stable quantity of signals. Through this process, at the time of application submission, students send a signal to indicate to a defined number of residency programs their particular interest in those programs. Such signals allow students to stand out to their favored programs and allow programs to receive a list of highly interested applicants. To our knowledge, this approach has not been used previously in the residency application process. Yet, articles advocating for signaling exist in the otolaryngology literature, 5-7 and the methodology, rationale, and results of preference signaling for graduates of economics PhD programs applying for faculty positions have been described. 8

"An OPDO working group, comprising the 4 authors, drove the establishment of a signaling process. In the spring of 2020, we held a series of meetings and webinars to engage stakeholders in the development and implementation of a signaling process. We included students, program directors, and specialty societies, such as the Society of University Otolaryngologists and the Association of Academic Departments of Otolaryngology Otolaryngology Chairs Organization. Additional discussions with the Association of American Medical Colleges, the Electronic Residency Application Service (ERAS), and the National Resident Matching Program also took place.

"In hindsight, establishing consensus across stakeholders proved to be the most challenging hurdle to successful implementation of our signaling process. Stakeholders had to accept this change and the inherent risks of implementing a "never before in medicine" process. These discussions, however, also provided a critical opportunity to refine our proposal and create an educational ecosystem that accepted this signaling process.

...

"All otolaryngology residency programs attested to the code of conduct, and none opted out of the signaling process. By October 21, 2020, the date that applications were released to programs, 611 students had submitted applications to otolaryngology residency programs, 559 applicants had submitted a Match list including an otolaryngology program, and 558 applicants had participated in the signaling process. Of 119 non-military otolaryngology residency programs, 118 received at least 1 signal. The number of signals received per program ranged from 0 to 71 with a mean of 22 (standard deviation 17) and a median of 16 

...

"Program directors most commonly reported using signals as a tiebreaker for similar applications and as part of an initial application review algorithm. One program required a signal to offer an applicant an interview.

...

"Applicants reported applying to a mean of 77 programs (standard deviation 21), including their 5 signaled programs. The rate of receiving an interview offer was significantly higher for signaled programs (58%, 670/1,150) compared with both nonsignaled programs (14%, 2,394/16,520; P < .001) and the comparative nonsignal program (23%, 53/230; P < .001; see Figure 2). To assess the impact of signaling across the spectrum of applicant competitiveness, we divided applicants into quartiles based on their overall likelihood of receiving an interview offer. Signals had a significant impact (P < .001) on interview offers across all quartiles (see Figure 3).

...

"The magnitude of signal impact we found likely represents both an increased rate of interview offers from signaled programs and a decreased rate of interview offers from nonsignaled programs. In contrast to our 5 signals, the American Economic Association provides 2 signals for graduates of economics PhD programs applying for faculty positions, suggesting that signal scarcity preserves its value and intent. 8 Decreasing the number of signals would force applicants to narrow their list of programs of primary interest and would disincentivize signaling "dream" programs. By increasing the number of signals, the lack of a signal becomes an indication of disinterest. If enough signals are provided, signaling could have a similar impact on the application process as an application cap. The ideal number of signals then must be explored.

"Our data demonstrated that signaling allowed applicants to influence their likelihood of receiving an interview offer from programs of particular interest. Given the distribution of signals received across programs, we believe that signaling also improved the distribution of interview offers among applicants, which could mitigate interview hoarding. 

...

"While applicants appear to benefit significantly from signaling, they also bear the responsibility of targeting their signals appropriately.

...

"Multiple specialties have expressed interest in adopting a similar program. However, otolaryngology is not representative of all medical or surgical specialties. It is a small, competitive surgical subspecialty with a 63% Match rate and no unmatched residency slots in the 2021 cycle. While the impact of signaling may vary significantly outside of these parameters, we are optimistic that the benefits will carry over to other specialties. Incorporating a signaling option within ERAS would facilitate both wide adoption and further analysis of such a process."

Signaling in medical residencies in the upcoming 2023 season

 One approach that is being adopted in applications for medical residencies is to allow signals of interest.

Here's an announcement from ERAS, the Electronic Residency Application Service® :

Supplemental ERAS® application for the ERAS 2023 cycle

"With the support of their specialty leadership organizations, programs from the following specialties may participate in the supplemental ERAS application for the 2023 ERAS season. "

************

Here's a 2010 paper that includes some of the experience of signaling in the Economics Ph.D. market. We used (and still use) only 2 signals, and we definitely rejected the idea of gold and silver signals...

The Job Market for New Economists: A Market Design Perspective, by Peter Coles, John Cawley, Phillip B. Levine, Muriel Niederle, Alvin E. Roth, and John J. Siegfried (Journal of Economic Perspectives—Volume 24, Number 4—Fall 2010—Pages 187–206)

Experimental Economics in the Tradition of John Kagel (video)

 In October there was an in-person celebration of John Kagel, which I was delighted to participate in, in Tucson, Arizona. (It was my first in-person conference since the beginning of the Covid pandemic, during a brief window of optimism.) Now it's been posted on YouTube by the hosts, at the Economic Science Lab of the University of Arizona:

Keynote lecture of Professor Alvin Roth at the Workshop in Honor of John Kagel, Tucson, Arizona, October 2021

My talk was called Experimental Economics in the Tradition of John Kagel, and I began by explaining this photograph, which has John in the middle.

I eventually focused on how the following experiment helped shape a good deal of practical market design:

Kagel, John H. and A.E. Roth, "The dynamics of reorganization in matching markets: A laboratory experiment motivated by a natural experiment," Quarterly Journal of Economics, February, 2000, 201-235.

And I concluded by giving John some unnecessary advice as he embarks on his tenth decade.

**********

You can see more from the 2021 North-American Economic Science Association Conference (including the above video) here.

An Interview Match for medical residents and fellows--a preliminary proposal

 There is a lot of concern in the graduate medical education community that too much time and treasure is being spent on too many unproductive interviews prior to the submission of rank order lists for the Match.  Here's discussion of a proposal for an interview match, to precede the interview stage before the actual NRMP Match.

Explaining a Potential Interview Match for Graduate Medical Education, by Irene Wapnir, MD; Itai Ashlagi, PhD; Alvin E. Roth, PhD; Erling Skancke, MS; Akhil Vohra, PhD; Irene Lo, PhD; Marc L. Melcher, MD, PhD, J Grad Med Educ (2021) 13 (6): 764–767.  https://doi.org/10.4300/JGME-D-20-01422.1

"Residency and fellowship candidates are applying to more programs to enhance their chances of securing interviews and matching favorably. The COVID-19 pandemic has shifted interviews to video formats, which lowers interview-associated costs for applicants but may further increase application numbers.1  While a candidate's application to a training program communicates some interest in the program, the relative amount of interest is obscured when candidates apply to large numbers of programs. We suspect that, as a result, programs host large numbers of low-yield interviews.

"The number of interviews is steadily increasing, and there is widespread agreement on the need to ease congestion in the pre-Match evaluation process.2  Proposals to reduce this burden include signaling (organized, centrally-controlled protocol for limited communication of interest),3–5  capping the number of applications or the number of interviews,6,7  and an early acceptance matching program as in college admissions.8,9 

"We propose another solution, an “interview match” to address the expanding number of interviews.10  An interview match enables candidates and programs to express preferences privately by ranking their interview choices individually or in tiers. This may ease congestion in the “marketplace,” reduce costs for candidates, favor interviews that are more likely to lead to a match in the final Match, and avoid interviews unlikely to convert to a match. An interview match algorithm would match based on the same “deferred-acceptance” algorithm currently used by the National Resident Matching Program but adapted to a “many-to-many” setting where candidates and programs receive multiple interviews."

Virtual interviewing increased the number of interviews by (and of) candidates for medical residencies

 Here's an article on virtual interviewing of candidates for medical residencies (about which, see also this post about Erling Skancke's work on interviewing):

Beshar I, Tate WJ, Bernstein D. Residency interviews in the digital era, Postgraduate Medical Journal Published Online First: 05 October 2021. doi: 10.1136/postgradmedj-2021-140897

"Halfway through the interview cycle, however, questions were raised about system-level equity of virtual interviewing. In December 2020, the Association of American Medical Colleges (AAMC) released an open letter citing ‘a maldistribution in residency interview invitations’, with the ‘highest tier applicants hav(ing) so many interviews’.15 The letter, addressed to both programme directors and students, called on programmes ‘to recruit a diverse pool of residents’ and encouraged students ‘to release[e] some interviews if you are holding more than needed, allowing your fellow students to access those interview opportunities’.15 Medical school deans began encouraging competitive students to forgo interviews. In the words of the Dean of students at the UC College of Medicine: ‘They need to identify a reasonable number to have a successful match and release others so their peers who need them can have them’.6 By some estimates, programmes invite the same pool of highly qualified applicants, with just 7%–21% of the applicant pool filling 50% of the interview slots in some specialties.16 Meanwhile, a survey of plastic surgery programme directors demonstrated nearly one in three increased the number of interview offers per available residency spot.

"At our institution—Stanford School of Medicine—and as applicants of the 2020–2021 cycle ourselves—we saw the effect of this firsthand. We administered a survey to all students participating in the match process in both the 2019 (in-person) and 2020 (virtual) years. In the survey, respondents identified the residency programme or programmes to which they applied as well as the number of interviews they attended. Of Stanford’s 2019 and 2020 graduating classes, 83.7% (n=72) and 62.3% (n=62), respectively, completed our survey.

"Of the 2019 applicants, 97.2% (n=70) reported residency interviews that required airline travel, compared with 0% of the 2020 applicants. The median number of interviews for the 2019 applicants was 8, compared with 14 for the 2020 applicants. Across the 2 years analysed, all fields showed an increase in the number of interviews accepted (table 1). Of the four fields with the most applicants, the largest per cent change was in anaesthesia (244%), followed by ophthalmology (216%), internal medicine (144%) and psychiatry (128%), respectively. On average, across all specialties, the number of accepted interviews changed by 160%.

What we know about labor market interviews

 Erling Skancke is on the job market from Stanford this year. You should interview him. If you do, you'll learn a lot about interviews, which are the subject of his job market paper, which provides a lot of new insight into an important practical set of isssues.

Skancke, Erling, Welfare and Strategic Externalities in Matching Markets with Interviews (November 10, 2021). Available at SSRN: https://ssrn.com/abstract=3960558 or http://dx.doi.org/10.2139/ssrn.3960558

Abstract: Recent debate in the medical literature has brought attention to issues with the pre-match interview process for residency and fellowship positions at hospitals. However, little is known about the economics of this decentralized process. In this paper, I build a game-theoretic model in which hospitals simultaneously decide on which doctors to interview, in order to learn their preferences over doctors. I show that increased interview activity by any hospital imposes an unambiguous negative welfare externality on all other hospitals. In equilibrium, both hospitals and doctors may be better off by a coordinated reduction in interview activity. The strategic externality is more subtle, and conditions are derived under which the game exhibits either strategic complementarities or substitutes. Moreover, an increase in market size may exacerbate the interview externalities, preventing agents from reaping the thick market benefits that would arise in the absence of the costly interviews. This effect increases participants' incentives to match outside of the centralized clearinghouse as markets become thicker, jeopardizing the long-term viability of the clearinghouse. The model also provides new insights into several market design interventions that have recently been proposed.

NRMP Statement On Interviewing

 The National Resident Matching Program (NRMP) has a statement on interviewing, which precedes the NRMP match for new American doctors, and that reflects concerns that the interviewing process has become congested.

NRMP Statement On Interviewing

"The National Resident Matching Program® (NRMP®) has heard the concerns of learners and programs in the medical education community about the interview process and wants to encourage equitable practices among applicants and programs. As such we have developed the following recommendations. Although especially true during the enduring times of the pandemic, the recommendations align with the NRMP’s long-standing commitment to maintaining a fair, efficient, reliable, and transparent process for all. Recommendations also foster well-being among all parties.

"NRMP Recommendations for Programs:

1. Programs should conduct all interviews virtually for the 2021-2022 cycle.

2. Programs should extend interview offers that equal, not exceed, the total number of available interview slots.

3. Applicants should be given a minimum of 48 hours to respond to an interview offer.

"NRMP Recommendations for Applicants:

1. Applicants should make timely decisions about interview offers and promptly notify programs in which they are no longer interested, freeing up interview slots.

2. Applicants should give ample, adequate (e.g., one week) notice to programs in the event they change their minds and decline accepted interviews."

Nervousness and confusion about residency applications

 Medpage Today has the story, which has been active on medical twitter for a while--some residency candidates are finding that their applications are incomplete, and blame the common application service ERAS, which in response reports that incomplete applications are the result of errors by the candidates.

Is ERAS Glitch Impeding Residency Interviews?  — Several students are missing letters of recommendation, but AAMC calls reports "unsubstantiated"

"MedPage Today spoke to several residency applicants who believe they experienced ERAS tech issues when submitting their applications. A majority of these applicants requested anonymity due to fear of retribution.

While many say a potential bug is not surprising, students are frustrated that they cannot pin down when "their documents went missing -- and whether or not it jeopardized their chances of matching with some programs.

"In each case, the glitch is almost identical: an applicant's letters of recommendation (or other documents) were uploaded to ERAS, and the applicant was certain they assigned the right materials to each program before they hit submit. But weeks later, applicants checked the platform to see that these documents had never been assigned nor delivered to the right programs -- rendering their applications incomplete."

************

Related:

Med Students Filed Nearly 70 Residency Applications Each— "I don't think this is sustainable at all" by Amanda D'Ambrosio, MedPage Today October 27, 2021

"Across all medical specialties, this cycle MD applicants submitted an average of 68 applications, DO applicants submitted 92, and international medical graduates submitted 139, according to data provided to MedPage Today by the Association of American Medical Colleges (AAMC).

"Medical graduates applying to orthopedic surgery in the 2022 cycle submitted an average of 88 applications -- the highest across all medical specialities. Candidates to urology, otolaryngology, and dermatology -- some of the most competitive specialties -- all applied to around 80 programs each.

"More than 46,000 medical graduates have applied for the 2022 residency match so far: approximately 23,000 MDs, 8,000 DOs, and 15,000 international graduates. The preliminary data from ERAS -- the centralized system that medical graduates use to apply to residency -- includes all applications submitted through the beginning of October."

Selection of Dutch doctors by lottery

 Here's a recent article describing the once and future Dutch selection of medical students by lottery. (I believe that residency positions may also have or have had selection by lottery.)

Rationales for a Lottery Among the Qualified to Select Medical Trainees: Decades of Dutch Experience by Olle ten Cate, J Grad Med Educ (2021) 13 (5): 612–615. https://doi.org/10.4300/JGME-D-21-00789.1

"The Dutch Lottery for Medical School Selection

"A lottery, as a method to determine who will be admitted to medical school or residency, may sound an absurd proposition to many. A lottery appears to devalue motivation, disregard high effort and talent, and randomly block freedom of career choice. However, The Netherlands has decades of experience with this method. The Dutch government applied a lottery system nationally for admission to all medical schools in 1972. This system was abandoned in 2017 after an appeal but will now be reinstalled in 2023 as a legitimate procedure for the selection of students.

"Until 1972, the admission to Dutch medical schools, which have a 6-year program not preceded by baccalaureate education, was freely accessible for applicants with the proper secondary schooling (note that the Dutch government pays for most of medical education). When applicants increased in number and their costs became substantial, the Dutch government introduced a numerus fixus, a restricted total number of positions, derived from predictions of future physician need. After years of debate, politicians settled on a “weighted lottery” system for admissions. The average score on a national final secondary examination determined the weighting. Students with an outstanding score would triple their chances compared to those with a just-pass score. Declined candidates could reenter the lottery for 2 subsequent years. For decades schools and the public were generally satisfied with this procedure to determine the one-third of all applicants (on average across decades) for whom there was space at a Dutch medical school. The lottery procedure was smoothly conducted by a government agency, until 1996. That year an outstanding high school graduate was turned down 3 times and appealed the decision. Political and societal anger arose and led to a gradual replacement of the lottery, initially with a local qualitative selection process in parallel with a national weighted lottery. In 2 decades, the national lottery system was abandoned altogether; legislation prohibited medical schools from using a lottery as of 2017. Surprisingly, in 2020, a parliamentary majority voted to allow schools to use a lottery system, and thus reinstalled lottery processes as a legitimate method of selection. The law is effective in 2023."

*************

Related:

Lottery Admissions System in the Netherlands, FL Meijler, J Vreeken - Science, 1975 - science.org

The weighted lottery in the admission to ‘restricted’ university studies in the Netherlands,      

Ben Wilbrink

The use of lottery systems in school admissions

C Stasz, C Von Stolk, R Europe, C Rand, TW Sutton - 2007 - researchgate.net

Unraveling of Forensic Psychiatry fellowships makes participants paranoid on both sides of the market

 Where lawyers speak naturally about rules, psychiatrists don't shy away from talking about feelings, and the current disorder in the market for forensic psychiatry fellowships is making many participants miserable.

The Fellowship Application Process Must Be Reformed  by Octavio Choi, Journal of the American Academy of Psychiatry and the Law Online September 2021, 49 (3) 300-310; DOI: https://doi.org/10.29158/JAAPL.210088-21

"These are unhappy days in the world of forensic psychiatry fellowship programs. Here is the crux of the problem: too much product, not enough customers. Agapoff and colleagues report that for the 2016–2017 academic year, forensic psychiatry fellowships achieved a 56 percent fill rate, with 65 residents spread over 44 programs offering a total of 116 positions.1 Since then, the number of forensic programs has continued to grow, up to 48 ACGME-accredited programs offering 127 positions in 2018–2019. Seventy-three of those positions were filled, equating to a 57.5 percent fill rate.2 Things were better in the older days. According to ACGME records, in 2012–2013 there were about the same number of active residents (70) in just 36 programs.3

"The implications are clear: forensic fellowship programs are increasingly desperate to recruit a small number of applicants. From the perspective of program directors such as me, the rational strategy to pursue in this situation is to identify promising applicants early and try to sign them up before anyone else can get to them. Indeed, in recent years, fierce competition has led programs to make earlier and earlier offers that are time-limited (also known as the “exploding offer”). Paranoia is high. Given the nontransparent nature of most transactions in the applications process (no one really knows what anyone else is up to), and lack of objective referees, it only takes the slightest hint of malfeasance for outrage and fear of missing out to amplify.

"The overriding fear of many program directors is that they will not fill their available positions. In addition to bruised egos, being left with open positions means contracts will be left unfilled, possibly leading to cancellation and, ultimately, reduction or elimination of programs. Literally, to not fill risks death (of the program). The imperative, then, is to avoid not filling at all costs.

"On the other side is a paradox. For applicants, low fill rates should translate into a buyer's market, yet because the market is unregulated, the current system inflicts much suffering on them. As one recent applicant succinctly described the process: “it's a hot mess.” Competition by programs for the limited number of applicants has led to earlier and earlier offers being made with shorter and shorter times to decide; too short to adequately assess and receive offers from other programs. Indeed, the whole point of an exploding offer, from the program's point of view, is to curtail assessment of other programs by forcing applicants to make decisions before they might otherwise be ready. In marketing parlance, the idea is to pick up a bargain by taking a good off the market before it can be fairly priced.

...

"The failure of the current system is not about program directors being bad people. It is about the fragile nature of voluntary agreements during difficult times. The math is simple. If each program director has a 95 percent chance of behaving ethically over the course of the applications cycle, and there are 48 programs, there is only a .95 to the forty-eighth power probability (=8.5% chance) that all 48 directors will behave ethically in any given year. A single program director acting less-than-fully ethically is enough to kickstart a paranoid feedback loop that devolves into chaos: “If program X isn't playing by the rules, I don't see why I need to keep playing by the rules, especially if it's going to hurt me.”

"But note that system failure does not even require any actual unethical behavior; all that is required is the perception that others are behaving unethically, a perception that is encouraged to flourish in the context of desperation and lack of transparency"

Market design in Tokyo

 Fuhito Kojima and Hiroaki Odahara report on some of the projects presently underway at the University of Tokyo Market Design Center (UTMD), which include matching for child care, for medical residencies, and for internal labor markets.

Kojima, F., Odahara, H. Toward market design in practice: a progress report. Japanese Economic Review, (2021). https://doi.org/10.1007/s42973-021-00103-w

Abstract: In recent years, many developments have been made in matching theory and its applications to market design. This paper surveys some selected topics from this research area and describe our own work. We also describe the newly established University of Tokyo Market Design Center (UTMD), which works as a vehicle for practical implementation.

Redesigning the market for American medical residencies

 The medical community has spent a good deal of effort over the past two years conferring about difficulties in the transition from medical school to residency, i.e. in transiting from what is called undergraduate medical education (in medical school, recognized with a degree) to graduate medical education (working in a hospital, recognized by Board certification in some medical specialty).  

The resulting document, below, represents a lot of work by a lot of people, and is worth a close look. (It's long: 276 pages, three dozen or so more or less specific recommendations, public comments, etc.). 

I'll have more to say about some specifics in future posts, in due time. 

The Coalition for Physician Accountability’s Undergraduate Medical Education-Graduate Medical Education Review Committee (UGRC): Recommendations for Comprehensive Improvement of the UME-GME Transition

"Overview: In 2020, the Coalition for Physician Accountability (Coalition) formed a new committee to examine the transition from undergraduate medical education (UME) to graduate medical education (GME). The UME-GME Review Committee (the “UGRC” or the “Committee”) was charged with the task of recommending solutions to identified challenges in the transition. These challenges are well known, but the complex nature of the transition together with the reality that no single entity has responsibility over the entire ecosystem has perpetuated the problems and thwarted attempts at reform.

"Using deliberate and thoughtful methods, the UGRC spent 10 months exploring, unpacking, discussing, and debating all aspects of the UME-GME transition. The Committee envisioned a future ideal state, performed a rootcause analysis of the identified challenges, repeatedly sought stakeholder input, explored the literature, sought innovations being piloted across the country, and generated a preliminary set of potential solutions to the myriad problems associated with the transition. Initial recommendations were widely released in April 2021, and feedback was obtained from organizational members of the Coalition as well as interested stakeholders through a public  call for comment. This feedback was instrumental to refining, altering, and improving the recommendations into their final form. The UGRC also responded to feedback by consolidating similar recommendations, organizing them into more cogent themes, and sequencing them to guide implementation. "

Some history of the National Resident Matching Program

 Here's a short history of the resident match, including the recent merger of MD and DO student applicants, and some thoughts about current issues.

Acad Med. 2021 Aug; 96(8): 1116–1119.

The Single Match: Reflections on the National Resident Matching Program’s Sustained Partnership With Learners  by Zaid I. Almarzooq, MBBCh, Heather A Lillemoe, MD, Ebony White-Manigault, MPH, Thomas Wickham, DO, MPH, and Laurie S. Curtin, PhD

Here's the concluding paragraph:

"The NRMP has come a long way, but we recognize that the residency selection process still is fraught with stress and uncertainty, albeit for reasons different from those that prompted creation of the Match. Application inflation, debt, and a disproportionate reliance on licensure exam scores have contributed to a climate that makes the transition to residency perhaps as stressful as when the Match was created nearly 70 years ago. 18,19 However, as the NRMP moves beyond achievement of the Single Match milestone and we reflect on the organization’s history of responding to the needs of its constituents, we believe the NRMP will continue to evolve and identify innovative and meaningful ways to address learner needs. We hope learners of all kinds value that commitment and stand ready to support the NRMP’s efforts to continually improve the transition to residency."

An interview in (not on) Clinical Orthopaedics and Related Research, largely about resident matching

 For those  readers of this blog who may have missed the May issue of Clinical Orthopaedics and Related Research, here's an interview by the editor

A Conversation with ... Alvin E. Roth PhD, Economist, Game Theorist, and Nobel Laureate Who Improved the Modern Residency Match  by Leopold, Seth S. MD, Clinical Orthopaedics and Related Research, Volume 479(5), May 2021, p 863-866   doi: 10.1097/CORR.0000000000001758

The interview includes this long answer to a short question about the resident Match:

Seth S. Leopold MD: Many readers will dispute the idea that the Match is resistant to strategic manipulation (“gaming the system”); why do you believe it is, and why do you think this perception persists?

Alvin E. Roth PhD: That question requires a somewhat complicated answer. The Match is built around an idea of how to organize a simple labor market, and that idea had to be adapted to the complex structure of the modern medical labor force. A simple labor market would be one in which graduating medical students each seek a single position, positions are well described in advance, and applicants and residency programs can each rank-order all of their possible matches; that is, applicants can rank programs and programs can rank applicants. That simple market can be modeled mathematically, and it can be shown that a deferred acceptance algorithm with applicants proposing makes it a dominant strategy for all applicants to submit rank-order lists corresponding to their true preferences. (A dominant strategy is one in which regardless of what rank-order lists others submit, no applicant can do better than to rank residency programs in order of his or her true preferences. For instance, your chance of getting your second-choice program if your first choice rejects you is exactly the same as if you had listed your second choice first.)

"That’s a theoretical answer about a market that is quite a bit simpler than the modern market for residencies. The deferred acceptance algorithm for that simple market was studied by Gale and Shapley [8], for which Shapley shared the 2012 Nobel Prize in Economics. (I had earlier shown that in a simple market, applicants can’t profitably manipulate their rank-order lists [16].)

"The actual modern market for residencies differs from that simple market in several ways. For one thing, not all applicants are seeking a single position. This can happen for several reasons, the most important of which is that couples can enter the Match looking for pairs of jobs; in 2020, for example, more than one thousand couples submitted rank-order lists consisting of pairs of jobs. There are also many more residency programs than an applicant can submit on a rank-order list, and many more applicants than programs can interview, so decisions have to be made beforehand that are more complicated than how to order the rank-order list. These complications may also add to confusion about the Match and about how the Match algorithm works.

"Computational studies of the Match nevertheless confirm that once interviews are over and an applicant has decided what programs to apply to, it is perfectly safe to submit a rank-order list that corresponds to the applicant’s true preferences [18]. To put it another way, there is no advantage to submitting a rank-order list that differs from an applicant’s preferences (and there is a danger in submitting a different rank-order list, because the Match will use the submitted list to make matches, in order).

"This fact doesn’t seem to have yet penetrated to everyone who participates in the Match [13]. For this reason, all those who advise medical students entering the Match should increase their advising efforts around this point.

"Note that the Match is only the final part of the transition to residency (or to fellowships). That transition starts with applications and interviews and includes various kinds of signals, like exam scores and transcripts and letters of reference. While the dominant-strategy property of the Match makes that part of the process strategically simple (that is, we can confidently advise students to submit rank-order lists in order of their true preferences), the other parts of the process (what rotations to take before applying, where to apply, how to conduct yourself at interviews) are not simple at all."

NRMP Position Statement On The (In)Feasibility Of An Early Match

 There has been some suggestion that dividing the resident match into early and late matches might be a way to address the congestion in applications and interviews that has bedeviled the transition from medical school to residency in recent years.  The NRMP now has a statement pointing out that there are serious problems with that idea.

NRMP Position Statement On The Feasibility Of An Early Match

"For the past eighteen months the National Resident Matching Program® (NRMP®) has been working closely with other national medical education organizations to examine the current state of the transition to residency. Conversations have focused on mitigating burdens for both applicants and programs in the selection and recruitment process and addressing uncertainty in the future of the interview cycle.

...

"Among the proposed solutions to current challenges in the transition to residency are calls for an early match. Specifically, NRMP has been asked to implement the Early Result and Acceptance Program (ERAP) pilot program proposed for Obstetrics and Gynecology, created through American Medical Association’s Reimagining Residency Grant, “Transforming the UME to GME Transition: Right Resident, Right Program, Ready Day One”. The stated goals of the ERAP pilot are to allow applicants to engage in strategic decision-making, reduce burden on programs while hypothesizing that the change will result in holistic review, and reduce necessary applications and interviews. ERAP calls for an early match to begin in September 2022 for the 2023 Match cycle. ERAP permits applicants to apply to a maximum of three programs in the early match with programs including up to 50% of their positions if they choose to participate. This statement outlines NRMP’s concerns about the structure of the ERAP pilot program, the lack of evidence supporting the proposed changes to the Match, the implications of an early match for the matching process, and preliminary findings of modeling an early match being conducted by experts in market design and the matching algorithm.

"The NRMP has reviewed the ERAP pilot program with consideration for whether changes to the matching process have the potential to inadvertently disadvantage Match participants. It is through that lens NRMP remains concerned with the following aspects of the ERAP pilot:

"Although voluntary, applicants may feel pressured to participate in an early match where up to half the available positions in a specialty may fill before the Main Residency Match® opens.

"There exists no mechanism for demonstrating how an early match will make visible less competitive applicants and those underrepresented in medicine, which is hypothesized in the project document.

"The proposed limit of three applications per applicant could force applicants to make compromises not present in the Match today. ...While the ERAP investigation team hypothesized that the application limit will increase holistic review by programs, there are no mandates to ensure that programs conduct holistic review nor are there restrictions on the number of applications programs may accept, interviews they may offer, or applicants they may rank. With no objective evidence to support the hypothesis, we cannot conclude that the proposed application limit would increase holistic review of applications.

"There exists no mechanism for safeguarding an applicant’s failure to match in the early match from programs as they enter the Main Residency Match, which could result in the applicant being viewed as less competitive.

"In addition to concerns about disadvantaging applicants, NRMP is mindful of possible behavior changes resulting from changes to the Match process that could affect Match outcomes for all Match participants.

• "The structure of an early match does not allow for mixed-specialty couples ranking or multispecialty individual ranking, which may cause applicants to reconsider their specialty choices, fundamentally changing their career path.
• "Programs may have insufficient information (e.g., clinical evaluations, MSPE, LORs) to evaluate applicants fully and fairly in the early match.
• Programs may see a surge in non-traditional applicants as the early match provides three opportunities to enter training through either the early match, the Main Residency Match, or SOAP®. This may result in an increased number of applications or applicants who may otherwise not select the specialty.
• Not matching in the early match is likely to increase the number of applications per individual in the Main Residency Match, as applicants enter a matching cycle with only half of the positions remaining available. This may increase stress, cost, and could adversely affect the wellness of applicants.

...

"it is important to first outline the core concepts of the match as a stable “market”. The Match was established in 1952, to solve a “congestion” problem in medical residencies involving applications, offers, and acceptances. In a May 2021 pre-submission working paper, Itai Ashlagi, Ph.D. and Alvin Roth, Ph.D. describe the consequences of congestion as “unraveling” where programs initially responded to congestion by making “exploding offers” that prevented applicants from considering many programs because they were pressed to accept an early offer, before knowing whether an offer from a more preferred program might be forthcoming if they waited. The authors note that NRMP’s matching process, in its current form, has four distinct properties that are relevant to managing the problems of congestion and unraveling and maintaining a stable matching market. Specifically, the NRMP matching process

"1. Is Uncongested: participants make all decisions (on Rank Order Lists) in advance, so there is no delay in processing offers, rejections, and acceptances, which is done by the computerized Roth-Peranson algorithm.

"2. Defers acceptances: preferences of applicants and programs are not finalized until all preferences have been considered, thereby producing stable matching: i.e., matching in which there are no “blocking pairs” of applicants and programs not matched to one another but who both would prefer to be.

"3. Promotes true preferences: it is safe for participants to state their true preferences when they submit their Rank Order Lists (ROLs).

"4. Establishes a “thick” market: most residency programs in most specialties participate in the NRMP Match, which also allows for multi-specialty applications and couple matching (including for mixed-specialty couples).

"The authors opine that an early match such as the proposed ERAP pilot followed by the Main Residency Match would not share three of the four important properties of the Match:

"1. An early match would dilute the thick market: not all positions would be available at the same time (and further, it would not allow applicants to express multi-specialty preferences, nor would it accommodate mixed-specialty couples).

"2. early match would introduce complicated strategic decisions into the formulation of ROLs: it would no longer be safe for participants to submit ROLs straightforwardly corresponding to their preferences.

"3. An early match would not produce a stable matching: there would be mutually disappointed blocking pairs of mismatched applicants and programs. This would also make it less safe to report ROLs that straightforwardly corresponded to preferences."

Congestion in interviews for pediatric surgery fellowships

 In recent years, pediatric surgery has been a very popular subspecialty among Stanford surgical residents (upon completion of their 5 year general surgery residency).  A lot of time and treasure is spent interviewing for these relatively few fellowship positions: except in 2020 when interviews were remote, fellowship applicants pay for their own travel, etc.  And Stanford hospitals pull back on elective surgeries while the surgical residents are on the road interviewing.  Is so much interviewing inefficient?  Many think so, and here are some data.

Analysis of the pediatric surgery fellowship application process using the Thalamus™ database, by  Saunders Lin, Jason Reminick, Ephy Love, Benedict Nwomeh, Sanjay Krishnaswami, Journal of Pediatric Surgery, Volume 56, Issue 6, June 2021, Pages 1095-1100

"Background: The pediatric surgery fellowship interview process is costly and time intensive. We hypothesized that the increasing number of interviews completed by applicants and programs have become inefficient over time.

...

"Results: Our dataset included 34, 41, and 45 programs, which represented 81%, 91%, and 97% of all programs in 2018, 2019, and 2020, respectively. The median number of interviews completed per program remained constant, while the median number of interviews per applicant increased from 9.0 in 2018 to 13.0 in 2020. For 75% of programs, a program required only 4 or less candidates to fill their position. On average, 96% of program interviews do not result in a matched candidate.

"Conclusions: Programs offer interviews out of proportion to the number of positions available, and most applicants attend all interviews offered. We recommend an initial program goal of 20 interviews, which may be achieved by increased use of virtual interviews and the creation of program-level data on ideal applicant profiles.

...

"1. Introduction: With the advent of computer scheduling software and electronic interview platforms, data collection regarding the pediatric surgery fellowship interview process on a national level is now possible. One such platform is Thalamus, a scheduling software currently used for pediatric surgery fellowship interviews [1].

"The pediatric surgery match remains one of the most competitive fellowship application processes, with a total of 43 available positions for 78 applicants in the 2020 match cycle [2]. Published data show that extensive time and monetary resources are used every applicant cycle, with the average candidate spending around 14% of pretax salary and using up to three full weeks of residency days to complete interviews [3]. Despite these costs, however, programs continue to place considerable value on in-person interviews.

...

"2.1. Data source and methods: Thalamus is a comprehensive online and mobile Graduate Medical Education (GME) scheduling and communication software currently used in the pediatric surgery interview process. For applicants, features include a real-time scheduling system with online and mobile compatibility that allows applicants to self-schedule and instantly confirm their interview dates. From a program perspective, Thalamus is able to handle all interview confirmations, cancellations and rescheduling, and allows for comprehensive collection of applicant and program data both on the aggregate and individual levels.

"Thalamus was founded in 2013 and has been used in pediatric surgery since December 2016. The software is also currently used by more than 2200 residency and fellowship programs at more than 200 hospital systems across more than 100 specialties. It segments each institution by institutional ID and each program within each institution by program + ACGME ID (or a similar number for non-ACGME accredited programs). This is a cloud hosted database on the Microsoft Azure/SQL Server. Thalamus maintains several IRB approved/exempt research relationships with various specialties and other leadership organizations in Graduate Medical Education. This data is not shared between programs nor any other organization outside of Thalamus.

"We performed a retrospective investigation using Thalamus to identify population-level parameters regarding the pediatric surgery match between 2018 and 2020. This study was deemed exempt from approval by the Oregon Health and Sciences University Institutional Review Board as it did not contain patient data and applicant data was de-identified.

"3.2. Individual program and applicant data: With regards to individual program and applicant match data, the mean number of interviews offered and completed per program were similar in all three years (Table 2). The highest number of interviews a program completed was 44 in 2020. The number of interviews offered and completed per program have remained constant during the time-period. In contrast, both the mean and median number of interviews received and completed by applicants have increased. The median number of interviews completed per applicant increased 33.3% between 2018 and 2019 and an additional 12.5% between 2019 and 2020. Furthermore, the number of applicants who complete three or less interviews have been decreasing in the past three years: 25% in 2018, 20.6% in 2019, and 11.4% in 2020. Conversely, the number of applicants who completed more than 20 interviews has also been increasing in the past three years.

Strategic issues with (combined) early and late matching, by Mumcu and Saglam

  Here's a paper on early decision in college admissions. I read it with particular interest because of related discussions going on right now about early and late matching to medical residencies in the U.S. In their model, early matching introduces either or both strategic behavior and instabilities.

Strategic Issues in College Admissions with Early Decision by Ayse Mumcu and Ismail Saglam, Economics Bulletin, Volume 41, Issue 1, 2021

Abstract: In this paper, we consider college admissions with early decision (ED) using a many-to-one matching model with two periods. As in reality, each student commits to only one college in the ED period and agrees to enroll if admitted. Under responsive and consistent preferences for both colleges and students, we show that there exists no stable matching system, consisting of ED and regular decision (RD) matching rules, which is nonmanipulable via ED quotas by colleges or ED preferences by colleges or students. We also show that when colleges or students have common preferences and each student applies early only to the top-ranked college with respect to her RD preference, then no college has a strict incentive to offer a single-choice ED program. On the other hand, if students compromise in the ED market and make early application to colleges that are not top-ranked, then colleges may become better off when they offer ED programs than when they do not.