Showing posts with label interviews. Show all posts
Showing posts with label interviews. Show all posts

Wednesday, December 6, 2023

Applying for medical residencies: a consensus statement from Internal Medicine

 The Alliance for Academic Internal Medicine has released a "consensus statement" with many proposals about application and interview caps, and signaling.

Catalanotti, Jillian S., Reeni Abraham, John H. Choe, Kelli A. Corning, Laurel Fick, Kathleen M. Finn, Stacy Higgins et al. "Rethinking the Internal Medicine Residency Application Process to Prioritize the Public Good: A Consensus Statement of the Alliance for Academic Internal Medicine." The American Journal of Medicine (2023).

It also includes a call for data and analysis:

"AAIM proposes increasing internal medicine program preference signals to 15, using tiered signaling with three “gold” and 12 “silver” signals, and setting an interview cap of 15 in the 2024-2025 recruitment season, with participation by all internal medicine programs. The Alliance recommends that all internal medicine programs participate in ACI. AAIM recommends that programs transparently share information about their use of preference signals and other application screening methods and calls for real-time data analysis to explore impact, inform future iterations and identify potential harms.

"The Alliance calls upon ERAS and NRMP as well as Thalamus® and other interview scheduling platforms to transparently share data, to embrace change, and to perform analyses needed to inform this process. For example, recent modeling with eight years of retrospective NRMP data in OBGYN demonstrated that an early match round may increase the number of “mutually dissatisfied applicant-program pairs” and that a multiple-round match process could introduce potential rewards for gamesmanship, a prime factor addressed by the current process.35 AAIM applauds this analysis and hopes that the new collaboration between ERAS and Thalamus® may provide useful interview data to inform this proposal and further interventions."

And here is reference 35 in that last paragraph, about which I've blogged before.

I Ashlagi, E Love, JI Reminick, AE. Roth
Early vs Single Match in the Transition to Residency: Analysis Using NRMP Data From 2014 to 2021
J Grad Med Educ, 15 (2) (Apr 2023), pp. 219-227, 10.4300/JGME-D-22-00177.1

Friday, August 4, 2023

AEA Guidance on the 2023-24 Economics Job Market Cycle

In an effort to forestall/halt/slow unravelling in the market for new Ph.D. economists, the American Economic Association has released some guidelines regarding interviews and offers.

 AEA Guidance on the 2023-24 Economics Job Market Cycle


July 27, 2023

To: Members of the American Economic Association and Economics Department Chairs
From: Peter L. Rousseau, Secretary-Treasurer
Subject: AEA Guidance on the 2023-24 Economics Job Market Cycle

AEA Guidance on the 2023-24 Economics Job Market Cycle

The AEA Executive Committee, in conjunction with its Committee on the Job Market, recognizes that it is to the benefit of the profession if the job market for economists is thick, with many employers and job candidates participating in the same stages at the same time.  Moreover, the AEA's goals of diversity, equity, and inclusion are fostered by having a timeline that remains widely known and accepted, ensuring that candidates can correctly anticipate when each stage will occur.  The AEA has a role to establish professional norms, which includes ensuring fair treatment of job candidates, including that they have enough time to consider job offers.

With these goals in mind, and in light of inquiries from both job candidates and employers about how to proceed, the AEA asks that departments and other employers consider the following timeline for initial interviews and “flyouts” in the upcoming job cycle (2023-24). 

Timing of interview invitations
The AEA suggests that employers wait to extend interview invitations until the day after job market signals are transmitted to employers (roughly December 1).

Rationale: the AEA created the signaling mechanism to reduce the problem of asymmetric information and allow job candidates to credibly signal their interest to two employers. The AEA asks that employers wait to extend interview invitations until those signals have been transmitted, and to use that information to finalize their set of candidates to interview. This helps the job market in several ways: it reduces the problem of imperfect information, it helps ensure a thick market at each stage, and it promotes the AEA’s goals of diversity, equity, and inclusion. Job candidates from historically under-represented groups may lack informal networks and thus may especially rely on the signals to convey their interest. Waiting to review the signals before issuing invitations promotes a fairer, more equitable process.

We also ask that all employers indicate on EconTrack when they have extended interview invitations. This allows candidates to learn about the status of searches without visiting websites posting crowd-sourced information and potentially inappropriate other content.

Timing of interviews
Initial interviews may take place any time after the AEA signals are sent (roughly December 1). The AEA recommends that all initial interviews take place virtually (e.g. by Zoom). We suggest that interviews not take place during the AEA meeting itself (January 5-7, 2024).

Rationale: In the past, interviews were conducted in-person at the AEA/ASSA meetings. This promoted thickness of the market, because most candidates and employers were present at the meetings, but had the disadvantage of precluding both job candidates and interviewers from fully participating in AEA/ASSA sessions. 

Initial job interviews went online during COVID, and feedback indicated that the benefits of virtual first-round interviews (e.g. low monetary cost, zero cost in travel time, scheduling flexibility, convenience) outweighed the limitations (e.g. less rich interaction).

We ask that interviews NOT take place during the AEA/ASSA meetings (January 5-7, 2024) in order to allow job candidates and interviewers to participate in the conference.

Timing of flyouts
Flyouts have historically happened at times appropriate for the employer, and the AEA sees no reason to suggest otherwise.  We ask that all employers indicate on EconTrack when they have extended flyout invitations. Unlike with interviews, the AEA does not take a position on whether flyouts should be virtual or in-person.

Timing of offers and “exploding” offers
In order to ensure that the job market remains sufficiently synchronized and thick, and that candidates have a chance to compare offers, the AEA recommends that employers leave job offers open (i.e. do not require candidates to accept or decline) until at least January 31.

The AEA also strongly recommends that employers give candidates at least two weeks to consider their job offer.  We recognize that offers made late in the job market season (e.g., March or later) may be of shorter duration.  In some circumstances, employers are under heavy pressure to give less time to candidates for various reasons.  If that is absolutely necessary, we recommend that employers give candidates a minimum of one week to consider the offer, and that candidates be given advance notice of this (e.g. at the flyout stage) whenever possible. 

Rationale: Recently, there is concern about a rise in “exploding offers” – i.e., offers for which candidates are given too few days to sufficiently consider the offer and their alternatives. This can prevent candidates from learning about their options or comparing offers, and at the extreme can be coercive.  Giving candidates two weeks (or, late in the job market season, at least one week) to consider an offer is a reasonable standard.

We also ask that all employers indicate on EconTrack when they have completed or closed their search.

Job market institutions and mechanisms
Please keep in mind the various job market institutions and mechanisms created by the AEA to improve the job market:

We encourage all employers to review and abide by Best Practices for Economists to Build a More Diverse, Inclusive, and Productive Profession, and in particular those for conducting a fair recruiting process.

Thank you for helping to ensure a transparent and equitable job market for new Ph.D. economists.  

Friday, April 28, 2023

Interesting development in the transition from medical school to residency: connecting applications and interviews

 The market for new doctors has been suffering from congestion in applications and interviews, in the runup to the resident Match (see recent post with a diagram). The American Association of Medical Colleges runs the main application server, ERAS. A private company called Thalamus runs a growing interview scheduling service. Now they are looking to collaborate.

 Here's  yesterday's press release from Thalamus:

AAMC, Thalamus Announce New Collaboration to Improve Transition to Residency

Collaboration will increase transparency and make the residency process easier for applicants and programs  

Washington, D.C., April 27, 2023—Today the AAMC (Association of American Medical Colleges) and Thalamus announced a strategic collaboration to accelerate innovation and ease the transition to residency for medical students, medical schools, and residency programs. The collaboration will combine the AAMC’s long-established leadership in innovation along the continuum from medical school to residency training and continuing medical education with Thalamus’ market-leading product and software development expertise. 

“The transition from undergraduate medical education to graduate medical education is a critical period in any learner’s journey to becoming a physician,” said David J. Skorton, MD, AAMC president and CEO. “We know the community is seeking enhanced tools and integrated services that better support application and recruitment processes. We listened, we have made improvements, and, with Thalamus, we are excited to make this vision a reality.” 

The organizations will collaborate to leverage their data, technology, and expertise to transform the medical residency and fellowship recruitment processes for applicants and programs. Their efforts will focus on increasing transparency, supporting equity through holistic review, and improving the learner experience by consolidating the fragmented interview management process. 

“We are thrilled to be collaborating with the AAMC to provide a comprehensive solution that will streamline graduate medical education recruitment processes,” said Jason Reminick, MD, MBA, MS, CEO and founder of Thalamus. “But even more, we are looking forward to building new and innovative tools that improve the experience, are cost-effective, and leverage data for the benefit of the medical education community and the advancement of our collective missions.” Dr. Reminick applied to residency in 2012 during an eventful recruitment season disrupted by Hurricane Sandy. “I’m particularly excited to provide applicants with a comprehensive platform to manage their interview season.” 

The collaboration between the AAMC and Thalamus will enable data-sharing and innovative research that will benefit the undergraduate to graduate medical education community and advance both organizations’ missions. The initiative also demonstrates the commitment of both organizations to addressing the concepts and themes outlined in the 2021 report from the Coalition for Physician Accountability’s Undergraduate Medical Education-Graduate Medical Education Review Committee.

In recent years, the AAMC has completed significant in-depth research and upgraded technology to enhance the Electronic Residency Application Service® (ERAS®) suite of application and selection tools, such as updating the MyERAS® application content, building analytics tools for institutions, and partnering on collaborative research initiatives. Thalamus has completed unique research related to the physician workforce, including how geography influences The Match® and specialty-specific interview practices. The Thalamus technology will continue the upgrade of the ERAS suite of application and selection tools. The AAMC and Thalamus remain committed to future innovations that will enable the ERAS program to continue to evolve faster and better. 

Beginning in June 2023, all ERAS residency and fellowship programs will receive complimentary access to Thalamus’ leading interview management platform, Thalamus Core and Itinerary Wizard, as well as Cerebellum, a novel data and analytics dashboard to assess recruitment outcomes, specifically from a diversity, equity, inclusion, and geographic perspective. Programs may also elect to purchase Thalamus’ video interview platform and Cortex, its technology-assisted holistic application review and screening platform. 

According to AAMC data, the U.S. is expected to experience a shortage of up to 124,000 physicians by 2034. Given the burnout and other challenges to the health care system caused by the COVID-19 pandemic, the AAMC and Thalamus look to use their collective expertise to promote a diverse and representative workforce that will enhance health care and patient outcomes. 

The data and research the AAMC and Thalamus have amassed to identify resident, fellow, and physician recruitment trends can potentially have a major impact on diversity in medicine and begin to address several well-established and longstanding systemic challenges. These efforts will support not only the application and selection processes in graduate medical education but also aim to improve the experiences of the U.S. physician workforce over the long term. 

Related Resources 

###

Note for editors: Leaders from the AAMC and Thalamus are available to speak with media about this new collaboration and what it means for residency programs and applicants. 

The AAMC (Association of American Medical Colleges) is a nonprofit association dedicated to improving the health of people everywhere through medical education, health care, medical research, and community collaborations. Its members are all 157 U.S. medical schools accredited by the Liaison Committee on Medical Education; 13 accredited Canadian medical schools; approximately 400 teaching hospitals and health systems, including Department of Veterans Affairs medical centers; and more than 70 academic societies. Through these institutions and organizations, the AAMC leads and serves America’s medical schools and teaching hospitals and the millions of individuals across academic medicine, including more than 193,000 full-time faculty members, 96,000 medical students, 153,000 resident physicians, and 60,000 graduate students and postdoctoral researchers in the biomedical sciences. Following a 2022 merger, the Alliance of Academic Health Centers and the Alliance of Academic Health Centers International broadened the AAMC’s U.S. membership and expanded its reach to international academic health centers. Learn more at aamc.org

Thalamus is the premier, cloud-based interview management platform designed specifically for application to Graduate Medical Education (GME) training programs. The software streamlines communication by eliminating unnecessary phone calls/emails allowing applicants to book interviews in real-time, while acting as a comprehensive applicant tracking system for residency and fellowship programs. Thalamus provides comprehensive online interview scheduling and travel coordination via a real-time scheduling system, video interview platform, AI application screening/review tool (Cortex) providing technology-assisted holistic review, and first-in-class DEI-focused analytics dashboard (Cerebellum). Featured nationally at over 300+ institutions and used by >90% of applicants, Thalamus is the most comprehensive solution in GME interview management. For more information on Thalamus, please visit https://thalamusgme.com or connect with us on LinkedInFacebookInstagramTwitter, or YouTube

Wednesday, November 23, 2022

Improving the transition to (surgical) residency

The transition from medical school to residency is presently troubled by congestion involving (too) many applications and interviews.  It's a subject of considerable discussion in the medical community, sometimes hampered between the parts of the process that proceed the Match, and the Match itself (which is the clearinghouse run by the NRMP that, after all applications and interviews have been processed, solicits rank order lists and turns them into a matching of doctors to residency programs)..  Here's a paper that focuses sensibly on the runup to the Match, even though its title follows the (unfortunately common) practice of calling the whole process the Match.

Designing the “match of the future”: challenges and proposed solutions in the interview and match phase of the UME–GME transition by Sophia K. McKinley, Maria S. Altieri, Olabisi Sheppard, Kimberly Hendershot, Keneeshia Williams, Brigitte K. Smith on behalf of the ASE Graduate Surgical Education Committee, Global Surgical Education - Journal of the Association for Surgical Education : 17 November

Table 1 Challenges and proposed solutions in the surgical resident selection process (click to embiggen)




Tuesday, August 2, 2022

American Finance Association guidelines to prevent unravelling of the job market

 Zoom is changing interview practices, and there's concern that academic markets for new Ph.D grads could unravel. The AFA is on the case for the finance market with these guidelines:

Guidelines for the AFA Rookie Recruiting Cycle

The AFA rookie job market cycle of 2021-2022 created uncertainty, confusion, and unneeded stress for job market candidates and for recruiters. In the interest of developing a more coordinated job market that benefits all involved, the AFA Board has the following suggested guidelines.

Timing of interviews:

Initial interviews can be virtual or in person, but the AFA recommends that the initial interviews should not begin before December 15, 2022, and that the timing of the “campus visit” should occur after the AFA meeting.

Timing of job offers:

In order to facilitate the best matching between candidates and positions, the AFA Board believes strongly that job offers should remain open until at least February 20. The AFA Board also encourages employers to abstain from giving exploding offers with too short of a time frame, since they are unfair to the candidates. Consequently, the AFA promotes the following professional norm: If a job candidate receives and accepts a coercive exploding offer (i.e., one that expires before February 20), the AFA does not consider such an acceptance to be binding.

These guidelines are designed for the AFA rookie recruiting cycle and do not pertain to recruiting cycles for other job markets such as the FMA or European job markets.


HT: Alex Chan

Thursday, July 7, 2022

Coordinating the timing of the market for new Economics Ph.D.s: guidance from the AEA

 Here's an email broadcast by the American Economic Association, aimed to promote market thickness by avoiding unraveling and dealing with congestion:

AEA Guidance on Timeline for 2022-23 Economics Job Cycle

 July 1, 2022

To: Members of the American Economic Association
From: Peter L. Rousseau, Secretary-Treasurer
Subject: AEA Guidance on Timeline for 2022-23 Economics Job Cycle

The AEA Executive Committee, in conjunction with its Committee on the Job Market, recognizes that it is to the benefit of the profession if the job market for economists is thick, with many employers and job candidates participating in the same stages at the same time.  Moreover, the AEA's goals of diversity, equity, and inclusion are fostered by having a timeline that remains widely known and accepted, ensuring that candidates can correctly anticipate when each stage will occur. With these goals in mind, and in light of inquiries from both students and departments about how to proceed, the AEA asks that departments and other employers consider the following timeline for initial interviews and “flyouts” in the upcoming job cycle (2022-23).  

Interview invitations
The AEA suggests that employers wait to extend interview invitations until the day after job market signals are transmitted to employers.

Rationale: the AEA created the signaling mechanism to reduce the problem of asymmetric information and allow job candidates to credibly signal their interest to two employers. The AEA asks that employers wait to extend interview invitations until those signals have been transmitted, and to use that information to finalize their set of candidates to interview. This helps the job market in several ways: it reduces the problem of imperfect information, it helps ensure a thick market at each stage, and it promotes the AEA’s goals of diversity, equity, and inclusion. Job candidates from historically under-represented groups may lack informal networks and thus may especially rely on the signals to convey their interest. Waiting to review the signals before issuing invitations promotes a fairer, more equitable process.

We also ask that all employers indicate on EconTrack when they have extended interview invitations; this allows candidates to learn about the status of searches without visiting websites posting crowd-sourced information and potentially inappropriate other content.

Interviews
The AEA recommends that employers conduct initial interviews starting on Monday, January 2, 2023, and strongly recommends that all interviews take place virtually (e.g. by Zoom). We suggest that interviews not take place during the AEA meeting itself (January 6-8, 2023).

Rationale: In the past, interviews were conducted in person at the AEA/ASSA meetings. This promoted thickness of the market, because most candidates and employers were present at the in-person meetings, but had the disadvantage of precluding both job candidates and interviewers from fully participating in AEA/ASSA sessions. 

Interviews should now be conducted virtually to prevent risk of exposure to COVID, and to promote equity among the candidates. Informal feedback to the AEA committee on the job market indicated that the benefits of virtual first-round interviews (e.g. low monetary cost, zero cost in travel time, convenience) outweighed the limitations (e.g. less rich interaction).

We recommend that employers wait until January 2 to interview candidates because job candidates may have teaching or TA responsibilities in December. Moreover, having a clear start date for interviews will help candidates to have accurate expectations of the timing of the stages of the market. An unraveling of the market works against the AEA’s goal of having a thick market at each stage and also works against candidates having uniform expectations of the timing of each stage of the market.

We ask that interviews NOT take place during the AEA/ASSA meetings (January 6-8, 2023) in order to allow job candidates and interviewers to participate in the conference.

Flyouts and offers
Flyouts and offers have historically happened at times appropriate for the employer, and the AEA sees no reason to suggest otherwise.  We ask that all employers indicate on EconTrack when they have extended flyout invitations and closed their searches. Unlike with interviews, the AEA does not take a position on whether flyouts should be virtual or in-person.

Job market institutions and mechanisms
Please keep in mind the various job market institutions and mechanisms created by the AEA to improve the job market:

·       The JOE Network includes a database of job openings for economists.

o   Employers may sign up here: https://www.aeaweb.org/joe/employer.

o   Job candidates may search the database here: https://www.aeaweb.org/joe/listings.

o   The JOE Network has an electronic clearinghouse for job candidates to submit job applications. Job candidates may register here: https://www.aeaweb.org/joe/candidate.

·       The AEA Committee on the Job Market releases data and guidance on the job market here: https://www.aeaweb.org/joe/communications.

·       EconTrack: a board on which employers can indicate when they have extended interview and flyout invitations, and closed their search: https://www.aeaweb.org/econtrack.


Thank you for helping to ensure a transparent and equitable job market for new Ph.D. economists.  

Tuesday, May 24, 2022

Erling Skancke defends his dissertation

 Erling Skancke defended his dissertation last week:


Here's his job market paper:


Welfare and Strategic Externalities in Matching Markets with Interviews (Job Market Paper)
Recent debate in the medical literature has raised concerns about the pre-match interview process for residency and fellowship positions at hospitals. However, little is known about the economics of this decentralized process. In this paper, I build a game-theoretic model in which hospitals conduct costly interviews in order to learn their preferences over doctors. I show that increased interview activity by any hospital imposes an unambiguous negative welfare externality on all other hospitals. In equilibrium, both hospitals and doctors may be better off by a coordinated reduction in interview activity. The strategic externality is more subtle, and conditions are derived under which the game exhibits either strategic complementarities or substitutes. Moreover, an increase in market size may exacerbate the inefficiencies of the interview process, preventing agents from reaping the thick market benefits that would arise in the absence of the costly interviews. This effect increases participants' incentives to match outside of the centralized clearinghouse as markets become thicker, jeopardizing the long-term viability of the clearinghouse. The model also provides new insights into several market design interventions that have recently been proposed.

Congratulations, Erling! 
Welcome to the club.

Sunday, December 19, 2021

An Interview Match for medical residents and fellows--a preliminary proposal

 There is a lot of concern in the graduate medical education community that too much time and treasure is being spent on too many unproductive interviews prior to the submission of rank order lists for the Match.  Here's discussion of a proposal for an interview match, to precede the interview stage before the actual NRMP Match.

Explaining a Potential Interview Match for Graduate Medical Education, by Irene Wapnir, MD; Itai Ashlagi, PhD; Alvin E. Roth, PhD; Erling Skancke, MS; Akhil Vohra, PhD; Irene Lo, PhD; Marc L. Melcher, MD, PhD, J Grad Med Educ (2021) 13 (6): 764–767.  https://doi.org/10.4300/JGME-D-20-01422.1

"Residency and fellowship candidates are applying to more programs to enhance their chances of securing interviews and matching favorably. The COVID-19 pandemic has shifted interviews to video formats, which lowers interview-associated costs for applicants but may further increase application numbers.1  While a candidate's application to a training program communicates some interest in the program, the relative amount of interest is obscured when candidates apply to large numbers of programs. We suspect that, as a result, programs host large numbers of low-yield interviews.

"The number of interviews is steadily increasing, and there is widespread agreement on the need to ease congestion in the pre-Match evaluation process.2  Proposals to reduce this burden include signaling (organized, centrally-controlled protocol for limited communication of interest),3–5  capping the number of applications or the number of interviews,6,7  and an early acceptance matching program as in college admissions.8,9 

"We propose another solution, an “interview match” to address the expanding number of interviews.10  An interview match enables candidates and programs to express preferences privately by ranking their interview choices individually or in tiers. This may ease congestion in the “marketplace,” reduce costs for candidates, favor interviews that are more likely to lead to a match in the final Match, and avoid interviews unlikely to convert to a match. An interview match algorithm would match based on the same “deferred-acceptance” algorithm currently used by the National Resident Matching Program but adapted to a “many-to-many” setting where candidates and programs receive multiple interviews."

Wednesday, December 15, 2021

Virtual interviewing increased the number of interviews by (and of) candidates for medical residencies

 Here's an article on virtual interviewing of candidates for medical residencies (about which, see also this post about Erling Skancke's work on interviewing):

Beshar I, Tate WJ, Bernstein D. Residency interviews in the digital era, Postgraduate Medical Journal Published Online First: 05 October 2021. doi: 10.1136/postgradmedj-2021-140897

"Halfway through the interview cycle, however, questions were raised about system-level equity of virtual interviewing. In December 2020, the Association of American Medical Colleges (AAMC) released an open letter citing ‘a maldistribution in residency interview invitations’, with the ‘highest tier applicants hav(ing) so many interviews’.15 The letter, addressed to both programme directors and students, called on programmes ‘to recruit a diverse pool of residents’ and encouraged students ‘to release[e] some interviews if you are holding more than needed, allowing your fellow students to access those interview opportunities’.15 Medical school deans began encouraging competitive students to forgo interviews. In the words of the Dean of students at the UC College of Medicine: ‘They need to identify a reasonable number to have a successful match and release others so their peers who need them can have them’.6 By some estimates, programmes invite the same pool of highly qualified applicants, with just 7%–21% of the applicant pool filling 50% of the interview slots in some specialties.16 Meanwhile, a survey of plastic surgery programme directors demonstrated nearly one in three increased the number of interview offers per available residency spot.

"At our institution—Stanford School of Medicine—and as applicants of the 2020–2021 cycle ourselves—we saw the effect of this firsthand. We administered a survey to all students participating in the match process in both the 2019 (in-person) and 2020 (virtual) years. In the survey, respondents identified the residency programme or programmes to which they applied as well as the number of interviews they attended. Of Stanford’s 2019 and 2020 graduating classes, 83.7% (n=72) and 62.3% (n=62), respectively, completed our survey.

"Of the 2019 applicants, 97.2% (n=70) reported residency interviews that required airline travel, compared with 0% of the 2020 applicants. The median number of interviews for the 2019 applicants was 8, compared with 14 for the 2020 applicants. Across the 2 years analysed, all fields showed an increase in the number of interviews accepted (table 1). Of the four fields with the most applicants, the largest per cent change was in anaesthesia (244%), followed by ophthalmology (216%), internal medicine (144%) and psychiatry (128%), respectively. On average, across all specialties, the number of accepted interviews changed by 160%.





Sunday, December 12, 2021

What we know about labor market interviews

 Erling Skancke is on the job market from Stanford this year. You should interview him. If you do, you'll learn a lot about interviews, which are the subject of his job market paper, which provides a lot of new insight into an important practical set of isssues.

Skancke, Erling, Welfare and Strategic Externalities in Matching Markets with Interviews (November 10, 2021). Available at SSRN: https://ssrn.com/abstract=3960558 or http://dx.doi.org/10.2139/ssrn.3960558

Abstract: Recent debate in the medical literature has brought attention to issues with the pre-match interview process for residency and fellowship positions at hospitals. However, little is known about the economics of this decentralized process. In this paper, I build a game-theoretic model in which hospitals simultaneously decide on which doctors to interview, in order to learn their preferences over doctors. I show that increased interview activity by any hospital imposes an unambiguous negative welfare externality on all other hospitals. In equilibrium, both hospitals and doctors may be better off by a coordinated reduction in interview activity. The strategic externality is more subtle, and conditions are derived under which the game exhibits either strategic complementarities or substitutes. Moreover, an increase in market size may exacerbate the interview externalities, preventing agents from reaping the thick market benefits that would arise in the absence of the costly interviews. This effect increases participants' incentives to match outside of the centralized clearinghouse as markets become thicker, jeopardizing the long-term viability of the clearinghouse. The model also provides new insights into several market design interventions that have recently been proposed.

Friday, November 26, 2021

NRMP Statement On Interviewing

 The National Resident Matching Program (NRMP) has a statement on interviewing, which precedes the NRMP match for new American doctors, and that reflects concerns that the interviewing process has become congested.

NRMP Statement On Interviewing

"The National Resident Matching Program® (NRMP®) has heard the concerns of learners and programs in the medical education community about the interview process and wants to encourage equitable practices among applicants and programs. As such we have developed the following recommendations. Although especially true during the enduring times of the pandemic, the recommendations align with the NRMP’s long-standing commitment to maintaining a fair, efficient, reliable, and transparent process for all. Recommendations also foster well-being among all parties.

"NRMP Recommendations for Programs:

1. Programs should conduct all interviews virtually for the 2021-2022 cycle.

2. Programs should extend interview offers that equal, not exceed, the total number of available interview slots.

3. Applicants should be given a minimum of 48 hours to respond to an interview offer.

"NRMP Recommendations for Applicants:

1. Applicants should make timely decisions about interview offers and promptly notify programs in which they are no longer interested, freeing up interview slots.

2. Applicants should give ample, adequate (e.g., one week) notice to programs in the event they change their minds and decline accepted interviews."

Friday, August 20, 2021

Preference Signaling and Worker-Firm Matching: Evidence from Interview Auctions, by Laschever and Weinstein

 Here's a recent working paper from the IZA Institute of Labor Economics concerned with the importance of signals of interest in labor market matching:

Preference Signaling and Worker-Firm Matching: Evidence from Interview Auctions, by Ron A. Laschever and Russell Weinstein,  IZA DP No. 14622

Abstract: "We study whether there are improvements in worker-firm matching when employers and applicants can credibly signal their interest in a match. Using a detailed résumé dataset of more than 400 applicants from one university over five years, we analyze a matching process in which firms fill some of their interview slots by invitation and the remainder are filled by an auction. Consistent with the predictions of a signaling model, we find the auction is valuable for less desirable firms trying to hire high desirability applicants. Second, we find evidence that is consistent with the auction benefiting overlooked applicants. Candidates who are less likely to be invited for an interview (e.g., non-U.S. citizens) are hired after having the opportunity to interview through the auction. Among hires, these candidates are more represented among auction winners than invited interviewees, and this difference is more pronounced at more desirable firms. Finally, counterfactual analysis shows the auction increases the number and quality of hires for less desirable firms, and total hires in the market

...

"Auctions for interview slots may address two important frictions in the matching process: uncertainty over applicant quality, and uncertainty over the likelihood that an applicant accepts an offer. Even if employers can successfully identify desirable applicants, there remains the challenge of identifying which candidates are truly interested in the job and would accept an o↵er with high probability. In recent years the cost of job applications has fallen as more postings and applications are online. This
further raises the potential that applicants will have a low likelihood of accepting an offer.
...
"Though not common, there are a few markets in which all applicants have an equal opportunity to credibly signal their preferences for an employer. One example is the American Economic Association (AEA) job signaling mechanism, which allows candidates to send a signal of interest to two departments. Importantly, there is no requirement that employers interview the applicants sending the signal. In contrast, in our setting an employer is compelled to meet with some signaling job seekers.

"A second example, and the focus of this paper, is the auction system used in the market for professional master’s degree students, most commonly MBA students, at many top-ranked programs. These programs allow employers to choose some percentage of the applicants they interview, but require the remainder of the interview slots are allocated through an auction. Typically, firms first invite applicants for interviews, before applicants have had the opportunity to signal. Next, there is an auction for the remaining interview slots, and thus auction participants are students who were not invited for an interview by the firm. Each student is provided with an equal allotment of “bid points,” and the auction winners are guaranteed interviews
with the firm."

Monday, July 5, 2021

NRMP Position Statement On The (In)Feasibility Of An Early Match

 There has been some suggestion that dividing the resident match into early and late matches might be a way to address the congestion in applications and interviews that has bedeviled the transition from medical school to residency in recent years.  The NRMP now has a statement pointing out that there are serious problems with that idea.

NRMP Position Statement On The Feasibility Of An Early Match

"For the past eighteen months the National Resident Matching Program® (NRMP®) has been working closely with other national medical education organizations to examine the current state of the transition to residency. Conversations have focused on mitigating burdens for both applicants and programs in the selection and recruitment process and addressing uncertainty in the future of the interview cycle.

...

"Among the proposed solutions to current challenges in the transition to residency are calls for an early match. Specifically, NRMP has been asked to implement the Early Result and Acceptance Program (ERAP) pilot program proposed for Obstetrics and Gynecology, created through American Medical Association’s Reimagining Residency Grant, “Transforming the UME to GME Transition: Right Resident, Right Program, Ready Day One”. The stated goals of the ERAP pilot are to allow applicants to engage in strategic decision-making, reduce burden on programs while hypothesizing that the change will result in holistic review, and reduce necessary applications and interviews. ERAP calls for an early match to begin in September 2022 for the 2023 Match cycle. ERAP permits applicants to apply to a maximum of three programs in the early match with programs including up to 50% of their positions if they choose to participate. This statement outlines NRMP’s concerns about the structure of the ERAP pilot program, the lack of evidence supporting the proposed changes to the Match, the implications of an early match for the matching process, and preliminary findings of modeling an early match being conducted by experts in market design and the matching algorithm.

"The NRMP has reviewed the ERAP pilot program with consideration for whether changes to the matching process have the potential to inadvertently disadvantage Match participants. It is through that lens NRMP remains concerned with the following aspects of the ERAP pilot:

"Although voluntary, applicants may feel pressured to participate in an early match where up to half the available positions in a specialty may fill before the Main Residency Match® opens.

"There exists no mechanism for demonstrating how an early match will make visible less competitive applicants and those underrepresented in medicine, which is hypothesized in the project document.

"The proposed limit of three applications per applicant could force applicants to make compromises not present in the Match today. ...While the ERAP investigation team hypothesized that the application limit will increase holistic review by programs, there are no mandates to ensure that programs conduct holistic review nor are there restrictions on the number of applications programs may accept, interviews they may offer, or applicants they may rank. With no objective evidence to support the hypothesis, we cannot conclude that the proposed application limit would increase holistic review of applications.

"There exists no mechanism for safeguarding an applicant’s failure to match in the early match from programs as they enter the Main Residency Match, which could result in the applicant being viewed as less competitive.

"In addition to concerns about disadvantaging applicants, NRMP is mindful of possible behavior changes resulting from changes to the Match process that could affect Match outcomes for all Match participants.

  • "The structure of an early match does not allow for mixed-specialty couples ranking or multispecialty individual ranking, which may cause applicants to reconsider their specialty choices, fundamentally changing their career path.
  • "Programs may have insufficient information (e.g., clinical evaluations, MSPE, LORs) to evaluate applicants fully and fairly in the early match.
  • Programs may see a surge in non-traditional applicants as the early match provides three opportunities to enter training through either the early match, the Main Residency Match, or SOAP®. This may result in an increased number of applications or applicants who may otherwise not select the specialty.
  • Not matching in the early match is likely to increase the number of applications per individual in the Main Residency Match, as applicants enter a matching cycle with only half of the positions remaining available. This may increase stress, cost, and could adversely affect the wellness of applicants.

...

"it is important to first outline the core concepts of the match as a stable “market”. The Match was established in 1952, to solve a “congestion” problem in medical residencies involving applications, offers, and acceptances. In a May 2021 pre-submission working paper, Itai Ashlagi, Ph.D. and Alvin Roth, Ph.D. describe the consequences of congestion as “unraveling” where programs initially responded to congestion by making “exploding offers” that prevented applicants from considering many programs because they were pressed to accept an early offer, before knowing whether an offer from a more preferred program might be forthcoming if they waited. The authors note that NRMP’s matching process, in its current form, has four distinct properties that are relevant to managing the problems of congestion and unraveling and maintaining a stable matching market. Specifically, the NRMP matching process

"1. Is Uncongested: participants make all decisions (on Rank Order Lists) in advance, so there is no delay in processing offers, rejections, and acceptances, which is done by the computerized Roth-Peranson algorithm.

"2. Defers acceptances: preferences of applicants and programs are not finalized until all preferences have been considered, thereby producing stable matching: i.e., matching in which there are no “blocking pairs” of applicants and programs not matched to one another but who both would prefer to be.

"3. Promotes true preferences: it is safe for participants to state their true preferences when they submit their Rank Order Lists (ROLs).

"4. Establishes a “thick” market: most residency programs in most specialties participate in the NRMP Match, which also allows for multi-specialty applications and couple matching (including for mixed-specialty couples).

"The authors opine that an early match such as the proposed ERAP pilot followed by the Main Residency Match would not share three of the four important properties of the Match:

"1. An early match would dilute the thick market: not all positions would be available at the same time (and further, it would not allow applicants to express multi-specialty preferences, nor would it accommodate mixed-specialty couples).

"2. early match would introduce complicated strategic decisions into the formulation of ROLs: it would no longer be safe for participants to submit ROLs straightforwardly corresponding to their preferences.

"3. An early match would not produce a stable matching: there would be mutually disappointed blocking pairs of mismatched applicants and programs. This would also make it less safe to report ROLs that straightforwardly corresponded to preferences."



Thursday, June 10, 2021

Congestion in applications and interviews, by Arnosti, Johari and Kanoria

 Here's a paper modeling the issue that some labor markets may face congestion related to large numbers of applications followed by costly interviews.

Nick Arnosti, Ramesh Johari, Yash Kanoria (2021) Managing Congestion in Matching Markets. Manufacturing & Service Operations Management 23(3):620-636. https://doi.org/10.1287/msom.2020.0927

Abstract. "Problem definition: Participants in matching markets face search and screening costs when seeking a match. We study how platform design can reduce the effort required to find a suitable partner. Practical/academic relevance: The success of matching platforms requires designs that minimize search effort and facilitate efficient market clearing.

"Methodology: We study a game-theoretic model in which “applicants” and “employers” pay costs to search and screen. An important feature of our model is that both sides may waste effort: Some applications are never screened, and employers screen applicants who may have already matched. We prove existence and uniqueness of equilibrium and characterize welfare for participants on both sides of the market. Results: We identify that the market operates in one of two regimes: It is either screening-limited or application-limited. In screening-limited markets, employer welfare is low, and some employers choose not to participate. This occurs when application costs are low and there are enough employers that most applicants match, implying that many screened applicants are unavailable. In application-limited markets, applicants face a “tragedy of the commons” and send many applications that are never read. The resulting inefficiency is worst when there is a shortage of employers. We show that simple interventions—such as limiting the number of applications that an individual can send, making it more costly to apply, or setting an appropriate market-wide wage—can significantly improve the welfare of agents on one or both sides of the market. 

"Managerial implications: Our results suggest that platforms cannot focus exclusively on attracting participants and making it easy to contact potential match partners. A good user experience requires that participants not waste effort considering possibilities that are unlikely to be available. The operational interventions we study alleviate congestion by ensuring that potential match partners are likely to be available.

And from the Conclusion:

"We also compare the effects of an application limit to those of other available levers: either raising application costs or lowering the wage paid to applicants. Although these interventions can lead to thesame aggregate welfare as an application limit, they differ in how they distribute this welfare. Charging fees and lowering wages both increase aggregate welfare at the expense of applicants. Although these interventions may be appropriate for a platform looking to monetize its services or attract more employers, an application limit can yield Pareto improvements in welfare and may be more suitable if the platform is primarily concerned with applicant welfare. These considerations might explain why the tutoring platform TutorZ charges tutors for each potential client that they contact, whereas the dating platforms Coffee Meets Bagel and Tinder limit the number of likes/right swipes permitted in a certain period."