Signaling for Otolaryngology residency programs

 Here's a report on the use of signaling for residency positions, from the Oto match.

Pletcher, Steven, Chang, C.W., Thorne, Marc, MD, MPH & Malekzadeh, Sonya. (2022). The Otolaryngology Residency Program Preference Signaling Experience. Academic Medicine, 97, 664-668. https://doi.org/10.1097/ACM.0000000000004441

"The average applicant to Otolaryngology-Head and Neck Surgery in the 2021 residency application cycle applied to more than 50% of otolaryngology programs nationwide, submitting 77 applications, 1 a 34% increase over 5 years. 2 This surge has made it difficult for residency programs to holistically review applications and has limited opportunities for applicants to stand out to programs of particular interest.

...

"A formal preference signaling process provides all applicants with access to a known and stable quantity of signals. Through this process, at the time of application submission, students send a signal to indicate to a defined number of residency programs their particular interest in those programs. Such signals allow students to stand out to their favored programs and allow programs to receive a list of highly interested applicants. To our knowledge, this approach has not been used previously in the residency application process. Yet, articles advocating for signaling exist in the otolaryngology literature, 5-7 and the methodology, rationale, and results of preference signaling for graduates of economics PhD programs applying for faculty positions have been described. 8

"An OPDO working group, comprising the 4 authors, drove the establishment of a signaling process. In the spring of 2020, we held a series of meetings and webinars to engage stakeholders in the development and implementation of a signaling process. We included students, program directors, and specialty societies, such as the Society of University Otolaryngologists and the Association of Academic Departments of Otolaryngology Otolaryngology Chairs Organization. Additional discussions with the Association of American Medical Colleges, the Electronic Residency Application Service (ERAS), and the National Resident Matching Program also took place.

"In hindsight, establishing consensus across stakeholders proved to be the most challenging hurdle to successful implementation of our signaling process. Stakeholders had to accept this change and the inherent risks of implementing a "never before in medicine" process. These discussions, however, also provided a critical opportunity to refine our proposal and create an educational ecosystem that accepted this signaling process.

...

"All otolaryngology residency programs attested to the code of conduct, and none opted out of the signaling process. By October 21, 2020, the date that applications were released to programs, 611 students had submitted applications to otolaryngology residency programs, 559 applicants had submitted a Match list including an otolaryngology program, and 558 applicants had participated in the signaling process. Of 119 non-military otolaryngology residency programs, 118 received at least 1 signal. The number of signals received per program ranged from 0 to 71 with a mean of 22 (standard deviation 17) and a median of 16 

...

"Program directors most commonly reported using signals as a tiebreaker for similar applications and as part of an initial application review algorithm. One program required a signal to offer an applicant an interview.

...

"Applicants reported applying to a mean of 77 programs (standard deviation 21), including their 5 signaled programs. The rate of receiving an interview offer was significantly higher for signaled programs (58%, 670/1,150) compared with both nonsignaled programs (14%, 2,394/16,520; P < .001) and the comparative nonsignal program (23%, 53/230; P < .001; see Figure 2). To assess the impact of signaling across the spectrum of applicant competitiveness, we divided applicants into quartiles based on their overall likelihood of receiving an interview offer. Signals had a significant impact (P < .001) on interview offers across all quartiles (see Figure 3).

...

"The magnitude of signal impact we found likely represents both an increased rate of interview offers from signaled programs and a decreased rate of interview offers from nonsignaled programs. In contrast to our 5 signals, the American Economic Association provides 2 signals for graduates of economics PhD programs applying for faculty positions, suggesting that signal scarcity preserves its value and intent. 8 Decreasing the number of signals would force applicants to narrow their list of programs of primary interest and would disincentivize signaling "dream" programs. By increasing the number of signals, the lack of a signal becomes an indication of disinterest. If enough signals are provided, signaling could have a similar impact on the application process as an application cap. The ideal number of signals then must be explored.

"Our data demonstrated that signaling allowed applicants to influence their likelihood of receiving an interview offer from programs of particular interest. Given the distribution of signals received across programs, we believe that signaling also improved the distribution of interview offers among applicants, which could mitigate interview hoarding. 

...

"While applicants appear to benefit significantly from signaling, they also bear the responsibility of targeting their signals appropriately.

...

"Multiple specialties have expressed interest in adopting a similar program. However, otolaryngology is not representative of all medical or surgical specialties. It is a small, competitive surgical subspecialty with a 63% Match rate and no unmatched residency slots in the 2021 cycle. While the impact of signaling may vary significantly outside of these parameters, we are optimistic that the benefits will carry over to other specialties. Incorporating a signaling option within ERAS would facilitate both wide adoption and further analysis of such a process."