Update on the Orthopaedic Sports Medicine Fellowship Match

From the Orthopaedic Journal of Sports Medicine:
Outcomes in the Orthopaedic Sports Medicine Fellowship Match, 2010-2017
by Mary K. Mulcahey, MD*, Meghan K. Hayes, BS, Christopher M. Smith, MD, Matthew J. Kraeutler, MD, Jeffrey D. Trojan, BA, Eric C. McCarty, MD

"Together with an increase in the number of applicants for orthopaedic fellowships, the process of applying to fellowship programs has evolved over the past several years. Currently, the majority of orthopaedic fellowship programs utilize a centralized, formal matching process.2 Sports medicine fellowship programs utilized the National Resident Matching Program until 2005.2 After the discontinuation of the formal matching process, residents were often asked to commit to a position during their third year of residency, before receiving adequate exposure to all subspecialties, or they were forced to accept or reject an offer before they could compare programs.
...
"A recent study assessed the match process and the Accreditation Council for Graduate Medical Education (ACGME) status of fellowships in the 9 orthopaedic subspecialties (adult reconstructive orthopaedics, foot and ankle orthopaedics, hand surgery, musculoskeletal oncology, orthopaedic sports medicine, orthopaedic surgery of the spine, orthopaedic trauma, pediatric orthopaedics, and shoulder and elbow surgery).3 This study discovered that 25% of available orthopaedic fellowship positions are devoted to sports medicine.3,12 Sports medicine is also the most popular orthopaedic subspecialty among current AAOS members, with the percentage of members who completed a sports medicine fellowship rising from 27% in 2004 to 49% in 2010.16 Additionally, orthopaedic sports medicine was found to have the highest proportion of ACGME-accredited fellowship programs, with 93.1% of programs and 97.3% of positions receiving accreditation.
...
:A 2014 study by Daniels et al3 investigated orthopaedic subspecialty fellowships in terms of the match process, characteristics, and ACGME accreditation. Fellowships were assessed by searching subspecialty society webpages and individual program websites. This study found that among the 9 orthopaedic subspecialty fellowships, there were collectively more positions offered than there were graduating orthopaedic residents.3 In 2013, there were 792 allopathic and osteopathic resident graduates and 897 total fellowship positions.3 The current study demonstrates that the opposite trend exists for applicants to sports medicine fellowships. In each year, excluding 2014, there were more sports medicine fellowship applicants than positions available."
***********
See my previous posts on orthopaedics, most of which are about the fellowship match.

The Fellowship Matches in Orthopedic Surgery

The Journal of Bone and Joint surgery has a new article on the experience of the fellowship matches in orthopedic surgery, many of which started after a study of the (then unraveled) match process in the 2008 article,
Harner, Christopher D., Anil S. Ranawat, Muriel Niederle, Alvin E. Roth, Peter J. Stern, Shepard R. Hurwitz, William Levine, G. Paul DeRosa, Serena S. Hu, "Current State of Fellowship Hiring: Is a universal match necessary? Is it possible?," Journal of Bone and Joint Surgery, 90, 2008,1375-1384.


The new report, by Lisa K. Cannada, MD, Scott J. Luhmann, MD, Serena S. Hu, MD, and Robert H. Quinn, MD is
The Fellowship Match Process: The History and a Report of the Current Experience, 2015-01-01Z, Volume 97, Issue 1, Pages e3(1)-e3(7), The Journal of Bone and Joint Surgery.

It's gated, so here are some relevant paragraphs:

"Beginning in 2007, there was substantial movement from the American Academy of Orthopaedic Surgeons (AAOS) and the American Orthopaedic Association (AOA) to promote a coordinated match process for orthopaedic fellowships. It is estimated that at least 90% of all orthopaedic surgery residents participate in a year of fellowship training 1 . The results of a survey at the 2007 AOA Symposium on Fellowships found that 79% of attendees believed that the current process was unacceptable and 87% believed that the process was unfair to residents 2 . The situation of those disciplines that were not in an organized match process was compared with problems often seen in a decentralized labor market 2 . A survey of residents indicated that 80% of residents were in favor of an organized match for fellowship and wanted a later date in their fourth postgraduate year for the decisions 2 .
...
"There have been previous attempts at a formalized match process for fellowship positions. However, the process for most subspecialties unraveled over time. The failure of the match process in the past was due to a variety of reasons: fewer applicants than positions, interviews in the third postgraduate year, early offering of positions, and the lack of a regulated process with a central agency for applications with deadlines 

The Orthopaedic Hand Surgery Fellowship Match is administered by the National Resident Matching Program (NRMP) and has been so since 1990.

The American Shoulder and Elbow Surgeons (ASES) made arrangements to administer their own match, which they have done since 2005.

The Sports Match was run through the NRMP until 2005. Sports rejoined the formal match process in 2008, using the San Francisco Match (SF Match).

The Adult Reconstruction Match joined SF Match in 2009, and the match is now run with the same applications and timeline as the Tumor Match. There has been no formal match in place for tumor fellowships in the past.

The Pediatric Orthopaedic Society of North America (POSNA) had a previous match that had failed, in part, because of noncompliance by the fellowship programs and directors. POSNA ran another match from 2008 to 2009 and joined SF Match in 2010.

The Spine Match involves cooperation among multiple societies: the North American Spine Society, the Cervical Spine Research Society, and the Scoliosis Research Society. They joined SF Match in 2009.

The American Orthopaedic Foot & Ankle Society was the pioneer in the new match process, initially beginning in 2006 through the NRMP. Subsequently, the American Orthopaedic Foot & Ankle Society joined SF Match in 2007.

The Orthopaedic Trauma Association (OTA) had a match program in the 1990s that dissolved. The OTA reinstated the match in 2007, which was initially administered through the OTA. In 2008, the OTA formalized the match process through SF Match."

Another important aspect is the time away from work and the financial burden of interviewing. As mentioned, residents have an average of ten interviews. This number seems to be consistent between the subspecialties and to be representative of the number of interviews for the fellowship match process. The subspecialty societies have different approaches to the process. The OTA previously offered interviewing at its annual meeting in the fall. However, many programs still require on-site interviews. Currently, the OTA annual meetings offer information sessions from the programs. In this way, the applicants can meet and can interact with faculty and can decide if the program would be suitable for them. The meeting affords the applicants the ability to talk to the fellowship program faculty and current and past fellows before spending several hundred dollars on an interview. Sports fellowships attempt to offer regional interviews so that the applicant can attend several interviews in a short time period, saving time and the added expense of additional flights.POSNA permits interviews at the International Pediatric Orthopaedic Symposium. The society encourages applicants to attend formal interviews at the fellowship location, but it is not a requirement.The Board of Specialty Societies Match Committee has offered interview space to each subspecialty society during the AAOS Annual Meeting. One perceived limitation of regional or national meeting interviews is the inability of the applicant to see the program site firsthand.The cost of the interviewing process associated with the match process has been raised as a concern by applicants from almost every subspecialty society. The costs cited by applicants in the post-match survey response from the applicants ranged from $600 for the interview process to more than $5000.".."A previous reason cited for the failure of the previous matches was the lack of process regulation. To ensure the integrity of a match process, guidelines need to exist. The biggest concerns lie in the area of communication between applicants and programs after the interview. The precedent for the current strict rules could possibly be traced back to the failure of the previous matches in the 1990s and early 2000s. There was no universal match process at that time. The ASES rules state: “No communication between the applicant and program director/staff after the interview.” Likewise, the spine and sports subspecialties have similar strict rules of no communication. The sanctions that each society has in place are available on their web sites. The subspecialty society for the respective match imposes any sanction necessary. Most sanctions to the program involve restriction from participation in the match for a specific time period to fellowship faculty not being allowed to serve on subspecialty boards of directors and/or committees or to the program being banned from making podium presentations or receiving research grants. There have been no major sanctions reported by any subspecialty society.In conclusion, with the advent of a fellowship match and the increased number of applicants, the fellowship application process is not so different from the residency application process. 

Market for Orthopaedic surgeons

I recently broke an ankle in Maastricht and flew home for surgery in Boston. In both places I visited the emergency room. In both cases the orthopaedic surgery resident who I was treated by in the ER was a young woman.

In Boston, I remarked that, when I was much younger, orthopaedic surgeons were almost all men, and that back then they claimed that orthopaedic surgery had a lot in common with carpentry, and required significant upper body strength. The resident told me that the situation had indeed changed, she had senior mentors who were women.

When she and her colleagues apply for subspecialty fellowships, they will face not only a more gender-integrated market but also a much more orderly market than in the recent past.

A "match" (a centralized clearinghouse) is coming for Orthopaedic surgery subspecialties--see the following preparatory study by two economists (Muriel Niederle and myself) and seven surgeons. (As it happens--small world--the surgeon who put in the many new titanium parts I now am growing new bone around had heard me give an Orthopaedic Surgery Grand Rounds on this subject.)

Harner, Christopher D., Anil S. Ranawat, Muriel Niederle, Alvin E. Roth, Peter J. Stern, Shepard R. Hurwitz, William Levine, G. Paul DeRosa, Serena S. Hu, "Current State of Fellowship Hiring: Is a universal match necessary? Is it possible?," Journal of Bone and Joint Surgery, 90, 2008,1375-1384.

Signaling for Orthopaedic surgery residencies

 The Journal of Bone and Joint Surgery reports a small survey about signaling behavior among applicants for orthopaedic surgery residencies.

Deckey, David G., Eugenia Lin, Coltin RB Gerhart, Joseph C. Brinkman, Karan A. Patel, and Joshua S. Bingham. "Decoding the Signals: An Analysis of Preference Signaling in the 2023 Orthopaedic Surgery Residency Match." JBJS Open Access 8, no. 3 (2023).

"While previously used in other specialties, the preference signaling program (PSP) was implemented in the 2022 to 2023 orthopaedic surgery residency application process for the first time. The PSP allowed for 30 signaling tokens to be sent by applicants to programs of their choice to indicate particular interest in a program.

...

"An anonymous electronic survey was emailed to all orthopaedic surgery residency applicants who applied to the authors' institution during the 2022 to 2023 application cycle. The survey was sent after match lists were submitted and closed before the release of match results. 

...

"The survey was completed by 101 applicants. Applicants applied to a mean of 90 programs (range: 10-197) and received an average of 12 interview invitations (range: 0-39). Applicants almost uniformly used all 30 signals, with nearly two-thirds signaling their home programs (65%, 49/76), and nearly all applicants sending signals to programs at which they performed away rotations (95.7%, 88/92). Applicants received a mean of 9 invitations from programs they signaled, compared with 2 invitations from programs they did not signal."

#######

Applicants report sending signals to all the programs that would have been expected to automatically give them interviews even in the absence of a signaling mechanism--namely their home programs and those which they have spent time visiting in 'away rotations.'

In the Economics job market, which may have been the first to introduce signaling, we limited applicants to 2 signals, and advised them not to signal jobs in which they already had well established mutual interests, i.e. not to signal jobs which they felt would interview them without signals. One of the ideas behind the Econ signaling mechanism is that there are many ways applicants can send signals of interest within a network to which they are well connected. Since some applicants are better connected than others, we were offering some signals that could be sent out of network.

It will be interesting to understand if signals to Ortho residencies are helping promote out of network interviews, or are largely strengthening the network connections already established by medical schools (home programs) and away rotations.

How can medical residency candidates be evaluated more reliably?

 Standardized tests as measures of physician aptitude are falling into disrepute and disuse.  Consequently the medical profession needs to develop better ways for evaluators (e.g. med school professors) to communicate information about applicants to residency programs.

Here are two reflections on the current state of afairs in Orthopaedic surgery.

Are Narrative Letters of Recommendation for Medical Students Interpreted as Intended by Orthopaedic Surgery Residency Programs?  by Egan, Cameron R. MD; Dashe, Jesse MD; Hussein, Amira I. PhD; Tornetta, Paul III MD

Clinical Orthopaedics and Related Research: February 25, 2021 - doi: 10.1097/CORR.0000000000001691

"Background: Narrative letters of recommendation are an important component of the residency application process. However, because narrative letters of recommendation are almost always positive, it is unclear whether those reviewing the letters understand the writer’s intended strength of support for a given applicant.

"Questions/purposes: (1) Is the perception of letter readers for narrative letters of recommendation consistent with the intention of the letter’s author? (2) Is there inter-reviewer consistency in selection committee members’ perceptions of the narrative letters of recommendation?

"Methods: Letter writers who wrote two or more narrative letters of recommendation for applicants to one university-based orthopaedic residency program for the 2014 to 2015 application cycle were sent a survey linked to a specific letter of recommendation they authored to assess the intended meaning regarding the strength of an applicant. A total of 247 unstructured letters of recommendation and accompanying surveys were sent to their authors, and 157 surveys were returned and form the basis of this study (response percentage 64%). The seven core members of the admissions committee (of 22 total reviewers) at a university-based residency program were sent a similar survey regarding their perception of the letter.

...

"Conclusion :Our results demonstrate that the reader’s perception of narrative letters of recommendation did not correlate well with the letter writer’s intended meaning and was not consistent between letter readers at a single university-based urban orthopaedic surgery residency program.

"Clinical Relevance: Given the low correlation between the intended strength of the letter writers and the perceived strength of those letters, we believe that other options such as a slider bar or agreed-upon wording as is used in many dean’s letters may be helpful."

**********

CORR Insights®: Are Narrative Letters of Recommendation for Medical Students Interpreted as Intended by Orthopaedic Surgery Residency Programs? by Zywiel, Michael G. MD, MSc, Clinical Orthopaedics and Related Research: April 29, 2021 - doi: 10.1097/CORR.0000000000001780

"With the upcoming transition of the USMLE Step 1 to a pass/fail score, and as we continue to gather more evidence calling into question the current selection criteria used for surgical training, programs are increasingly left to wonder how they can select learners that are most likely to succeed. Similarly, learners are increasingly left wondering how they can appropriately determine whether they are likely to succeed in a chosen specialty.

...

"Going forward, we need more research within the domain of selection criteria for training. This includes identifying more reliable predictors of technical skill, nontechnical skill, as well as performance in independent practice. The failure of most current selection criteria to adequately predict performance suggests that novel, specialty-specific instruments may need to be developed, evaluated, and ultimately incorporated at the medical student level to better predict future performance."

Applications and interviews prior to matching in Orthopaedic Surgery

It takes many applications and interviews to match for orthopaedic surgery residencies...

Matching in Orthopaedic Surgery
Chen, Antonia F. MD, MBA; Secrist, Eric S. MD; Scannell, Brian P. MD; Patt, Joshua C. MD, MPH
Journal of the American Academy of Orthopaedic Surgeons: February 15, 2020 - Volume 28 - Issue 4 - p 135-144
doi: 10.5435/JAAOS-D-19-00313

Abstract
In 2016, 1,137 fourth year medical students submitted applications for orthopedic surgery residency positions. Students applied to an average of 79 programs, resulting in in a total of 89,846 applications being submitted for 727 first year residency positions. This ratio of 124 applications per position is two SDs above the mean relative to other medical specialties. The average applicant for orthopaedic surgery residency attends 2.4 away rotations, as attending 2 away rotations increases an applicant's odds of matching, and submits 83 applications. This excessive number of applications overburdens programs, subjects applicants to considerable costs, and diminishes the quality of fit between interviewees and programs. Eighty-three percent of program directors use step 1 United States Medical Licensing Examination scores as a screening tool to decrease the number of applications necessary for review. The average matched applicant attended 11.5 interviews, and Step 1 scores, research productivity, and Alpha Omega Alpha (AOA) status can be used to predict the number of applications necessary to obtain 12 interviews. AOA membership has the strongest influence on interview yield. Applicants report spending an average of approximately $7,000 on the interview process, and 72% borrow money to cover these costs. Post-interview contact, although forbidden by the National Resident Matching Program , has been reported by 60% to 64% of applicants.

The Orthopedic Surgery Fellowships

I recently occupied the attention of some knee surgeons, with whom I had interacted in a very different way when writing about how fellowship matches might be implemented for orthopedic surgeons (Harner, Christopher D., Anil S. Ranawat, Muriel Niederle, Alvin E. Roth, Peter J. Stern, Shepard R. Hurwitz, William Levine, G. Paul DeRosa, Serena S. Hu, "Current State of Fellowship Hiring: Is a universal match necessary? Is it possible?," Journal of Bone and Joint Surgery, 90, 2008,1375-1384.)

So I read this state-of-the-match report with interest...

Orthopedic Surgery Fellowships: The Effects of Interviewing and How Residents Establish a Rank List

Matthew C. Niesen, MD; Jeffrey Wong, MD; Edward Ebramzadeh, PhD; Sophia Sangiorgio, PhD; Nelson Fong SooHoo, MD; James V. Luck, MD; Jeffrey Eckardt, MD
Orthopedics
March 2015 - Volume 38 · Issue 3: 175-179

"Prior to the establishment of the Orthopaedic Fellowship Match in 2008, orthopedic residents had to decide where to complete fellowship training in an environment without a formal match system.1 Annual meetings were “free-for-alls” with both programs and residents anxious to make a deal. If interviews occurred, they were uncoordinated and happened earlier and earlier in the process as programs competed for top candidates and residents tried to secure positions at the best programs. Often, both sides involved in the process had to make a decision without knowing what alternatives might exist or arise in the future. Residents frequently were uncertain whether they would receive additional offers and were pressured to commit to a fellowship, while fellowship programs had high interview cancellation rates because residents had already taken other positions. This system may have caused residents and fellowship programs to settle for a less than ideal “match.” A 2008 American Orthopaedic Association symposium indicated that more than half of all residents surveyed accepted their first fellowship offer. The same survey showed that 78% of residents favored a transition to a centralized match.2

One of the primary goals of the Orthopaedic Fellowship Match was to create a fairer and more coordinated process in which both applicants and fellowship programs had time to evaluate and consider their training options. However, with the establishment of the match, new complications arose. Fellowship programs require additional planning for interviewing and ranking a large number of residents, residencies need to find coverage for residents away from their clinical duties, and residents face the additional issues of cost and time away from service. The magnitude of these factors is currently unknown. A primary goal of this study was to establish benchmark values for these factors and to quantify the financial impact and time away from service, specifically for residents. This information will be valuable to residents and both residency and fellowship programs as they complete the fellowship match process.

After applying to and interviewing for fellowships, residents have the task of establishing a final rank list. Several studies have identified the factors medical students in the United States and Canada consider most important when establishing a rank list for residency positions.3–5 The factors that are commonly ranked the highest for residency positions include clinical experience, location, and academic reputation. After a brief review of the literature, it is apparent that limited data exist identifying what factors are most important for residents of all specialties in medicine and surgery when choosing a fellowship program or establishing a rank list. A recently published study evaluated what factors residents pursuing pediatric otolaryngology positions considered to be most important when establishing a rank list for fellowship positions.6For these applicants, appropriate experience, faculty reputation, and location were ranked as the most important factors when choosing a program. Accreditation Council for Graduate Medical Education (ACGME) accreditation, fellowship longevity, and salary were less important. The factors orthopedic surgery residents consider most important when establishing a rank list for fellowships have not been investigated. Thus, this was a second goal of this study.

...

Conclusion

The Orthopaedic Fellowship Match has created an environment in which residents can consider a greater number of options without feeling pressure to make a quick decision. The vast majority, nearly 96%, of orthopedic surgery residents pursue fellowship training; sports medicine, hand, and adult reconstruction were the most popular subspecialties in this study. The interview process has financial implications and draws residents away from clinical services. The estimated cost of the orthopedic fellowship interview process was $4671±$2454, with a median of 10 days off service required to interview. These are the first benchmark values established for these variables.

Residents should attempt to be more selective with their application choices to minimize unnecessary costs, time off service, and associated stress and fatigue. Finally, this study is the first to demonstrate what factors orthopedic surgery residents consider most important when establishing a final rank list for fellowships. Residents value operative experience, autonomy, and fellowship staff members the most and place the least importance on research opportunities and salary. Fellowship programs may find this information useful as they choose which aspects of their programs to highlight during recruitment activities.

Signaling in resident and fellowship matches to reduce interview congestion

An article by Dr. Joseph Bernstein in Clinical Orthopaedics and Related Research (December 2017, Volume 475, Issue 12, pp 2845–2849 ) argues that a signaling mechanism might help deal with congestion in the Orthopaedic Surgery match.
"Not the Last Word: Want to Match in an Orthopaedic Surgery Residency? Send a Rose to the Program Director," (gated)

I was invited to write the comment, below, which appeared along with two other (less favorable) comments from surgeons. (All of the comments appear, without titles, in the same Not the Last Word column at the link...)

Roth, Alvin E. “Congestion and signaling in residency matching,” Clinical Orthopaedics and Related Research, December 2017, 475: 2847, 2849

Now that applying to many residency programs is easy, programs receive so many applications that they have trouble deciding whom to interview, particularly because receiving an application is no longer as strong a signal of interest as it was when applying was harder [1]. The same could be said for how residents applied to colleges when they were younger, and how they will apply to fellowships when they are older. The internet and common application tools make sending applications easier, and evaluating them harder. (This is the common problem of congestion: e.g. it’s harder to use email when we get too many emails, etc.…)

In congested markets, in which not every interesting applicant can be interviewed, signals are important. An application itself is a signal about an applicant’s accomplishments. Like a peacock’s tail, it shows how desirable a candidate is, i.e. why the program should be interested in the applicant. When a program receives too many applications it becomes more costly to read them all, but each one continues to convey the applicant’s accomplishments.

What is lost when applications are easy to send is how interested the applicant is in the program. And, in a congested market, it helps to be able to signal not only how interesting you are, but also how interested, because programs that can’t interview every attractive applicant need to devote much of their interviewing to applicants who might ultimately be interested.

In Economics, the AEA’s signaling system allows each candidate to send no more than two signals of particular interest in being interviewed, for free [2].  Why [only] two? Because while it can be shown how one signal can unambiguously improve the process of selecting candidates for interviews [3], too many signals could harm the process.  Suppose we allowed 50 signals: then the absence of a signal would start to be a signal itself (“this candidate must not be interested in us at all if he didn’t even send us one of his 50 signals…”)  Signals get much of their value by being scarce. So when you can send only two, a program which receives one knows that you targeted them as one of only two recipients.

To which programs should a candidate signal? We advise candidates not to send either of their signals to the very top programs in their field. Those programs can simply interview the candidates they like best, since they have good reason to believe that every application signals genuine interest. Signals will do the most good if sent to programs that should be interested in the candidate, but to whom it might not be obvious that he or she is interested in them.

The resident match removes congestion from the process of making offers and accepting or rejecting them, since each participant can submit a long rank order list that is processed centrally [4]. But interviewing remains congested, because interviews take time. It is worthwhile considering how changes in the market design [5] could smooth the process.  Organizing a signaling system—and then monitoring how it works--seems like a promising step.

[1] Bernstein, Joseph, “Want to Match in an Orthopaedic Surgery Residency? Send a Rose to the Program Director,” this journal, this issue [?]

[2] Coles, Peter, John H. Cawley, Phillip B. Levine, Muriel Niederle, Alvin E. Roth, and John J. Siegfried, “The Job Market for New Economists: A Market Design Perspective,” Journal of Economic Perspectives, 24,4, Fall 2010, 187-206.

[3]  Coles, Peter, Alexey Kushnir and Muriel Niederle, “Preference Signaling in Matching Markets”, American Economic Journal: Microeconomics, 2013, 5(2), 99-134

[4] Roth, Alvin E.  “The origins, history, and design of the resident match,” JAMA. Journal of the American Medical Association, vol. 289, No. 7, February 19, 2003, 909-912.

[5] Roth, Alvin E. Who Gets What—and Why: The New Economics of Matchmaking and Market Design, An Eamon Dolan Book, Houghton Mifflin Harcourt, Boston, New York, 2015.

Signaling for medical residencies: the first few years

Two papers report on signaling in Otolaryngology and Orthopaedic surgery.

 The Otolaryngology societies have a summary of the current state of affairs that's worth hearing. A number of specialties (including Oto) allow many signals, and these seem to be acting as a soft cap on applications, rather than as a signal of special interest as in specialties that (like Economics) allow only a small number of signals.

Preference Signaling in Otolaryngology—Past, Present, and Future: A Comment From the Society of University Otolaryngologists (SUO), Association of Academic Departments in Otolaryngology (AADO), and the Otolaryngology Program Directors Organization (OPDO)  by Steven D. Pletcher MD, Bradley F. Marple MD, David J. Brown MD, The Laryngoscope Early View,  First published: 04 July 2024  https://doi.org/10.1002/lary.31613

"The year 2020 was a year of change. The residency application process, already suffering from spiraling application numbers,1 now faced the COVID-19 pandemic with a loss of away rotations and apprehension about virtual interviews. In the face of change, the Otolaryngology Program Directors Organization Council (OPDO) approached the leadership of the Association of Academic Departments in Otolaryngology (AADO) and the Society of University Otolaryngologists (SUO) with a recommendation to implement preference signaling. This system, originally described in the economics PhD marketplace,2 allows students a set number of signals (Otolaryngology used 5 in its inaugural year) to send to programs of particular interest. 

...

"Following the lead of Otolaryngology, Urology, General Surgery, Internal Medicine, and Dermatology implemented preference signaling the following year. Since that time, signaling has grown exponentially and is now utilized in the residency application process of nearly every specialty. 

...

"In the 2024-2025 residency application cycle, the evolution of preference signaling continues. Building on Otolaryngology's experience, in the 2023 application cycle Orthopaedic Surgery implemented a high-signal approach, providing applicants with 30 signals. This transition shows promise for reversing the vexing problem of spiraling application numbers—“Big Signaling” has now been adopted by Otolaryngology and four additional specialties the majority of whom have shown a 25%–30% decrease in applications submitted per student saving students a combined $2.5 million in application fees alone. Obstetrics and Gynecology has piloted a tiered signaling system, providing three gold and 15 silver signals to their students. 

...

"Because the number of signals received by programs is not publicized, students are unable to reliably target programs where their signals are less likely to be diluted by competing signals. Specialties should consider providing voluntary “signal cohort” (i.e., my program received between 75 and 100 signals in the 2024 application cycle) data to help applicants make more informed signal decisions and programs with low signal numbers will likely attract additional candidates. 

...

"One of the key statistics to guide applicants in high signal specialties is the interview offer rate for non-signal applications: this helps define the value of applications beyond the set number of signals."

########

Preference Signaling in the Orthopaedic Surgery Match: Applicant and Residency Program Attitudes, Behaviors, and Outcomes, by Guthrie, Stuart Trent MD, FAOA1,a; Dagher, Tanios BSE2; Essey-Stapleton, Jodi MS, MEd3; Balach, Tessa MD, FAOA2,  JBJS Open Access 9(2):e23.00146, April-June 2024. | DOI: 10.2106/JBJS.OA.23.00146

"In the first year of preference signaling, applicants reported applying to 16% fewer programs than if preference signaling had not been available. These results align with AAMC data, which report applications per program dropping 17.4% (from an average of 639.6 to 774.6), and applications per applicant dropping to 76.9 from 86.07,8. Further changes could occur in future cycles as students become more accustomed to the influence of signaling on their application."

Signals and interviews in the transition from medical school to residency

Late last year I was interviewed by Dr. Seth Leopold, who is a Professor in the Department of Orthopaedics and Sports Medicine at the University of Washington School of Medicine, and Editor-in-Chief of the journal Clinical Orthopaedics and Related Research.   That interview has just appeared ahead of print on the journal's website: 

A Conversation with … Alvin E. Roth PhD, Economist, Game Theorist, and Nobel Laureate Who Improved the Modern Residency Match  by Leopold, Seth S. MD, Clinical Orthopaedics and Related Research: April 7, 2021 - Publish Ahead of Print - doi: 10.1097/CORR.0000000000001758

Here's one part of our Q&A:

Dr. Leopold:You once commented in a Not the Last Word column in CORR® that the Match might be improved if a bit more room could be made for candidates to send “signals” to programs that indicate particular interest[5]; if you could make one change to the Match right now to make it fairer all around, what would that change be?

Dr. Roth: I don’t yet know enough about the whole pre-Match process of applications and interviews to answer that confidently. I’m hoping to gain access to data that will illuminate more clearly how applications lead to interviews, and how interviews interact with other kinds of information to influence what rank-order lists are submitted by applicants and programs. Some of that process is surely in flux, between the pandemic causing interviews to be conducted remotely and the United States Medical Licensing Examination Step 1 going pass/fail. Signaling is a way to address miscoordination in interviewing (such as whether too many interviews are concentrating on too few candidates), but there are other ways the interview process might be broken that might better be addressed by other tweaks in how interviews are organized.

Dr. Leopold:I believe the study you’re proposing here would find a very attentive audience, both in medical schools and residency programs across the country, especially competitive ones like orthopaedic surgery. Based on other kinds of markets you’ve evaluated—I recognize I’m asking you to speculate—what do you think you might find here?

Dr. Roth: Presently, in at least some specialties, many interviews are conducted for each residency and fellowship position. It could be that interviews play a critical role in allowing programs and applicants to assess each other, regardless of the other information they may have. But it could also be that at least some interviews are being conducted “defensively,” because all the interviews that others are participating in make it hard for each program or applicant to predict how likely any interview will lead to a position being offered and accepted in the Match. So, it is possible that there is “too much” interviewing, in the sense that in perhaps predictable ways, some programs are interviewing some candidates they can virtually never hire, and some candidates they would never want to hire. Conversely, applicants are interviewing for some jobs they have hardly any chance of being offered, and some they sensibly think they won’t need to take. Of course, some things can be predictable even if they can’t be predicted by individual applicants and programs with the information they now have available. It might therefore be possible to suggest institutional reforms that would help reduce the uncertainty in deciding which interviews to offer. That might also reduce the number (and costs) of interviews. (In just such a way, the Match helped solve the problem of uncertainty involved in offers and acceptances, back when offers were exploding.) And there’s a possibility that fewer interviews could make everyone better off in terms of expectations, particularly if participants on both sides of the market will feel a reduced need to do so many interviews if everyone else reduces the number they do. But as you say, until we can look into this carefully, I’m just speculating.

Money and medicine in Britain's National Health Service

The London Times reports NHS bars woman after she saw private doctor

"A WOMAN has been denied an operation on the NHS after paying for a private consultation to deal with her severe back pain.
Jenny Whitehead, a breast cancer survivor, paid £250 for an appointment with the orthopaedic surgeon after being told she would have to wait five months to see him on the NHS. He told her he would add her to his NHS waiting list for surgery.
She was barred from the list, however, and sent back to her GP. She must now find at least £10,000 for private surgery, or wait until the autumn for the NHS operation to remove a cyst on her spine. "

The managing of waiting lists for scarce resources is a tough business.

Unraveling

Many markets have trouble coordinating on the timing of transactions, and this has led to market failures in markets as diverse as the market for college football bowls, and the labor market for federal court clerks, and in various medical markets, such as (most recently) gastroenterologists and orthopaedic surgeons.

Why do transactions in some markets happen inefficiently early? Here are the concluding paragraphs from our recent NBER working paper Unraveling Results from Comparable Demand and Supply: An Experimental Investigation
by Muriel Niederle, Alvin E. Roth, M. Utku Unver - #15006 (LS)

" It has been known at least since Roth and Xing (1994) that many markets unravel, so that offers become progressively earlier as participants seek to make strategic use of the timing of transactions. It is clear that unraveling can have many causes, because markets are highly multidimensional and time is only one dimensional (and so transactions can only move in two directions in time, earlier or later). So there can be many different reasons that make it advantageous to make transactions earlier. There can also be strategic reasons to delay transactions; see e.g. Roth and Ockenfels (2002) on late bidding in internet auctions.
Thus the study of factors that promote unraveling is a large one, and a number of distinct causes have been identified in different markets or in theory, including instability of late outcomes (which gives blocking pairs an incentive to identify each other early), congestion of late markets (which makes it difficult to make transactions if they are left until too late), and the desire to mutually insure against late-resolving uncertainty. There has also been some study of market practices that may facilitate or impede the making of early offers, such as the rules and customs surrounding "exploding" offers, which expire if not accepted immediately.
In this paper we take a somewhat different tack, and consider conditions related to supply and demand that will tend to work against unraveling, or to facilitate it. There seems to be a widespread perception, in markets that have experienced it, that unraveling is sparked by a shortage of workers.
But for inefficient unraveling to occur, firms have to be willing to make early offers and workers have to be willing to accept them. Our experiment supports the hypothesis that a shortage of workers is not itself conducive to unraveling, since workers who know that they are in short supply need not hurry to accept offers by lower quality firms. Instead, in the model and in the experiment, it is comparable supply and demand that leads to unraveling, in which attention must be paid not only to the overall demand and supply, but to the supply and demand of workers and firms of the highest quality. This seems to reflect what we see in many unraveled markets, in which competition for the elite firms and workers is fierce, but the quality of workers may not be reliably revealed until after a good deal of hiring has already been completed."

http://papers.nber.org/papers/W15006

Postscript: Skip Sauer over at The Sports Economist has a post about a 9th grader offered a college football scholarship in what is becoming a seriously unraveled market.

Signaling in the markets for new doctors

 Signaling of interest is catching on in medical labor markets for residents and fellows.

Here's some material from Thalamus (which describes itself as "Complete GME interview management solution for applicants & programs. Easy, secure, and automated interview scheduling to optimize in-person & virtual recruitment.")

The Ultimate Guide to Preference Signaling for Medical Residency Applicants and Programs 2022-2023.

• Part 1: Introduces the history of preference signaling.
• Part 2: Describes its introduction to GME.
• Part 3: Discusses outcomes to date.
• Part 4: Provides advice for applicants and programs.
• Part 5: Describes how Thalamus will support this initiative.

It all seems to have started with the signaling mechanism we use in Economics.

From Part 1: 

"The Emergence of Preference Signaling:

Preference signaling was first implemented in 2006, as part of the recruitment process for economics graduate students administered through the American Economics Association (AEA). Since then, there have been several useful studies analyzing this process by leading economists at institutions including Harvard and Stanford. These include “Preference Signaling in Matching Markets” and “The Job Market for New Economists: A Market Design Perspective.”

"Of note, one of the authors on the latter article is Dr. Alvin E. Roth, who won the Nobel Prize in Economics for proving certain key attributes of the matching algorithm that is used today by the National Resident Matching Program (NRMP), where Dr. Roth currently serves as a board member. This article has been cited in papers throughout GME that examine preference signaling in specialties including Otolaryngology and Orthopaedic Surgery."

********

Earlier:

Sunday, November 15, 2009

Signaling for interviews in the Economics Job Market: Registration opens tomorrow

Incentives in (British) medicine

The Telegraph reports on the Annual Congress of the British Orthopaedic Association: Severely injured patients are being delayed specialist care because Government targets mean hospitals are encouraged to operate on routine cases first, experts have warned.

"Routine cases such as hip and knee replacements have to be carried out within Government waiting time targets meaning urgent trauma cases are delayed which can jeopardise their recovery."

School choice in SF moves forward

Yesterday Muriel Niederle and Clayton Featherstone were among the presenters to the San Francisco Board of Education, speaking about possible designs for a new school choice system there. It seems that they are well on the way to a good outcome.

One of the Board members, Rachel Norton, has a blog on which she posted before and after accounts of the meeting:
Tonight’s student assignment meeting should be interesting!
Recap: Closing in on a student assignment policy

Here is a video of the whole meeting (but you can navigate a bit so you don't have to watch the full 3 hours: Muriel's testimony, from her slide presentation through answering of questions from the board is from 1:09 to 2:09 on the video).

For the technically inclined, papers about our prior work on school choice systems in NYC and Boston are here.

It has been mentioned in the SF discussions that our team of market designers has worked on a number of problems aside from school choice, so here are background links on some of them for SF readers who are interested:
National Resident Matching Program and related medical labor markets
Gastroenterologists, Orthopaedic surgeons
Kidney Exchange
AEA market for new economists

Market for health care: no law of one price

The Boston Globe ran a story about healthcare costs at different Boston area hospitals, in which fees are negotiated between insurers and individual hospitals: A healthcare system badly out of balance

""The same service delivered the same way with the same outcome can vary in cost from one provider to the next by as much as 300 percent," said Charles Baker, president of the state's second-largest health insurer, Harvard Pilgrim Health Care. "There is no other sector of the economy anywhere in this country in which that kind of price variability with no appreciable difference in service or product quality can sustain itself over time.""

Pricing certainly serves different function in health care than in other parts of the economy, and tertiary care teaching hospitals do more than provide simple patient services (e.g. they also train future docs, about which see my previous post today, on Orthopaedic surgeons). So, as the healthcare system is brought into better balance, some attention will have to be paid to paying for some of the things that now may be paid for with hidden cross-subsidies.


HT Paul Kominers (younger brother of the remarkable if less cool SK)

Discriminatory quotas in admissions to universities and graduate schools (and some black humor from my father's generation)

 History doesn't exactly repeat itself, but it rhymes. (Mark Twain apparently missed the opportunity often attributed to him of saying that.) Here's a story about the history of ethnic quotas for Jews, as it played out in medicine in the 20th Century. (And at the bottom, some old jokes about coping with it...)

The History of Discriminatory Jewish Quotas in American Medical Education and Orthopaedic Training, by Solasz, Sara J. BA1; Zuckerman, Joseph D. MD1; Egol, Kenneth A. MD1,a, The Journal of Bone and Joint Surgery 105(4):p 325-329, February 15, 2023. | DOI: 10.2106/JBJS.22.00466

"By the early 1920s, formal quota systems were put in place to limit the number of Jewish applicants admitted to many American medical schools. This quota was partly a result of the second wave of Jewish immigration and the subsequent rise in antisemitism in the country. As a sign of the growing antisemitism in America, in 1920, Henry Ford published a weekly series called “The International Jew: The World’s Problem” on the front page of his newspaper, The Dearborn Independent. In this series, which continued for years, Ford fueled antisemitism both at home and abroad. The effect at the time was enormous: the feeling was that if an American icon as rich and powerful as Ford could hold this conviction so strongly, then it must be credible5,6. In addition to the effect on medical school admissions, measures were taken to deny Jewish people access to social institutions, neighborhoods, swimming pools and beaches, and employment.

...

"Harvard was known to have quotas restricting the number of Jewish students admitted to the college under the leadership of Lowell. In a letter to a philosophy professor, Lowell wrote that admitting Jewish students would ruin Harvard, “not because Jews of bad character have come; but the result follows from the coming in large numbers of Jews of any kind, save those few who mingle readily with the rest of the undergraduate body.”8

"It is within this social and political climate that the Jewish quota system appeared in medical schools throughout the U.S. Although medical school officials have always denied the existence of Jewish quotas, records from schools across the country reveal a systematic and intentional anti-Jewish prejudice. The medical historian Henry Sigerist wrote that Jewish students were subject to a “tacit, but nevertheless highly effective, quota system and in most schools the number of Jewish students rarely exceeds 10 per cent.”9 Many mainstream thinkers in higher education argued for further reductions in the acceptance rate for Jewish students, advocating for discrimination against Jews under the guise of keeping the “national ratio correct,” which would bring down the number of accepted students to represent only 3% to 4% of the total class.

...

"The most significant decrease in the number of admitted students occurred at Columbia University, which asked for the applicant’s religion, parents’ birthplaces, racial origin, and mother’s maiden name11. At the Columbia College of Physicians and Surgeons, the rate of enrollment of Jewish students between 1920 and 1940 dropped from 47% to 6%; during the same period, the rate dropped from 40% to 5% at Cornell University Medical College3. Throughout the Northeast, where the concentration of Jewish applicants was the highest, quotas appeared at schools such as Harvard Medical School, Yale School of Medicine, and the Woman’s Medical College of Pennsylvania3.

...

Facing this widespread sentiment, Jewish students hoping to gain admission to medical school were forced to take action—with some even changing their last names to avoid discrimination. Medical schools identified “Jewish names” on applications, especially when the applicants were from areas with large Jewish populations, to indirectly discriminate against Jewish students. Soon, schools expanded applications to require completion of a “change of name” section. *

...

"Prior to the establishment of the current U.S. residency “match” system, each residency position was sought individually with an application and interview and was typically followed by a near-immediate offer of a position. This system certainly provided a biased and unfair method for filling training programs.

...

"In New York State, the Education Practices Act (1948) set a precedent for other states to pass legislation to eliminate discriminatory admissions practices. As the wave of antisemitism began to fade and the need for physicians grew, medical schools and graduate medical education programs started to remove the quota systems, which came to a complete end in the 1970s."

##############

*Black humor was common in my dad's generation, including jokes about name changing (sometimes told in Yiddish). Here are two, approximately remembered.

1. Shmuel and Moshe, friends from Odessa, meet in New York after both have immigrated to America.  Moshe spots him from a distance and rushes over, calling out "Shmuel!"  They embrace, and Shmuel says, "I'm called Sam now, in America.  How about you?"

Moshe says "my American name is Sean Ferguson."  Sam is astonished, and asks "how come?"  "Well," says Moshe/Sean, I had picked out a good American name, but I was so nervous when I got to Ellis Island that I couldn't remember it. So when the immigration officer asked me my name, all I could think of to say was "I've already forgotten/ ikh hab shoyn fargesn, which is what he wrote down."  (איך האב שוין פארגעסן)

2. A few years later, Sean Ferguson goes to court to change his name again, to John McMillan. The judge asks him why he wants to do that.  He says "When I apply for positions, people ask what my name used to be..."