Wednesday, December 2, 2020

Chinese college admissions reform: some consequences, by Yan Chen, Ming Jiang and Onur Kesten in PNAS

 An empirical evaluation of Chinese college admissions reforms through a natural experiment by Yan Chen, Ming Jiang, and Onur Kesten

PNAS first published November 24, 2020;

Abstract: College admissions policies affect the educational experiences and labor market outcomes for millions of students each year. In China alone, 10 million high school seniors participate in the National College Entrance Examination to compete for 7 million seats at various universities each year, making this system the largest centralized matching market in the world. The last 20 years have witnessed radical reforms in the Chinese college admissions system, with many provinces moving from a sequential (immediate acceptance) mechanism to some version of the parallel college admissions mechanism, a hybrid between the immediate and deferred acceptance mechanisms. In this study, we use a natural experiment to evaluate the effectiveness of the sequential and parallel mechanisms in motivating student college ranking strategies and providing stable matching outcomes. Using a unique dataset from a province that implemented a partial reform between 2008 and 2009, we find that students list more colleges in their rank-ordered lists, and more prestigious colleges as their top choices, after the province adopts the parallel mechanism in its tier 1 college admissions process. These listing strategies in turn lead to greater stability in matching outcomes, consistent with our theoretical prediction that the parallel mechanism is less manipulable and more stable than the sequential mechanism.

Tuesday, December 1, 2020

The labor market that is the military: a report from the National Academies of Science, Engineering, and Medicine

 I recently served as a member of a National Academies committee on the issues facing the Air Force in managing its human capital, i.e. its labor force.  It resulted in a long report covering many aspects of Air Force policy.  

One of the fun things about that assignment was meeting with people throughout the U.S. armed forces (and some allies).  Yesterday I had the privilege of attending the promotion ceremony for a very thoughtful Air Force officer, Tobias Switzer. Congratulations, Colonel!.*

Readers of this blog are likely to be most interested in the evolving ways that military personnel are matched to new assignments, from time to time during their careers.  A first order problem has to do with retention, since the armed services compete with the private sector for highly trained people (e.g. pilots, cyber warriors, special operators, to name a few), which is most visible at times when the person in question doesn't have any further military service obligation.  Below, I excerpt some paragraphs concerning one new aspect of this process, called the Talent Marketplaces. (The page numbers refer to sections of the report, e.g. "p2-7" refers to page 7 of section 2.)

Here's the report:

A Consensus Study Report, from the National Academies of Sciences, Engineering, and Medicine 2020:

Strengthening U.S. Air Force Human Capital Management: A Flight Plan for 2020-2030. Washington, DC: The National Academies Press.

Committee on Strengthening U.S. Air Force Human Capital Management, Board on Human-Systems Integration, Division of Behavioral and Social Sciences and Education

Committee and Staff: Julie J.C.H. Ryan (Co-Chair), William J. Strickland (Co-Chair), Terry A. Ackerman, David S.C. Chu, Lt. Gen. Gina M. Grosso, Usaf (Retired), Brig. Gen. Leon A. Johnson, Usafr (Retired), Judith S. Olson, Dan J. Putka, Alvin E. Roth, Ann Marie Ryan, Stephen Stark, Cherie Chauvin, Study Director, Elizabeth T. Cady, Senior Program Officer, Daniel Talmage, Program Officer, Margaret Kelly, Sr. Program Assistant, Tina M. Latimer, Program Coordinator

Below are some snippets drawn from various sections of the report.

"since 2014, the Air Force has lost more fighter pilots annually than its annual production rate, and forecasts of the commercial airline pilot marketplace are an important variable in long-term workforce planning to develop and sustain the career field." (p2-7)

"A lack of clear data on why Airmen are leaving the force diminishes the Air Force’s ability to do such evaluations or establish an “early warning system” to identify recruits at risk of early attrition. Increased use of assignment tools like the Talent Marketplace will allow relevant longitudinal data to be collected on the extent to which Airmen’s separation decisions may be related to their preferences over the positions to which they could have been assigned as compared to the one to which they were assigned (see Appendix D)."  (p4-22)

"the recently implemented Talent Marketplace provides an innovative online means for matching Airmen with assignments based on expressed preferences of Airmen and position owners for post-accession job assignments. However, the usefulness of expressed preferences depends a good deal on how much information is available to Airmen about positions and to position owners about Airmen (through the Talent Marketplace), given that such information profoundly shapes preferences. Consequently, the most effective version of the Talent Marketplace would also serve as an information marketplace that allows position owners and Airmen to make appropriate information available to each other to form informed, accurate preferences (e.g., realistic job previews, using video and written descriptions). The key point is that matching, and the overall functioning of the USAF HCM system, may be improved by developing new methods of sharing preferences as well as new algorithms for taking preferences into account (see Appendix D)" (p4-26)

"In 2019, the Air Force began using the web-based platform, the Talent Marketplace, for making officer assignments (Lieutenant Colonel and below). The Talent Marketplace was initiated in an effort to meet a stated goal: “To the maximum extent possible, assign individuals on a voluntary basis and in the most equitable manner feasible while meeting mission and commander needs” (USAF, 2018a, p.2)." (p5-10)

"The Air Force is investigating implementing the Talent Marketplace to manage enlisted assignments, but a decision is still pending at this time."(p5-11)

"There are two officer assignment cycles per year. To support the Officer Assignment System for all assignments of Lieutenant Colonel and below (except for Judge Advocate General officers), the Air Force uses its newly developed Talent Marketplace. The web-based system provides transparency for available positions, provides visibility on an officer’s preferences to their commander, and incorporates gaining commander input as well for the first time. The technology behind it examines officer assignment solutions by incorporating specific prioritizations from both the officers who are eligible to move and the gaining unit (see Appendix D for discussion of preference informed matching). Officers on the vulnerable-to move list use the Talent Marketplace to indicate a desirability rating for assignment location preferences using a list of locations with jobs to fill, in alphabetic order. There is limited information on each position in the system: duty title, command, and location, but the officer can see how many other officers are interested in the position and make their decision accordingly.17 Any additional information the officer would like to know about the position is gathered through their own independent research. After the window closes for officers to bid for positions, the position owners access the system to see the final list of volunteers to fill their positions. The results of the matching algorithm are used as a first step in the process, which is further adjusted as needed and finalized by AFPC." (p5-11)

"As the Air Force expands its use of the Talent Marketplace for officers and develops a more modern approach to the antiquated Enlisted Quarterly Assignments List, it could benefit from considering the research conducted on and the implementation and results of similar marketplace initiatives (see for example, Malia, 2020). The U.S. Army, in particular, recently implemented its Army Officer Assignment Marketplace via Assignment Interactive Module (AIM).30 AIM is a centralized clearinghouse that requires officers and units to finalize preferences for the other side of the market at the same time, typically 6–9 months before officers are expected to move to their next assignment. The Army studies its assignment system in the Office of Economic and Manpower Analysis (OEMA), Department of Social Sciences, West Point.31 Information provided to the committee on the design and administration of AIM indicated that it is something of a hybrid model that involves a lot of hand-processing of assignments by assignment officers. However, it was reported that the “Human Resources Command received permission from the Assistant Secretary of the Army for Manpower and Reserve Affairs to test the concept of matching officers to jobs according to a deferred acceptance algorithm similar to that of the National Resident Matching Program,”32 which matches new doctors to residency programs (Roth, 1984; Roth and Peranson, 1999). 

"One aspect of the Army Cadet Branching process (used to assign new West Point graduates to branches) is that Cadets can bid for preferred assignments by offering to extend their service obligation. Although a detailed review of AIM was outside the scope of this study, the committee notes that analysis has suggested that its use of a deferred acceptance algorithm is seriously flawed in implementation (e.g. Switzer, 2011), and this may be important for the designers and administrators of the Air Force Talent Marketplace to study and understand." 

"As of May 15, 2019, the Army reports that 90,952 officers of all ranks had participated in AIM. Officers are encouraged to submit resumes, but the majority have not done so (from 7% of Colonels, up to 42% of Majors). Resume-writers are encouraged to include unique Knowledge, Skills, and Behaviors (KSBs), but almost none have done so. Units seeking to fill jobs are encouraged to submit (long) preference lists of officers for each job they are seeking to fill, and so far, participation has been spotty: “The best participation is from [the 10 percent of] units who submit preferences for many jobs, and for multiple officers on each job” (U.S. Army, 2019, p. 2). Although no explanation was provided to the committee, the Army’s overall low participation numbers suggest that the Air Force and Army could benefit from joint efforts to understand the reaction Airmen and Soldiers have had to these initiatives as well as to develop approaches to improve participation and therefore its overall utility.

"Turning back to the Air Force, the present (preliminary) use of deferred acceptance algorithms in the Talent Marketplace is not an evidence-based decision; rather it arises from a perhaps too-hasty parallel with the operation of the private-sector clearinghouse for American physicians (see discussion of preference informed matching in Appendix D). To guide Air Force research into post-accession assignment decisions to improve and expand the effectiveness and validity of the Talent Marketplace, the committee identified the following research questions:

 How do assignments affect separation decisions? 33 (Collect data on “Who stays, who leaves, where they go, would they have stayed for a better assignment?”)

 How are preference data related to family data? (Collect family data on jobs of spouses, age, and schooling of children)

 How is the new blended retirement system changing separation choices?

 How do pilot separations interact with airline hiring demands? (E.g., could new arrangements be initiated between the Air Force and commercial airlines to better meet fluctuating needs of both?34)

 Can exit interviews be combined with interventions that might prevent or delay separation by the most effective Airmen?

 How should the Talent Marketplace be organized for minimizing early separation of the most effective Airmen?

 Does the Talent Marketplace assure an appropriate distribution of talent across units?

 How does the Talent Marketplace affect individual and unit performance?" (pp5-27-28)

"Similar to the Army, the Air Force is implementing a “Talent Marketplace” that gives Airmen increased agency in decisions about their assignments, which should improve “fit” and career satisfaction. But early results suggest the Air Force could benefit from a better understanding of how Airmen view this initiative, perhaps working jointly with the Army which seems also to be encountering early implementation problems. At a minimum, any review should re-consider whether the deferred acceptance algorithm the Air Force Talent Marketplace currently employs is being deployed as effectively as possible, or whether some other preference-informed matching procedure might better fit Air Force needs. In any case, a better understanding of what information participants require to form their preferences reliably will help support the information exchange needed by any matching and assignment system that incorporates preferences." (p5-38)

Among the recommendations for the Talent Marketplace, the report includes:

"—Initiate a Talent Marketplace promotion campaign across the entire service to stimulate use and buy-in through formal training, consumer feedback, and success stories.

—Expand the use of the Talent Marketplace, or a conceptually similar technology, to modernize the approach to enlisted Airmen assignments.

—Leverage data and create processes to further enable the operational goals of the Talent Marketplace for both officers and enlisted Airmen.

Ensure that the Talent Marketplace is also an information marketplace that gives position holders and candidates enough information about one another to form informed preferences.

 For job openings already using the Talent Marketplace, encourage position owners to post detailed job descriptions, and review many candidates (i.e., submit long preference lists), and encourage candidates to review many jobs.

 Incentivize true preference revelation for both “hiring” and “being hired” parties (i.e., make it safe to rank opportunities in their honest order of desirability.

 Use data to predict and recommend person-job match in a contextual manner, including preferences on both sides. Better leverage exit survey and other data for insights such as hidden reasons for attrition, the influence of preferences on separation decisions, and diversity concerns related to retention.

—Expand the Talent Marketplace to strategically fill hard-to-fill jobs and improve retention, especially in critical career fields.

Analyze whether and why certain job assignments predictably cause top choices to resign rather than take the assignment.

 Consider alternative approaches and incentives to offer declined jobs to someone who would prefer the position.

 Develop flexible procedures that preserve the possibility of retaining candidates who have chosen to separate rather than accept an assignment, by exploring whether other assignments would cause them to reconsider." (pp6-16-17)

From Appendix D: Preference-Informed Matching in Job Assignment

"Many of the current Air Force assignment procedures have grown out of the historical low-tech assignment tool consisting of a whiteboard covered with colored sticky notes, a longstanding system later augmented by spreadsheets. Often assignment teams worked with very little information about job requirements and candidate preferences. Although candidates with particular preferences or special needs could sometimes have these recognized by having their current commander (i.e., the “losing commander” who would be losing them, but who knew them) advocate for them to the assignment team, there were few ways of communicating preferences in a general and easily-used way.

"In recent years the United States Air Force and other services have moved towards somewhat more market-oriented assignment procedures, such as the Talent Marketplace developed for use by the officer assignment system,1 that make it easier for candidates and also for hiring authorities to share information and express preferences. This approach is shaped by the idea that sometimes the mutual needs and preferences of the candidate and the hiring authority could be better expressed and met. But the equivalent of whiteboards and sticky-notes still plays a role, as the information needed for hiring authorities and candidates to gain information with which to form and express preferences is still limited." (pD-1)

"Benefits of Deferred Acceptance Algorithms: DA algorithms use the information contained in the preferences of both candidates and hiring authorities, and they produce what are called stable matchings, which don’t have “blocking pairs” (i.e. there is never a service member and Air Force job that would have both, mutually, preferred each other) or “justified envy” (in which a lower-priority candidate receives a job preferred by a higher priority candidate with equal qualifications). This approach also renders it safe for members of the proposing side to reveal their preferences truthfully. Deferred acceptance algorithms have been used to match new doctors to their first positions in the U.S., and in other health care labor markets, and to match children to schools in a number of American cities (see Roth 2002, 2008 and the references cited there, and Roth 2015).

"Drawbacks of Deferred Acceptance Algorithms: The blocking pairs the Air Force needs to be most concerned with for retention don’t involve Air Force positions and service members. They instead involve private-sector jobs and service members who might choose to separate from the Air Force to take a private-sector job instead of the offered Air Force assignment. Because of this, it isn’t clear that the form of stability produced by deferred acceptance algorithms is the best goal for an Air Force assignment system. Eliminating blocking pairs involving Airmen and alternative assignments within the Air Force comes at a cost, since a stable matching (i.e., one with no such blocking pairs) may not be Pareto optimal for candidates (i.e., it may be possible to give all of some groups of candidates assignments for which they are all qualified and which they all prefer, which might better facilitate retention of service members who have no further military obligation). This is worth further study, particularly if (as is now the case) deferred acceptance algorithms are being employed to generate an initial matching that is then modified by assignment teams." (ppD6-7)

"To summarize, the Talent Marketplace has to also become an information marketplace that allows position owners and Airmen to make appropriate information available to each other in order to develop informed, accurate preferences. The key point is that matching, and the overall functioning of the human capital system, may be improved by developing new methods of sharing preferences as well as new algorithms for taking preferences into account.#" (pD-10)

#"Much of the academic literature on matching assumes that institutions that allow participants to form accurate preferences already exist. One of the tasks facing the Air Force is to develop such institutions in parallel with the development of the Talent Marketplace."


*In these complicated times it's good to remember that the oath that U.S. military officers take is to defend the Constitution of the United States.

Monday, November 30, 2020

Philippe Steiner on matching and romance, and transplants

 The French economic sociologist Philippe Steiner, who studies (among other things) how markets and gift giving can coexist, has a short piece about dating platforms.

Plateformes d’appariement, rencontres amoureuses et mondes marchands ("Matching platforms, romantic encounters and trading worlds") by Philippe Steiner, Dans Revue Française de Socio-Économie 2020/2 (n° 25), pages 161 à 166

Via google translate:

"Two elements can serve to close this brief reflection on the meeting of economic sociology and the sociology of sexuality.

"The appearance of a commercial intermediary modifies the social conditions of the romantic encounter. However, is it of a commercial nature? The use of the term matrimonial market, in which it is a question of "making a choice, maximizing your options and using calculation techniques in terms of costs and profits, and efficiency" [Illouz, 2006, p. 252], might lead one to believe. This interpretation is doubtful: if the market implies the idea of ​​choice, the converse is not true. The market relationship is characterized by monetary power, that is, the ability to obtain the desired good by paying more - it is not for nothing that auction technology is often taken as the example of the market. Also, once the relationship connecting individuals to the platform has brought together two potential partners, it is not the ability to pay that will make the match between them."


"Finally, the matching technologies that are at work in the platforms are not necessarily associated with the market world [Steiner, 2016, chap. 7]. Matching platforms using deferred acceptance or optimal trading cycle technologies can serve as well to reproduce the market functioning as to enable non-market matches. Alvin Roth's economic engineering applies to the labor market (pairing of medical interns and hospitals) as well as to organ transplantation, in which the commercial relationship is banned by national laws as well as by international declarations of professionals."


The following interview may also be of interest to readers of this blog:

“Organic” Gift-Giving and Organ Transplantation, the Development of Economic Sociology and Morality in a Super-Monetized World: An Interview with Philippe Steiner Journal of Economic Sociology, 2014, vol. 15, issue 1, 11-19

 "when I studied the issue of organ transplantation, in full agreement with Healy’s approach, the organizational setting appeared to be very, very important. Accordingly, organ donation is a gift that individual actors provide to organizational actors. And then, with this gift, the organization conducts an extensive and very important process to ensure that the kidney does not convey illness, AIDS, cancer. In addition, the degree of compatibility between the organ and the body is checked. And they do this very rapidly. Then, they allocate the gift to a new individual actor. However, the important thing, in my opinion, is that between the first individual actor and the second one there is a large organization. More precisely, a plurality of organizations. This is something that I refer to in my present book as  organizational gift-giving”. To parallel the Durkheimian distinction between mechanical and organic solidarity, I would call this “organic” gift-giving and thus draw a distinction between the usual story about people in Melanesia who give gifts according to Malinowski and Mauss. 


" I am trying to map gift-giving, inheritance, and the exchange of symbolic goods, which are at the frontiers of usual market exchanges, to provide a broad view of what exchange at large means in our present society. Considering market exchange as a limited element of all the transactions in the world is my way to escape this super-monetized world.


"Social forces are pushing in the direction of a fullblown market society, whereas others are resisting and devoting their energy to maintaining a frontier between market exchanges and other forms of exchange. In that sense, political issues remain central, as in Polanyi’s time. To return to my research on organ transplantation, I would like to stress that the last chapter of the book concerns what is usually referred to as transplant tourism — is it good to have transplant tourism? Should it be fully legalized? Is the creation of a biomarket in India for Americans suffering from final-stage kidney failure a good thing? You must say yes or no. You cannot escape a political decision. And my answer was “Definitely, no biomarkets”. However, of course, this is not an easy position because as you know there are individuals who are dying because of the lack of kidneys. Therefore, this (response) is uncertain, difficult. However, in the end, not giving an answer is a boon to those pushing for the commodification of body parts. So, finally, I decided to stay on the Maussian–Polanyian side — “limit the market.”

Sunday, November 29, 2020

Market design in Philosophy of the Social Sciences, by Philippe van Basshuysen

Philippe van Baßhuysen has an article now online in the journal Philosophy of the Social Sciences, focusing on market design, and taking as his main example the design of the labor clearinghouse for American doctors, the National Resident Matching Program. (He also includes some remarks about spectrum auctions.)  He comments a bit on the gap between how some parts of the philosophy of science literature think about market design, and how market designers thing about it, and his article brings some of the latter thought to the attention of philosophers.

How to Build an Institution, by Philippe van Basshuysen

Philosophy of the Social Sciences. November 2020. doi:10.1177/0048393120971545.

"Abstract: How should institutions be designed that “work” in bringing about desirable social outcomes? I study a case of successful institutional design—the redesign of the National Resident Matching Program—and argue that economists assume three roles when designing an institution, each of which complements the other two: first, the designer combines positive and normative modeling to formalize policy goals and to design possible mechanisms for bringing them about. Second, the engineer refines the design by conducting experiments and computational analyses. Third, the plumber implements the design in the real world and mends it as needed."

Saturday, November 28, 2020

Convalescent plasma for Covid-19 may not be as effective as hoped

 Here's a recent article from the New England Journal of Medicine: they conclude that treatment of Covid-19 patients with convalescent plasma is no better than a placebo treatment (for a group of seriously ill patients with over a 10% mortality rate).

A Randomized Trial of Convalescent Plasma in Covid-19 Severe Pneumonia

by Ventura A. Simonovich, M.D., Leandro D. Burgos Pratx, M.D., Paula Scibona, M.D., María V. Beruto, M.D., Marcelo G. Vallone, M.D., Carolina Vázquez, M.D., Nadia Savoy, M.D., Diego H. Giunta, M.D., M.P.H., Ph.D., Lucía G. Pérez, M.D., Marisa del L. Sánchez, M.D., Andrea Vanesa Gamarnik, Ph.D., Diego S. Ojeda, Ph.D., et al., for the PlasmAr Study Group

RESULTS: A total of 228 patients were assigned to receive convalescent plasma and 105 to receive placebo. The median time from the onset of symptoms to enrollment in the trial was 8 days (interquartile range, 5 to 10), and hypoxemia was the most frequent severity criterion for enrollment. The infused convalescent plasma had a median titer of 1:3200 of total SARS-CoV-2 antibodies (interquartile range, 1:800 to 1:3200]. No patients were lost to follow-up. At day 30 day, no significant difference was noted between the convalescent plasma group and the placebo group in the distribution of clinical outcomes according to the ordinal scale (odds ratio, 0.83 (95% confidence interval [CI], 0.52 to 1.35; P=0.46). Overall mortality was 10.96% in the convalescent plasma group and 11.43% in the placebo group, for a risk difference of −0.46 percentage points (95% CI, −7.8 to 6.8). Total SARS-CoV-2 antibody titers tended to be higher in the convalescent plasma group at day 2 after the intervention. Adverse events and serious adverse events were similar in the two groups.

CONCLUSIONS: No significant differences were observed in clinical status or overall mortality between patients treated with convalescent plasma and those who received placebo. 

HT: Irene Wapnir

Friday, November 27, 2020

Market Design Job Market Candidates in Econ and Computer Science (or a combination of the two) from SIGecom Exchanges

 Assaf Romm writes:

"The editors of SIGecom Exchanges were kind enough to host on their November issue a list that I compiled containing profiles of market designers currently on the job market. The profiles contain links to their homepage and CVs, and a short summary of their research and job market papers. There are twelve excellent candidates with very interesting papers!

"The list of candidate profiles is here. "

And here's the list of people described at greater length at the link--you could hire one of them.

Delacretaz, David, Goldner, Kira,  Gonczarowski, Yannai A., Imamura, Kenzo, Koren, Moran, Larroucau, Tomas, Qian, Pengyu, Raghavan, Madhav, Sayedahmed, Dilek, Sullivan, Colin. Thakur, Ashutosh , Vohra, Akhil.

Thursday, November 26, 2020

Madhav Raghavan on transparency (and on the European job market)

Madhav Raghavan is completing a postdoc at the University of Lausanne this year (working with Bettina Klaus), and is on the job market, primarily in Europe.  I got to know him when he visited us briefly at Stanford this year (in person, before having to retreat back to Switzerland, as Covid surged...)

His job market paper concerns the verifiability of the algorithms that run centralized clearinghouses, and how that can be improved by changing the kinds of feedback that participants receive.

Transparency in Centralised Allocation: Theory and Experiment  by Rustamdjan Hakimov and Madhav Raghavan

Abstract: Many algorithmic allocation mechanisms suffer from a verifiability problem: participants cannot check if their assignments are correct. This problem is compounded if there are suspicions that the designer has deviated from the true allocation. We formalise these concerns and propose solutions in an information-based framework. A participant's assignment is `verifiable' by her if any other assignment contradicts her information. A stronger requirement is `transparency', where the designer cannot deviate from the true allocation without being detected. We show how the communication of `terminal-cutoffs' and the use of `predictable' multi-stage mechanisms each provide information to participants that verifies their assignments. Even though the information from predictable mechanisms and terminal-cutoffs can each be manipulated by a dishonest designer without detection, in our main result we show that they nevertheless achieve transparency if used together. We suggest transparent environments for use in school admissions, single-object auctions and house allocation. We support the effectiveness of our solutions via a school admissions laboratory experiment.


The paper develops the theory in some generality, and also focuses on the particular case of the deferred acceptance algorithm, which is the subject of the experiment they report.  

If you're hiring, he's worth a close look.