I recently served as a member of a National Academies committee on the issues facing the Air Force in managing its human capital, i.e. its labor force. It resulted in a long report covering many aspects of Air Force policy.
One of the fun things about that assignment was meeting with people throughout the U.S. armed forces (and some allies). Yesterday I had the privilege of attending the promotion ceremony for a very thoughtful Air Force officer, Tobias Switzer. Congratulations, Colonel!.*
Readers of this blog are likely to be most interested in the evolving ways that military personnel are matched to new assignments, from time to time during their careers. A first order problem has to do with retention, since the armed services compete with the private sector for highly trained people (e.g. pilots, cyber warriors, special operators, to name a few), which is most visible at times when the person in question doesn't have any further military service obligation. Below, I excerpt some paragraphs concerning one new aspect of this process, called the Talent Marketplaces. (The page numbers refer to sections of the report, e.g. "p2-7" refers to page 7 of section 2.)
Here's the report:
A Consensus Study Report, from the National Academies of Sciences, Engineering, and Medicine 2020:
Strengthening U.S. Air Force Human Capital Management: A Flight Plan for 2020-2030. Washington, DC: The National Academies Press. https://doi.org/10.17226/25828
Committee on Strengthening U.S. Air Force Human Capital Management, Board on Human-Systems Integration, Division of Behavioral and Social Sciences and Education
Committee and Staff: Julie J.C.H. Ryan (Co-Chair), William J. Strickland (Co-Chair), Terry A. Ackerman, David S.C. Chu, Lt. Gen. Gina M. Grosso, Usaf (Retired), Brig. Gen. Leon A. Johnson, Usafr (Retired), Judith S. Olson, Dan J. Putka, Alvin E. Roth, Ann Marie Ryan, Stephen Stark, Cherie Chauvin, Study Director, Elizabeth T. Cady, Senior Program Officer, Daniel Talmage, Program Officer, Margaret Kelly, Sr. Program Assistant, Tina M. Latimer, Program Coordinator
Below are some snippets drawn from various sections of the report.
"since 2014, the Air Force has lost more fighter pilots annually than its annual production rate, and forecasts of the commercial airline pilot marketplace are an important variable in long-term workforce planning to develop and sustain the career field." (p2-7)
"A lack of clear data on why Airmen are leaving the force diminishes the Air Force’s ability to do such evaluations or establish an “early warning system” to identify recruits at risk of early attrition. Increased use of assignment tools like the Talent Marketplace will allow relevant longitudinal data to be collected on the extent to which Airmen’s separation decisions may be related to their preferences over the positions to which they could have been assigned as compared to the one to which they were assigned (see Appendix D)." (p4-22)
"the recently implemented Talent Marketplace provides an innovative online means for matching Airmen with assignments based on expressed preferences of Airmen and position owners for post-accession job assignments. However, the usefulness of expressed preferences depends a good deal on how much information is available to Airmen about positions and to position owners about Airmen (through the Talent Marketplace), given that such information profoundly shapes preferences. Consequently, the most effective version of the Talent Marketplace would also serve as an information marketplace that allows position owners and Airmen to make appropriate information available to each other to form informed, accurate preferences (e.g., realistic job previews, using video and written descriptions). The key point is that matching, and the overall functioning of the USAF HCM system, may be improved by developing new methods of sharing preferences as well as new algorithms for taking preferences into account (see Appendix D)" (p4-26)
"In 2019, the Air Force began using the web-based platform, the Talent Marketplace, for making officer assignments (Lieutenant Colonel and below). The Talent Marketplace was initiated in an effort to meet a stated goal: “To the maximum extent possible, assign individuals on a voluntary basis and in the most equitable manner feasible while meeting mission and commander needs” (USAF, 2018a, p.2)." (p5-10)
"The Air Force is investigating implementing the Talent Marketplace to manage enlisted assignments, but a decision is still pending at this time."(p5-11)
"There are two officer assignment cycles per year. To support the Officer Assignment System for all assignments of Lieutenant Colonel and below (except for Judge Advocate General officers), the Air Force uses its newly developed Talent Marketplace. The web-based system provides transparency for available positions, provides visibility on an officer’s preferences to their commander, and incorporates gaining commander input as well for the first time. The technology behind it examines officer assignment solutions by incorporating specific prioritizations from both the officers who are eligible to move and the gaining unit (see Appendix D for discussion of preference informed matching). Officers on the vulnerable-to move list use the Talent Marketplace to indicate a desirability rating for assignment location preferences using a list of locations with jobs to fill, in alphabetic order. There is limited information on each position in the system: duty title, command, and location, but the officer can see how many other officers are interested in the position and make their decision accordingly.17 Any additional information the officer would like to know about the position is gathered through their own independent research. After the window closes for officers to bid for positions, the position owners access the system to see the final list of volunteers to fill their positions. The results of the matching algorithm are used as a first step in the process, which is further adjusted as needed and finalized by AFPC." (p5-11)
"As the Air Force expands its use of the Talent Marketplace for officers and develops a more modern approach to the antiquated Enlisted Quarterly Assignments List, it could benefit from considering the research conducted on and the implementation and results of similar marketplace initiatives (see for example, Malia, 2020). The U.S. Army, in particular, recently implemented its Army Officer Assignment Marketplace via Assignment Interactive Module (AIM).30 AIM is a centralized clearinghouse that requires officers and units to finalize preferences for the other side of the market at the same time, typically 6–9 months before officers are expected to move to their next assignment. The Army studies its assignment system in the Office of Economic and Manpower Analysis (OEMA), Department of Social Sciences, West Point.31 Information provided to the committee on the design and administration of AIM indicated that it is something of a hybrid model that involves a lot of hand-processing of assignments by assignment officers. However, it was reported that the “Human Resources Command received permission from the Assistant Secretary of the Army for Manpower and Reserve Affairs to test the concept of matching officers to jobs according to a deferred acceptance algorithm similar to that of the National Resident Matching Program,”32 which matches new doctors to residency programs (Roth, 1984; Roth and Peranson, 1999).
"One aspect of the Army Cadet Branching process (used to assign new West Point graduates to branches) is that Cadets can bid for preferred assignments by offering to extend their service obligation. Although a detailed review of AIM was outside the scope of this study, the committee notes that analysis has suggested that its use of a deferred acceptance algorithm is seriously flawed in implementation (e.g. Switzer, 2011), and this may be important for the designers and administrators of the Air Force Talent Marketplace to study and understand."
"As of May 15, 2019, the Army reports that 90,952 officers of all ranks had participated in AIM. Officers are encouraged to submit resumes, but the majority have not done so (from 7% of Colonels, up to 42% of Majors). Resume-writers are encouraged to include unique Knowledge, Skills, and Behaviors (KSBs), but almost none have done so. Units seeking to fill jobs are encouraged to submit (long) preference lists of officers for each job they are seeking to fill, and so far, participation has been spotty: “The best participation is from [the 10 percent of] units who submit preferences for many jobs, and for multiple officers on each job” (U.S. Army, 2019, p. 2). Although no explanation was provided to the committee, the Army’s overall low participation numbers suggest that the Air Force and Army could benefit from joint efforts to understand the reaction Airmen and Soldiers have had to these initiatives as well as to develop approaches to improve participation and therefore its overall utility.
"Turning back to the Air Force, the present (preliminary) use of deferred acceptance algorithms in the Talent Marketplace is not an evidence-based decision; rather it arises from a perhaps too-hasty parallel with the operation of the private-sector clearinghouse for American physicians (see discussion of preference informed matching in Appendix D). To guide Air Force research into post-accession assignment decisions to improve and expand the effectiveness and validity of the Talent Marketplace, the committee identified the following research questions:
How do assignments affect separation decisions? 33 (Collect data on “Who stays, who leaves, where they go, would they have stayed for a better assignment?”)
How are preference data related to family data? (Collect family data on jobs of spouses, age, and schooling of children)
How is the new blended retirement system changing separation choices?
How do pilot separations interact with airline hiring demands? (E.g., could new arrangements be initiated between the Air Force and commercial airlines to better meet fluctuating needs of both?34)
Can exit interviews be combined with interventions that might prevent or delay separation by the most effective Airmen?
How should the Talent Marketplace be organized for minimizing early separation of the most effective Airmen?
Does the Talent Marketplace assure an appropriate distribution of talent across units?
How does the Talent Marketplace affect individual and unit performance?" (pp5-27-28)
"Similar to the Army, the Air Force is implementing a “Talent Marketplace” that gives Airmen increased agency in decisions about their assignments, which should improve “fit” and career satisfaction. But early results suggest the Air Force could benefit from a better understanding of how Airmen view this initiative, perhaps working jointly with the Army which seems also to be encountering early implementation problems. At a minimum, any review should re-consider whether the deferred acceptance algorithm the Air Force Talent Marketplace currently employs is being deployed as effectively as possible, or whether some other preference-informed matching procedure might better fit Air Force needs. In any case, a better understanding of what information participants require to form their preferences reliably will help support the information exchange needed by any matching and assignment system that incorporates preferences." (p5-38)
Among the recommendations for the Talent Marketplace, the report includes:
"—Initiate a Talent Marketplace promotion campaign across the entire service to stimulate use and buy-in through formal training, consumer feedback, and success stories.
—Expand the use of the Talent Marketplace, or a conceptually similar technology, to modernize the approach to enlisted Airmen assignments.
—Leverage data and create processes to further enable the operational goals of the Talent Marketplace for both officers and enlisted Airmen.
Ensure that the Talent Marketplace is also an information marketplace that gives position holders and candidates enough information about one another to form informed preferences.
For job openings already using the Talent Marketplace, encourage position owners to post detailed job descriptions, and review many candidates (i.e., submit long preference lists), and encourage candidates to review many jobs.
Incentivize true preference revelation for both “hiring” and “being hired” parties (i.e., make it safe to rank opportunities in their honest order of desirability.
Use data to predict and recommend person-job match in a contextual manner, including preferences on both sides. Better leverage exit survey and other data for insights such as hidden reasons for attrition, the influence of preferences on separation decisions, and diversity concerns related to retention.
—Expand the Talent Marketplace to strategically fill hard-to-fill jobs and improve retention, especially in critical career fields.
Analyze whether and why certain job assignments predictably cause top choices to resign rather than take the assignment.
Consider alternative approaches and incentives to offer declined jobs to someone who would prefer the position.
Develop flexible procedures that preserve the possibility of retaining candidates who have chosen to separate rather than accept an assignment, by exploring whether other assignments would cause them to reconsider." (pp6-16-17)
From Appendix D: Preference-Informed Matching in Job Assignment
"Many of the current Air Force assignment procedures have grown out of the historical low-tech assignment tool consisting of a whiteboard covered with colored sticky notes, a longstanding system later augmented by spreadsheets. Often assignment teams worked with very little information about job requirements and candidate preferences. Although candidates with particular preferences or special needs could sometimes have these recognized by having their current commander (i.e., the “losing commander” who would be losing them, but who knew them) advocate for them to the assignment team, there were few ways of communicating preferences in a general and easily-used way.
"In recent years the United States Air Force and other services have moved towards somewhat more market-oriented assignment procedures, such as the Talent Marketplace developed for use by the officer assignment system,1 that make it easier for candidates and also for hiring authorities to share information and express preferences. This approach is shaped by the idea that sometimes the mutual needs and preferences of the candidate and the hiring authority could be better expressed and met. But the equivalent of whiteboards and sticky-notes still plays a role, as the information needed for hiring authorities and candidates to gain information with which to form and express preferences is still limited." (pD-1)
"Benefits of Deferred Acceptance Algorithms: DA algorithms use the information contained in the preferences of both candidates and hiring authorities, and they produce what are called stable matchings, which don’t have “blocking pairs” (i.e. there is never a service member and Air Force job that would have both, mutually, preferred each other) or “justified envy” (in which a lower-priority candidate receives a job preferred by a higher priority candidate with equal qualifications). This approach also renders it safe for members of the proposing side to reveal their preferences truthfully. Deferred acceptance algorithms have been used to match new doctors to their first positions in the U.S., and in other health care labor markets, and to match children to schools in a number of American cities (see Roth 2002, 2008 and the references cited there, and Roth 2015).
"Drawbacks of Deferred Acceptance Algorithms: The blocking pairs the Air Force needs to be most concerned with for retention don’t involve Air Force positions and service members. They instead involve private-sector jobs and service members who might choose to separate from the Air Force to take a private-sector job instead of the offered Air Force assignment. Because of this, it isn’t clear that the form of stability produced by deferred acceptance algorithms is the best goal for an Air Force assignment system. Eliminating blocking pairs involving Airmen and alternative assignments within the Air Force comes at a cost, since a stable matching (i.e., one with no such blocking pairs) may not be Pareto optimal for candidates (i.e., it may be possible to give all of some groups of candidates assignments for which they are all qualified and which they all prefer, which might better facilitate retention of service members who have no further military obligation). This is worth further study, particularly if (as is now the case) deferred acceptance algorithms are being employed to generate an initial matching that is then modified by assignment teams." (ppD6-7)
"To summarize, the Talent Marketplace has to also become an information marketplace that allows position owners and Airmen to make appropriate information available to each other in order to develop informed, accurate preferences. The key point is that matching, and the overall functioning of the human capital system, may be improved by developing new methods of sharing preferences as well as new algorithms for taking preferences into account.#" (pD-10)
#"Much of the academic literature on matching assumes that institutions that allow participants to form accurate preferences already exist. One of the tasks facing the Air Force is to develop such institutions in parallel with the development of the Talent Marketplace."
**********
*In these complicated times it's good to remember that the oath that U.S. military officers take is to defend the Constitution of the United States.
Posts
