Peter Rousseau comments on our field experiment involving the Econ job market, in PNAS

 My post yesterday was about the experiment about social media and the job market for economists.  I only noticed later that the PNAS also posted a comment on our article, by Professor Peter Rousseau, the secretary of the American Economic Association, who has a long and intimate familiarity with that job market, which the AEA has played a giant role in organizing.

 Improving the job market in economics (and beyond…) by Peter L. Rousseau  PNAS   May 4, 2026  https://doi.org/10.1073/pnas.2609971123

Here is the part of his comment directly connected to our paper:

" the authors make a welcome and useful contribution to the market design literature with a fascinating experiment designed to substitute for and even improve upon the informal information channels lost to the economics job market in the new postpandemic normal. Given that some job candidates are less active self-promoters than others and that, conversely, excessive self-promotion can in some cases be viewed as a negative by prospective recruiters, the authors’ proposed mechanism offers serious promise for leveling the playing field, even if just modestly, for economics job candidates in terms of their visibilities, and perhaps even for expanding the number of jobs actually filled over the course of a recruiting season.
 

"In the experiment, an AI-based algorithm, supplemented with some human checking and reassignments, matched selected economists on social media (i.e., the “influencers”) with willing job candidates based on the closeness of their research. About 43 percent of willing candidates were selected for this treatment. The key to the experiment lies in the matches themselves, which were assigned in a manner that did not take the relative prominence or institutional ranking of an influencer directly into account. All candidate participants were invited to post a tweet about their job market papers on a social media site created for this purpose, and the influencers were asked to post neutral quote-tweets about the members of the treatment group to which they had been assigned. If executed according to design, recruiters viewing the quote-tweets receive information about the closeness of a given candidate’s research interests to those of the influencer. This may function as a partial substitute for the painstaking process of deducing such information across the hundreds of application packets that recruiters receive with only a brief period for making initial decisions. Knowing that a candidate’s research is close to that of Professor “X” is a tangible signal that could make that candidate more likely to be interviewed or receive a campus flyout or job offer from an institution seeking an entry-level economist like Professor X. The experiment indicates that individuals in the treatment group did indeed receive more campus visits and job offers than candidates assigned to the control group, and that the effect on job offers was especially strong for women. It also finds, however, that these effects were more pronounced for candidates matched to influencers with relatively higher citation counts than for those matched to influencers with relatively more followers, as these two measures of prominence in the profession are not that highly correlated. 

" The question of scalability then becomes paramount. Considering the experiment’s positive findings, it is natural to assume that, if universally available, all job candidates would choose to participate and receive the treatment. The process would otherwise go on as stated with perhaps additional influencers being selected by the organizers to serve the larger pool of candidates. Two observations seem reasonable at this point: first, in such a setup, better information about matches could lead to more open positions being filled, which would be a better aggregate outcome; and second, the treatment might in practice benefit candidates from outside the very top departments the most. This is because candidates from the highest ranked departments, who are often perceived by recruiters as having a higher probability of eventually becoming a star, will typically receive more interviews, campus visits, and offers, but in the end can still only accept one offer. With an enlarged set of viable matches, this means that some candidates who may have been otherwise overlooked will find jobs. Of course, the job market may take longer to clear under this mechanism as candidates will have more options to consider before departments go to second or third rounds of offers.
 

"Casual observations of the job market among economics departments and their chairs do suggest that a number of recruiters are unable to fill positions they have posted. The AEA does not currently collect information on just how many, but the very existence of the “AEA Job Market Scramble,” where recruiters and unmatched candidates can post their availabilities on an online message board each March, is indicative of the challenge (3). The design of a job signaling mechanism by the AEA and its implementation in December of each year (4), where job candidates can list two departments to which they would like to express interest in an interview, is another such intervention aimed at easing the congestion.
Another interesting result is that women appear to benefit most from the treatment, while this benefit does not extend to members of other groups traditionally underrepresented in economics. The authors point to existing evidence indicating that women on average tend to be less active promoters of their own research on social media than others and suggest that the additional visibility provided by the quote-tweets could be leading to more job offers. This potential channel, of course, could also be viable for any candidate with a tendency to self-promote less. To explain a special advantage for women, one could note the possibility of forces in the 2022–2023 job market where departments seeking to improve the gender balances of their faculties became aware of candidates through the mechanism who they may have otherwise overlooked. If this is the case, the next question to ask is why does the effect not carry over for members of other underrepresented groups? The answer, though no doubt a speculative one, may lie in the preexistence of other mechanisms and informal channels for promoting such candidates, rendering the marginal effects of the authors’ particular intervention not statistically significant.
Finally, while having the potential to increase the number of matches and raise their average quality, the effects of the authors’ intervention will be subject to some randomness based on the assignment of a given candidate’s influencer. For example, when any influencer posts a quote-tweet about a candidate who has been independently and objectively determined to have close research interests, that candidate’s post tends to receive more views and likes on X than those in the control group, and the extent of this visibility correlates with the size of the influencer’s following. Yet these effects do not seem to transfer downstream to job outcomes, where candidates receiving quote-tweets from highly cited influencers are the ones tending to see more offers. In a real sense, the adage “all publicity is good publicity,” often applied to economics research, may not be always true. The assignment of influencers to candidates, even if randomized, will matter for individual outcomes even though the aggregate effects of the intervention are positive. Given the potential individual benefits compared to nontreatment, however, job candidates would likely embrace the residual uncertainty and participate in the mechanism.
 

"The intervention designed by Qiu et al. may hold even greater promise outside of the economics discipline. In the natural sciences, for example, recruiting for scarce academic postdoctoral positions among new PhDs at a similar career stage, which are markets typically saturated with candidates, often moves directly to a very limited allocation of campus visits based in no small part on letters and other communications from mentors, some of whom could be less than ideally matched with their students or less well known than would-be assigned influencers. These cases are ones in which an enhanced visibility of candidates, when coupled with independent information about the closeness of their work to what senior researchers and their groups might be seeking, could lead to the greater advancement of science more generally.
 : 
"Competing interests P.L.R. has served since 2012 as Secretary-Treasurer of the American Economic Association, a 501(c)(3) non-profit deeply committed to improving the job market for new Ph.D. economists, and for which one of the companion article’s co-authors (A. E. Roth) served as President in 2017.

#######

 Peter's comment and our paper appeared online, but won't appear in print until next week in the May 12, 2026 | vol. 123 | no. 19 issue of PNAS.

 

Yesterday's post: 

Tuesday, May 5, 2026  Social media, job market outcomes, and ethics of field experiments, by Qiu, Chen, Cohn and Roth in PNAS

 

Social media, job market outcomes, and ethics of field experiments, by Qiu, Chen, Cohn and Roth in PNAS

 One of the fun things about our paper published in today's PNAS is that, as a working paper, it prompted a vigorous discussion of the ethics of doing field experiments in economics.  We discuss this more fully in the published version, below:

J. Qiu, Y. Chen, A. Cohn, & A.E. Roth, Social media promotion improves job market outcomes, Proc. Natl. Acad. Sci. U.S.A. 123 (19) e2528289123, https://doi.org/10.1073/pnas.2528289123 (2026). 

Abstract: Social media has transformed how academics disseminate research, but its effect on academic job outcomes remains unclear. Previous research has shown correlations between social media exposure and metrics like citation counts, but these relationships may be confounded by unobserved factors such as researcher quality or access to professional networks. We examine whether social media promotion causally affects job market outcomes in economics through a field experiment on Twitter (now X). We first collect tweets about job market papers from 519 candidates and post them from a dedicated account. We then randomize half of the posts to be quote-tweeted by established economists in the candidates’ fields, and measure the effects on both online visibility and hiring outcomes. We find that posts in the treatment group receive 441% more views and 303% more likes than those in the control group. Candidates whose posts were assigned to be quote-tweeted receive one additional flyout invitation compared to the control group average of 5.4 flyouts. Furthermore, women in the treatment group receive 0.9 more job offers than women in the control group, who receive 3 offers on average. Exploring mechanisms, we find that academic reputation drives these results, with stronger effects for quote-tweets from highly cited scholars and for candidates from top institutions. Our findings suggest social media promotion causally increases research visibility and improves academic job market outcomes.

 

  ...

"Ethical considerations.
"After the release of our working paper on the Social Science Research Network (SSRN) on May 20, 2024, a vigorous discussion arose on both social and mainstream media, particularly on Twitter, about the ethics of our experiment and of field experiments more generally (e.g., ref. 30). The main concern suggested that job markets are essentially constant sum, so that randomly promoting some candidates through having their JMPs quote-tweeted by influencers would necessarily (and unethically) disadvantage both those who were in the control condition of the experiment and those who did not participate in the experiment.
 

"We understand the importance of considering the ethical implications of any experiment and that ethicality is connected to the underlying economics of the job market. In this latter respect, given the information friction and congestion in the interview process, job markets are unlikely to be constant sum. Aside from the possibility of welfare gains from improved match quality, we note that, typically in matching markets, many employers fail to fill all their positions while at the same time qualified candidates fail to find one, so that welfare can also be improved by filling more positions. [In the 2022–2023 job market, the total number of jobs listed on JOE was 3,608, including 933 (1,083) full-time academic jobs in (outside) the United States and 718 full-time nonacademic jobs (any location). On the supply side, 1,386 Ph.D. students and postdocs applied to at least one job through JOE from August to December 2022 (31).] In economics, the job market often has unfilled positions by the end of February, leading to a scramble round each year starting in March. Similarly, the annual National Resident Matching Program (NRMP) for new physicians in the United States also leads to some positions being unfilled, despite having far more applicants than available positions. [For example, in 2024, 38,494 positions were offered to 44,853 active applicants and 2,510 positions were unfilled (6.5%), at the end of both the main match (a deferred acceptance algorithm, see ref. 32) and a centralized postmatch scramble called the Supplemental Offer and Acceptance Program (33).]
 

"The phenomenon of unfilled positions in a thick labor market may reflect congestion in the interview process. In such a market, since many positions receive more applications than the number of candidates who can feasibly be interviewed, the matching of interviews to jobs may be imperfect in the sense that an employer can find that none of the people interviewed can be successfully hired, but could have filled the position if more appropriate interviewees had been chosen. To mitigate this issue, signaling mechanisms have been introduced in both the economics and medical markets to facilitate a better matching of interviewees and employers (29, 34). In our context, the quote-tweeting of JMPs may similarly serve to help employers find better matches with their selection of interviewees who can be hired.
 

"We also propose that highlighting suitable candidates from underrepresented groups for a position could potentially expand the overall number of job openings. A notable example is the President’s Postdoctoral Fellowship Program, implemented across multiple institutions including the University of Michigan and the University of California system. This program seeks to recruit future faculty members “with the potential to bring to their research and undergraduate teaching the critical perspective that comes from their nontraditional educational background or understanding of the experiences of groups historically underrepresented in higher education.” (See, e.g., https://presidentspostdoc.umich.edu/, retrieved on August 29, 2025.)
 

"Finally, we consider trends in the broader context of job search in evaluating the ethical considerations related to our study. Social media has become a common channel for academics to advertise the JMPs of their students. Thus, we are not introducing a new channel for candidate promotion, nor are we excluding others outside of our experiment from availing themselves of this channel. Our goal is to understand the extent to which this channel may create visibility or improve outcomes for job candidates, especially since not all candidates may have equal access. Our paper belongs to the class of natural field experiment (35), a class that has seen a growing number of studies in which field experiments are used to assess the effects of market interventions. [A natural field experiment is one “where the subjects do not know that they are in an experiment” (35). In our context, participants were told only that we would arrange for their JMPs to be tweeted, but not that there would be a quote-tweet treatment.] One of the main benefits of conducting a natural field experiment is that it minimizes possible Hawthorne effects (36). These studies are widely accepted and even recognized, with the 2019 Nobel Prize for experiments in development economics. If it is ethical for economists to use experiments to evaluate interventions in other markets, it should also be ethical for economists to study the market for economists. And if it is ethical to promote students who are on the job market, then it should be ethical to study the effects of such promotion.
 

"In sum, from a normative perspective (should scholars promote candidates?), we argue that such promotion can reduce information friction and job market congestion, potentially leading to more efficient matching. From a positive perspective (does promotion matter?), we demonstrate in Results that it increases candidate visibility and improves job market outcomes, especially for women who are traditionally underrepresented in economics." 

#############

Earlier (a blog post about reference 30, above): 

Saturday, June 8, 2024  The ethics of field experiments in Economics, in the Financial Times

 

The Economic Experience: An Introduction through Experiments by Charles A. Holt and Erica Sprott

 Princeton University Press has a new economics textbook centered on experiments:

 The Economic Experience: An Introduction through Experiments by Charles A. Holt and Erica Sprott  

"An innovative introduction to economic behavior that uses interactive experiments to promote experience-based discovery"

Here's my blurb:

“Experiments have had a huge impact on behavioral economics, and Holt and Sprott’s book aims to make teaching economics, through experiments, easy on instructors and fun for students.”—Alvin Roth, Stanford University

 

John Henry Kagel (1942-2026), an incomparable experimental economist

 John Kagel will be buried this morning,  January 7, at  New Tifereth Israel Cemetery in Columbus, Ohio. He passed away yesterday.  I don't know the details, but my sense is that he hadn't been well for a while. He was 83.

In his prime, John was the best experimental economist in the world.

He was also my friend, colleague, coauthor, co-editor, and all-around mensch and role model. He was full of life.  

Words fail. 

Here's his Google scholar page. 

In 2023 the journal Experimental Economics had a special issue in John's honor: here's the Introduction. 

May his memory be a blessing. 

Experimental math (XKCD)

The Unreasonable Effectiveness of Mathematics

 https://xkcd.com/3180/

 

Nottingham's Centre for Decision Research and Experimental Economics celebrates its 25th anniversary

 In September, Nottingham's productive center for experimental economics celebrated a quarter century since it's founding: 

 CeDEx 25th Anniversary Workshop .

I just saw the announcement now, but maybe that's a sign of how successfully experimental economics has become established in the profession, over the last century and around the world.

 For some early history, see e.g. 

Roth, Alvin E. "The early history of experimental economics." Journal of the History of Economic Thought 15.2 (1993): 184-209.

 

Judd Kessler: the most helpful advice I ever got (YouTube short video)

 I was charmed by this very short video featuring Judd Kessler (and I'm very grateful to whoever gave him that advice):

 

 

https://youtube.com/shorts/zBTlYwvYE2M?si=q6_vRIflWvAmMDYb 

 

 

 

 

You can see his new book, Lucky By Design in the background 

“Lucky by Design is that rarest of things: an economics PAGE-TURNER.” —Lin-Manuel Miranda 

European Economic Review Summer School in Experimental and Behavioral Economics, June 1-4, 2026.

 

 It's never too late to learn about experiments:

3rd European Economic Review Summer School  in Experimental and Behavioral Economics

The European Economic Review is pleased to announce their third Summer School in Experimental and Behavioral Economics to be held at ISEG in Lisbon from June 1 to June 4, 2026. The School will feature lectures by leading researchers in state-of-the-art topics in experimental and behavioral economics in addition to a research workshop. Throughout the School the students will be able to present their own research in poster sessions and receive feedback from leading faculty and fellow participants. 

The 3rd European Economic Review Summer School is very privileged to feature lectures by leading figures such as Isabelle Brocas (University of Southern California), Juan D. Carrillo (University of Southern California), Vincent Crawford (Oxford University and University of California at San Diego), Michalis Drouvelis (University of Birmingham), Ernst Fehr (University of Zurich) and David Levine (Royal Holloway University of London). The goal of the School is to deepen attendants’ understanding and knowledge of recent advances in the field of Experimental and Behavioral Economics. The topics taught will cover a broad range of methodologies such as theory, laboratory and field experiments, as well as applications. The School will provide a unique environment where students can expand their knowledge on topical research issues and engage with leading figures in the field. Scholars who have been admitted to the School will be taught the following subjects:

• decision-making and its biological causal processes (Isabelle Brocas)

• decision-making in children and adolescents (Juan D. Carrillo) 

• behavioral game theory and topics in behavioral decision theory (Vincent Crawford)

• experimental games in the lab; emotions and human cooperation (Michalis Drouvelis)

• economics of social preferences; product obfuscation in markets (Ernst Fehr)

• quantitative behavioral theories of long term play in experiments (David Levine)

Further details on the content of the lectures, as well as background material, will be uploaded gradually in the Program of the School. 

The Summer School invites applications from Ph.D and MSc students in Economics, Business, Psychology, Behavioral Science, Political Science and related fields from all over the world. Faculty and professionals are also welcome. To apply to the School, please submit a CV using our Application Form. The deadline for applications is January 31, 2026. Decisions will be sent to applicants by February 14, 2026. 

Experimental economics at the University of Pittsburgh

 I recently had the opportunity to visit my old haunts at the University of Pittsburgh, and took part in a reopening and re-dedication of the famous experimental economics lab there. Here's the announcement:

 PEEL is officially back open after renovations

The Pittsburgh Experimental Economics Laboratory (PEEL) celebrated its grand reopening last week following substantial renovation. Led by PEEL Director Lise Vesterlund, the renovations allow Pitt to continue to lead as one of the preeminent experimental economics research universities in the country.

Alvin Roth, winner of the 2012 Nobel Memorial Prize in Economics, was on site to celebrate the event. Roth, who was the Andrew W. Mellon Professor of Economics at Pitt through the 1980s and 90s and is now a professor at Stanford University, founded the lab in 1988 alongside then-Pitt Professor John Kagel.  His original research at PEEL led to his Nobel Prize for market design and to his design of the National Residency Matching Program (NRMP) and the New England Program for Kidney Exchange.  

“Experiments are very powerful, and I think this [lab] is a huge advantage,” Roth said. “For a long time, economists thought of data as something that governments collected […] but when you do experiments, you can carefully control for what you think are relevant differences and understand causes much better.” 

 

 Vesterlund, who has held the Mellon chair in economics since 2007, has conducted countless experiments in the lab. Particularly influential has been her research assessing the impact of work assignment on gender differences in advancement, research that lead to her award-winning co-authored book The No Club: Putting a Stop to Women's Dead-End Work.  

Vesterlund says a well-functioning lab is the foundation for Pitt’s strong economics department. “PEEL is a world-renowned experimental economics laboratory.  It has been the home to countless seminal research findings. Findings that seeded fields in the profession and improved the way we design markets and organizations. Few institutions can maintain a well-functioning lab, and PEEL has been essential in drawing an exceptionally strong group of scholars to Pitt. The renovation will ensure that the University of Pittsburgh continues to be the center for cutting-edge research in experimental economics.” 

The ceremony included opening remarks from Dean Adam Leibovich before Vesterlund and Roth cut the ribbon, formally re-opening the lab. The renovated lab comprises 40 computer kiosks, where participants can engage with state-of-the-art research software and make choices that inform us about human decision-making.  

To learn more about research being conducted in the lab, you can visit the PEEL website.

 

Experiments and behavioral market design at Pitt (tomorrow)

 I'm flying  back to my old haunts in Pittsburgh today, for (among other things) two events at the University of Pittsburgh tomorrow:

 October 17, 2025 PEEL Reopening Ceremony with Professor Vesterlund & Professor Roth

"The History of PEEL: The Pittsburgh Experimental Economics Laboratory (PEEL) was founded by John Kagel and Alvin Roth, a Nobel Laureate in Economics. Since its inception, PEEL has served as a hub for pioneering research in experimental economics. Notably, the lab contributed to foundational work on market design, which played a significant role in Roth’s Nobel Prize-winning contributions. Over the decades, PEEL has maintained its reputation as a center of excellence, attracting top scholars and fostering innovation in economic research. +

 

October 17, 2025 BEDI Workshop (Behavioral Economics and Design Initiative)

BEDI Workshop - Friday, October 17th, 2025

Breakfast | 8:15 am – 8:45 am
Wesley W. Posvar Hall, 4130, 230 S Bouquet St, Pittsburgh, PA 15213 

Conference Welcome | 8:45 am – 9:00 am
Lise Vesterlund, BEDI Director, Andrew W. Mellon Professor, University of Pittsburgh

Session 1 | 9:00 am – 10:40 am
Erina Ytsma– Assistant Professor, Carnegie Mellon University, “Gender Differences in the
Response to Incentives: Evidence from Academia”
Stephanie Wang – Professor, University of Pittsburgh
Claire Duquennois – Assistant Professor, University of Pittsburgh, “Minority athletic performance,
racial attitudes, and racial hate”
Jonathan Woon – Professor & Associate Dean, University of Pittsburgh, “The Epistemology of
Justice: Awareness and Institutional Choice”

Refreshment Break | 10:40 am – 11:10 am

Session 2 | 11:10 am – 12:00 pm
Jenny Chang – Graduate Student, Carnegie Mellon University, “When Women Self-Promote:
Evidence on Beliefs and Downstream Consequences”
Aden Halpern – Graduate Student, University of Pittsburgh 

Brandon Williams – Graduate Student, University of Pittsburgh
Dhwani Yagnaraman – Graduate Student, Carnegie Mellon University, “Crowd-in and crowd-out of
climate policies”
Aaron Balleisen– Graduate Student, Carnegie Mellon University, “Cheap Talk and Pluralistic
Ignorance”

Lunch | 12:00 am – 1:15 pm

Session 3 | 1:15 pm – 2:30 pm
Osea Giuntella – Associate Professor, University of Pittsburgh, “Beliefs, Resilience, and Leadership: Evaluating Trauma-Informed Training”
John Conlon – Assistant Professor, Carnegie Mellon University, “Memory Rehearsal and Belief Biases”
Alex Chan – Professor, Harvard University, “Preference for Explainable AI”

PEEL Re-Opening + Refreshment Break | 2:30 pm to 3:30 pm

Session 4 | 3:30 pm – 4:45 pm
Alistair Wilson – Professor, University of Pittsburgh, “Veto Delegation: A Mechanism that works! (Kinda)”
Yucheng Liang – Assistant Professor, Carnegie Mellon University, “Asking the Right Questions: Information Acquisition for Choices under Risk”
Muriel Niederle - Professor, Stanford University

 

Investigating human and LLM psychology by prompting LLMs to play experimental economics games: Xie, Mei, Yuan, and Jackson in PNAS

 The great science fiction writer of my youth was Isaac Asimov, who not only wrote space opera (The Foundation Trilogy), but also wrote about intelligent robots, i.e. about robots with artificial general intelligence.  So, like you and me, they had complicated psychological lives, and one of the main characters in these stories was the robopsychologist  Dr. Susan Calvin (see e.g. the short story collection I, Robot, and also several of the robot novels).

I'm reminded of this by the several papers now reporting how large language models respond when asked to play games that have been used to study human behavior.  Those papers are framed as using LLMs to learn about the human behavior on which they were trained. But they can also be read as telling us about the 'psychology' of LLMs. Here's a good one from the PNAS. 

Xie, Yutong, Qiaozhu Mei, Walter Yuan, and Matthew O. Jackson. "Using large language models to categorize strategic situations and decipher motivations behind human behaviors." Proceedings of the National Academy of Sciences 122, no. 35 (2025): e2512075122. 

Abstract: By varying prompts to a large language model, we can elicit the full range of human behaviors in a variety of different scenarios in classic economic games. By analyzing which prompts elicit which behaviors, we can categorize and compare different strategic situations, which can also help provide insight into what different economic scenarios might induce people to think about. We discuss how this provides a step toward a nonstandard method of inferring (deciphering) the motivations behind the human behaviors. We also show how this deciphering process can be used to categorize differences in the behavioral tendencies of different populations. 

 

The effect of biological nudges may (also) have been overstated

 There have been a number of recent studies suggesting that the effectiveness of psychological "nudges" may have been substantially overstated. Here's a paper saying something similar about snorting testosterone:

 Dreber, Anna, Magnus Johannesson, Gideon Nave, Coren L. Apicella, Shawn N. Geniole, Taisuke Imai, Erik L. Knight et al. "Investigating the effects of single-dose intranasal testosterone on economic preferences in a large randomized trial of men." Proceedings of the National Academy of Sciences 122, no. 39 (2025): e2508519122. 

Abstract: "There is conflicting evidence on whether testosterone affects economic preferences such  as risk taking, fairness, and altruism, with most evidence coming from correlational  studies or small testosterone administration studies. To credibly test this hypothesis,  we conducted a large-scale, preregistered, double-blind randomized controlled trial with  1,000 male participants—10 to 20 times larger than typical prior studies. Participants  were randomly allocated to receive a single dose of either placebo or intranasal testosterone, and carried out a series of economic tasks capturing social preferences, competitiveness, and risk preferences. We find no evidence of a treatment effect for any of  our nine primary outcome measures, and no strong evidence of an association between  basal salivary testosterone and economic preferences within men. These results fail to  conceptually replicate previous high-impact publications reporting positive findings in  smaller samples, calling into question the idea that short-term testosterone fluctuations  are important drivers of men’s economic preferences. Our results do not rule out the  possibility that different effects might emerge under alternative dosages, administration  protocols, or task timings, or that behavioral effects differ between men and women. The  potential for developmental or long-term effects of testosterone also remains an open  question for future research, though such effects are ethically challenging to investigate  experimentally in humans." 

 

"The number  of participants in the studies reporting any statistically significant effects has ranged from N = 24 ( 18 ) to N = 118 ( 14 ). It is well  known that small, underpowered studies increase the risk that findings reported as statistically significant are false positives..."

...

"We fail to find evidence of a treatment effect of a single-dose of 
intranasal testosterone on any of our eight primary outcome measures or the hypothesized interaction effect for the ninth primary outcome measure. The 99.5% CI can be used to interpret which  effect sizes in the hypothesized direction we find strong evidence  against, see Figs. 1 – 3 . For our eight main effects primary hypotheses, we find strong evidence against effect sizes between about 0.15 Cohen’s d units (Investor Value and Loss Aversion) and 0.26 Cohen’s d units (Risk Aversion), which are considered small effect  sizes. We thus find strong evidence against the hypothesis that  single-dose intranasal testosterone administration has important  effects on economic preferences or behavior in men for all outcome measures in our study.

...

" Our study can be considered a highly powered conceptual replication of several previous results reported in high-impact journals, with a 10 to 20 times larger sample size than most previous  randomized controlled studies. However, our study does not constitute a direct replication of any specific previous study as we did  not base our design on one individual previous study ( 66 ). Our  study fails to conceptually replicate the following previous findings  about treatment effects of testosterone: that testosterone increases  offers in the ultimatum game ( 13 ); that testosterone decreases  offers in the ultimatum game ( 12 ); that testosterone decreases trust  in the trust game ( 16 ); that testosterone increases trustworthiness  (backtransfers) in the trust game ( 16 ), and that testosterone  increases risk taking (17). We furthermore fail to conceptually replicate previous correlational results that testosterone is positively correlated with economic risk-taking ( 26 , 29 , 67 ), that  testosterone is positively correlated with generosity in the dictator  game ( 30 ), and that testosterone is positively correlated with the  rejection of unfair offers in the ultimatum game ( 25 )."

Explaining economics can change the evaluation of policies, by Elias, Lacetera and Macis

 Perhaps economists should get involved in the discussion of public policies during political campaigns...

Is the Price Right? The Role of Economic Trade-Offs in Explaining Reactions to Price Surges
Julio Elías, Nicola Lacetera , Mario Macis    Management Science
Published Online:4 Jul 2025https://doi.org/10.1287/mnsc.2024.04555 

Abstract: Public authorities often introduce price controls following price surges, potentially causing inefficiencies and exacerbating shortages. A survey experiment with 7,612 Canadian and U.S. respondents shows that unregulated price surges raise moral objections and widespread disapproval. However, acceptance increases and demand for regulation declines when participants are prompted to consider economic trade-offs between controlled and unregulated prices, whereby incentives from higher prices lead to additional supply and enhance access to goods. Moreover, highlighting these trade-offs reduces polarization in moral judgments between supporters and opponents of unregulated pricing. Textual analysis of responses to open-ended questions provides further insights into our findings, and an incentivized donation task demonstrates consistency between stated preferences and real-stakes behavior. Although economic trade-offs do influence public support for price control policies, the evidence indicates that even when the potential gains in economic efficiency from unregulated prices are explicit, a significant divide persists between the utilitarian views that standard economic thinking implies and the nonutilitarian values held by the general population.

 

"Overall, therefore, we document widespread opposition to sudden price surges, motivated in large part by moral and ideological considerations. However, explicitly describing possible economic trade-offs between policy regimes does affect people’s reactions by making them more open to letting prices move freely. This result suggests that people do not immediately consider efficiency or equilibrium considerations when reacting to and expressing a judgment about price surges. When considerations about economic efficiency are missing, moral reactions are highly polarized; when economic trade-offs are explicit, views tend to converge. However, the fact that most respondents still support price control policies in this case suggests that this position derives from normative concerns and not necessarily from a lack of consideration for equilibrium effects and efficiency implications."

Experimental and behavioral econ at Stanford in August (registration open)

 Here is the program and links to register for the Stanford Experimental Economics session at SITE this summer: 

Session 13: Experimental Economics  Wed, Aug 20 2025, 8:00am - Thu, Aug 21 2025, 5:00pm PDT

Organized by
Christine Exley, University of Michigan
Muriel Niederle, Stanford University
Kirby Nielsen, California Institute of Technology
Al Roth, Stanford University
Lise Vesterlund, University of Pittsburgh

###########

And here's an announcement of a workshop that will immediately follow that session:

On August 21-23 Stanford will host the third instance of the Graduate Student Boot Camp in Behavioral Public Economics. The camp is organized by Doug Bernheim, Ben Lockwood, and Dmitry Taubinsky. It will feature instructional sessions from each of them, as well as sessions from four external speakers: Matthew Gentzkow, Botond Kőszegi, Olivia Mitchell, and Matthew Notowidigdo.

We would also like to invite other young scholars—graduate students, postdocs and assistant professors—to audit the boot camp. We will not be able to provide accommodations or cover travel expenses, but we welcome participation in our lectures/discussions and meals.  If you are a young scholar interested in attending some or all of the camp, please indicate your interest (and which sessions you would be able to attend, so that we can gauge attendance and our capacity constraints) via this form: https://forms.gle/dNBWgkrUXtYXgnMT9. We are aiming for 10-15 young scholars.

If you are interested in the content covered in previous instances of the boot camp, you can find additional information here and here.

All best,
Doug, Dmitry, and Ben