Tuesday, September 21, 2021

Modern peer review: how it evolved since the 1950's in Sociology

 Peer review is a big part of the design of modern academic publishing in scholarly journals. It wasn't always that way, and the current peer review system is pretty modern. Here's an account of its development in the discipline of Sociology, since the 1950's (which is similar to what we see in Economics, except that it appears Sociology relies substantially more on double-blind reviews).

Merriman, B. Peer Review as an Evolving Response to Organizational Constraint: Evidence from Sociology Journals, 1952–2018. The American Sociologist 52, 341–366 (2021). https://doi.org/10.1007/s12108-020-09473-x

"Abstract: Double-blind peer review is a central feature of the editorial model of most journals in sociology and neighboring social scientific fields, yet there is little history of how and when its main features developed. Drawing from nearly 70 years of annual reports of the editors of American Sociological Association journals, this article describes the historical emergence of major elements of editorial peer review. These reports and associated descriptive statistics are used to show that blind review, ad hoc review, the formal requirement of exclusive submission, routine use of the revise and resubmit decision, and common use of desk rejection developed separately over a period of decades. The article then argues that the ongoing evolution of the review model has not been driven by intellectual considerations. Rather, the evolution of peer review is best understood as the product of continuous efforts to steward editors’ scarce attention while preserving an open submission policy that favors authors’ interests."

From the introduction:

"In the main, editors are faculty members who operate a journal concurrently with ordinary work responsibilities; some receive modest, fixed remuneration, but editors have no strong financial interest in the journals they edit, and commonly serve for fixed or periodically renewed terms. (For most current journals, the economic interest rests primarily with one of a handful of large commercial publishers.) Journals do not restrict submissions by status criteria such as institutional affiliation or academic rank, and submission ordinarily carries little or no money cost, even at journals where authors assume a large part of the eventual expense of publication. Authors are expected to submit a given work exclusively to a single journal. After initial screening, submissions to a typical journal undergo double-blind review, in which the identities of authors and reviewers are not known to one another. Most evaluations of submitted manuscripts are produced by scholars who are not part of the appointed editorial staff of the journal. Work that is published has ordinarily undergone at least one formal round of revision and resubmission in response to the substance of external evaluations.

...

"At ASA journals, blind review, external review, exclusive submission, the formal revise and resubmit decision, and a developmental (rather than advisory) model of assessment developed in succession over a period of more than 30 years. In the twenty-first Century, persistent difficulties in obtaining timely reviews prompted a rapid, order of magnitude increase in frequency of rejections without review, commonly called desk rejections. Blind review was the only feature of the present model adopted at an ASA journal with an explicitly stated, unambiguously intellectual aim. This article argues that the other features of the current peer review model emerged as improvised efforts to balance two competing organizational imperatives: editors must steward scarce time and attention, but have also sought to render reasonably timely decisions without a priori exclusion of large numbers of prospective authors or capricious rejection of submissions. This pattern in journal operations in many ways reflects larger structural changes in sociology: rapid expansion of the field in the mid-to-late twentieth Century was succeeded by increasing competition in the academic labor market and heightened publishing expectations for tenure and promotion."

A snippet from the body of the paper:

"Ad hoc review, in which manuscripts are referred to scholars not formally affiliated with the journal organization, did not become an integral feature of the ordinary reviewing process until the early 1970s. Before then, virtually all evaluations were produced by members of journal editorial boards. Exclusive submission to a single journal also did not become a rule until the 1970s, and there is suggestive evidence that simultaneous submission to multiple journals may have been somewhat common until that time.

"At first, reviewing was plainly intended to aid to the work of the editor; the occasional value of reviews to authors was taken as an incidental benefit. The evolution of a developmental model of review oriented toward the author was gradual, as was the emergence of the revise and resubmit decision as a nearly unavoidable intermediate step on the path to publication."

...

And in conclusion:

"A primary constraint on editorial innovation is, of course, the professional and status structure of academic disciplines. An extensive body of research on disciplines, and on higher educational institutions more generally, has shown a powerful isomorphic tendency: such structures tend to converge on a given form of practice even if all the actors are wholly aware of its inadequacies. Further, change in such practices will, under most circumstances, be slow: individual academic advancement involves regularly submitting oneself to the judgment of the more experienced members of a discipline according to the standards those more experienced scholars impose. Those who may have the freshest view of an intellectual field, and perhaps a greater impulse to explore new lines of work, also face the strongest pressures to invest their time and effort conservatively in the oldest means of publicizing their work.

"Efforts to change publishing norms therefore stand a much greater chance of success if they are adopted first, or early, by actors who occupy central places in a field, or if they are given the strong, credible endorsement of such actors (Starbuck 2016: 178). Conversations about academic publishing models, especially their relative unresponsiveness to changing circumstances in the twenty-first Century, often possess a degree of fatalism. But the development of editorial peer review itself is an important reminder of how rapidly a good idea may spread."


HT: Retraction Watch

Monday, September 20, 2021

Keeping the market for new economists thick: AEA guidelines on timing of interviews

 Below are some guidelines suggested by the American Economic Association for the conduct of this year's job market for new Ph.D.s

AEA Guidance on Timeline for 2021-22 Economics Job Cycle

"The AEA Executive Committee, in conjunction with its ad hoc Committee on the Job Market, recognizes that it is to the benefit of the profession if the job market for economists is thick, with many employers and job candidates participating in the same stages at the same time.  Moreover, the AEA's goals of fairness, inclusion, and diversity are fostered by having a timeline that remains widely accepted, even as public health conditions necessitate a virtual ASSA meeting again this year. With these goals in mind, and in light of inquiries from both students and departments about how to proceed, the Association asks that departments and other employers consider the following timeline for initial interviews and “flyouts” in the upcoming job cycle.


Interview invitations
The AEA suggests that employers wait to extend interview invitations until at least Thursday, December 2, 2021.

Rationale: the AEA will deliver signals from job candidates to employers on December 1. We suggest that employers review those signals and incorporate them into their decision-making before extending interview invitations. Job candidates from under-represented groups may lack informal networks and thus, may especially rely on the signals to convey their interest. Waiting to review the signals before issuing invitations promotes a fairer, more equitable process.

We also ask that all employers indicate on EconTrack when they have extended interview invitations; this allows candidates to learn about the status of searches without visiting websites posting crowd-sourced information and potentially inappropriate other content.

Interviews
The AEA suggests that employers conduct initial interviews starting on Monday, January 3, 2022, and that all interviews take place virtually; i.e. either by phone or online (e.g. by Zoom). We suggest that interviews not take place during the AEA meeting itself (January 7-9, 2022).

Rationale: In the past, interviews were conducted in person at the AEA/ASSA meetings. This promoted thickness of the market, because most candidates and employers were present at the in-person meetings, but had the disadvantage of precluding both job candidates and interviewers from fully participating in AEA/ASSA sessions. Given that the 2022 AEA/ASSA meetings will be entirely virtual, we suggest that interviews NOT take place on the meeting dates to allow job candidates and interviewers to participate in the conference.

We recommend that employers wait until January 3 to interview candidates because job candidates may have teaching or TA responsibilities in December, and to ensure that candidates have accurate expectations of the timing of the stages of the market. An unraveling of the market works against the Association’s goal of having a thick market at each stage and also against candidates having uniform expectations of the market’s timing. All interviews should be conducted by phone or online to prevent risk of exposure to COVID, and to promote equity among the candidates.

Flyouts and offers
Flyouts and offers generally happen at times appropriate for the employer, and the AEA sees no reason to suggest otherwise.  We ask that all employers indicate on EconTrack when they have extended flyout invitations and closed their searches.

Job market institutions and mechanisms
Please keep in mind the various job market institutions and mechanisms created by the AEA to improve the job market:


Thank you for your attention to this initiative."

Sunday, September 19, 2021

Stanford welcomes students back to campus

Starting tomorrow, we'll have classes in person, indoors, masked.



Saturday, September 18, 2021

Surrogacy law under review in New Zealand

 From the U. of Canterbury:

Who are my parents? Why New Zealand’s ‘creaky’ surrogacy laws are overdue for major reform by Debra Wilson, Annick Masselot,  and Martha Ceballos 

"several separate pieces of legislation cover the two types of surrogacy: gestational, where the child is not genetically related to the surrogate parent; and traditional, where the child is genetically related.

The resulting legal confusion is now the subject of a Law Commission review, which proposes significant reform based on the guiding principle that “the best interests of the child should be paramount”.

Right now, that cannot be said of the way surrogate children and their parents are treated under law that even judges have described as “creaky” and “inadequate”.

 ...

Surrogacy is regulated through the Human Assisted Reproductive Technology Act, which prohibits commercial surrogacy and requires gestational surrogacy to be approved by an ethics committee.

But that act is silent on the legal parentage of the child, leaving this to be determined by the Status of Children Act. Effectively, the woman who gives birth and her partner (if the partner consents to the assisted reproduction) are the child’s legal parents.

This means the intended parents have no legal rights to the child – even if they are the genetic parents – until they adopt the child under the Adoption Act.

But legal parentage is important. Legal parents transfer citizenship to their children and act on their behalf, such as giving consent to medical treatment or travel."

Friday, September 17, 2021

Lectures on equilibrium in markets for indivisible goods at U. Tokyo by Teytelboym, Baldwin and Jagadeesan, Sept 21-24

 Towards a general theory of markets with indivisible goods: special lectures at The University of Tokyo Market Design Center (UTMD)

September 21-24, 2021 (Japan time) 

Organizers:

Fuhito Kojma, Director, UTokyo Market Design Center, and Professor, the University of Tokyo

Michihiro Kandori: Vice Director, UTokyo Market Design Center, and University Professor, the University of Tokyo

Yuichiro Kamada: Associate Professor, UC Berkeley, and Global Fellow, the University of Tokyo

Venue: Zoom online  Language: English

Program

*All times shown below are Japan time.

*Each lecture will be followed by 30 minutes Q&A session.


Lecture 1 9/21 (Tue) 16:00-17:30

Introduction to Markets for Indivisible Goods (by Alexander Teytelboym)

In many settings, such as auctions, the indivisibility of goods is a key market feature. But in markets with indivisible goods, competitive equilibria might not exist.  We explore conditions, such as substitutability of goods, that ensure existence of competitive equilibria. We also discuss connections between conditions for existence, tâtonnement, and cooperative properties of equilibria.


Lecture 2 9/22 (Wed) 16:00-17:30

The geometry of preferences: demand types, equilibrium with Indivisibilities, and bidding languages (by Elizabeth Baldwin)


An equivalence theorem between geometric structures and utility functions allows new methods for understanding preferences. Our classification of valuations into “Demand Types”, incorporates existing definitions regarding the comparative statics of demand (substitutes, complements, “strong substitutes”, etc.) and permits new ones. Our Unimodularity Theorem generalises previous results about when competitive equilibrium exists for any set of agents whose valuations are all of a “demand type”. Contrary to popular belief, equilibrium is guaranteed for more classes of purely-complements, than of purely-substitutes, preferences. Our Intersection Count Theorem checks equilibrium existence for combinations of agents with specific valuations by counting the intersection points of geometric objects. Applications include the “Product-Mix Auction” introduced by the Bank of England in response to the financial crisis. In that context, we show that all substitutes preferences can be represented, and no other preferences can be represented, by appropriate sets of permitted bids in the Substitutes Product-Mix Auction language; an analogous result holds for strong substitutes, when we refine the characteristics of the language. These languages thus also provide new characterizations of (all) substitutes, and of strong substitutes, respectively.


Lecture 3 9/23 (Thu) 16:00-17:30 The Equilibrium Existence Duality (by Alexander Teytelboym)

We show that, with indivisible goods, the existence of competitive equilibrium fundamentally depends on agents’ substitution effects, not their income effects. Our Equilibrium Existence Duality allows us to transport results on the existence of competitive equilibrium from settings with transferable utility to settings with income effects. One consequence is that net substitutability—which is a strictly weaker condition than gross substitutability—is sufficient for the existence of competitive equilibrium. Further applications give new existence results beyond the case of (net) substitutes. Our results have implications for auction design.


Lecture 4 9/24 (Fri) 09:30-11:00 Matching and Prices (by Ravi Jagadeesan)

Indivisibilities and budget constraints are pervasive features of many matching markets. But gross substitutability — a standard condition on preferences in matching models — typically fails in such markets. To accommodate budget constraints and other income effects, we instead assume that agents’ preferences satisfy net substitutability. Although competitive equilibria do not generally exist in our setting, we show that stable outcomes always exist and are efficient. We illustrate how the flexibility of prices is critical for our results. We also discuss how budget constraints and other income effects affect the properties of standard auction and matching procedures, as well as of the set of stable outcomes.


  Recorded lecture will be posted on UTMD’s YouTube channel within 6 hours.

 

Thursday, September 16, 2021

David Grether, 1938-2021

 David Grether, a pioneer in experimental economics at Caltech has died.

Here's the initial Caltech announcement, promising more to come: 

David Grether, Caltech Frank Gilloon Professor of Economics, Emeritus, passed away on September 12. He was 82.

The paper of his that I remember best (and that I have taught whenever I cover preference reversals) is his incentivized replication of the preference reversals first noticed in hypothetical choice psychology experiments:

Grether, David M., and Charles R. Plott. "Economic theory of choice and the preference reversal phenomenon." The American Economic Review 69.4 (1979): 623-638.

Here are the papers he listed prominently on his web page:

SELECTED PUBLICATIONS

"Mental Processes and Strategic Equilibration: An fMRI Study of Selling Strategies in Second Price Auctions" with C. Plott, D. Rowe, M. Sereno and J. Allman Experimental Economics Vol. 10 (2007) pp. 105-122

Sequencing strategies in large, competitive, ascending price automobile auctions: An experimental study with Charles R. Plott  Journal of Economic Behavior and Organization Vol. 71 (2009) pp.75-88 http://dx.doi.org/10.1016/j.jebo.2009.02.018

The preference reversal phenomenon: Response mode, markets and incentives, with James Cox. Economic Theory 7 (1996): 387-405.

Individual behavior and market performance. American Journal of Agricultural Economics 76 (1994): 1079--1083.

Are people Bayesian? Uncovering behavioral strategies, with Mahmoud A. El-Gamal. Journal of the American Statistical Association 90 (1995): 1127-1145.

Wednesday, September 15, 2021

School choice in Latin America, in BBC News Mundo

 A news story with an unusually detailed account of school choice algorithms discusses some of the issues in Latin America, in Spanish, in BBC News Mundo. (Google translate does an adequate job...).  One of the issues discussed is that geographic priorities for schools give wealthier families an advantage, and perpetuate geographic segregation.

Qué es el Mecanismo de Boston y por qué expertos denuncian que hay segregación en las asignaciones de escuelas en América Latina  Analía Llorente  BBC News Mundo

[G translate: What is the Boston Mechanism and why experts denounce that there is segregation in school assignments in Latin America]

Some snippets:

"The Boston mechanism allows for a lot of parenting strategy and that means that it generates a lot of inequalities " [says] Paula Jaramillo, Associate Professor in Economics at the Universidad de los Andes in Colombia

...

"The criticism is against the Boston Mechanism because it continues to be applied, but it is also against the deferred acceptance method because it is generating segregation at the neighborhood level," says Caterina Calsamiglia, a leader in the investigation of these methods.

"The specialist refers to the fact that a student obtains more points for his proximity to the preferred school, therefore he has a greater chance of being admitted to it.

"This creates one of the main strategies is the moving neighborhood, decision can only carry out families with middle income to high, creating inequality."

...

"In many places in Latin America the selection method is not even regulated, and where it is, they are not very transparent with parents in communicating the methods.

"We know a lot about what is done by cities in the United States, in Europe, in Asia, we don't know so much about Latin America," says Paula Jaramillo.

...

"In conclusion, the experts believe that there is no magic method that can be applied uniformly in the countries of the region to avoid segregation and inequality in school selection.

"They agree that the deferred acceptance method is the "fairest" but not perfect. There are many factors to take into account from the quality of the schools to their location."