Showing posts sorted by date for query "Einstein Foundation". Sort by relevance Show all posts
Showing posts sorted by date for query "Einstein Foundation". Sort by relevance Show all posts

Tuesday, October 8, 2024

An own-goal in replication science--retraction of a paper that reported high replicability

  A 2023 paper reporting high replicability of psychology experiments has been retracted from Nature Human Behavior. The retraction notice says in part 
"The concerns relate to lack of transparency and misstatement of the hypotheses and predictions the reported meta-study was designed to test; lack of preregistration for measures and analyses supporting the titular claim (against statements asserting preregistration in the published article); selection of outcome measures and analyses with knowledge of the data; and incomplete reporting of data and analyses."

RETRACTED ARTICLE: High replicability of newly discovered social-behavioural findings is achievable

This article was retracted on 24 September 2024

Matters Arising to this article was published on 24 September 2024

This article has been updated

Abstract

Failures to replicate evidence of new discoveries have forced scientists to ask whether this unreliability is due to suboptimal implementation of methods or whether presumptively optimal methods are not, in fact, optimal. This paper reports an investigation by four coordinated laboratories of the prospective replicability of 16 novel experimental findings using rigour-enhancing practices: confirmatory tests, large sample sizes, preregistration and methodological transparency. In contrast to past systematic replication efforts that reported replication rates averaging 50%, replication attempts here produced the expected effects with significance testing (P < 0.05) in 86% of attempts, slightly exceeding the maximum expected replicability based on observed effect sizes and sample sizes. When one lab attempted to replicate an effect discovered by another lab, the effect size in the replications was 97% that in the original study. This high replication rate justifies confidence in rigour-enhancing methods to increase the replicability of new discoveries.

########


############

In general, I'm more optimistic about replications than preregistrations for identifying replicable results and testing hypotheses about them.  In this case, preregistration apparently revealed that what was written up as a replication study had begun as something else, and that the goal posts had been moved ex post, apparently in inappropriate ways.
######
Somewhat related are my posts on the Einstein Foundation Award for Promoting Quality in Research.

Thursday, January 25, 2024

Call for nominations: Einstein Foundation Award for Promoting Quality in Research.

 Here's the call for the Einstein Foundation Award for Promoting Quality in Research. (I've finished a term and am no longer on the jury...)

THE 2024 CALL IS OPEN!

CALL FOR ENTRIES

DEADLINE 
APRIL 30, 2024
(10:00 pm UTC)

→ Submit online
 

 

The annual €500,000 Einstein Foundation Award for Promoting Quality in Research in cooperation with the BIH Quest Center for Responsible Research is inviting applications and nominations again. The international award is open to any researcher, or group of researchers, institution, organization, and early career researcher around the globe whose work helps to fundamentally advance the quality, transparency, and reproducibility of science and research. We warmly welcome applications and nominations from marginalized and underrepresented groups.

The Award will honor successful candidates in the following three categories: 

I Individual Award (€200,000): Individuals or small teams who have outstandingly contributed to fostering research quality can be nominated. Nominators are strongly encouraged to consider a diverse set of criteria, including gender, race/ethnicity, geography, and career stage.

II Institutional Award (€200,000): Governmental and non-governmental organizations, institutions, or other entities that have notably enhanced research quality can apply or be nominated. Successful governmental organizations or institutions will not receive any funds in addition to the award itself.

III Early Career Award (€100,000): Individuals or teams can submit a project proposal that seeks to foster research quality and value. Eligible candidates must hold a doctorate or have equivalent research experience and should not have been working as an independent researcher for more than five years. In the case of a team entry, the majority must be Early Career Researchers.

The deadline for entries is April 30, 2024. The awardees will be announced by the end of 2024.

Learn more about all past winners and finalists here.

Selection
An international, interdisciplinary, and diverse panel of researchers and research quality activists will evaluate submissions and select awardees. Meet the jury here
.

For questions, please contact Einstein Foundation Award Coordinator Dr. Ulrike Pannasch.


Wednesday, November 15, 2023

2023 EINSTEIN FOUNDATION AWARD FOR PROMOTING QUALITY IN RESEARCH

 Here's the press release from the Einstein Foundation in Berlin:

THE WINNERS OF THE 2023 EINSTEIN FOUNDATION AWARD FOR PROMOTING QUALITY IN RESEARCH

"THE EINSTEIN FOUNDATION BERLIN IS TO HONOR BELGIAN BIOINFORMATICIAN YVES MOREAU, THE BERKELEY INITIATIVE FOR TRANSPARENCY IN THE SOCIAL SCIENCES, AND THE RESPONSIBLE RESEARCH ASSESSMENT INITIATIVE WITH THIS YEAR’S EINSTEIN FOUNDATION AWARD FOR PROMOTING QUALITY IN RESEARCH 2023. 

The recipient of the Individual Award is Yves Moreau from the Katholieke Universiteit Leuven. Moreau ranks among the most ardent advocates for ethical standards in the utilization of human DNA data in the age of artificial intelligence and big data. He designs algorithms that protect personal privacy during the analysis of genetic data. This year’s Institutional Award recognizes the work of the Berkeley Initiative for Transparency in the Social Sciences (BITSS), which advocates for rigor, transparency, and reproducibility in social scientific research. The Institute achieves this through establishing open science practices, developing appropriate infrastructure, and conducting meta-research. The 2023 Early Career Award goes to the Responsible Research Assessment Initiative headed by Anne Gärtner (Dresden University of Technology). The project aims to identify, test, and establish novel criteria for the assessment of researchers and their output. Moving away from quantity of output and other unsuitable metrics, it will foreground quality of research by taking into account factors such as transparency, robustness, innovation, and cooperation. 

...

"Jury member Michel Cosnard, computer scientist at the UniversitĂ© Nice-Sophia-Antipolis, believes that Yves Moreau is highly deserving of the award, which recognizes his unwavering dedication on both professional and ethical fronts. “Moreau links deep research in DNA analysis and artificial intelligence with ethics, integrity, and human rights. His work and achievements serve as a cornerstone to help us confront the difficult social questions that arise from rapid technological developments.” 

"Fellow jury member and Stanford University economist Alvin Roth firmly endorses the chosen winner of the Institutional Award: “The Berkeley Initiative for Transparency in the Social Sciences plays an active, creative role in the ‘credibility revolution’ in science by promoting careful experimentation, and supporting efforts to make replication and verification commonplace.” 

###########

The two previous awards

Monday, December 5, 2022

Saturday, February 11, 2023

Monday, December 5, 2022

The 2022 Einstein Foundation Awards

 Here are the Recipients of the 2022 Einstein Foundation Awards

Einstein Foundation Individual Award Winner 2022:

Gordon Guyatt, for his seminal contributions to evidence based medicine (including the name)

"Einstein Foundation Award winner Gordon Guyatt improved the quality of clinical research and helped bring evidence to medicine. His next mission is bringing patients into the discussion"


Einstein Foundation Institutional Award 2022:

The Psychological Science Accelerator

"The Psychological Science Accelerator, winner of the 2022 Institutional Award, has made its name by facilitating large, crowdsourced international studies in all aspects of the discipline and by democratizing and diversifying big team psychological science."


Einstein Foundation Early Career Award 2022:

Ape Research Index (the name almost speaks for itself)


Here's the description of the Award(s): The Einstein Foundation Award for Promoting Quality in Research

"The Einstein Foundation Award for Promoting Quality in Research aims to provide recognition and publicity for outstanding efforts that enhance the rigor, reliability, robustness, and transparency of research in the natural sciences, the social sciences, and the humanities, and stimulate awareness and activities fostering research quality among scientists, institutions, funders, and politicians. To acknowledge the outstanding role early career researchers (ECRs) have in promoting research quality, ECRs will be invited to propose projects that foster research quality and value. Projects will be competitively selected for funding and internationally showcased."

Here are the members of this year's award jury. (I am one of the 15 jury members).


Here's my post on last year's prizes:

Thursday, November 25, 2021


Saturday, February 19, 2022

The Einstein Foundation Award for Promoting Quality in Research, call for nominations

 April 10 is the deadline for new nominations for the 2022 awards. (Some nominations from last year will be held over for consideration this year.

The Einstein Foundation Award for Promoting Quality in Research

Here's my post regarding the 2021 awards:

Thursday, November 25, 2021

Monday, February 14, 2022

Building capacity in science

 Here's a post from Nature about some of the projects highlighted by the recent Einstein Foundation awards for promoting quality in research


Science Should Value Building Research Capacity

T
The Source
Written by Patrick S. Forscher and Moreen Terer

Science values outstanding, already-completed scientific achievements and the people who make them. This priority is illustrated in the profusion of scientific awards, such the Nobel Prize, Fields Medal, and Breakthrough Prize, that reward these accomplishments. However, science places less emphasis on efforts to promote quality within the research ecosystem itself. Nor does science typically recognize the critical capacity-building activities that are necessary to create the robust scientific ecosystems necessary to produce high quality research in the first place.


On November 21, 2021, the Einstein Foundation broke with this trend by issuing a series of awards for efforts to promote quality within the research ecosystem itself. Even more unusually, one of its award categories, the Early Career category, recognized not past achievements but rather outstanding proposed projects that showed special promise in promoting future quality. Four projects were shortlisted for this €100,000 award.

The projects that were shortlisted for the Early Career award shared something in common that may be unexpected: they are unusually focused on building a strong scientific community, especially in groups and settings that science has neglected. Take the first author’s (Patrick Forscher’s) project, for example. This project aimed to grow behavioral science in Africa by building a website, called “Lab in a Box”, to make it easy to set up a new behavioral lab in Africa, enhancing an existing database of measures with existing measures that are adapted to African languages and contexts, and stocking that database with newly translated and adapted measures. These activities are feasible due to Patrick’s position at a research center, the Busara Center for Behavioral Economics, that is both headquartered in Africa and dedicated to advancing behavioral science in the Global South. As Africa currently produces 2% of all research output (Kasprowicz et al., 2020), capacity-building activities such as the ones proposed are critical if behavioral science is to establish a robust presence in Africa.

***********

Capacity building is important in medicine as well as in science.  Here's a paper on global chains of kidney exchange, that would contribute to building surgical capacity around the world.

Global kidney chains, by Afshin Nikzad, Mohammad Akbarpour, Michael A. Rees, Alvin E. Roth, Proceedings of the National Academy of Sciences Sep 2021, 118 (36) e2106652118; DOI: 10.1073/pnas.2106652118

Thursday, November 25, 2021

Promoting Quality in Research: the inaugural Einstein Foundation Awards, 2021

 I had the privilege of being a member of the jury for the Einstein Foundation Awards, which were awarded yesterday for the first time.

Here's the award booklet: 

EINSTEIN FOUNDATION AWARD 2021 PROMOTING QUALITY IN RESEARCH

Individual Award 2021:  Paul Ginsparg

"Preprints have been shared in the physics community since the early 1950s but mostly among well established professors. Physicist Paul Ginsparg, who receives the Einstein Foundation’s Individual Award, set out to democratize access to scientific results. Today, his preprint server arXiv has spread to many other fields—and made science progress more efficient and fairer."


Institutional Award 2021: The Center for Open Science

"Open research is on the rise, but a lot of research information still remains behind closed doors. The Center for Open Science, recipient of the Institutional Award, advocates for more transparency and open access, training scientists, publishers and funders, providing technology and policy recommendations. With this multi pronged approach, it has helped create an open science community which is becoming ever more self-sustaining."


Early Career Researcher Award 2021: "ManyBabies5: Teaming up for Developmental Science → manybabies.github.io/MB5

"Unlike adult subjects, babies cannot tell you what they are thinking. Therefore, researchers studying infant development use looking patterns to understand how babies think and feel about the world. “An overwhelming majority of studies use looking time to make inferences about infant cognition,” comments Martin Zettersten from Princeton University in the United States. But what exactly is it that drives babies to pay attention to different things? To come up with a statistically sound answer to this question, he and his colleague Jessica Kosie initiated ManyBabies5, a large international consortium of infant researchers. “We aim to increase diversity amongst researchers and test subjects alike and collaboratively want to come up with the best test possible on how different factors matter in infant looking time,” Jessica Kosie explains. To that end, the ManyBabies team will put together a diverse and sizeable sample of infants around the world. “To me, team science is a useful tool for producing high quality, robust science,” says Martin Zettersten. “Embracing this approach as a field as many groups are doing right now—that feels revolutionary to me.”


THE JURY The Einstein Foundation Council has convened an outstanding group of scholars representing the natural sciences, the humanities, and the social sciences. The international jury is presided over by Dieter Imboden and both defines the objectives of the award and selects the awardees. The following jury members have been appointed for the first three year term: DIETER IMBODEN (President), PhD, Theoretical Physics; Professor Emeritus of Environmental Physics, ETH ZĂĽrich; DOROTHY BISHOP, PhD, Neuropsychology; Professor of Developmental Neuropsychology, Oxford University; ALASTAIR BUCHAN, MD, PhD, Medicine; Professor of Stroke Medicine, Oxford University; MICHEL COSNARD, PhD, Computer Science; Professor Emeritus of Informatics, UniversitĂ© de CĂ´te d’Azur; LORRAINE DASTON, PhD, History of Science; Director Emerita of the Max Planck Institute for the History of Science Berlin; MOSHE HALBERTAL, PhD, Philosophy; Professor of Jewish Thought and Philosophy, Hebrew University; LENA LAVINAS, PhD, Economics; Professor of Welfare Economics, Universidade Federal do Rio de Janeiro; JULIE MAXTON, PhD, Law; Executive Director of the Royal Society, London; MARCIA MCNUTT, PhD, Geophysics; President of the National Academy of Sciences of the United States; EDWARD MIGUEL, PhD, Economics; Professor of Environmental and Resource Economics, University of California, Berkeley; ALVIN ROTH, PhD, Economics; Professor of Economics, Stanford University; SOAZIC ELISE WANG SONNE, Economist, World Bank Group; PhD Fellow, United Nations University; SUZY STYLES, PhD., Psychology; Professor of Psycholinguistics, Nanyang Technological University; E. JĂśRGEN ZĂ–LLNER, Dr. Dr. h.c. mult., Medicine; Chairman of the Board of Trustees of the Stiftung CharitĂ©, Senator (ret.), Berlin

**************

Here's my earlier description of the award:

Saturday, December 12, 2020


Monday, August 30, 2021

Symposium on Research Integrity: Replicability and outright malfeasance. November 24 in Berlin

 Research integrity has been in the news lately, concerning low levels of replicability in some kinds of research, together with more intentional (but probably/hopefully less widespread) problems.  Here's a forthcoming symposium:

Symposium on Integrity in Research

Place and time: hybrid event on November 24, 2021.

"Symposium topic: Research Integrity is a controversial topic within academia, but also in public discourse. Prominent cases of scientific misconduct capture the limelight, but recently a multitude of issues beyond the classical triad of plagiarism, falsification, and fabrication have taken center stage, many of which concern the quality and rigor of research and further challenge the trustworthiness of science. These include selective publication and file-drawer problems, various biases, as well as the general reproducibility or robustness of research results. At the same time, open, inclusive, and creative cultures of research in teams and organizations have been threatened by practices that prioritize outputs, as demonstrated by numerous examples of insufficient mentoring, unfair authorship practices, or intransparency about career progress for younger researchers. All these issues have long histories of discussions about improving scientific methods, however, views differ on how important these issues are, and whether all disciplines are affected equally. While some argue that there is only one scientific method, requiring universal standards for robust evidence, others emphasize the diversity of research cultures and the mutual criticism and learning that can result from this diversity.

"As scientific expertise becomes more important in and for the public, it becomes apparent that scientific findings are often provisional, subject to correction, and scientific experts may disagree. There are no simple either-or answers. While it seems indisputable that scientific evidence should be subject to the highest possible standards and be appropriate to the context, it is nevertheless necessary that these standards evolve as scientific methods and research questions progress. For urgent societal problems - such as pandemics - we may even be willing to lower these standards. For new problems, appropriate standards will only emerge after much experimentation and debate. The need for such constant debate is familiar to scientists, but can be disconcerting to the public. Standardization - both in the sense of setting standards and in the sense of homogenization - of research can therefore run the risk of undermining, rather than securing, the progress of knowledge. As a result, the integrity of research must remain a topic for debate, as it is expected to ensure both the robustness and innovation of research while meeting the expectations of different research cultures and the public.

"As a contested topic, research integrity encompasses a wide range of actors, platforms and organizations, policies and measures. The symposium will bring together participants from research, practice and policy to map this heterogeneous field, provide evidence of its effectiveness and analyze its (future) development.


"Keynotes and formats

"The symposium is organized as a hybrid event and will include keynotes by Professor Lorraine Daston (Max Planck Institute for the History of Science Berlin / University of Chicago) and Professor Dava J. Newman (Massachusetts Institute of Technology / MIT Media Lab).

"Other international and national experts will be invited together with the event's guests to discuss and reflect on the current state of research integrity and its future development. Interactive and inclusive formats connecting on-site participants and digital guests worldwide will ensure a sustained exchange on a topic of central importance for the future of science.

The symposium is organized in concurrence with the Einstein Foundation Berlin’s “Einstein Award for Promoting Quality in Research” that will take place later the same evening.

Registration, access and program

Participation in the symposium is free of charge. Further details on registration, access and the program of the event will be published on this page soon.

Contact  Nele Albrecht, Scientific Coordinator for Research Quality  Email: core@berlin-university-alliance.de 

Saturday, December 12, 2020

The Einstein Foundation Award for Promoting Quality in Research (apply or nominate now...)

 Here's the announcement of a new award, for promoting research integrity, reproducibility, and or other elements of research quality.

The Einstein Foundation Award for Promoting Quality in Research

"Objective

The Einstein Foundation Award for Promoting Quality in Research aims to provide recognition and publicity for outstanding efforts that enhance the rigor, reliability, robustness, and transparency of research, and stimulate awareness and activities fostering research quality among scientists, institutions, funders, and politicians. To acknowledge the outstanding role early career researchers (ECRs) have in promoting research quality, ECRs will be invited to propose projects that foster research quality and value. Projects will be competitively selected for funding and internationally showcased.

Award Categories

Individual Award: Individual scientist or small teams of collaborating scientists can be nominated. The laureate will be awarded €200,000.

 

Institutional Award: Governmental and non-governmental organizations, institutions, or other entities can apply or be nominated. The award-winning organization or institution will receive €200,000. If governmental organizations or institutions are the recipients of the award, they will not receive any funds in addition to the award itself.

 

Early Career Award: Early career researcher can submit a project proposal for an award of €100,000.