Not so much, is the answer given in the article below by Doucouliago and Ioannidis in the December issue of J of Economic Surveys (see, even the date isn't credible:). John Ioannidis is quite well known for his critiques of the reliability of published medical research.
Ioannidis, J. and Doucouliagos, C. (2013), WHAT'S TO KNOW ABOUT THE CREDIBILITY OF EMPIRICAL ECONOMICS?. Journal of Economic Surveys, 27: 997–1004. doi: 10.1111/joes.12032
Abstract
The scientific credibility of economics is itself a scientific question that can be addressed with both theoretical speculations and empirical data. In this review, we examine the major parameters that are expected to affect the credibility of empirical economics: sample size, magnitude of pursued effects, number and pre-selection of tested relationships, flexibility and lack of standardization in designs, definitions, outcomes and analyses, financial and other interests and prejudices, and the multiplicity and fragmentation of efforts. We summarize and discuss the empirical evidence on the lack of a robust reproducibility culture in economics and business research, the prevalence of potential publication and other selective reporting biases, and other failures and biases in the market of scientific information. Overall, the credibility of the economics literature is likely to be modest or even low.
...
4. Conclusions: Unanswered Questions and Possibilities for Improvement
Despite the aforementioned empirical evidence, much remains unknown. How credible is economics research in different subfields? Is credibility rising or falling over time? How can this market be improved? What is the impact of corporate and institutional interests, journals, funding agencies, and other stakeholders and how can we utilize this potential impact to improve the credibility of economics?
However some paths forward seem clear: strengthen the reproducibility culture with emphasis on independent replication; conduct larger, better studies; promote collaborative efforts rather than siloed, one-investigator research; and reduce biases and conflicts. The exact interventions, which might achieve these changes are not clear, and perhaps there is room for conducting experimental studies on different potential interventions. Progress may be difficult to achieve unless the rewards and incentives system of conducting and publishing research is modified. Little progress is likely if investigators get rewarded and promoted for publishing significant results and for perpetuating theories and claims even when they are wrong. Conversely, one might expect better outcomes, if replication research is encouraged, reproducibility is rewarded, and/or irreproducibility is penalized.
Answering these questions requires more meta-research (i.e. empirical research on research). Empirical studies that have been performed in other scientific disciplines may be extrapolated and conducted in economics. Tests for bias may also be adapted for economics and embraced by leading journals. Of course, meta-research is also susceptible to bias and errors of its own. The extent of bias and possible errors needs to be evaluated also for emerging meta-research tools. Nonetheless, tests for small-study effects and publication or selective reporting biases need to be applied increasingly, further developed and evaluated (Stanley and Doucouliagos, 2012).
************
See also
Why Most Published Research Findings Are False
John P. A. Ioannidis
Published: Aug 30, 2005DOI: 10.1371/journal.pmed.0020124
Ioannidis, J. and Doucouliagos, C. (2013), WHAT'S TO KNOW ABOUT THE CREDIBILITY OF EMPIRICAL ECONOMICS?. Journal of Economic Surveys, 27: 997–1004. doi: 10.1111/joes.12032
Abstract
The scientific credibility of economics is itself a scientific question that can be addressed with both theoretical speculations and empirical data. In this review, we examine the major parameters that are expected to affect the credibility of empirical economics: sample size, magnitude of pursued effects, number and pre-selection of tested relationships, flexibility and lack of standardization in designs, definitions, outcomes and analyses, financial and other interests and prejudices, and the multiplicity and fragmentation of efforts. We summarize and discuss the empirical evidence on the lack of a robust reproducibility culture in economics and business research, the prevalence of potential publication and other selective reporting biases, and other failures and biases in the market of scientific information. Overall, the credibility of the economics literature is likely to be modest or even low.
...
4. Conclusions: Unanswered Questions and Possibilities for Improvement
Despite the aforementioned empirical evidence, much remains unknown. How credible is economics research in different subfields? Is credibility rising or falling over time? How can this market be improved? What is the impact of corporate and institutional interests, journals, funding agencies, and other stakeholders and how can we utilize this potential impact to improve the credibility of economics?
However some paths forward seem clear: strengthen the reproducibility culture with emphasis on independent replication; conduct larger, better studies; promote collaborative efforts rather than siloed, one-investigator research; and reduce biases and conflicts. The exact interventions, which might achieve these changes are not clear, and perhaps there is room for conducting experimental studies on different potential interventions. Progress may be difficult to achieve unless the rewards and incentives system of conducting and publishing research is modified. Little progress is likely if investigators get rewarded and promoted for publishing significant results and for perpetuating theories and claims even when they are wrong. Conversely, one might expect better outcomes, if replication research is encouraged, reproducibility is rewarded, and/or irreproducibility is penalized.
Answering these questions requires more meta-research (i.e. empirical research on research). Empirical studies that have been performed in other scientific disciplines may be extrapolated and conducted in economics. Tests for bias may also be adapted for economics and embraced by leading journals. Of course, meta-research is also susceptible to bias and errors of its own. The extent of bias and possible errors needs to be evaluated also for emerging meta-research tools. Nonetheless, tests for small-study effects and publication or selective reporting biases need to be applied increasingly, further developed and evaluated (Stanley and Doucouliagos, 2012).
************
See also
Why Most Published Research Findings Are False
John P. A. Ioannidis
Published: Aug 30, 2005DOI: 10.1371/journal.pmed.0020124
Posts
No comments:
Post a Comment