Non-standard errors

Albert J. Menkveld, Anna Dreber, Felix Holzmeister, Jürgen Huber, Magnus Johannesson, Michael Kirchler, Michael Razen, Utz Weitzel, David Abad Díaz, Menachem Abudy, Tobias Adrian, Yacine Ait-Sahalia, Olivier Akmansoy, Jamie Alcock, Vitali Alexeev, Arash Aloosh, Livia Amato, Diego Amaya, James Angel, Amadeus Bach, Edwin Baidoo, Gaetan Bakalli, Andrea Barbon, Oksana Bashchenko, Parampreet Christopher Bindra, Geir Hoidal Bjonnes, Jeff Black, Bernard S. Black, Santiago Bohorquez, Oleg Bondarenko, Charles S. Bos, Ciril Bosch-Rosa, Elie Bouri, Christian Brownlees, Anna Calamia, Viet Nga Cao, Gunther Capelle-Blancard, Laura Capera, Massimiliano Caporin, Allen Carrion, Tolga Caskurlu, Bidisha Chakrabarty, Mikhail Chernov, William M. Cheung, Ludwig Boris Chincarini, Tarun Chordia, Sheung Chi Chow, Benjamin Clapham, Jean-Edouard Colliard, Carole Comerton-Forde, Edward Curran, Thong Dao, Wale Dare, Ryan J. Davies, Riccardo De Blasis, Gianluca De Nard, Fany Declerck, Oleg Deev, Hans Degryse, Solomon Deku, Christophe Desagre, Mathijs van Dijk, Chukwuma Dim, Thomas Dimpfl, Yun Jiang Dong, Philip Drummond, Tom Dudda, Ariadna Dumitrescu, Teodor Dyakov, Anne Haubo Dyhrberg, Michał Dzieliński, Asli Eksi, Izidin El Kalak, Saskia ter Ellen, Nicolas Eugster, Martin D. D. Evans, Michael Farrell, Ester Félez-Viñas, Gerardo Ferrara, El Mehdi Ferrouhi, Andrea Flori, Jonathan Fluharty-Jaidee, Sean Foley, Kingsley Y. L. Fong, Thierry Foucault, Tatiana Franus, Francesco Franzoni, Bart Frijns, Michael Frömmel, Servanna Fu, Sascha Füllbrunn, Baoqing Gan, Thomas Gehrig, Dirk Gerritsen, Javier Gil-Bazo, Lawrence R. Glosten, Thomas Gomez, Arseny Gorbenko, Ufuk Güçbilmez, Joachim Grammig, Vincent Gregoire, Björn Hagströmer, Julien Hambuckers, Erik Hapnes, Jeffrey H. Harris, Lawrence Harris, Simon Hartmann, Jean-Baptiste Hasse, Nikolaus Hautsch, Xuezhong He, Davidson Heath, Simon Hediger, Terrence Hendershott, Ann Marie Hibbert, Erik Hjalmarsson, Seth A. Hoelscher, Peter Hoffmann, Craig W. Holden, Alex R. Horenstein, Wenqian Huang, Da Huang, Christophe Hurlin, Alexey Ivashchenko, Subramanian Rama Iyer, Hossein Jahanshahloo, Naji Jalkh, Charles M. Jones, Simon Jurkatis, Petri Jylha, Andreas Kaeck, Gabriel Kaiser, Arzé Karam, Egle Karmaziene, Bernhard Kassner, Markku Kaustia, Ekaterina Kazak, Fearghal Kearney, Vincent van Kervel, Saad Khan, Marta Khomyn, Tony Klein, Olga Klein, Alexander Klos, Michael Koetter, Jan Pieter Krahnen, Aleksey Kolokolov, Robert A. Korajczyk, Roman Kozhan, Amy Kwan, Quentin Lajaunie, FY Eric C Lam, Marie Lambert, Hugues Langlois, Jens Lausen, Tobias Lauter, Markus Leippold, Vladimir Levin, Yijie Li, Hui Li, Chee Yoong Liew, Thomas Lindner, Oliver B. Linton, Jiacheng Liu, Anqi Liu, Guillermo Llorente, Matthijs Lof, Ariel Lohr, Francis A. Longstaff, Alejandro Lopez-Lira, Shawn Mankad, Nicola Mano, Alexis Marchal, Charles Martineau, Francesco Mazzola, Debrah Meloso, Roxana Mihet, Vijay Mohan, Sophie Moinas, David Moore, Liangyi Mu, Dmitriy Muravyev, Dermot Murphy, Gabor Neszveda, Christian Neumeier, Ulf Nielsson, Mahendrarajah Nimalendran, Sven Nolte, Lars L. Norden, Peter O'Neill, Khaled Obaid, Bernt Arne Ødegaard, Per Östberg, Marcus Painter, Stefan Palan, Imon Palit, Andreas Park, Roberto Pascual, Paolo Pasquariello, Lubos Pastor, Vinay Patel, Andrew J. Patton, Neil D. Pearson, Loriana Pelizzon, Aleksandra Rzeźnik, Anthony Sanford, Harikumar Sankaran, Asani Sarkar, Lucio Sarno, O. Scaillet, Stefan Scharnowski, Klaus Reiner Schenk-Hoppé, Andrea Schertler, Michael Schneider, Florian Schroeder, Norman Schürhoff, Philipp Schuster, Marco A. Schwarz, Mark S. Seasholes, Norman Seeger, Or Shachar, Andriy Shkilko, Jessica Shui, Mario Sikic, Giorgia Simion, Lee A. Smales, Paul Söderlind, Elvira Sojli, Konstantin Sokolov, Laima Spokeviciute, Denitsa Stefanova, Marti G. Subrahmanyam, Sebastian Neusüss, Barnabas Szaszi, Oleksandr Talavera, Yuehua Tang, Nicholas Taylor, Wing Wah Tham, Erik Theissen, Julian Thimme, Ian Tonks, Hai Tran, Luca Trapin, Anders B. Trolle, Giorgio Valente, Robert A. Van Ness, Aurelio Vasquez, Thanos Verousis, Patrick Verwijmeren, Anders Vilhelmsson, Grigory Vilkov, Vladimir Vladimirov, Sebastian Vogel, Stefan Voigt, Wolf Wagner, Thomas Walther, Patrick Weiss, Michel van der Wel, Ingrid M. Werner, Joakim Westerholm, Christian Westheide, Evert Wipplinger, Michael Wolf, Christiaan Cornelis Petrus Wolff, Leonard Wolk, Wing Keung Wong, Jan Wrampelmeyer, Shuo Xia, Dacheng Xiu, Ke Xu, Caihong Xu, Pradeep Yadav, José Yagüe, Cheng Yan, Antti Yang, Woongsun Yoo, Wenjia Yu, Shihao Yu, Bart Zhou Yueshen, Darya Yuferova, Marcin Zamojski, Abalfazl Zareei, Stefan Zeisberger, Sarah Zhang, Xiaoyu Zhang, Zhuo Zhong, Z. Ivy Zhou, Chen Zhou, Xingyu Sonya Zhu, Marius Zoican, Remco C. J. Zwinkels, Jian Chen, Teodor Duevski, Ge Gao, Roland Gemayel, Dudley Gilder, Paul Kuhle, Emiliano Pagnotta, Michele Pelli, Jantje Sönksen, Lu Zhang, Konrad Ilczuk, Dimitar Bogoev, Ya Qian, Hans C. Wika, Yihe Yu, Lu Zhao, Michael Mi, Li Bao, Andreea Vaduva, Marcel Prokopczuk, Alejandro Avetikian, Zhen-Xing Wu

  • In statistics, samples are drawn from a population in a data-generating process (DGP). Standard errors measure the uncertainty in sample estimates of population parameters. In science, evidence is generated to test hypotheses in an evidence-generating process (EGP). We claim that EGP variation across researchers adds uncertainty: non-standard errors. To study them, we let 164 teams test six hypotheses on the same sample. We find that non-standard errors are sizeable, on par with standard errors. Their size (i) co-varies only weakly with team merits, reproducibility, or peer rating, (ii) declines significantly after peer-feedback, and (iii) is underestimated by participants.

Download full text files

Export metadata

Metadaten
Author:Albert J. Menkveld, Anna Dreber, Felix Holzmeister, Jürgen Huber, Magnus Johannesson, Michael Kirchler, Michael Razen, Utz Weitzel, David Abad Díaz, Menachem Abudy, Tobias Adrian, Yacine Ait-Sahalia, Olivier Akmansoy, Jamie Alcock, Vitali Alexeev, Arash Aloosh, Livia Amato, Diego Amaya, James Angel, Amadeus Bach, Edwin Baidoo, Gaetan Bakalli, Andrea Barbon, Oksana Bashchenko, Parampreet Christopher Bindra, Geir Hoidal Bjonnes, Jeff Black, Bernard S. Black, Santiago Bohorquez, Oleg Bondarenko, Charles S. Bos, Ciril Bosch-Rosa, Elie Bouri, Christian Brownlees, Anna Calamia, Viet Nga Cao, Gunther Capelle-Blancard, Laura Capera, Massimiliano CaporinORCiDGND, Allen Carrion, Tolga Caskurlu, Bidisha Chakrabarty, Mikhail Chernov, William M. Cheung, Ludwig Boris Chincarini, Tarun Chordia, Sheung Chi Chow, Benjamin Clapham, Jean-Edouard Colliard, Carole Comerton-Forde, Edward Curran, Thong Dao, Wale Dare, Ryan J. Davies, Riccardo De Blasis, Gianluca De Nard, Fany Declerck, Oleg Deev, Hans Degryse, Solomon Deku, Christophe Desagre, Mathijs van Dijk, Chukwuma Dim, Thomas Dimpfl, Yun Jiang Dong, Philip Drummond, Tom Dudda, Ariadna Dumitrescu, Teodor Dyakov, Anne Haubo Dyhrberg, Michał Dzieliński, Asli Eksi, Izidin El Kalak, Saskia ter Ellen, Nicolas Eugster, Martin D. D. Evans, Michael Farrell, Ester Félez-Viñas, Gerardo Ferrara, El Mehdi Ferrouhi, Andrea Flori, Jonathan Fluharty-Jaidee, Sean Foley, Kingsley Y. L. Fong, Thierry Foucault, Tatiana Franus, Francesco Franzoni, Bart Frijns, Michael Frömmel, Servanna Fu, Sascha Füllbrunn, Baoqing Gan, Thomas Gehrig, Dirk Gerritsen, Javier Gil-Bazo, Lawrence R. Glosten, Thomas Gomez, Arseny Gorbenko, Ufuk Güçbilmez, Joachim Grammig, Vincent Gregoire, Björn Hagströmer, Julien Hambuckers, Erik Hapnes, Jeffrey H. Harris, Lawrence Harris, Simon Hartmann, Jean-Baptiste Hasse, Nikolaus Hautsch, Xuezhong He, Davidson Heath, Simon Hediger, Terrence Hendershott, Ann Marie Hibbert, Erik Hjalmarsson, Seth A. Hoelscher, Peter Hoffmann, Craig W. Holden, Alex R. Horenstein, Wenqian Huang, Da Huang, Christophe Hurlin, Alexey Ivashchenko, Subramanian Rama Iyer, Hossein Jahanshahloo, Naji Jalkh, Charles M. Jones, Simon Jurkatis, Petri Jylha, Andreas Kaeck, Gabriel Kaiser, Arzé Karam, Egle Karmaziene, Bernhard Kassner, Markku Kaustia, Ekaterina Kazak, Fearghal Kearney, Vincent van Kervel, Saad Khan, Marta Khomyn, Tony Klein, Olga Klein, Alexander Klos, Michael Koetter, Jan Pieter KrahnenORCiDGND, Aleksey Kolokolov, Robert A. Korajczyk, Roman Kozhan, Amy Kwan, Quentin Lajaunie, FY Eric C Lam, Marie Lambert, Hugues Langlois, Jens Lausen, Tobias Lauter, Markus Leippold, Vladimir Levin, Yijie Li, Hui Li, Chee Yoong Liew, Thomas Lindner, Oliver B. Linton, Jiacheng Liu, Anqi Liu, Guillermo Llorente, Matthijs Lof, Ariel Lohr, Francis A. Longstaff, Alejandro Lopez-Lira, Shawn Mankad, Nicola Mano, Alexis Marchal, Charles Martineau, Francesco Mazzola, Debrah Meloso, Roxana Mihet, Vijay Mohan, Sophie Moinas, David Moore, Liangyi Mu, Dmitriy Muravyev, Dermot Murphy, Gabor Neszveda, Christian Neumeier, Ulf Nielsson, Mahendrarajah Nimalendran, Sven Nolte, Lars L. Norden, Peter O'Neill, Khaled Obaid, Bernt Arne Ødegaard, Per Östberg, Marcus Painter, Stefan Palan, Imon Palit, Andreas Park, Roberto Pascual, Paolo Pasquariello, Lubos Pastor, Vinay Patel, Andrew J. Patton, Neil D. Pearson, Loriana PelizzonORCiDGND, Aleksandra Rzeźnik, Anthony Sanford, Harikumar Sankaran, Asani Sarkar, Lucio Sarno, O. Scaillet, Stefan Scharnowski, Klaus Reiner Schenk-Hoppé, Andrea Schertler, Michael Schneider, Florian Schroeder, Norman Schürhoff, Philipp Schuster, Marco A. Schwarz, Mark S. Seasholes, Norman Seeger, Or Shachar, Andriy Shkilko, Jessica Shui, Mario Sikic, Giorgia Simion, Lee A. Smales, Paul Söderlind, Elvira Sojli, Konstantin Sokolov, Laima Spokeviciute, Denitsa Stefanova, Marti G. Subrahmanyam, Sebastian Neusüss, Barnabas Szaszi, Oleksandr Talavera, Yuehua Tang, Nicholas Taylor, Wing Wah Tham, Erik TheissenORCiDGND, Julian Thimme, Ian Tonks, Hai Tran, Luca Trapin, Anders B. Trolle, Giorgio Valente, Robert A. Van Ness, Aurelio Vasquez, Thanos Verousis, Patrick Verwijmeren, Anders Vilhelmsson, Grigory Vilkov, Vladimir Vladimirov, Sebastian Vogel, Stefan Voigt, Wolf Wagner, Thomas Walther, Patrick Weiss, Michel van der Wel, Ingrid M. Werner, Joakim Westerholm, Christian Westheide, Evert Wipplinger, Michael Wolf, Christiaan Cornelis Petrus Wolff, Leonard Wolk, Wing Keung Wong, Jan Wrampelmeyer, Shuo Xia, Dacheng Xiu, Ke Xu, Caihong Xu, Pradeep Yadav, José Yagüe, Cheng Yan, Antti Yang, Woongsun Yoo, Wenjia Yu, Shihao Yu, Bart Zhou Yueshen, Darya Yuferova, Marcin Zamojski, Abalfazl Zareei, Stefan Zeisberger, Sarah Zhang, Xiaoyu Zhang, Zhuo Zhong, Z. Ivy Zhou, Chen Zhou, Xingyu Sonya Zhu, Marius Zoican, Remco C. J. Zwinkels, Jian Chen, Teodor Duevski, Ge Gao, Roland Gemayel, Dudley Gilder, Paul Kuhle, Emiliano Pagnotta, Michele Pelli, Jantje Sönksen, Lu Zhang, Konrad Ilczuk, Dimitar Bogoev, Ya Qian, Hans C. Wika, Yihe Yu, Lu Zhao, Michael Mi, Li Bao, Andreea Vaduva, Marcel Prokopczuk, Alejandro Avetikian, Zhen-Xing Wu
URN:urn:nbn:de:hebis:30:3-616364
URL:https://ssrn.com/abstract=3961574
DOI:https://doi.org/10.2139/ssrn.3961574
Parent Title (English):SAFE working paper ; No. 327
Series (Serial Number):SAFE working paper series (327)
Publisher:SAFE
Place of publication:Frankfurt am Main
Document Type:Working Paper
Language:English
Year of Completion:2021
Year of first Publication:2021
Publishing Institution:Universitätsbibliothek Johann Christian Senckenberg
Release Date:2021/11/30
Psyndex Keyword:liquidity; multi-analyst approach; non-standard errors
Issue:November 23, 2021
Page Number:56
Note:
The coordinators are grateful for financial support from (Dreber) the Knut and Alice Wallenberg Foundation, the Marianne, Marcus Wallenberg Foundation, the Jan Wallander, Tom Hedelius Foundation, (Huber) an FWF grant P29362, (Huber and Kirchler) FWF SFB F63, and (Menkveld) NWO-Vici.
Institutes:Wirtschaftswissenschaften / Wirtschaftswissenschaften
Wissenschaftliche Zentren und koordinierte Programme / House of Finance (HoF)
Wissenschaftliche Zentren und koordinierte Programme / Center for Financial Studies (CFS)
Wissenschaftliche Zentren und koordinierte Programme / Sustainable Architecture for Finance in Europe (SAFE)
Dewey Decimal Classification:3 Sozialwissenschaften / 33 Wirtschaft / 330 Wirtschaft
JEL-Classification:C Mathematical and Quantitative Methods / C1 Econometric and Statistical Methods: General / C12 Hypothesis Testing
Sammlungen:Universitätspublikationen
Licence (German):License LogoDeutsches Urheberrecht