On the challenges of drawing conclusions from p-values just below 0.05

1640 days ago
@DrDominikVogel @ingorohlfing @Chengxin_M_Xu @Journal_BPA As I explain in https://t.co/HNyInPv1xY and https://t.co/2H8XIkaaf3, the pattern of p-values is influenced by many factors, especially in a heterogeneous set of studies like this. If there were only true effects, the curve should be steeper.
1846 days ago
@MarkZobeck @ErikVanZwet @aidybarnett That's the rub - you are already grouping pub bias and researchers DoF together, and you assume you can say anything about them in this graph. I explain in this 2015 paper that this is very difficult: https://t.co/s2Rq4iX7wE.
1848 days ago
@shilaan01 That's a real hornet's nest. People have tried to identify p-hacking. Here's a comment of mine explaining why that doesn't work: https://t.co/2H8XIkaaf3 and here's a paper explaining why it is practically impossible even with perfect data: https://t.co/s2Rq4iX7wE
1926 days ago
@wuthrich_k @jhaushofer Very cool - you might be interested in some early criticism I wrote on papers using p-curves to conclude widespread bias, where I also pointed out it's not that easy: https://t.co/s2Rq4iX7wE and https://t.co/2H8XIkaaf3.
2226 days ago
@HammadSheikh Sure, but people who try to find it can not provide proof it happens on a widespread scale https://t.co/SlHZ1IKdZQ but feel free to try.
2226 days ago
@profjmb See https://t.co/SlHZ1IKdZQ - if you want to make specific clsims you will need to model those and as my papers shows that is not easy. People widely misunderstand this.
2226 days ago
@ignaziano Because https://t.co/SlHZ1IKdZQ
2279 days ago
@rubenarslan @RogertheGS @richarddmorey @ceptional @LeonidTiokhin Exactly. I probably wrote the best paper in the literature on the challenges of drawing conclusions of p-values just below .05 https://t.co/SlHZ1IKdZQ And I am still ok with using this as a general heuristic for non-prereg papers in journal we know commit massive publication bias
2552 days ago
2019 resolution- rely less on p-values and more on persuasive science. Great read by ⁦@lakens⁩ - another reason we should rethink how we use p values when formulating hypothesis. Just because something isn’t<.05 doesn’t make it true, and vice versa. https://t.co/haj7VxcRnh
RT @pash22: On the challenges of drawing conclusions from p-values just below 0.05 https://t.co/GpLVb3vri6 via @lakens
RT @pash22: On the challenges of drawing conclusions from p-values just below 0.05 https://t.co/GpLVb3vri6 via @lakens
RT @pash22: On the challenges of drawing conclusions from p-values just below 0.05 https://t.co/GpLVb3vri6 via @lakens
RT @pash22: On the challenges of drawing conclusions from p-values just below 0.05 https://t.co/GpLVb3vri6 via @lakens
2553 days ago
RT @pash22: On the challenges of drawing conclusions from p-values just below 0.05 https://t.co/GpLVb3vri6 via @lakens
RT @pash22: On the challenges of drawing conclusions from p-values just below 0.05 https://t.co/GpLVb3vri6 via @lakens
RT @pash22: On the challenges of drawing conclusions from p-values just below 0.05 https://t.co/GpLVb3vri6 via @lakens
2553 days ago
https://t.co/k73KJWPf8U
RT @pash22: On the challenges of drawing conclusions from p-values just below 0.05 https://t.co/GpLVb3vri6 via @lakens
RT @pash22: On the challenges of drawing conclusions from p-values just below 0.05 https://t.co/GpLVb3vri6 via @lakens
RT @pash22: On the challenges of drawing conclusions from p-values just below 0.05 https://t.co/GpLVb3vri6 via @lakens
2553 days ago
RT @pash22: On the challenges of drawing conclusions from p-values just below 0.05 https://t.co/GpLVb3vri6 via @lakens
2553 days ago
RT @pash22: On the challenges of drawing conclusions from p-values just below 0.05 https://t.co/GpLVb3vri6 via @lakens
2553 days ago
RT @pash22: On the challenges of drawing conclusions from p-values just below 0.05 https://t.co/GpLVb3vri6 via @lakens
2553 days ago
On the challenges of drawing conclusions from p-values just below 0.05 https://t.co/GpLVb3vri6 via @lakens
RT @flourneuro: Why hello problematic distribution of test statistics in economics. Also, tools to help thinking about underlying models of…