"The backpack fallacy rears its ugly head once again"

Never heard of the backpack fallacy before, but it's a nice name for an all-too-common problem.

https://statmodeling.stat.columbia.edu/2023/08/22/the-backpack-fallacy-rears-its-ugly-head-once-again/

#Statistics #Fallacy #pvalue

"A Sensitivity Test Does Everything That a Significance Test Does, And Better". The former
- uses the same information as the latter's #pValue
- doesn't assume perfect randomization or measurement
- can deal with missing data

...and unless #stats software offers sensitivity analysis by default, the status quo will continue.

PDF: http://www.iosrjournals.org/iosr-jrme/papers/Vol-13%20Issue-2/Ser-4/G1302045056.pdf

Issue: http://www.iosrjournals.org/iosr-jrme/pages/vol13-issue2-Series-4.html

“Over time it appears the #pValue has become a gatekeeper for whether work is publishable, at least in some fields,” said Jessica Utts, ASA president. “This apparent editorial bias leads to the ‘#FileDrawerEffect,’ in which research with #StatisticallySignificant outcomes are much more likely to get published, while other work that might well be just as important scientifically is never seen in print.”

https://bird.makeup/users/jmcrookston/statuses/1443291322052489222 “The #pValue was never intended to be a substitute for scientific reasoning,” said Ron Wasserstein, the ASA’s executive director. “Well-reasoned #statistical arguments contain much more than the value of a single number and whether that number exceeds an arbitrary threshold. The ASA statement is intended to steer research into a ‘post p<0.05 era.’”

The original #metastudy (coauthored by Stanford people, lest you think statistics quackery is the exclusive realm of second tier universities), after excluding self-reported data, and when looking at the time/length correlation for the European population, gets a p-value of… <drumroll> 0.4…

Which (especially when you take into account the wild number of tests performed) is… 😂🤣😂🤡

https://wjmh.org/DOIx.php?id=10.5534/wjmh.220203

#pvalue #urology

Came across this interesting paper on the limitations of permutation as a method for computing the p-value of a test: https://academic.oup.com/bioinformatics/article/22/18/2244/317881
#stats #pvalue #permutation

Part II of my series on what to do with non-significant results is up now.

In this post, I focus on how to determine if your data is compatible with the claim of "no effect" (and why relying on p-values is wrong). It covers TOST equivalence tests and how to interpret and write up your results for publication. Link below 🔗

#statistics #frequentist #NHST #pvalue #equivalencetests #TOST #null #hypothesistesting #hypothesis

https://mzstats.blogspot.com/2023/01/what-to-do-with-null-results-part-ii.html

Does anyone know a good paper/article/video that explains that (and why) the p value follows a uniform distribution when the null is true (i.e., when there is no association/effect in the population)? @lakens (please correct me if I got this wrong) #pvalue #pvalues #stats #statistics

I'm ending the year by starting a blog - Finding Suff Out - a space where I can work out statistical and methodological problems I encounter in my job (it will contain guides, tutorials, rants, etc.)

Published my first post now. It's on what to do with non-significant results when you have a small sample size. See link below.

#statistics #frequentist #NHST #pvalue #newstatistics #estimation

https://mzstats.blogspot.com/2022/12/what-to-do-with-null-results-part-i.html

@cetra3 The web app seems to auto-decide on its own whether or not I'm caught up and then either display new posts above or below my current notional position in the feed. About half the time it guesses correctly. That's actually slightly better than my car does with deciding whether or not I want all the doors locked or just the driver's... #pvalue

“Accept uncertainty. Be thoughtful, open, and modest.” Remember “ATOM.” #statistics #datascience #pvalue

About null hypothesis testing (again)
#science #statistics #pvalue #significance #oddratio #xkcd
🔗 https://xkcd.com/2599/

About the statistical (in)significance debate, Andrew Gelman is very relevant, as usual:
🔗 https://statmodeling.stat.columbia.edu/2019/04/02/thinking-about-abandon-statistical-significance-p-values-etc/
#science #statistics #pvalue

Les ravages de la fétichisation des p-valeurs et du biais de publication, en une diapo.
(via http://audimath.math.cnrs.fr/paradoxes-aleatoires.html )
#pvalue #publicationBias