Multiple Testing:

New article discusses the “use and misuse of corrections for multiple testing.”

“In general, avoid corrections for multiple testing if statistical claims are to be made for each individual test...”

https://doi.org/10.1016/j.metip.2023.100120

#Stats
#Statistics
#MultipleTesting
#MultipleComparisons
#NHST

Opinion:

If your #Null seems ridiculous, so is your H1.

#NHST #stats

Psych Methods Book:

Forthcoming psychology methods book by someone who’s not afraid to challenge the status quo and always makes me think hard:

David Trafimow. (Aug 2023). Methodological Issues in Psychology: Concept, Method, and Measurement

https://www.taylorfrancis.com/books/mono/10.4324/9781003365167/methodological-issues-psychology-david-trafimow

#Science
#Psychology
@psychology
#ReplicationCrisis
#Statistics
#NHST
#Methodology
#Philsci
#PhilosophyOfScience

Online Talks & Workshops, Feb 8-11: Stats Reform, Open Science, R

Archaeology?! Yes, but it's for everyone--registrations are coming from across science, and from students to seasoned researchers. All welcome. Opening keynote by Nicole Lazar of ASA 'beyond p < .05' fame. I'm speaking later on the first day.

More

https://thenewstatistics.com/itns/2023/01/22/online-talks-workshops-feb-8-11-stats-reform-open-science-r/

#NHST #OpenScience #Teaching #TheNewStatistics #Uncategorized

Part II of my series on what to do with non-significant results is up now.

In this post, I focus on how to determine if your data is compatible with the claim of "no effect" (and why relying on p-values is wrong). It covers TOST equivalence tests and how to interpret and write up your results for publication. Link below 🔗

#statistics #frequentist #NHST #pvalue #equivalencetests #TOST #null #hypothesistesting #hypothesis

https://mzstats.blogspot.com/2023/01/what-to-do-with-null-results-part-ii.html

I'm ending the year by starting a blog - Finding Suff Out - a space where I can work out statistical and methodological problems I encounter in my job (it will contain guides, tutorials, rants, etc.)

Published my first post now. It's on what to do with non-significant results when you have a small sample size. See link below.

#statistics #frequentist #NHST #pvalue #newstatistics #estimation

https://mzstats.blogspot.com/2022/12/what-to-do-with-null-results-part-i.html

One commenter noted that they believe it is not necessarily #NHST that is the problem, but that #CS have limited experience w/ design of #experiments or #philosophyofscience.

I agree that most tend to use a grab bag of recipes without much understanding and it's understandable. I believe this is because it is just hard. It is hard to have only a semester or less in these types of methods and to understand how to use them without shooting yourself in the foot no matter the philosophy

It really is fun to objectively choose a level of alpha for NHT using Mudge's optimal alpha. It's so simple with a little simulation and elbow grease. And then you feel kinda boss-level when someone asks you why you did not use 0.05. https://journals.plos.org/plosone/article/comments?id=10.1371/journal.pone.0032734 #NHST #PowerAnalysis

Are p-values convoluted and arcane? Are confidence intervals hopelessly confusing? No! These ideas can be challenging to teach and learn, but they represent an invaluable way of thinking about scientific results. Once they're properly understood, they are more intuitive than they get credit for. Here is an attempt at a very brief explanation of why I love the logic of null hypothesis significance testing. [1/7]
#Statistics #Frequentist #NHST #Inference

What do you think about the statement that "the point null hypothesis is always false" (Cohen 1990, Am. Psych.), and therefore we would be better off either 1) abandoning statistical hypotheses (scientific hypotheses and CI + EF are enough) or 2) the null hypothesis should reflect the scientific hypothesis? (see Zumbo & Kroc 2016, J. Mod. App. Stat. Meth.).

@david_colquhoun @lakens @DenisMeuthen
#statistics #nhst

@lakens Why is the “new statistics” (I assume that of Geoff Cumming) a failed alternative? Doesn’t it just say use effect sizes and confidence intervals instead of NHST? What’s wrong with that (except reviewers asking “where are you p-values”)? #statistics #nhst #biostatistics

Nuzzo (2014) wrote a short opinion paper about P-values, criticizing significance testing (#nhst). The author argues that using P-values to make black-and-white decisions is not used as Fisher intended. Scientists should not ask "is there an effect" (nhst) but "how much of an effect is there" (using confidence intervals and effect sizes).

#statistics

Nuzzo, R 2014, Nature, "P-values, the 'gold standard of statistical validity, are not as reliable as many scientists assume"