Hello #statstodon! A reviewer asks me to perform a post-hoc #PowerAnalysis. I know this is generally not advised because if you replace the a priori effect size by the effect size measured in the experiment, this will introduce an erroneous relationship between the significance level of the test and the measured power.
… but does that mean that there is no proper way of measuring power retrospectively? For example, if you refrain from using the measured effect size and instead simulate a range of “a priori” effect size unrelated to the results of the test, then the dependency of the power to the significance level should not happen?
#stats #statschat @lakens

Had a lot of fun figuring out how modals work in Shiny in order to add this pop-up window of individual interaction simulations to my power analysis web app for interactions/moderation: https://david-baranger.shinyapps.io/InteractionPoweR_analytic/
#rstats #shiny #PowerAnalysis

Can anyone point me to a paper/post explaining why you should not base your sample size calculation on the effect found in the literature or a pre-test? Pretty sure I read it once but cannot find it anymore. Maybe I saw it in @lakens's course? #poweranalysis #samplesize

Andrew Gelman famously said that "you need 16 times the sample size to estimate an interaction than to estimate a main effect" https://statmodeling.stat.columbia.edu/2018/03/15/need-16-times-sample-size-estimate-interaction-estimate-main-effect/. Where does 16x come from? If you're used to thinking about interactions as regressions, his process is confusing! A 🧵... #stats #PowerAnalysis

I've seen posts about conducting power analyses by using pilot data to guide simulations/bootstraps. Does anyone know of any resources/papers/blog posts on how this works? #stats #rstats #PowerAnalysis #simulation

Suppose you've designed a study and found a "null" result, and you'd like to argue that you've found evidence against the phenomenon in question producing a large effect. Should you emphasize how statistically powerful your study was?
You can, but there are much better ways to describe the evidence. Provide an upper bound on the plausible magnitude of the effect given the theory tests your data permit. For example, report a confidence interval. [1/3]
#Statistics #StatsTeaching #PowerAnalysis

Suppose you've designed a study and found a "null" result, and you'd like to argue that you've found evidence against the phenomenon in question producing a large effect. Should you emphasize how statistically powerful your study was?
You can, but there are much better ways to describe the evidence. Provide an upper bound on the plausible magnitude of the effect given the theory tests your data permit. For example, report a confidence interval. [1/3]
#Statistics #StatsTeaching #PowerAnalysis

It really is fun to objectively choose a level of alpha for NHT using Mudge's optimal alpha. It's so simple with a little simulation and elbow grease. And then you feel kinda boss-level when someone asks you why you did not use 0.05. https://journals.plos.org/plosone/article/comments?id=10.1371/journal.pone.0032734 #NHST #PowerAnalysis

https://biol607.github.io/lectures/power_analysis.html#1 for anyone who wants these #PowerAnalysis slides or https://biol607.github.io/lab/power_analysis_lab.html for a lab

I know not everyone of my students knows who this is anymore, but I really cannot resist making this a title slide for #PowerAnalysis

Fricken Lazer beans!

Interesting read, and video, attacking a cryptoprocessor with power analysis and laser glitching. Thought I'd pass this along

#cryptography #cryptanalysis #cryptoprocessor #laserglitch #poweranalysis

https://hackaday.com/2022/11/26/defeating-a-cryptoprocessor-with-laser-beams/

Fricken Lazer beans!

Interesting read, and video, attacking a cryptoprocessor with power analysis and laser glitching. Thought I'd pass this along

#cryptography #cryptanalysis #cryptoprocessor #laserglitch #poweranalysis

https://hackaday.com/2022/11/26/defeating-a-cryptoprocessor-with-laser-beams/

Updated preprint for @jlfossum's awesome paper on #poweranalysis in #mediationanalysis. The main takeaway is that if you’re simulating normal or nonnormal homoskedastic data you can use joint significance test for power analysis instead of bootstrapping but not if you simulate heteroskedadtic data. #psychology #quantmethods #psychmethods #quantpsych #psychresearch https://psyarxiv.com/5tm2x/