Critical Metascience:

2022 has been a bumper year for what I’d call “critical metascience” - work that takes a step back and offers a critical perspective in the field.

My Top 10 papers of 2022 in this area are, in alphabetical order… 🥁 🧵👉

#OpenScience
#MetaScience
#MetaResearch
#PsychMethods
#ReplicationCrisis
#SociologyofScience
#ScienceofScience
#PhilosophyOfScience
#PhilSci
#PhilScidon

1/12

For #FollowFriday, can I recommend some fairly new additions to our Mastodonian family:

@devezer theoretical metascientist

@itaiyanai co-host of the Night Science podcast

@orlamuldoon professor of social psychology

@qjep The Quarterly Journal of Experimental Psychology

#Metascience
#NightScience
#SocialPsychology
#Psychology
@socialpsych
#MetaResearch
#ReplicationCrisis
#PhilosophyOfScience
#PhilSci
#PhilScidon
#AcademicChatter
@academicchatter
#PolPsych
#PoliticalScience

Can we kill theories?

Social psychology consists of “a vast graveyard of undead theories” (Ferguson & Heene, 2012).

 “No experimental result can ever kill a theory" (Lakatos, 1976).

#PhilosophyOfScience​
#PhilSci​
#PhilScidon​
#SocialPsychology
@socialpsych

How interested are scientists in the truth?

Mizrahi (2021) analysed the text of scientific papers and found that “scientists use the terms ‘knowledge’ and ‘understanding’ significantly more often than the term ‘truth’ when they talk about the aims or goals of scientific research in their published works."

#PhilosophyOfScience
#PhilSci
#PhilScidon
#MetaScience
#MetaResearch
#SociologyofScience
#ScienceofScience

https://doi.org/10.1007/s11229-020-02889-5

Replicability and Theory:

“Our results suggest that many of the practices that have been proposed as a means to improve the replicability of psychological research—such as open data and methods…preregistration and Registered Reports…and basing conclusions on Bayesian inference…or p < .005 rather than p < .05…—do indeed improve confidence in replicability among our sample.”

Continued 🙂 🧵👉

#MetaScience
#MetaResearch
#PsychMethods
#ReplicationCrisis
#PhilosophyOfScience
#PhilSci
#PhilScidon

@MarkRubin This is massively simplistic. Hypotheses include the criteria for delineating phenomena in need of explanation, satisfaction criteria for success, disciplinary standards and practices, and taxonomies of subjects under investigation. IMO.
#MetaScience
#MetaResearch
#PsychMethods
#ReplicationCrisis
#PhilosophyOfScience
#PhilSci
#PhilScidon

What’s a hypothesis?

“A hypothesis is not simply a guess about the result of an experiment. It is a proposed explanation that can predict the outcome of an experiment. A hypothesis has two components: (1) an explanation and (2) a prediction. A prediction simply isn’t useful on its own.” (Haroz, 2014)

Blog post: http://steveharoz.com/blog/2014/mysterious-origins-of-hypotheses-in-visualization-and-chi/

#MetaScience
#MetaResearch
#PsychMethods
#ReplicationCrisis
#PhilosophyOfScience
#PhilSci
#PhilScidon

A “quietist” response to the replication crisis:

“The quietist approach proposes that we should just accept that it is in the nature of science that we get things wrong, and that this is particularly true with sciences in early stages of development.”

Bird (2021). Understanding the replication crisis as a base rate fallacy.

🔒 https://doi.org/10.1093/bjps/axy051

🔓 https://kclpure.kcl.ac.uk/portal/files/95454096/Replication_base_rate_fallacy_REV2.pdf

#MetaScience
#MetaResearch
#PsychMethods
#ReplicationCrisis
#PhilosophyOfScience
#PhilSci
#PhilScidon
@philosophy

Crypto-positivism:

“A central dimension of our argument is that many of the tenets of positivism are so embedded within Western culture, academia, and the world of education in particular that they are often invisible to researchers and those who consume their research.”

Kincheloe and Tobin (2015): https://doi.org/10.1163/9789463000765_003

#PhilosophyOfScience
#PhilSci
#PhilScidon
@philosophy
#Positivism

Philosophy-free science?

"There is no such thing as philosophy-free science, just science that has been conducted without any consideration of its underlying philosophical assumptions."

(Dennett, 2014, p. 20, https://www.google.co.uk/books/edition/Intuition_Pumps_And_Other_Tools_for_Thin/9SduAwAAQBAJ?hl=en&gbpv=0)

#PhilosophyOfScience
#PhilSci
#PhilScidon
@philosophy

“Do predictions, and what kind of predictions, justify belief in the truth of scientific theories?”

Brilliant new PhD thesis by Pekka Syrjänen considers “the epistemic role of prediction in science,” provides a clear, easy-to-follow discussion of the issues in this area, and develops a new “predictive virtues approach.”

PhD thesis: https://helda.helsinki.fi/bitstream/handle/10138/350314/Syrj%C3%A4nen_Pekka_dissertation_2022.pdf

Six quotes follows 👉 /1

#PhilosophyOfScience
#PhilSci
#PhilScidon
#PhD
@philosophy

Hypothesis testing:

“There is no problem with a test having a high or even a maximal probability of passing the hypothesis it tests; there is only a problem if it has a high probability of passing hypotheses erroneously.”

Deborah Mayo (1996, p. 275) “Error and the growth of experimental knowledge”

https://errorstatistics.files.wordpress.com/2020/10/egek-pdf-red.pdf

#PhilosophyOfScience
#PhilSci
#PhilScidon
#Statistics

A sculpting metaphor of theorizing in experimental psychology.

From Roland Pfister (2022). Operationalization and generalization in experimental psychology: A plea for bold claims. In Gozli & Valsiner (Eds.), Experimental psychology: Ambitions and possibilities.

https://link.springer.com/book/10.1007/978-3-031-17053-9

#Psychology
#PhilosophyOfScience
#PhilSci
#PhilScidon
#Theory
#Science

New book on “New Philosophical Perspectives on Scientific Progress”

A collection of original essays by experts in the field considering different types of progress in different disciplines, including #physics, #chemistry, #biology, #seismology, #psychology, #sociology, #economics, and #medicine.

https://www.routledge.com/New-Philosophical-Perspectives-on-Scientific-Progress/Shan/p/book/9780367760557#

#PhilosophyOfScience
#PhilSci
#PhilScidon
@philosophy

“We are not only in a replication but an interpretation crisis, a crisis of theory building.”

Benjamin Krämer (@benjkraemer) (2022, November). Why are most published research findings under-theorized? In Questions of Communicative Change and Continuity.

🔓 https://www.nomos-elibrary.de/10.5771/9783748928232-23.pdf

#OpenScience
#MetaScience
#PsychMethods
#ReplicationCrisis
#ScienceofScience
#PhilosophyOfScience
#PhilSci
#PhilScidon
#Communication

“Scientific Progress: By-Whom or For-Whom?”

“Scientific progress occurs when scientific information of the right sort is made available to relevant members of society at large, including scientists themselves but not excluding various groups of non-scientists who rely on that information for different purposes.”

New preprint by Finnur Dellsén (forthcoming in Studies in History and Philosophy of Science): http://philsci-archive.pitt.edu/21453/

#philosophyofscience
#philsci
#philscidon
@philosophy

Happy #WorldPhilosophyDay!

Here’s a summary of six ways in which philosophy is useful to science...

From De Haro (2020): “Science and Philosophy: A Love–Hate Relationship” https://doi.org/10.1007/s10699-019-09619-2

#philosophyofscience
#philsci
#philscidon
@philosophy

Forthcoming book by Nancy Cartwright and colleagues:

“The Tangle of Science: Reliability Beyond Method, Rigour, and Objectivity”

https://global.oup.com/academic/product/the-tangle-of-science-9780198866343

#philosophyofscience
#philsci
#philscidon
@philosophy

Objectivity in science:

"The objectivity of individuals in this scheme consists in their participation in the collective give-and-take of critical discussion and not in some special relation (of detachment, hardheadedness) they may bear to their observations."

Helen Longino (1990, p. 79): Science as Social Knowledge: https://press.princeton.edu/books/paperback/9780691020518/science-as-social-knowledge

#socialpsychology
#openscience
#philosophyofscience
#philsci
#philscidon

With thanks to @jdagg for letting me know, I wanted to say hi to #histodon folks - i research philosophy and history of science, especially history of organisms in biological research (humans and non-humans, with a soft spot for plants🪴) and history of data / data science - would love to meet other ppls interested in history of science here! Also: any chance of meeting other philosophers of science #philsci #philscidon ?