and it kind of saddens me that genuinely excellent thinkers like Neel Nanda are putting their writeups on #interpretability on LW, which ranges from noobish to terrible.

#machinelearning #ai #mechanisticinterpretability

Unlock the Black Box by Interpreting Graph Convolutional Networks via Additive Decomposition

https://openreview.net/forum?id=sroF8hhbzW

#gnn #subgraph #interpretability

TabCBM: Concept-based Interpretable Neural Networks for Tabular Data

Mateo Espinosa Zarlenga, Zohreh Shams, Michael Edward Nelson, Been Kim, Mateja Jamnik

Action editor: Pin-Yu Chen.

https://openreview.net/forum?id=TIsrnWpjQ0

#concepts #explanations #interpretability

🔍 Process supervision takes us closer to understanding the black box of AI. By rewarding each step, we unravel the model's decision-making process. It's a step forward in transparency and interpretability. Exciting times ahead! #ProcessSupervision #Transparency #Interpretability #AIInsights 🧠🔓🔍

Interpretable Mixture of Experts

Aya Abdelsalam Ismail, Sercan O Arik, Jinsung Yoon, Ankur Taly, Soheil Feizi, Tomas Pfister

Action editor: Frederic Sala.

https://openreview.net/forum?id=DdZoPUPm0a

#interpretability #interpretable #experts

Peering Inside #AI’s Black Boxes https://www.quantamagazine.org/cynthia-rudin-builds-ai-that-humans-can-understand-20230427/
#interpretability #explainability #aiEthics

TabCBM: Concept-based Interpretable Neural Networks for Tabular Data

https://openreview.net/forum?id=TIsrnWpjQ0

#concepts #interpretability #explanations

Interpretable Mixture of Experts

https://openreview.net/forum?id=DdZoPUPm0a

#interpretability #interpretable #experts

#Computerphile - #Glitch #Tokens In #LargeLanguageModels

#RobMiles talks about '#GlitchTokens', those mysterious words, which result in gibberish when entered into some large #LanguageModels.

https://www.youtube.com/watch?v=WO2X3oZEJOA&ab_channel=Computerphile

#GPT #ChatGPT #Language #Interpretability #OpenAI #AI #ArtificialIntelligence #Representation #Representations

The TAYSIR competition about extracting small, interpretable models from neural language models will also be hosted at ICGI!

https://remieyraud.github.io/TAYSIR/

The first CFP is available: https://remieyraud.github.io/TAYSIR/TAYSIR-Call_for_participation_1.pdf

#NLP #NLProc #CFP #Interpretability #automata #LanguageModels

There are a four invited speakers, but I am only personally familiar with three (Cyril Allauzen @ Google, Will Merrill @ NYU/Google, and Dana Angluin @ Yale).

These three talks should be fantastic, especially if you are interested in #automata, #FormalLanguages, and #Interpretability in neural language models!

(Plugging https://flann.super.site/ if those sound cool to you)

OSS community doing good work on #interpretability in the quest for realistic eyes and hands in diffusion models— they’ve traced the problem back to the encoder needing better training

https://stable-diffusion-art.com/how-to-use-vae/

My new article in @towardsdatascience

IML can increase/improve:
- Accuracy
- Performance in production
- Trust
- The reach of ML
- Storytelling
- Knowledge

Would you add anything?

#DataScience #MachineLeaning #IML #XAI #Interpretability #Explainability

No paywall link:
https://towardsdatascience.com/the-6-benefits-of-interpretable-machine-learning-e32fb8b60e9?sk=109f1fb39c5f18082c14105785b8e2af

An interesting upcoming challenge on extracting simple and interpretable models (e.g., finite-state machines, random forests, etc.) from already-trained neural language models:

https://remieyraud.github.io/TAYSIR/

Some potential background reading (please suggest more!):

- https://arxiv.org/abs/1711.09576
- https://direct.mit.edu/coli/article/47/2/221/98517/Approximating-Probabilistic-Models-as-Weighted
- https://aclanthology.org/W19-3112/
- https://link.springer.com/article/10.1007/s10994-021-05948-1
- https://link.springer.com/article/10.1007/s10994-013-5409-9

#ML #automata #interpretability

RT @oacore
Here it is. @PLOSBiology describes 10 straightforward writing #tips and a #webtool guiding authors to help address the most common cases that remain difficult for #textmining tools https://journals.plos.org/plosbiology/article?id=10.1371%2Fjournal.pbio.3000716&utm_source=feedburner&utm_medium=email&utm_campaign=Feed%3A+OATP-Primary+%28OATP+primary%29.
#openaccess #openscience #discoverability #interpretability

Our #WACV23 paper on the topic of evaluation for model interpretation methods is available online https://openaccess.thecvf.com/content/WACV2023/html/Behzadi-Khormouji_A_Protocol_for_Evaluating_Model_Interpretation_Methods_From_Visual_Explanations_WACV_2023_paper.html

Congrats to @behzadikhormuji for the good work.
#XAI #Interpretability #interpretableml #UAntwerp #imec

Working on a new article "Debugging a PyTorch Image Model with SHAP"

A mini-car takes images of a track as input. Using these, a model is used to make predictions which then turn the car.

Using SHAP, we can see the model uses background pixels when making predictions. This causes problems when we move the car to a new location.

Should be out next week :)

#DataScience #MachineLeaning #SHAP #IML #XAI #Interpretability

We present *context length probing*, an embarrassingly simple, model-agnostic, #blackbox explanation technique for causal (#GPT-like) language models.

The idea is simply to check how predictions change as the left-hand context is extended token by token. This allows assigning "differential importance scores" to different contexts as shown in the demo.

Paper: https://arxiv.org/abs/2212.14815
Demo: https://cifkao.github.io/context-probing/

#explainability #interpretability #Transformer #NLProc

🧵1/4

What do LLMs learn over the course of training? How do these patterns change as you scale? To help answer these questions, we are releasing a Pythia, suite of LLMs + checkpoints designed for research on interpretability and training dynamics!

The models have sizes ranging from 19M to 13B parameters, contain 143 intermediate checkpoints, and were trained on the same exact data in the same exact order.

#ml #ai #nlproc #interpretability #EleutherAI

https://github.com/EleutherAI/pythia

Heard I should write an #introduction? Ok.

Hi! I'm Jenn. I do research on #responsibleAI at Microsoft Research NYC. I'm in the FATE group and co-chair Microsoft's Aether working group on transparency.

My research background is in #machinelearning theory and algorithmic econ, but since my mid-career crisis I've focused on human-centered approaches to #transparency, #interpretability, & #fairness of AI.

I'm into #AI that augments, rather than replaces, human abilities.

http://www.jennwv.com