Bridging Graph Position Encodings for Transformers with Weighted Graph-Walking Automata

Patrick Soga, David Chiang

https://openreview.net/forum?id=tE2NiMGd07

#graphs #automata #graph

One more CFP:

Finite-State Methods and Natural Language Processing (FSMNLP)

https://fsmnlp2023.github.io/

This is a venue for results on automata (and similar formalisms) that have a clear relation to NLP. Held this year in Stellenbosch, South Africa after skipping a cycle due to COVID.

Late May deadline, conference is August 28-30.

#nlproc #nlp #automata #cfp

Bridging Graph Position Encodings for Transformers with Weighted Graph-Walking Automata

https://openreview.net/forum?id=tE2NiMGd07

#graphs #automata #graph

The TAYSIR competition about extracting small, interpretable models from neural language models will also be hosted at ICGI!

https://remieyraud.github.io/TAYSIR/

The first CFP is available: https://remieyraud.github.io/TAYSIR/TAYSIR-Call_for_participation_1.pdf

#NLP #NLProc #CFP #Interpretability #automata #LanguageModels

There are a four invited speakers, but I am only personally familiar with three (Cyril Allauzen @ Google, Will Merrill @ NYU/Google, and Dana Angluin @ Yale).

These three talks should be fantastic, especially if you are interested in #automata, #FormalLanguages, and #Interpretability in neural language models!

(Plugging https://flann.super.site/ if those sound cool to you)

An interesting upcoming challenge on extracting simple and interpretable models (e.g., finite-state machines, random forests, etc.) from already-trained neural language models:

https://remieyraud.github.io/TAYSIR/

Some potential background reading (please suggest more!):

- https://arxiv.org/abs/1711.09576
- https://direct.mit.edu/coli/article/47/2/221/98517/Approximating-Probabilistic-Models-as-Weighted
- https://aclanthology.org/W19-3112/
- https://link.springer.com/article/10.1007/s10994-021-05948-1
- https://link.springer.com/article/10.1007/s10994-013-5409-9

#ML #automata #interpretability