Recently I've gotten really excited about #Enzyme as the future of #autodiff in #JuliaLang, in particular because it supports more language features than #Zygote (e.g. mutation, fast control flow, and preserving structural sparsity). I've started getting acquainted with its rules system, and I have some first impressions by comparison to #ChainRules. 🧵

AutoBound computes polynomial upper and lower bounds on a given function, valid in a user-defined interval. The application of AutoBound to deep learning optimizers can be used to derive the meta-optimizer SafeNet which automatically adjust the learning rate (at the cost of an additional forward pass): https://ai.googleblog.com/2023/04/beyond-automatic-differentiation.html

#autobound #safenet #autodiff #jax #ai

#chatgpt explaining #autodiff in various ways. I am very impressed by this #llm, not just its knowledge, but also that it can understand subtle forms of humor in its way of explanations.

On Thursday I'll be at #NeurIPS2022 presenting a paper on our new system for #autodiff of implicit functions. A 🧵on the paper (https://arxiv.org/abs/2105.15183)

Next week I'll be at #NeurIPS2022 presenting a couple of papers. The first one is on #autodiff through #optimization (aka #unrolling) and its bizarre convergence properties. A 🧵 (1/9)

Next week I'll be at #NeurIPS2022 presenting a paper on #autodiff through #optimization (aka #unrolling) and its bizarre convergence properties

A 🧵 (1/9)

I know it is "old", butI think I just fell in love with automatic differentiation #autodiff

Huge potential (and force) to speed up stuff in #compchem. Also: it is the cute math that stole my ❤️ (as always)

source:
https://twitter.com/Michielstock/status/1593582416328892416?s=20&t=OQcTmRfNrgiT-4WBUNiiHg
(in dutch, #birdsite)

Need to dig in further, but #chemtwoops insights/thoughts/pointers are strongly appreciated!

Another #JuliaLang #autodiff banger

https://youtu.be/DfKKTE9XU0Q

Technical Q. Anyone know how to do recursive binary checkpointing ("treeverse") over a number of steps that isn't determined until runtime? E.g. for an adaptive ODE solver.

Classically, the number of steps is assumed to be known in advance, I think.

#autodiff
#machinelearning
#honestly_I_have_no_idea_what_hashtag_to_use_for_obscure_technical_questions

Technical Q. Anyone know how to do recursive binary checkpointing ("treeverse") over a number of steps that isn't determined until runtime? E.g. for an adaptive ODE solver.

Classically, the number of steps is assumed to be known in advance, I think.

#autodiff
#machinelearning
#honestly_I_have_no_idea_what_hashtag_to_use_for_obscure_technical_questions

I just migrated to bayes.club, so here is another (and hopefully the last for a while) introduction

I'm a #MachineLearning engineer with a focus on probabilistic programming at @unituebingen where I help scientists use ML for their research. In the office and out, one of my main passions is #FOSS, and I work on a number of #OpenSource packages, mostly in #JuliaLang :julia: with a focus on #ProbProg, #manifolds, and #AutoDiff.

#introduction