A condensation of a few perspectives on the rising topic of diffusion models, focusing less on my usual areas of comfort (i.e. Markov processes and friends) and more on the other practical core, (i.e. denoising operators).

'Denoising-Centric Diffusions'
https://hackmd.io/@sp-monte-carlo/SJWAZBtr2

#spmathsblog

Various sampling algorithms can (implicitly or explicitly) involve some recursive structure, which enables certain tools for their theoretical analysis. Here is a small write-up of some basic thoughts on the topic.

https://hackmd.io/@sp-monte-carlo/BkHYt7tBn
'Nested Structure in MCMC Algorithms'

#spmathsblog

A proper write-up of the ramblings above, for easier reference: (I can't remember how much, if at all, I edited things)

https://hackmd.io/@sp-monte-carlo/ByR1IR3Fi

#spmathsblog

Adding some details to an earlier post (https://mathstodon.xyz/@sp_monte_carlo/109542712262310989): deriving and describing some near-Gaussian distributions which are relatively { explicit, tractable } in some (mildly) useful ways:

'Isoperimetric Surrogates for the Gaussian'

https://hackmd.io/@sp-monte-carlo/HJX65vjKo

#spmathsblog

Taking stock of the past week (with apologies for the dense formatting).

'#BayesComp2023 Recap'

https://hackmd.io/@sp-monte-carlo/SJVL1tvln

#spmathsblog

Earlier this year, I learned one of my current favourite 'concise results', i.e. results where you can concisely communicate the substance of the result and why it is true. In this post, I offer a snappy presentation to this effect.

'Univariate Log-Concave Rejection Sampling is Solved'

https://hackmd.io/@sp-monte-carlo/HktN0PjYo

#spmathsblog

So-called 'implicit' statistical models raise interesting opportunities for their ability to accommodate complex mechanistic data-generating processes, while posing a number of computational challenges. Many approaches to inference in these settings involves the systematic formation and refinement of approximations to both the likelihood and posterior. In this post, I outline some abstract descriptions of this approach.

'Iterative Likelihood Approximation'

https://hackmd.io/@sp-monte-carlo/BJ5F4C2Fo

#spmathsblog

One of the few things which all Markov chains have in common with one another is the use of a time variable. As such, one can probe the structure of general Markov chains by examining how they look both forwards and backwards in time. This post collects some very brief thoughts on the matter.

'Forwards, Backwards, and Stochastic Formulations of Optimal Transport'

https://hackmd.io/@sp-monte-carlo/Sy45m0nFo

#spmathsblog

Control variates are a useful tool for variance reduction in various Monte Carlo contexts. However, depending on the context, the methodology for fitting control variates ought to change slightly; I describe some of the changes here.

'Fitting Control Variates in MCMC'

https://hackmd.io/@sp-monte-carlo/BylA8u0nFi

#spmathsblog

There are various different streams of Monte Carlo methods, and it is sometimes tempting to frame them as being in competition with one another (RS vs IS, MCMC vs SMC, etc.). In this short post (prompted by a relatively old tweet of mine, from which my views have since shifted slightly), I argue against this framing, and advocate for celebrating the synergies and similarities between the sub-fields.

'Competition between Monte Carlo Methods'

https://hackmd.io/@sp-monte-carlo/rk5VjR3Kj

#spmathsblog

In analysis, one often wants to compare two quantities which arise as the outcome of time-varying processes, perhaps different ones. There are some nice tools which allow for these comparisons to be made, particularly for processes with good stability properties. The Alekseev-Groebner decomposition is one such tool.

'Decompositions à la Alekseev-Groebner'

https://hackmd.io/@sp-monte-carlo/BkwugRnti

#spmathsblog

Two recurrent difficult scenarios in probabilistic computations are dealing with multimodality and heavy-tailedness. Multimodality is difficult, but comes with some reasonable 'default' solutions, whereas heavy tails are perhaps less damaging, but require more care to resolve. In this blog post, I expand upon this distinction somewhat.

'Computation of Heavy-Tailed Measures'

https://hackmd.io/@sp-monte-carlo/SkZQOPoFi

#spmathsblog

It is well-understood that many sampling algorithms can mix more rapidly when some of the components of the target measure are marginalised out. In this blog post, I provide a quick proof of this ordering in a specific setting.

'On the Benefits of Marginalisation for Langevin Diffusions'

https://hackmd.io/@sp-monte-carlo/SygEqAnFo

#spmathsblog

A short note on a technique for remembering how to compute the Schur complement of a PSD matrix, which can arise in computing conditional laws of Gaussian random variables (which I think was the motivation for writing this).

'A Mnemonic for Schur Complements'

https://hackmd.io/@sp-monte-carlo/SymBw03Ki

#spmathsblog

Exponential families are often held up as being a very nice collection of probability distributions with which to work, particularly in a statistical context. This characterisation is very frequently accurate in practice, but there can be more to the story than just the presence of exponential families:

'Convenient Exponential Families'

https://hackmd.io/@sp-monte-carlo/B1VToR3Fj

#spmathsblog

Many tasks require demonstrating that a system is eventually exponentially convergent to an equilibrium state. In this post, I describe how statements of this form can be systematically improved to show uniform exponential convergence to equilibrium for a suitable related function.

'Contractivity from Eventual Exponential Convergence'

https://hackmd.io/@sp-monte-carlo/BkHqFPiFi

#spmathsblog

Some notes on the challenge of diagnosing convergence for MCMC algorithms, and how much of the algorithms' structure should be brought into this task.

'Convergence Diagnostics for MCMC'

https://hackmd.io/@sp-monte-carlo/H1jQEPoFi

#spmathsblog

A fun puzzle based on the idea of replacing the squared displacement loss in optimal transport with a Bregman divergence, and how this modification impacts the structure of optimal couplings:

'Bregman Cost Functions in Optimal Transport'

https://hackmd.io/@sp-monte-carlo/ry5L8woFj

#spmathsblog

Cobbling together some thoughts about transport maps, optimal and not:

'Non-Optimal Transport Maps'

https://hackmd.io/@sp-monte-carlo/BkybBPoFj

#spmathsblog

Some notes on some more 'quiet assumptions' in MCMC, this time with a focus on Gibbs sampling:

'Scans in Gibbs Sampling'

https://hackmd.io/@sp-monte-carlo/HkjK1PiKo

#spmathsblog