An update on the support of #Markdown for #Mastodon toots: things aren't as easy as I thought.

I initially thought of simply slapping #commonmarker in the HtmlAwareFormatter, but that's just not feasible. The HtmlAwareFormatter already gets an HTML text that has already been rendered by the source instance and pushed to the feed. That means that all the rendered HTML will be scraped when used in SAFE mode. The alternative is to use commonmarker in UNSAFE mode. That worked in some tests I ran yesterday, but that's, of course, very unsafe - anybody could inject some <script> tags into a toot.

I could of course add my own Markdown parser after the HTML has been processed, but that would look quite ugly, as it would have to figure out the source post's Markdown elements among a mess of HTML tags.

So the cleanest option would be to add it to the TextFormatter, so the instance would render the HTML directly from the source text. But that's not as immediate as it sounds like. There's already a quite messed process of entity extraction from the source text in the TextFormatter (e.g. to render mentions and anchors), and the Markdown extraction and rendering process will have to somehow tap into the existing entity extraction logic - for example, Markdown links should be extracted before the standard links are extracted and rendered, but rendered after the "normal" links have been rendered, or it'll mess up the overall rendered HTML.

Of course an option would be to just use the glitch-soc fork, but that fork has diverged quite a lot from the vanilla Mastodon codebase (it's almost 4000 commits ahead), migration may be tricky, and I would also have to migrate all the mods that I have already applied to my instance. So I'd rather patch the original codebase, so anyone running a vanilla instance can easily import my PR (and also the chances of it being merged upstream are higher).

But there's quite a lot of work involved to patch the current TextFormatter, and my #Ruby skills aren't that fresh either. So if any volunteers want to help, I'm currently working on this fork https://github.com/BlackLight/mastodon/tree/v3.5.2-mine.

Embracing CommonMark as the One True Markdown
https://sandfox.me/2016/04/05/embracing-commonmark.html
#rubygems #ruby #commonmark #commonmarker #rouge #commonmarkerrouge #myprojects