I was confused about #KolmogorovComplexity, the shortest computer program one can write to generate, say, a given sequence of bytes.

The "length of a computer program" is a dumb-looking metric. Devs know what a "code smell" is. The "length of a computer program" has a "Measurements & Standards smell."

So I thought - what would you use instead? How do YOU argue that A="49388531058673284058" is any more complex than B="33333333333333333333"?

My first go-to: B can be compressed to a much shorter string than B.

Then I think, through what compression algorithm?

"Well", I respond: "whatever, pkzip; who cares?!" The length of the algorithm is not part of the answer; the answer is 'the length of the string when it is compressed using the optimal algorithm'. And confidently I announce, "It is super-dumb to make the length of the algorithm part of the metric."

"Aha," I respond, "in that case I use an algorithm where "49388531058673284058" compresses to "1". Want me to code that for you?"

So I lose this round to myself.

But still, there's a stink in this KC metric. Really what you want to know is something closer to "what is the length of the sequence when it is compressed using the optimal algorithm, but NO CHEATING".

Perhaps it should be flipped around: what's the maximum information you can store in the sequence. But then I might claim that A="493885310586732840583024583823" happens to correspond to the the Book of Leviticus. So again, when the length of my algorithm is removed from the metric, I can cheat.

And yet.

I just don't think the length of a computer program ought to be running around town, calling itself a measurement in and of itself.

So I am confused about #KolmogorovComplexity.

@ldebowsk Adding variants of the #HashTags ...

#MutualInformation
#InformationTheory
#StochasticProcess
#ExcessEntropy
#KolmogorovComplexity
#AlgorithmicRandomness
#LargeLanguageModel
#NeuralNetwork
#PowerLaw
#UniversalCoding

(Just to help discoverability)

Kolmogorov Twitter

Twitter is what happens when interactions, low-latency, amplification, and images (memes are pretty much precisely cartoons of complex ideas) are cheap, while nuance, deliberation, and complexity are expensive. It drives all the intelligence from the discussion, absent very deliberate effort.

It’s also all plaza and no warrens. ...

https://joindiaspora.com/posts/0b3481508d2e0139a746002590d8e506

#Twitter #Nuance #KolmogorovComplexity #VlogBrothers #WarrensAndPlazas #CodeAndOtherLaws #LawrenceLessig #Alamogordo