Text Compression Gets Weirdly Efficient With LLMs - It used to be that memory and storage space were so precious and so limited of a r... - https://hackaday.com/2023/08/27/text-compression-gets-weirdly-efficient-with-llms/ #artificialintelligence #losslesscompression #lossycompression #textcompression #softwarehacks #neuralnetwork #compression #winzip #llm

We then investigated two #LossyCompression strategies: bit truncation and WavPack Hybrid mode. Lossy compression can dramatically boost compression performance, but we must first assess how it affects downstream analysis (i.e., spike sorting).

(5/n)

Via @metafilter: Ted Chiang on #ChatGPT .

"Think of ChatGPT as a blurry JPEG of all the text on the Web. It retains much of the information on the Web, in the same way that a JPEG retains much of the information of a higher-resolution image, but, if you’re looking for an exact sequence of bits, you won’t find it; all you will ever get is an approximation.

But, because the approximation is presented in the form of grammatical text, which ChatGPT excels at creating, it’s usually acceptable. You’re still looking at a blurry JPEG, but the blurriness occurs in a way that doesn’t make the picture as a whole look less sharp."

#ArtificialIntelligence #AI #LossyCompression

https://web.archive.org/web/20230209152247/https://www.newyorker.com/tech/annals-of-technology/chatgpt-is-a-blurry-jpeg-of-the-web

We use the Fast Fourier Transform (#FFT) #algorithm to do #LossyCompression for things like images (e.g. jpeg). Most of the information is dropped and only the most important information is retained, which when reversed provides a noisy but recognisable version of the original image.

If this happens with a simple matrix of multi-dimensional vectors (e.g. a bitmap) could it not also be done with word embeddings like #word2vec to perform lossy compression on text? Is this a thing?