[Previously](https://social.sdf.org/@mediocreatbest/110961879786553484)

I was able to get my #TinyStories #LLM model trained. I let it run for around 6 or 8 hours, I believe, so I ran it on about half of all the data points I had.

My training loss looks normal to me (attached).

Given that the training loss stopped improving by much around 10k steps (it improved, but not drastically), then I don't plan to train the NNMF decomposed model as much.

I'm already seeing the training loss for this new model decreasing appropriately.

Another day, another Google #Colab experiment.

Today, I am going to try to train my own #TinyStories #LLM from scratch.

This is part of my ongoing attempt to apply non-negative matrix factorization to the weights of dense linear layers within the model.

I'm hoping to pretty closely reproduce the training of the original TinyStories model, but using my own training code. Then, once I can reproduce it normally, I'll try out the factorization approach and compare them.

Over the last week, I spent some more time on my idea to apply non-negative matrix factorization to the weight layers of the #TinyStories #LLM.

To summarize: my theory is that factorization can yield: 1) better inference throughput, 2) better insights into how the LLM works.

Non-Negative factorization is significant because you can work in a concept space that is only additive: a little of this yields a little of that; instead of a little of this minus some of this yields that.

...because the demons are trapped with you inside?

#ArtistsOnMastodon #acrylic #MastoArt #painting #tinyStories #neuhier

"What are you protecting"
...asked the voice.
"Yourself? From what?
Or something else from you?

#ToBeContinued #art #acrylic #ArtistsOnMastodon #FediArt #MastoArt #painting #tinyStories

An accident...
...became an experiment.
...became soul-food.
...became an idea.
...became the fantasy of a dream...
...to be loved again.

#art #acrylic #tinyStories #artist #mastoArt #ArtistsOnMastodon #FediArt

Hey guys,

I've been working on a wee creative project, where I expand my micro-fics written over the years into tiny stories and also, I make an image to accompany them.

Sometimes the stories came first, sometimes the images.

I'd love for you to check them out😊
https://reeimaginedworlds.com/snapshot-stories/

#Writing #AmWriting #Creativity #WritingCommunity #WritingInspiration #TinyStories #SnapshotStories

You scrawl her name. You choose to remember her as evil through and through. She changed you. Worse, you let her in. There were no chains. You might have run. See your only witness veiled in shadows. She could have, but she made choices too. Hers was not to rescue you. You stood still as death and she commanded your compliant silence. Then in slow motion she leaned in close and said, now I will #open your head. Reach in. Clear your mind.🪶

#MastoPrompt (open) #microfiction #tinystories #writing