I made a first test of Llama 2 13B, in a 6-bits quantized version (thanks, #ggml )

It's good-to-excellent in various tasks: summarization, translation (I tried EN, IT, FR), NER with semantic filters.

AND it runs on a CPU-only installation on an Intel, at decent speed. 👏

https://huggingface.co/localmodels/Llama-2-13B-Chat-ggml

#llm #webui

Probably one of the best #ai model I tested so far. Really good for #roleplay, more logical than #wizard-vicuna. Can't wait for a 8K #context . :cat_typing:

#koboldcpp #ggml

https://huggingface.co/TheBloke/MythoLogic-13B-GGML

Just did a bunch of merges of upstream #ggml repo and managed to get the StarCoder and WizardCoder running in #turbopilot - there are definitely some opportunities to accelerate it to make it more useful.