Just published V0.2.0 of #turbopilot - now with full GPU offload support and interoperable with the Huggingface VSCode plugin https://github.com/ravenscroftj/turbopilot/releases/tag/v0.2.0 #copilot

New major release of #turbopilot today - v0.1.0

- Refactor of the codebase into a single unified binary that supports different model types (codegen, wizardcoder, starcoder, santacoder)
- Support for CUDA 11 and 12
- Improved simplified API, should make it easy to add new types of models in future.

https://github.com/ravenscroftj/turbopilot

#CodeCompletion #copilot

Just did a bunch of merges of upstream #ggml repo and managed to get the StarCoder and WizardCoder running in #turbopilot - there are definitely some opportunities to accelerate it to make it more useful.

Turbopilot v0.0.5 is here - CUDA support was added which means it is much faster at responding to long prompts and now windows builds are provided out of the box too! https://github.com/ravenscroftj/turbopilot/releases/tag/v0.0.5 #turbopilot #copilot #ai

More turbopilot experiments today - the new nvidia/cublas support added to the upstream repo has meant that suggestions can be generated a lot faster on long prompts. It's still quite slow to actually generate a reply due to all the byte shuffling but I've got some ideas for possible solutions #turbopilot #copilot #llm #autocomplete #nvidia https://github.com/ravenscroftj/turbopilot

I've done a bit more work on #turbopilot this weekend - v0.0.4 includes a universal MacOS binary with ARM support out of the box and I've also created a fork of vscode-fauxpilot with a status indicator while the model is "thinking". #copilot #selfhosted https://github.com/ravenscroftj/turbopilot/releases/tag/v0.0.4