TurboPilot

Turbopilot is deprecated/archived as of 30/9/23. There are other mature solutions that meet the community's needs better. Please read my blog post about my decision to down tools and for recommended alternatives.
-----------------------------------
TurboPilot is a self-hosted copilot clone which uses the library behind llama.cpp to run the 6 Billion Parameter Salesforce Codegen model in 4GiB of RAM. It is heavily based and inspired by on the fauxpilot project.
***NB: This is a proof of concept right now rather than a stable tool. Autocompletion is quite slow in this version of the project. Feel free to play with it, but your mileage may vary.***
**✨ Now Supports StableCode 3B Instruct** simply use TheBloke's Quantized GGML models and set -m stablecode.
**✨ New: Refactored + Simplified**: The source code has been improved to make it easier to extend and add new models to Turbopilot. The system now supports multiple flavours of model
**✨ New: Wizardcoder, Starcoder, Santacoder support** - Turbopilot now supports state of the art local code completion models which provide more programming languages and "fill in the middle" support.
🤝 Contributing
PRs to this project and the corresponding GGML fork are very welcome.
Make a fork, make your changes and then open a PR.
👋 Getting Started
The easiest way to try the project out is to grab the pre-processed models and then run the server in docker.
Getting The Models
You have 2 options for getting the model
Option A: Direct Download - Easy, Quickstart
You can download the pre-converted, pre-quantized models from Huggingface.
For low RAM users (4-8 GiB), I recommend StableCode and for high power users (16+ GiB RAM, discrete GPU or apple silicon) I recomnmend WizardCoder.
Turbopilot still supports the first generation codegen models from v0.0.5 and earlier builds. Although old models do need to be requantized.
You can find a full catalogue of models in MODELS.md.
Option B: Convert The Models Yourself - Hard, More Flexible
Follow this guide if you want to experiment with quantizing the models yourself.
⚙️ Running TurboPilot Server
Download the latest binary and extract it to the root project folder. If a binary is not provided for your OS or you'd prefer to build it yourself follow the build instructions
Run:
The application should start a server on port 18080, you can change this with the -p option but this is the default port that vscode-fauxpilot tries to connect to so you probably want to leave this alone unless you are sure you know what you're doing.
If you have a multi-core system you can control how many CPUs are used with the -t option - for example, on my AMD Ryzen 5000 which has 6 cores/12 threads I use:
To run the legacy codegen models. Just change the model type flag -m to codegen instead.
**NOTE: Turbopilot 0.1.0 and newer re-quantize your codegen models old models from v0.0.5 and older. I am working on providing updated quantized codegen models**
📦 Running From Docker
You can also run Turbopilot from the pre-built docker image supplied here
You will still need to download the models separately, then you can run:
Docker and CUDA
As of release v0.0.5 turbocode now supports CUDA inference. In order to run the cuda-enabled container you will need to have nvidia-docker enabled, use the cuda tagged versions and pass in --gpus=all to docker with access to your GPU like so: