LlamaGPT Installation on AWS - Step by Step Tutorial

 LlamaGPT is a self-hosted, offline, ChatGPT-like chatbot, powered by Llama 2. 100% private, with no data leaving your device.

It supported following models at the moment:

Model name	Model size	Model download size	Memory required
Nous Hermes Llama 2 7B Chat (GGML q4_0)	7B	3.79GB	6.29GB
Nous Hermes Llama 2 13B Chat (GGML q4_0)	13B	7.32GB	9.82GB
Nous Hermes Llama 2 70B Chat (GGML q4_0)	70B	38.87GB	41.37GB
Code Llama 7B Chat (GGUF Q4_K_M)	7B	4.24GB	6.74GB
Code Llama 13B Chat (GGUF Q4_K_M)	13B	8.06GB	10.56GB
Phind Code Llama 34B Chat (GGUF Q4_K_M)	34B	20.22GB	22.72GB

Commands Used:

git clone https://github.com/getumbrel/llama-gpt.git cd llama-gpt/ sudo chmod 666 /var/run/docker.sock ./run.sh --model 7b Then access it in browser either using your IP or localhost: http://localhost:3000