Thursday, August 31, 2023

LlamaGPT Installation on AWS - Step by Step Tutorial

 LlamaGPT is a self-hosted, offline, ChatGPT-like chatbot, powered by Llama 2. 100% private, with no data leaving your device.

It supported following models at the moment:

Model nameModel sizeModel download sizeMemory required
Nous Hermes Llama 2 7B Chat (GGML q4_0)7B3.79GB6.29GB
Nous Hermes Llama 2 13B Chat (GGML q4_0)13B7.32GB9.82GB
Nous Hermes Llama 2 70B Chat (GGML q4_0)70B38.87GB41.37GB
Code Llama 7B Chat (GGUF Q4_K_M)7B4.24GB6.74GB
Code Llama 13B Chat (GGUF Q4_K_M)13B8.06GB10.56GB
Phind Code Llama 34B Chat (GGUF Q4_K_M)34B20.22GB22.72GB

Commands Used:

git clone cd llama-gpt/ sudo chmod 666 /var/run/docker.sock ./ --model 7b Then access it in browser either using your IP or localhost: http://localhost:3000

No comments: