How to Install a Fully Local Private GPT on Ubuntu 20.04

In this article I will show how to install a fully local version of the PrivateGPT on Ubuntu 20.04 machine.

PrivateGPT is a project developed by Iván Martínez, which allows you to run your own GPT model trained on your data, local files, documents and etc. It is free and can run without internet access in local setup mode. Project is open-sourced and available at https://github.com/imartinez/privateGPT

My local setup includes the following components of the PrivateGPT:

• Vector store: Qdrant

• Embeddings: Ollama

• LLM: Ollama

Prerequisites:

Before installation ensure that you have satisfied these prerequisites:

1. Python version 3.11. To install and set the 3.11 version you can use pyenv tool:

   💡

   To install pyenv on your machine checkout this article: How to install 'pyenv' Python version manager on Ubuntu 20.04

    pyenv install 3.11
    pyenv global 3.11
   

2. For dependency management install Poetry:

    curl -sSL https://install.python-poetry.org | python3 -
   

3. Ensure that you have make tool installed:

    make --version
   

Install PrivateGPT:

1. Clone PrivateGPT repository, and navigate to it:

    git clone https://github.com/imartinez/privateGPT
    cd privateGPT
   

2. Install Ollama for LLM and Embeddings:

    curl -fsSL https://ollama.com/install.sh | sh
   

3. Ensure that your ollama service is running:

    ollama serve
   

    sudo systemctl status ollama.service
   

4. Now in a separate terminal install the default models and settings from settings-ollama.yaml file:

    sudo ollama pull mistral
    sudo ollama pull nomic-embed-text
   

   If you want to run ollama without sudo, then add your user account to the render group:

    sudo usermod -aG render <your_username>
   

   after adding the group log out and log back in to make the changes take effect.

   To list the groups your user belongs to use:

    groups
   

5. Install PrivateGPT with the following command:

    poetry install --extras "ui llms-ollama embeddings-ollama vector-stores-qdrant"
   

6. Once installed, you can run PrivateGPT. Before that ensure that you have a working Ollama running locally.

    PGPT_PROFILES=ollama make run
   

Now you can access your PrivateGPT UI http://localhost:8001/

References:

1. Private GPT: Main Concepts

2. Private GPT: Installation

3. https://github.com/imartinez/privateGPT

4. Ollama

5. Poetry installation

6. Qdrant

7. YouTube: Run your own AI but private

8. https://t.me/devops_orbit