Proxmox Virtual Environment (VE) 8.2.4 – how to use your first local ‚Meta Llama 3‘ Large Language Model (LLM) project without the need for a GPU and now with Open WebUI


## Docker Key & Repo hinzufügen ##
apt-get update
apt-get install ca-certificates curl
install -m 0755 -d /etc/apt/keyrings
curl -fsSL https://download.docker.com/linux/ubuntu/gpg -o /etc/apt/keyrings/docker.asc
chmod a+r /etc/apt/keyrings/docker.asc
echo \
„deb [arch=$(dpkg –print-architecture) signed-by=/etc/apt/keyrings/docker.asc] https://download.docker.com/linux/ubuntu \
$(. /etc/os-release && echo „$VERSION_CODENAME“) stable“ | \
tee /etc/apt/sources.list.d/docker.list > /dev/null
apt-get update

Hit:1 http://archive.ubuntu.com/ubuntu noble InRelease
Hit:2 http://archive.ubuntu.com/ubuntu noble-updates InRelease
Hit:3 http://archive.ubuntu.com/ubuntu noble-security InRelease
Reading package lists… Done
Reading package lists… Done
Building dependency tree… Done
Reading state information… Done
ca-certificates is already the newest version (20240203).
curl is already the newest version (8.5.0-2ubuntu10.1).
0 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.
Hit:1 http://archive.ubuntu.com/ubuntu noble InRelease
Hit:2 http://archive.ubuntu.com/ubuntu noble-updates InRelease
Hit:3 http://archive.ubuntu.com/ubuntu noble-security InRelease
Get:4 https://download.docker.com/linux/ubuntu noble InRelease [48.8 kB]
Get:5 https://download.docker.com/linux/ubuntu noble/stable amd64 Packages [9522 B]
Fetched 58.4 kB in 1s (111 kB/s)
Reading package lists… Done
## Docker Installation ##
root@pve-ai-llm-01:~# apt-get install docker-ce docker-ce-cli containerd.io docker-buildx-plugin docker-compose-plugin
## Open WebUI Container erstellen ##
root@pve-ai-llm-01:~# docker run -d –network=host -v open-webui:/app/backend/data -e OLLAMA_BASE_URL=http://127.0.0.1:11434 –name open-webui –restart always ghcr.io/open-webui/open-webui:main
root@pve-ai-llm-01:~# netstat -tulpn | grep LISTEN
tcp 0 0 127.0.0.1:11434 0.0.0.0:* LISTEN 282/ollama
tcp 0 0 0.0.0.0:8080 0.0.0.0:* LISTEN 661/python3
root@pve-ai-llm-01:~#
## Open WebUI aufrufen ##
http://pve-ai-llm-01:8080

Leave a Reply

You must be logged in to post a comment.