Open WebUI#
All configurations presented here use docker-compose. Read the Docker instructions first.
Warning
This contents of this page have not been tested for all Open WebUI versions.
Open WebUI on Docker setup (CPU only)#
Variable name |
Description |
---|---|
|
the base URL where the Ollama Docker instance is listening on, usually |
Open WebUI on a server, Ollama on another#
This method is used to reduce the load on the main server CPU by delegating the AI work to another computer. Three steps are necessary:
configuring an SSH port forwarding between the main server and the secondary one. The main server connects via autossh, using an SSH key, to the secondary server
setting up Ollama on the secondary server
setting up Open WebUI on the main server
SSH setup#
See also
SSH port forwarding is one possible method used by the server running Open WebUI to communicate with the server running Ollama. The advantages of this are:
encrpyted communication
Ollama server can be placed on a remote, non-LAN host easily
install the dependencies
apt-get install autossh
create the SSH key
mkdir /root/.ssh chmod 700 .ssh cd .ssh ssh-keygen -t rsa -b 16384 -C "$(whoami)@$(uname -n)-$(date -I)"
Use
otherserver_root
when prompted for the file nameImportant
Do not set a passphrase when prompted since a Systemd script will need to interact with this key.
copy the local SSH pubkey to the remote server in
/root/.ssh/authorized_keys
update the
local SSH configuration
Match host otherserver user root IdentityFile=/root/.ssh/otherserver_root
add this section to the OpenSSH configuration of the secondary server
# [ ... ] rest of the file # Enable SSH port forwarding for port 11434 (Ollama). Match user root PasswordAuthentication no IPQoS throughput AllowTcpForwarding yes PermitTunnel no X11Forwarding no PermitOpen 127.0.0.1:11434
on the main server, test the SSH connection: you should not be prompted for a password
ssh root@otherserver
create a
Systemd unit file
[Unit] Description=AutoSSH service for otherserver After=network.target [Service] User=root Group=root Environment="AUTOSSH_GATETIME=0" ExecStart=/usr/bin/autossh -M 0 -N -L 0.0.0.0:11434:127.0.0.1:11434 -o TCPKeepAlive=yes root@otherserver [Install] WantedBy=multi-user.target
run the deploy script
Ollama setup#
See the Ollama page. This setup is to be done on the secondary server.
Open WebUI setup#
This setup is to be done on the main server.
See also
follow the Docker instructions
create the jobs directories. See reference
mkdir -p /home/jobs/scripts/by-user/root/docker cd /home/jobs/scripts/by-user/root/docker
clone the repository
git clone https://github.com/open-webui/open-webui.git openwebui
create a
Docker compose file
version: '3' services: open-webui: build: context: . args: OLLAMA_BASE_URL: '/ollama' dockerfile: Dockerfile image: ghcr.io/open-webui/open-webui:main container_name: open-webui volumes: - ./open-webui:/app/backend/data ports: - 4018:8080 environment: - 'OLLAMA_BASE_URL=${OLLAMA_BASE_URL}' - 'OLLAMA_API_BASE_URL=${OLLAMA_BASE_URL}/api' - 'OLLAMA_ORIGINS=*' # This value needs to remain empty. - 'WEBUI_SECRET_KEY=' # See # https://github.com/open-webui/open-webui/blob/main/TROUBLESHOOTING.md#error-on-slow-responses-for-ollama - 'AIOHTTP_CLIENT_TIMEOUT=900' - 'SCARF_NO_ANALYTICS=true' - 'DO_NOT_TRACK=true' - 'ANONYMIZED_TELEMETRY=false' extra_hosts: - host.docker.internal:host-gateway healthcheck: test: ["CMD", "curl", "-f", "${OLLAMA_BASE_URL}"] interval: 1m30s timeout: 10s retries: 2 restart: unless-stopped volumes: open-webui: {}
Note
Replace these variables with the appropriate values
OLLAMA_BASE_URL
: since we are using SSH port forwarding, the address used must be the same as theinet
orinet6
address you get from$ ip a
on the main ethernet interface. For example, if your address is192.168.0.2
,OLLAMA_BASE_URL
might behttp://192.168.0.2:11434
remove the original docker-compose file from the repository
rm docker-compose.yaml
build the Docker image
docker-compose build
create a
Systemd unit file
. See also the Docker compose services section[Unit] Requires=docker.service Requires=network-online.target After=docker.service After=network-online.target [Service] Type=simple WorkingDirectory=/home/jobs/scripts/by-user/root/docker/openwebui ExecStart=/usr/bin/docker-compose up --remove-orphans ExecStop=/usr/bin/docker-compose down --remove-orphans Restart=always [Install] WantedBy=multi-user.target
run the deploy script
modify the reverse proxy port of your web server configuration with
4018
Footnotes