Open WebUI

Open WebUI#

All configurations presented here use docker-compose. Read the Docker instructions first.

Avvertimento

This contents of this page have not been tested for all Open WebUI versions.

Open WebUI on Docker setup (CPU only)#

Variable name

Description

OLLAMA_BASE_URL

the base URL where the Ollama Docker instance is listening on, usually http://${ADDR}:11434.

Open WebUI on a server, Ollama on another#

This method is used to reduce the load on the main server CPU by delegating the AI work to another computer. Three steps are necessary:

  1. configuring an SSH port forwarding between the main server and the secondary one. The main server connects via autossh, using an SSH key, to the secondary server

  2. setting up Ollama on the secondary server

  3. setting up Open WebUI on the main server

SSH setup#

Vedi anche

  • SSH keys - ArchWiki [1]

  • OpenSSH - ArchWiki - Forwarding other ports [2]

  • OpenSSH - ArchWiki - Run autossh automatically at boot via systemd [3]

SSH port forwarding is one possible method used by the server running Open WebUI to communicate with the server running Ollama. The advantages of this are:

  • encrpyted communication

  • Ollama server can be placed on a remote, non-LAN host easily

  1. install the dependencies

    apt-get install autossh

  2. create the SSH key

    mkdir /root/.ssh
chmod 700 .ssh
cd .ssh
ssh-keygen -t rsa -b 16384 -C "$(whoami)@$(uname -n)-$(date -I)"

    Use otherserver_root when prompted for the file name

    Importante

    Do not set a passphrase when prompted since a Systemd script will need to interact with this key.

  3. copy the local SSH pubkey to the remote server in /root/.ssh/authorized_keys

  4. update the local SSH configuration

    /root/.ssh/config#
    Match host otherserver user root
    IdentityFile=/root/.ssh/otherserver_root

  5. add this section to the OpenSSH configuration of the secondary server

    /etc/ssh/sshd_config#
    # [ ... ] rest of the file

# Enable SSH port forwarding for port 11434 (Ollama).
Match user root
    PasswordAuthentication no
    IPQoS throughput
    AllowTcpForwarding yes
    PermitTunnel no
    X11Forwarding no
    PermitOpen 127.0.0.1:11434

  6. on the main server, test the SSH connection: you should not be prompted for a password

    ssh root@otherserver

  7. create a Systemd unit file

    /home/jobs/services/by-user/root/autossh.otherserver.service#
    [Unit]
Description=AutoSSH service for otherserver
After=network.target

[Service]
User=root
Group=root
Environment="AUTOSSH_GATETIME=0"
ExecStart=/usr/bin/autossh -M 0 -N -L 0.0.0.0:11434:127.0.0.1:11434 -o TCPKeepAlive=yes root@otherserver

[Install]
WantedBy=multi-user.target

  8. run the deploy script

Ollama setup#

See the Ollama page. This setup is to be done on the secondary server.

Open WebUI setup#

This setup is to be done on the main server.

Vedi anche

  • Open WebUI [4]

  • GitHub - open-webui/open-webui: User-friendly WebUI for LLMs (Formerly Ollama WebUI) [5]

  • GitHub - open-webui/open-webui: User-friendly WebUI for LLMs (Formerly Ollama WebUI) - Error on Slow Responses for Ollama [6]

  1. follow the Docker instructions

  2. create the jobs directories. See reference

    mkdir -p /home/jobs/scripts/by-user/root/docker
cd /home/jobs/scripts/by-user/root/docker

  3. clone the repository

    git clone https://github.com/open-webui/open-webui.git openwebui

  4. create a Docker compose file

    /home/jobs/scripts/by-user/root/docker/openwebui/docker-compose.yml#
    version: '3'

services:
  open-webui:
    build:
      context: .
      args:
        OLLAMA_BASE_URL: '/ollama'
      dockerfile: Dockerfile
    image: ghcr.io/open-webui/open-webui:main
    container_name: open-webui
    volumes:
      - ./open-webui:/app/backend/data
    ports:
      - 4018:8080
    environment:
      - 'OLLAMA_BASE_URL=${OLLAMA_BASE_URL}'
      - 'OLLAMA_API_BASE_URL=${OLLAMA_BASE_URL}/api'
      - 'OLLAMA_ORIGINS=*'

      # This value needs to remain empty.
      - 'WEBUI_SECRET_KEY='

      # See
      # https://github.com/open-webui/open-webui/blob/main/TROUBLESHOOTING.md#error-on-slow-responses-for-ollama
      - 'AIOHTTP_CLIENT_TIMEOUT=900'

      - 'SCARF_NO_ANALYTICS=true'
      - 'DO_NOT_TRACK=true'
      - 'ANONYMIZED_TELEMETRY=false'
    extra_hosts:
      - host.docker.internal:host-gateway
    healthcheck:
      test: ["CMD", "curl", "-f", "${OLLAMA_BASE_URL}"]
      interval: 1m30s
      timeout: 10s
      retries: 2
    restart: unless-stopped

volumes:
  open-webui: {}

    Nota

    Replace these variables with the appropriate values

    • OLLAMA_BASE_URL: since we are using SSH port forwarding, the address used must be the same as the inet or inet6 address you get from $ ip a on the main ethernet interface. For example, if your address is 192.168.0.2, OLLAMA_BASE_URL might be http://192.168.0.2:11434

  5. remove the original docker-compose file from the repository

    rm docker-compose.yaml

  6. build the Docker image

    docker-compose build

  7. create a Systemd unit file. See also the Docker compose services section

    /home/jobs/services/by-user/root/docker-compose.openwebui.service#
    [Unit]
Requires=docker.service
Requires=network-online.target
After=docker.service
After=network-online.target

[Service]
Type=simple
WorkingDirectory=/home/jobs/scripts/by-user/root/docker/openwebui

ExecStart=/usr/bin/docker-compose up --remove-orphans
ExecStop=/usr/bin/docker-compose down --remove-orphans

Restart=always

[Install]
WantedBy=multi-user.target

  8. run the deploy script

  9. modify the reverse proxy port of your web server configuration with 4018

Footnotes

Contenuti