Installing Sherpa AI Server#

Before you begin#

You need a server with Docker and Docker Compose v2, enough space for archives, models and containers, and either access to the distribution or a complete set of files transferred in advance. For a local LLM, choose a GPU server or CPU-only installation. Prepare a domain name, IP address and SSL certificates. If the LLM runs on another computer, see connecting a separate LLM server.

Download the client-files archive for the published version you need and its checksum file. Verify SHA-256 using the distribution download guide. All Docker image archives must be the same version. The installer checks the version, checksums and image tags before starting containers. Do not select archives independently by latest and do not run load_all_docker_images.sh: that older helper selects the latest file for each image independently.

If the server cannot access the download server, transfer client-files, the required image archives and matching .SHA256SUM files into one directory. The download guide explains the required set and how to check available disk space.

Install using the script#

The example uses version 2.5.1. It is necessary to enter the name of your already verified archive and run these commands in the installation directory:

CLIENT_ARCHIVE=docker_aiserver_client_files_2.5.1.tgz
tar -xzf "$CLIENT_ARCHIVE"
chmod +x sh_scripts/*.sh
sudo ./sh_scripts/install.sh

The installer performs these steps:

1. Checks Docker, Docker Compose v2, the .env file and the APP_VERSION included in the package.
2. Offers GPU (vLLM) or CPU (llama.cpp) mode, optional Whisper and BGE Reranker services, and an available LLM model. Whisper and reranker require a GPU. If you choose several models, account for available video memory.
3. Finds the required Docker image archives for APP_VERSION. When connected, it downloads missing files, verifies SHA-256 and Docker manifest tags, loads the images and extracts the selected models. Without access, it reports which files are missing.
4. Backs up .env and prompts for HOST_IP, NGINX_DOMAIN_NAME, MAX_TOKENS_MESSAGE, the PostgreSQL password, model and other settings. Replace the default password with a strong, unique one. Do not keep passwords in unsecured notes. The installer restricts access to .env.
5. Offers HTTPS for the main server and the connection to the embedding server. Prepare trusted certificates and keys. Self-signed certificates can be used on a test server. See the SSL guide for details.
6. Shows the resulting settings and Docker Compose launch plan. It is necessary to check the mode, model, domain and profiles, then confirm launch.

The installer saves the selected profiles in COMPOSE_PROFILES. If installation is interrupted, resolve the reported cause and run install.sh again. Do not mix files from different versions to bypass a failed check.

Services and profiles#

Core services start without a separate profile:

Additional profiles select LLM and processing services:

When managing profiles yourself, specify only the services you selected. For example, GPU with Whisper:

docker compose --profile gpu --profile whisper up -d

For CPU without a GPU:

docker compose --profile cpu up -d

Check the installation#

It is necessary to check container status and logs. Running services should have status Up:

docker compose ps
docker compose logs --tail=100 aiserver

It is necessary to open https://<your-domain> in a browser and check that you can sign in. To check the web server response, replace the example domain with yours:

curl -I https://aiserver.example.com

Expect a successful HTTP response or a redirect to the sign-in page. It is necessary to check that the browser trusts the certificate without a warning. If you use a local LLM, send a test chat message and confirm that the model responds. If problems occur, inspect the relevant service logs:

docker compose logs --tail=100 aiserver-llm-server
docker compose logs --tail=100 aiserver-embed

For CPU mode, use aiserver-llm-server-cpu instead of aiserver-llm-server. If a container does not start, check disk space and logs. For sign-in or database errors, check .env. For HTTPS, check certificates and the domain name. For GPU, check the driver, GPU availability and VRAM.

Maintenance#

Run commands from the directory containing docker-compose.yml and .env:

docker compose ps
docker compose logs -f aiserver
docker compose restart aiserver
docker stats

For a GPU, also use nvidia-smi. docker compose down stops and removes containers; do not add -v if you want to retain data in Docker volumes. For updates, use the update guide. Before updating from 2.5.0 to 2.5.1, complete the required index preparation. If the current version is below 2.5.0, agree on an update path with support.