Installing Sherpa AI Server#

Before you begin#

You need a server with Docker and Docker Compose v2, enough space for archives, models and containers, and either access to the distribution or a complete set of files transferred in advance. For a local LLM, choose a GPU server or CPU-only installation. Prepare a domain name, IP address and SSL certificates. If the LLM runs on another computer, see connecting a separate LLM server.

Download the client-files archive for the published version you need and its checksum file. Verify SHA-256 using the distribution download guide. All Docker image archives must be the same version. The installer checks the version, checksums and image tags before starting containers. Do not select archives independently by latest and do not run load_all_docker_images.sh: that older helper selects the latest file for each image independently.

If the server cannot access the download server, transfer client-files, the required image archives and matching .SHA256SUM files into one directory. The download guide explains the required set and how to check available disk space.

Install using the script#

The example uses version 2.5.1. It is necessary to enter the name of your already verified archive and run these commands in the installation directory:

CLIENT_ARCHIVE=docker_aiserver_client_files_2.5.1.tgz
tar -xzf "$CLIENT_ARCHIVE"
chmod +x sh_scripts/*.sh
sudo ./sh_scripts/install.sh

The installer performs these steps:

1. Checks Docker, Docker Compose v2, the .env file and the APP_VERSION included in the package.
2. Offers GPU (vLLM), CPU (llama.cpp) or External LLM and Embedding mode, optional Whisper and BGE Reranker services, and an available LLM model. Whisper and reranker require a GPU. If you choose several models, account for available video memory.
3. Finds the required Docker image archives for APP_VERSION. When connected, it downloads missing files, verifies SHA-256 and Docker manifest tags, loads the images and extracts the selected models. Without access, it reports which files are missing.
4. Backs up .env and prompts for HOST_IP, NGINX_DOMAIN_NAME, MAX_TOKENS_MESSAGE, the PostgreSQL password, model and other settings. Replace the default password with a strong, unique one. Do not keep passwords in unsecured notes. The installer restricts access to .env.
5. Offers HTTPS for the main server and the connection to the embedding server. Prepare trusted certificates and keys. Self-signed certificates can be used on a test server. See the SSL guide for details.
6. Shows the resulting settings and Docker Compose launch plan. It is necessary to check the mode, model, domain and profiles, then confirm launch.

The installer saves the selected profiles in COMPOSE_PROFILES. If installation is interrupted, resolve the reported cause and run install.sh again. Do not mix files from different versions to bypass a failed check.

External LLM and Embedding#

Use this option when both services already run on another server. The main server needs network access to their APIs. The LLM must provide a compatible API, and Embedding must provide the Sherpa Embedding API. Prepare trusted certificates for HTTPS. If Embedding requires a token, obtain it from the service administrator before installation.

1. Run sudo ./sh_scripts/install.sh from the installation directory and choose [3] External LLM and Embedding.
2. Enter the LLM address, for example http://llm.example.local:8000, the Embedding address, for example https://embed.example.local:443, and the model name returned by the LLM API. Enter the Embedding token if authentication is required. Do not include /v1 in either address.
3. At the archive step, provide only the images requested by the installer. Local LLM and Embedding images and model archives are unnecessary in this mode.
4. Check the summary and the service list. .env must contain COMPOSE_PROFILES=external. The command docker compose --profile external config --services must not list aiserver-embed, aiserver-llm-server or aiserver-llm-server-cpu.
5. After startup, check docker compose ps, open Chat and send a test request. Check document search if you use it. A response and search results confirm connectivity to the external services.

The installer saves the URLs, hosts, ports, LLM scheme and model name in .env. If the LLM address changes, update its URL, host, port and scheme in .env. Then open the corresponding instance under Models and update Host, Port and Scheme: Chat uses the saved instance settings. If the Embedding address or token changes, update .env as well. After editing .env, run docker compose up -d --force-recreate aiserver: a regular restart does not reload environment variables. If requests fail, check network access from the main server and the aiserver logs. The updater keeps external mode and does not require local images or models for these services.

Services and profiles#

Core services start without a separate profile:

Additional profiles select LLM and processing services:

When managing profiles yourself, specify only the services you selected. For example, GPU with Whisper:

docker compose --profile embed --profile gpu --profile whisper up -d

For CPU without a GPU:

docker compose --profile embed --profile cpu up -d

Check the installation#

An administrator needs Docker Compose access on the server to diagnose the installation. Run this command from the directory containing .env and docker-compose.yml:

sudo ./sh_scripts/diagnostics.sh

The script checks the core containers, the local LLM, Whisper and BGE Reranker when their profiles are enabled, PostgreSQL, free disk space, and the API response. A successful check prints Диагностика завершена без замечаний and exits with code 0. On failure, it names the affected component and exits with code 1. Run the docker compose logs command shown in the message for details.

To have the installation wizard offer diagnostics after a successful start, launch it with SHERPA_DIAGNOSTICS_OFFER=true. Example: sudo env SHERPA_DIAGNOSTICS_OFFER=true ./sh_scripts/install.sh. The prompt is disabled by default. Diagnostics does not modify installation data.

It is necessary to check container status and logs. Running services should have status Up:

docker compose ps
docker compose logs --tail=100 aiserver

It is necessary to open https://<your-domain> in a browser and check that you can sign in. To check the web server response, replace the example domain with yours:

curl -I https://aiserver.example.com

Expect a successful HTTP response or a redirect to the sign-in page. It is necessary to check that the browser trusts the certificate without a warning. If you use a local LLM, send a test chat message and confirm that the model responds. If problems occur, inspect the relevant service logs:

docker compose logs --tail=100 aiserver-llm-server
docker compose logs --tail=100 aiserver-embed

For CPU mode, use aiserver-llm-server-cpu instead of aiserver-llm-server. If a container does not start, check disk space and logs. For sign-in or database errors, check .env. For HTTPS, check certificates and the domain name. For GPU, check the driver, GPU availability and VRAM.

Maintenance#

Run commands from the directory containing docker-compose.yml and .env:

docker compose ps
docker compose logs -f aiserver
docker compose restart aiserver
docker stats

For a GPU, also use nvidia-smi. docker compose down stops and removes containers; do not add -v if you want to retain data in Docker volumes. For updates, use the update guide. Before updating from 2.5.0 to 2.5.1, complete the required index preparation. If the current version is below 2.5.0, agree on an update path with support.