# Requirements for Sherpa AI Server

### System Requirements

* **OS**: Ubuntu 20.04+ is recommended, however, there are generally no issues with other Linux distributions; compatibility has been verified with RedOS, AstraLinux, AltLinux, Debian
* **CPU**: x86\_64 with AVX2
* **RAM**: 16 GB minimum, 32 GB+ recommended
* **Disk**: 100 GB+ of free space
* **GPU**: NVIDIA with CUDA 11.8+ (recommended)
* **Network**: Stable internet connection
* **Access**: sudo privileges for installation

**Important:**

* Installation takes hours due to downloading AI models (10-50 GB)
* After installation, internet is not required

### Preparing the Server

#### Checking Resources

The following uses Ubuntu syntax; if the command does not fit, you need to change the syntax according to your OS.

```bash
# Check system resources
df -h          # Disk space
free -h        # RAM
nvidia-smi     # GPU (if installed)
```

<details>

<summary>💡 Resource Check Comments</summary>

**df -h** - shows disk space usage in human-readable format **free -h** - shows information about RAM **nvidia-smi** - shows information about NVIDIA GPU (if installed)

</details>

#### Installing Basic Tools

```bash
# Update the system and install tools
sudo apt update
sudo apt install -y ca-certificates curl tar
```

<details>

<summary>💡 Comments on Installing Basic Tools</summary>

**sudo apt update** - updates the list of available packages from repositories **sudo apt install -y ca-certificates curl tar** - installs necessary tools:

* `ca-certificates` - root certificates for SSL verification
* `curl` - tool for downloading files
* `tar` - utility for working with archives
* `-y` - automatic confirmation of installation

</details>

### Installing Docker (skip if already installed)

Sherpa AI Server runs in Docker containers.

#### Installing Docker CE

```bash
# Add the official Docker GPG key
sudo install -m 0755 -d /etc/apt/keyrings
sudo curl -fsSL https://download.docker.com/linux/ubuntu/gpg -o /etc/apt/keyrings/docker.asc
sudo chmod a+r /etc/apt/keyrings/docker.asc

# Add the Docker repository
echo "deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.asc] https://download.docker.com/linux/ubuntu \
  $(. /etc/os-release && echo "${UBUNTU_CODENAME:-$VERSION_CODENAME}") stable" | \
  sudo tee /etc/apt/sources.list.d/docker.list > /dev/null

# Install Docker
sudo apt update
sudo apt install -y docker-ce docker-ce-cli containerd.io docker-buildx-plugin docker-compose-plugin
```

<details>

<summary>💡 Comments on Installing Docker</summary>

**Adding the GPG key:**

* `sudo install -m 0755 -d /etc/apt/keyrings` - creates a directory for keys with the correct permissions
* `sudo curl -fsSL https://download.docker.com/linux/ubuntu/gpg -o /etc/apt/keyrings/docker.asc` - downloads the Docker GPG key
* `sudo chmod a+r /etc/apt/keyrings/docker.asc` - sets read permissions for all

**Adding the repository:**

* `echo "deb [...]` - adds the official Docker repository to the APT sources list
* Uses environment variables to determine architecture and Ubuntu version

**Installing packages:**

* `sudo apt update` - updates the package list after adding a new repository
* `sudo apt install -y docker-ce docker-ce-cli containerd.io docker-buildx-plugin docker-compose-plugin` - installs Docker and components

</details>

#### Configuration and Verification

```bash
# Add user to the docker group (optional)
sudo usermod -aG docker $USER

# Check installation
docker --version
docker compose version
```

<details>

<summary>💡 Comments on Docker Configuration</summary>

**sudo usermod -aG docker $USER** - adds the current user to the docker group

* `-a` - append (adds to existing groups)
* `-G docker` - adds to the docker group
* `$USER` - variable with the current user's name

**docker --version** - shows the Docker version **docker compose version** - shows the Docker Compose version

**Expected result:** Docker successfully runs a test container.

</details>

**Expected result:** Docker successfully runs a test container.

### Installing GPU Support (NVIDIA + Toolkit) (skip if already installed)

If the server has an NVIDIA GPU, install the drivers and Container Toolkit:

> **💡 Offline Installation:** If the server does not have internet access, use the Offline Installation of GPU Support (NVIDIA) section below.

#### Installing NVIDIA Drivers

```bash
# Remove old drivers and install new ones
sudo apt purge 'nvidia-*'
sudo apt autoremove
sudo apt install -y nvidia-driver-580

# Reboot the server
sudo reboot
```

<details>

<summary>💡 Comments on Installing NVIDIA Drivers</summary>

**sudo apt purge 'nvidia-\*'** - removes all NVIDIA packages

* `purge` - removes packages and their configuration files
* `'nvidia-*'` - pattern for finding all packages starting with nvidia

**sudo apt autoremove** - removes unnecessary dependencies **sudo apt install -y nvidia-driver-580** - installs NVIDIA drivers version 580

**sudo reboot** - reboots the system to apply changes

</details>

#### Checking GPU

```bash
# After reboot, check GPU
nvidia-smi
```

<details>

<summary>💡 Comments on Checking GPU</summary>

**nvidia-smi** - System Management Interface for NVIDIA GPU

* Shows information about GPU, drivers, processes
* Used for diagnostics and monitoring

**Expected output:**

```
NVIDIA-SMI 580.xx.xx
Driver Version: 580.xx.xx
```

</details>

#### Installing NVIDIA Container Toolkit

```bash
# Add the repository and install the toolkit
curl -fsSL https://nvidia.github.io/libnvidia-container/gpgkey | \
  sudo gpg --dearmor -o /usr/share/keyrings/nvidia-container-toolkit.gpg

curl -fsSL https://nvidia.github.io/libnvidia-container/stable/deb/nvidia-container-toolkit.list | \
  sed 's#deb https://#deb [signed-by=/usr/share/keyrings/nvidia-container-toolkit.gpg] https://#g' | \
  sudo tee /etc/apt/sources.list.d/nvidia-container-toolkit.list

sudo apt update
sudo apt install -y nvidia-container-toolkit

# Configure Docker for GPU
sudo nvidia-ctk runtime configure --runtime=docker
sudo systemctl restart docker
```

<details>

<summary>💡 Comments on Installing NVIDIA Container Toolkit</summary>

**Adding the NVIDIA Container Toolkit repository:**

* `curl -fsSL https://nvidia.github.io/libnvidia-container/gpgkey` - downloads the GPG key
* `sudo gpg --dearmor` - converts the key to binary format
* `curl ... nvidia-container-toolkit.list` - downloads the repository configuration
* `sed 's#deb https://#deb [signed-by=...] https://#g'` - adds key signature to the configuration

**sudo apt update** - updates the package list **sudo apt install -y nvidia-container-toolkit** - installs the toolkit

**sudo nvidia-ctk runtime configure --runtime=docker** - configures Docker runtime for NVIDIA **sudo systemctl restart docker** - restarts the Docker daemon to apply changes

</details>

### Offline Installation of GPU Support (NVIDIA)

If the server does not have internet access, perform the installation of NVIDIA drivers and Container Toolkit offline. Package preparation can be done in parallel with downloading Docker images.

#### Preparing Packages on an Internet-Connected Machine

**Requirements:** A machine with Ubuntu/Debian and internet access.

**Important:**

* Package preparation can be done on any machine with Ubuntu/Debian (not necessarily on the target server)
* Ensure you have enough disk space (about 1 GB for all packages)
* All commands are executed in one terminal session, keep track of the current directory

On a machine with internet access (Ubuntu/Debian), prepare an archive with NVIDIA packages:

**Downloading NVIDIA Drivers**

**Important:** Choose one of two methods - either download the .run file from the NVIDIA website or use .deb packages via apt download.

**Method 1: Downloading the .run file from the official NVIDIA website (recommended for beginners)**

1. Go to the official NVIDIA website: <https://www.nvidia.com/Download/index.aspx>
2. Select options:
   * **Product Type**: GeForce / Quadro / Tesla (depending on your graphics card)
   * **Product Series**: select the series of your graphics card
   * **Product**: select the specific model
   * **Operating System**: Linux 64-bit
   * **Download Type**: Linux Driver
   * **Language**: English (US)
3. Click "Search" and download the driver file (e.g., `NVIDIA-Linux-x86_64-580.XX.XX.run`)
4. Create a directory for packages and save the .run file:

   ```bash
   mkdir -p nvidia-offline-packages/drivers
   # Move the downloaded .run file to this directory
   mv ~/Downloads/NVIDIA-Linux-x86_64-*.run nvidia-offline-packages/drivers/
   ```

**Method 2: Downloading .deb packages via apt (for advanced users)**

If you are using Ubuntu/Debian and want to use .deb packages:

```bash
# Create a directory for packages
mkdir -p nvidia-offline-packages/drivers
cd nvidia-offline-packages/drivers

# Download the driver via apt download (if the repository is available)
# First, add the NVIDIA repository (if not already added)
sudo add-apt-repository ppa:graphics-drivers/ppa
sudo apt update

# Download the driver package
apt download nvidia-driver-580

# Download driver dependencies (execute the command line by line)
# First, find out the list of dependencies:
apt-cache depends nvidia-driver-580 | grep "Depends:" | cut -d: -f2 | tr -d ' ' > /tmp/deps.txt

# Then download each dependency:
while read dep; do
	apt download "$dep" 2>/dev/null || echo "Skipped dependency: $dep"
done < /tmp/deps.txt

# Remove the temporary file
rm /tmp/deps.txt
```

**Downloading NVIDIA Container Toolkit**

**About GPG keys:**

* GPG keys are used to verify the authenticity of packages when installing via `apt install`
* When installing offline via `dpkg -i`, GPG keys **ARE NOT REQUIRED** - packages are installed directly
* GPG keys are only needed if you plan to use `apt install` or set up the repository for future updates
* For simplicity, you can skip downloading GPG keys and install packages via `dpkg -i`

```bash
# Go back to the packages directory
cd nvidia-offline-packages

# Create a directory for the Container Toolkit
mkdir -p toolkit

# Add the NVIDIA Container Toolkit repository (if not already added)
curl -fsSL https://nvidia.github.io/libnvidia-container/gpgkey | \
  sudo gpg --dearmor -o /usr/share/keyrings/nvidia-container-toolkit.gpg

curl -fsSL https://nvidia.github.io/libnvidia-container/stable/deb/nvidia-container-toolkit.list | \
  sed 's#deb https://#deb [signed-by=/usr/share/keyrings/nvidia-container-toolkit.gpg] https://#g' | \
  sudo tee /etc/apt/sources.list.d/nvidia-container-toolkit.list

sudo apt update

# Download the Container Toolkit package and all its dependencies
cd toolkit
apt download nvidia-container-toolkit

# Download Container Toolkit dependencies (execute the command line by line)
# First, find out the list of dependencies:
apt-cache depends nvidia-container-toolkit | grep "Depends:" | cut -d: -f2 | tr -d ' ' > /tmp/toolkit-deps.txt

# Then download each dependency:
while read dep; do
  apt download "$dep" 2>/dev/null || echo "Skipped dependency: $dep"
done < /tmp/toolkit-deps.txt

# Remove the temporary file
rm /tmp/toolkit-deps.txt

# Optionally: Download the GPG key and repository file (only needed if you plan to use apt install instead of dpkg)
# For offline installation via dpkg -i, GPG keys ARE NOT REQUIRED
cd ..
mkdir -p keys
curl -fsSL https://nvidia.github.io/libnvidia-container/gpgkey -o keys/nvidia-container-toolkit.gpg
curl -fsSL https://nvidia.github.io/libnvidia-container/stable/deb/nvidia-container-toolkit.list -o keys/nvidia-container-toolkit.list
```

**Packaging the Archive**

```bash
# Go back to the root directory of the packages
cd nvidia-offline-packages

# Ensure all files are in place:
# - drivers/ - contains .deb packages or .run driver file
# - toolkit/ - contains .deb packages of the Container Toolkit
# - keys/ - contains GPG keys and repository files (optionally, if available)

# Create an archive with all packages
# If the keys/ directory is empty or missing, you can create the archive without it:
if [ -d "keys" ] && [ "$(ls -A keys)" ]; then
    tar -czf nvidia-packages-offline.tar.gz drivers/ toolkit/ keys/
else
    tar -czf nvidia-packages-offline.tar.gz drivers/ toolkit/
    echo "Warning: GPG keys are not included in the archive (not required for installation via dpkg)"
fi

# Check the size of the archive
ls -lh nvidia-packages-offline.tar.gz
```

<details>

<summary>💡 Comments on Preparing Packages</summary>

**Downloading Drivers:**

* The official NVIDIA website provides the latest driver versions for all GPU models
* The driver version should meet the requirements (recommended 580+)
* .deb packages are more convenient for installation via dpkg (recommended for Ubuntu/Debian)
* .run files are universal installers from NVIDIA, work on any Linux distributions, but require stopping the graphical server during installation

**Downloading Container Toolkit:**

* `apt download` downloads the package without installation
* Dependencies are downloaded separately for a complete offline installation
* GPG keys and repository files are optional - they are only needed if you plan to use `apt install` instead of `dpkg -i`, or if you want to set up the repository for future updates

**Archive Size:** The expected size of the archive is about 200-500 MB depending on the versions of the packages.

</details>

#### Transferring the Archive to the Target Server

Transfer the archive `nvidia-packages-offline.tar.gz` to the target server by any convenient method (SCP, SFTP, USB drive, etc.). It is recommended to transfer the archive along with Docker images.

#### Installation on the Target Server

After transferring the archive to the target server, perform the installation:

**Installing NVIDIA Drivers from .deb Packages**

If you downloaded .deb driver packages:

```bash
# Extract the archive
tar -xzf nvidia-packages-offline.tar.gz
cd nvidia-offline-packages

# Check for .deb files in the drivers directory
ls -la drivers/*.deb

# Install NVIDIA drivers
cd drivers
sudo dpkg -i *.deb

# Check the installation result
if [ $? -eq 0 ]; then
    echo "Drivers installed successfully"
else
    echo "Dependency errors detected, fixing..."
    # If there are dependency errors, install them from the cache or fix dependencies
    sudo apt install --fix-broken -y
    
    # If fix-broken did not help, try installing dependencies manually
    # Ensure that all dependencies were downloaded to the drivers directory
    echo "Check that all dependencies are downloaded to the drivers/ directory"
    ls -la *.deb
fi

# Reboot the server to apply the drivers
echo "Rebooting the server in 10 seconds... Press Ctrl+C to cancel"
sleep 10
sudo reboot
```

**If the installation failed:**

* Check which packages did not install: `dpkg -l | grep nvidia`
* Ensure all dependencies are downloaded: `ls -la drivers/*.deb | wc -l` (should be several files)
* Try installing packages one by one: `sudo dpkg -i package_name.deb`

**Installing NVIDIA Drivers from .run File**

If you downloaded the .run driver file from the official NVIDIA website:

**Important before starting:**

* Ensure you have SSH access to the server or physical access (in case of issues)
* If a graphical interface is installed on the server, find out which display manager is used (gdm3, lightdm, sddm)
* For servers without a graphical interface, the steps for stopping the graphical server can be skipped

```bash
# Extract the archive (if the .run file was added to the archive)
tar -xzf nvidia-packages-offline.tar.gz
cd nvidia-offline-packages/drivers

# Find the .run driver file (e.g., NVIDIA-Linux-x86_64-580.XX.XX.run)
ls -la *.run

# Make the file executable
chmod +x NVIDIA-Linux-x86_64-*.run

# Determine which graphical server is used (execute one of the commands):
# Check for GNOME/GDM3:
systemctl status gdm3 > /dev/null 2>&1 && echo "Using gdm3"
# Check for LightDM:
systemctl status lightdm > /dev/null 2>&1 && echo "Using lightdm"
# Check for SDDM (KDE):
systemctl status sddm > /dev/null 2>&1 && echo "Using sddm"

# Stop the graphical server (execute ONLY one command corresponding to your display manager):
# For Ubuntu with GNOME:
sudo systemctl stop gdm3
# OR for Ubuntu with LightDM:
# sudo systemctl stop lightdm
# OR for systems with KDE:
# sudo systemctl stop sddm
# OR if there is no graphical interface, skip this step

# Install the driver (use flags for automatic installation)
sudo ./NVIDIA-Linux-x86_64-*.run \
  --silent \
  --no-nouveau-check \
  --no-opengl-files \
  --disable-nouveau

# Check the installation result (should be a success message)
echo $?

# If the graphical server was stopped, start it back up (use the same command as for stopping):
sudo systemctl start gdm3  # or lightdm/sddm depending on your system

# Reboot the server to apply the drivers
sudo reboot
```

**If the installation failed:**

* Check the logs: `sudo cat /var/log/nvidia-installer.log`
* Ensure the nouveau module is disabled: `lsmod | grep nouveau`
* Try the installation with the `--no-opengl-files` flag if the error is related to OpenGL

<details>

<summary>💡 Comments on Installing the .run File</summary>

**Installer Flags:**

* `--silent` - automatic installation without interactive questions
* `--no-nouveau-check` - skips the nouveau module check (open NVIDIA driver)
* `--no-opengl-files` - does not install OpenGL libraries (important for servers without GUI)
* `--disable-nouveau` - disables the nouveau module before installation

**Stopping the Graphical Server:**

* This step is not required on servers without a graphical interface
* If the graphical server is not stopped, the installation may fail
* After installation, the graphical server can be started back up

**Alternative Method (without stopping GUI):** If stopping the graphical server is not possible, you can use a text console:

1. Switch to a text console (Ctrl+Alt+F1)
2. Log in
3. Perform the installation of the .run file
4. Return to graphical mode (Ctrl+Alt+F7 or reboot)

</details>

**Checking Driver Installation**

After rebooting, check the driver installation:

```bash
# Check GPU (this command should show information about the graphics card)
nvidia-smi
```

**Expected result:** You should see a table with information about the GPU, for example:

```
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 580.XX       Driver Version: 580.XX       CUDA Version: 12.X  |
|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|===============================+======================+======================|
|   0  NVIDIA GeForce ...  Off  | 00000000:01:00.0 Off |                  N/A |
| ...                                                                         |
+-----------------------------------------------------------------------------+
```

**If the nvidia-smi command does not work or shows an error:**

* Check that the driver is loaded: `lsmod | grep nvidia`
* Check the logs: `dmesg | grep -i nvidia`
* Ensure the GPU is visible to the system: `lspci | grep -i nvidia`

**Installing NVIDIA Container Toolkit**

After successfully installing the drivers, install the Container Toolkit:

**Important:**

* Docker must be installed before installing the Container Toolkit
* Ensure Docker is running: `sudo systemctl status docker`

```bash
# Go back to the packages directory (replace with the actual path where you extracted the archive)
# For example, if the archive was extracted in the home directory:
cd ~/nvidia-offline-packages
# Or if in another location, specify the full path:
# cd /full/path/to/nvidia-offline-packages

# Check that all files are in place
ls -la toolkit/*.deb

# Optionally: Install the GPG key (NOT required for installation via dpkg)
# GPG keys are only needed if:
# 1. You plan to use apt install instead of dpkg -i
# 2. You want to set up the repository for future updates (when internet is available)
# For simple offline installation via dpkg -i, you can skip this step
if [ -f "keys/nvidia-container-toolkit.gpg" ]; then
    echo "Installing GPG key (optional)..."
    sudo mkdir -p /usr/share/keyrings
    sudo cp keys/nvidia-container-toolkit.gpg /tmp/nvidia-container-toolkit.gpg
    sudo gpg --dearmor /tmp/nvidia-container-toolkit.gpg -o /usr/share/keyrings/nvidia-container-toolkit.gpg
    sudo rm /tmp/nvidia-container-toolkit.gpg
    echo "GPG key installed"
else
    echo "GPG key not found - skipping installation (not critical for dpkg -i)"
fi

# Install the Container Toolkit from local packages
cd toolkit
sudo dpkg -i *.deb

# Check the installation result
if [ $? -eq 0 ]; then
    echo "Container Toolkit installed successfully"
else
    echo "Dependency errors detected, fixing..."
    # Fix dependencies if necessary
    sudo apt install --fix-broken -y
    
    # If fix-broken did not help, check that all dependencies are downloaded
    echo "Check for all dependencies in the toolkit/ directory"
    ls -la *.deb
fi

# Configure Docker for GPU
sudo nvidia-ctk runtime configure --runtime=docker

# Restart Docker to apply changes
sudo systemctl restart docker

# Check that Docker is running
sudo systemctl status docker

# Check GPU installation in Docker
nvidia-smi
```

**If there were issues with the installation:**

* Check that Docker sees the GPU: `docker info | grep -i runtime`
* Check the configuration: `cat /etc/docker/daemon.json`
* Ensure nvidia-container-toolkit is installed: `dpkg -l | grep nvidia-container-toolkit`
* Check Docker logs: `sudo journalctl -u docker -n 50`

<details>

<summary>💡 Comments on Installation on the Target Server</summary>

**Installing Drivers:**

* `dpkg -i *.deb` - installs all .deb packages from the directory
* `apt install --fix-broken` - fixes dependencies using the local package cache (if available)
* Rebooting is mandatory to activate NVIDIA drivers

**Installing Container Toolkit:**

* When installing via `dpkg -i`, GPG keys are NOT required - packages are installed directly without signature verification
* GPG keys are only needed if you use `apt install` or plan to set up the repository for updates
* `dpkg -i` is a simpler method for offline installation, does not require key setup

**Possible Issues:**

* If `apt install --fix-broken` does not work without internet, ensure all dependencies are downloaded to the packages directory
* Check the compatibility of driver and Container Toolkit versions with your version of Ubuntu/Debian

</details>

**Expected result:** After installation, the `nvidia-smi` command should show information about the GPU, and Docker containers should have access to the GPU.


---

# Agent Instructions: Querying This Documentation

If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter:

```
GET https://docs.sherparpa.ru/en/sherpa-ai/sherpa-ai-server/trebovaniya-k-serveru-sherpa-ai-server.md?ask=<question>
```

The question should be specific, self-contained, and written in natural language.
The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.