Prerequisites#

Flux-CD
Execute these commands to install the NVIDIA drivers and container toolkit on the host machine. This will allow the GPU Operator to manage the NVIDIA GPUs on the cluster.

Add the NVIDIA package repositories

curl -fsSL https://nvidia.github.io/libnvidia-container/gpgkey | sudo gpg --dearmor -o /usr/share/keyrings/nvidia-container-toolkit-keyring.gpg \
  && curl -s -L https://nvidia.github.io/libnvidia-container/stable/deb/nvidia-container-toolkit.list | \
    sed 's#deb https://#deb [signed-by=/usr/share/keyrings/nvidia-container-toolkit-keyring.gpg] https://#g' | \
    sudo tee /etc/apt/sources.list.d/nvidia-container-toolkit.list

sudo sed -i -e '/experimental/ s/^#//g' /etc/apt/sources.list.d/nvidia-container-toolkit.list

Update the package lists

sudo apt-get update

Install packages

sudo apt install nvidia-driver nvidia-container-toolkit nvidia-container-toolkit-base

Configure the container runtime to use the NVIDIA runtime.

sudo nvidia-ctk runtime configure --runtime=containerd

Installation#

Create the following directory structure for NVIDIA GPU Operator:

nvidia-gpu-operator/
├── gpu-operator.yml
├── gpu-operator-cm.yml
└── gpu-operator-repo.yml

Add the following content to nvidia-gpu-operator/gpu-operator.yml:

---
apiVersion: helm.toolkit.fluxcd.io/v2
kind: HelmRelease
metadata:
  name: gpu-operator
  namespace: gpu-operator
spec:
  interval: 6h
  chart:
    spec:
      chart: gpu-operator
      version: "v25.3.2"
      sourceRef:
        kind: HelmRepository
        name: nvidia
        namespace: flux-system
      interval: 6h
  install:
    createNamespace: true
  upgrade:
    remediation:
      remediateLastFailure: true
  values:
    driver:
      enabled: false
    toolkit:
      env:
      - name: CONTAINERD_SOCKET
        value: /run/k3s/containerd/containerd.sock
      - name: CONTAINERD_CONFIG
        value: /var/lib/rancher/k3s/agent/etc/containerd/config.toml

Add the following content to nvidia-gpu-operator/gpu-operator-cm.yml:

---
apiVersion: v1
kind: ConfigMap
metadata:
  name: time-slicing-config
  namespace: gpu-operator
data:
  any: |-
    version: v1
    flags:
      migStrategy: none
    sharing:
      timeSlicing:
        resources:
        - name: nvidia.com/gpu
          replicas: 4

# remember to patch the cluster policy to use this configmap
# kubectl patch clusterpolicy/cluster-policy -n gpu-operator --type merge -p '{"spec": {"devicePlugin": {"config": {"name": "time-slicing-config", "default": "any"}}}}'

Add the following content to nvidia-gpu-operator/gpu-operator-repo.yml:

---
apiVersion: source.toolkit.fluxcd.io/v1
kind: HelmRepository
metadata:
  name: nvidia
  namespace: flux-system
spec:
  interval: 6h
  url: https://helm.ngc.nvidia.com/nvidia

Commit and push these files to your Git repository. The Flux CD will automatically detect the changes and deploy the NVIDIA GPU Operator to your cluster.

Nvidia

Prerequisites#

Installation#