k8s/roles/vllm/README.md

vLLM

Installs vLLM via Helm for GPU-enabled inference workloads.

Requirements

• Kubernetes cluster with NVIDIA GPU workers
• NVIDIA device plugin deployed
• NGC API key for downloading models (stored in secret vllm-ngc-secret)

Role Variables

Variable	Default	Description
vllm_chart_version	0.1.0	vLLM Helm chart version
vllm_namespace	vllm-system	Kubernetes namespace
vllm_num_replicas	2	Number of replicas for HA
vllm_image.repository	vllm/vllm-openai	Container image repository
vllm_image.tag	latest	Container image tag
vllm_image.pull_policy	IfNotPresent	Image pull policy
vllm_resources.limits.nvidia.com/gpu	1	GPU limit per pod
vllm_resources.requests.nvidia.com/gpu	1	GPU request per pod
vllm_env	[]	Environment variables (e.g., NGC_API_KEY)
vllm_extra_args	[]	Additional CLI arguments
vllm_kubeconfig	/home/{{ ansible_user }}/.kube/config	Path to kubeconfig

Example Playbook

- hosts: k8s_manager
  become: false
  roles:
    - vllm

Prerequisites

• Create secret with NGC API key:

  kubectl create secret generic vllm-ngc-secret \
    --from-literal=api-key=your-ngc-api-key \
    -n vllm-system
  

• Ensure GPU workers are labeled:

  kubectl label nodes <node> nvidia.com/gpu.present=true