49 lines
1.4 KiB
Markdown
49 lines
1.4 KiB
Markdown
# vLLM
|
|
|
|
Installs vLLM via Helm for GPU-enabled inference workloads.
|
|
|
|
## Requirements
|
|
|
|
- Kubernetes cluster with NVIDIA GPU workers
|
|
- NVIDIA device plugin deployed
|
|
- NGC API key for downloading models (stored in secret `vllm-ngc-secret`)
|
|
|
|
## Role Variables
|
|
|
|
| Variable | Default | Description |
|
|
|----------|---------|-------------|
|
|
| `vllm_chart_version` | `0.1.0` | vLLM Helm chart version |
|
|
| `vllm_namespace` | `vllm-system` | Kubernetes namespace |
|
|
| `vllm_num_replicas` | `2` | Number of replicas for HA |
|
|
| `vllm_image.repository` | `vllm/vllm-openai` | Container image repository |
|
|
| `vllm_image.tag` | `latest` | Container image tag |
|
|
| `vllm_image.pull_policy` | `IfNotPresent` | Image pull policy |
|
|
| `vllm_resources.limits.nvidia.com/gpu` | `1` | GPU limit per pod |
|
|
| `vllm_resources.requests.nvidia.com/gpu` | `1` | GPU request per pod |
|
|
| `vllm_env` | `[]` | Environment variables (e.g., NGC_API_KEY) |
|
|
| `vllm_extra_args` | `[]` | Additional CLI arguments |
|
|
| `vllm_kubeconfig` | `/home/{{ ansible_user }}/.kube/config` | Path to kubeconfig |
|
|
|
|
## Example Playbook
|
|
|
|
```yaml
|
|
- hosts: k8s_manager
|
|
become: false
|
|
roles:
|
|
- vllm
|
|
```
|
|
|
|
## Prerequisites
|
|
|
|
- Create secret with NGC API key:
|
|
```bash
|
|
kubectl create secret generic vllm-ngc-secret \
|
|
--from-literal=api-key=your-ngc-api-key \
|
|
-n vllm-system
|
|
```
|
|
|
|
- Ensure GPU workers are labeled:
|
|
```bash
|
|
kubectl label nodes <node> nvidia.com/gpu.present=true
|
|
```
|