Skip to main content
Environment variables let you configure your vLLM workers without rebuilding your Docker image. You can customize model behavior, performance settings, and other deployment options to suit your needs. To set environment variables, go to your endpoint settings and add them under Environment Variables. The latest vLLM worker release, v2.30.3, is built on vLLM v0.32.0. The worker starts vllm serve with flags built from your environment variables. The tables below list common settings, but you’re not limited to them:
  • Any environment variable named after a vllm serve flag in upper snake case becomes that flag. For example, MAX_MODEL_LEN=4096 becomes --max-model-len 4096.
  • Boolean variables take true or false.
  • VLLM_EXTRA_ARGS passes extra flags straight to vllm serve. They’re added last, so they override any earlier setting. Use it for flags that don’t have an environment variable, for example VLLM_EXTRA_ARGS="--tool-call-parser qwen3_coder --enable-auto-tool-choice".
You can use any tool-call parser or speculative decoding method that vLLM v0.32.0 supports, such as MTP, even if the tables below don’t list it.
A variable named after a flag that vLLM has removed is ignored. The worker logs a warning at startup and continues. If a setting from an older deployment has no effect, check Removed variables.

LLM settings

These variables control the core language model configuration.

LoRA settings

Configure LoRA (Low-Rank Adaptation) adapters for your model.

Speculative decoding settings

Configure speculative decoding to improve performance. Pass the whole configuration as one JSON string in SPECULATIVE_CONFIG, which vLLM parses directly. For example, to use a draft model with three speculative tokens:
The per-field variables from earlier releases, such as SPECULATIVE_MODEL and NUM_SPECULATIVE_TOKENS, are no longer read. See Removed variables.

System performance settings

Configure GPU memory and system resource utilization.

Tokenizer settings

Customize tokenizer behavior and chat templates.

OpenAI compatibility settings

Configure OpenAI API compatibility features.
TOOL_CALL_PARSER tells vLLM how to interpret a model’s tool-call output. If the parser doesn’t match the model’s format, tool calls may not be detected (or may error during parsing).

Serverless and concurrency settings

Configure concurrency, timeouts, and logging for Serverless deployments. The worker reads MAX_CONCURRENCY, the two timeouts, VLLM_EXTRA_ARGS, and VLLM_CONFIG_FILE itself. The logging variables go to vLLM.

Advanced settings

Additional configuration options for specialized use cases.

Docker build arguments

These arguments are used when building a custom worker image, for example to bake a model in. They’re set at build time, so changing them on an endpoint has no effect.

Removed variables

The worker no longer reads the following variables. Setting one has no effect. Variables named after a flag that vLLM removed also log a warning at startup. For the complete list, see the worker’s configuration reference.
Last modified on November 14, 2026