llm-d contributors announce version 1.0 with new serving controls

llm-d project contributors announced version 1.0 on October 20, 2026, adding controls for production inference on Kubernetes. For operators using DisaggregatedSet, the release notes specify a concrete prerequisite: LeaderWorkerSet v0.11.1 or newer, with the DisaggregatedSet API enabled.

What llm-d 1.0 changes

llm-d is an open-source stack that coordinates distributed model inference on Kubernetes, working above model servers such as vLLM and SGLang. The project was founded by Red Hat, IBM Research, Google Cloud, CoreWeave and NVIDIA, and is licensed under Apache License 2.0.

Version 1.0 focuses on running inference workloads in production:

• Multi-tenant flow control applies priority and fairness rules to requests from different tenants, with controlled load shedding when capacity is constrained.
• Router high availability adds a hot-standby router to support continued routing if the active router fails.
• Coordinated updates for disaggregated serving let prefill and decode pools be managed as one logical unit. These are the separate stages that process an input prompt and generate a response.
• A v1 router configuration API provides the release’s version 1 interface for configuring the router.
• Agent-aware session handling supports session affinity and turn-priority scheduling to reuse cached context across turns. The Responses API also supports previous_response_id.

Together, these changes address traffic management, router resilience and upgrades across split serving pools, alongside session handling for agent workloads.

The prerequisite for DisaggregatedSet deployments

A deployment using DisaggregatedSet needs LeaderWorkerSet v0.11.1 or newer and the DisaggregatedSet API enabled. The requirement applies to DisaggregatedSet deployments, rather than every llm-d setup. Teams planning that upgrade can check the official v1.0.0 release notes for the exact dependency.

What changes for ROCm images

The release notes deprecate the project-built llm-d-rocm image builds and describe a temporary upstream vLLM ROCm image for disaggregated-serving support. They also list AMD ROCm deployment work. The image-build change is the specific operational note for teams using that path.

Organizations describing production use

DigitalOcean said it uses llm-d as foundational inference software for DigitalOcean Serverless Inference and Managed Agents. JetBrains said llm-d runs in production behind Junie Lite. Google Cloud said it was integrating llm-d 1.0 into Google Kubernetes Engine (GKE) across Google TPUs and NVIDIA GPUs.

These examples show how the project’s contributors and adopters describe using the stack: within cloud infrastructure, managed inference services and a production software plan.