---
title: '50611'
pageLayout: default
slug: thoughts/inference/vllm-prs/50611
permalink: https://aarnphm.xyz/thoughts/inference/vllm-prs/50611.md
generator:
  quartz: v4.6.0
  hostedProvider: Cloudflare
  baseUrl: aarnphm.xyz
full: https://aarnphm.xyz/llms-full.txt
---
# [#50611](https://github.com/aarnphm/aarnphm.github.io/issues/50611) \[Nixl\]\[PD\] DCP support for MLA models

[vllm-project/vllm#50611](https://github.com/vllm-project/vllm/pull/50611) · NickLucche · MERGED · created 2026-07-31T18:30:04Z · merged 2026-08-29T16:03:54Z

Alternative to [vllm-project/vllm#38433](https://github.com/vllm-project/vllm/pull/38433) as we iterate on the design with @pisceskkk .

I feel this version is much closer to the changes we should see to the core files in terms of code structure and modifications to the workflow - it should result closer to injecting DCP login into current abstraction, rather than building something on the side.
However, it makes some significant assumptions to do that, described below.

---

NIXL P/D disaggregation assumed MLA’s KV cache is fully duplicated across TP, so a remote rank only needs to be read once. Decode Context Parallel (DCP) breaks that: it shards KV by block across ranks.



```
  Producer  (dcp_size=4) -- KV sharded into 4 slices, one per rank
  +------+------+------+------+
  |  P0  |  P1  |  P2  |  P3  |
  +--+---+--+---+--+---+--+---+
     |      |      |      |
     v      v      v      v
  +------+------+------+------+
  |  D0  |  D1  |  D0  |  D1  |   <-  remote_rank % local_dcp_size == local_dcp_rank
  +------+------+------+------+
  Decoder   (dcp_size=2) -- each rank owns 2x the slice, reads 2 remote ranks

    D0 reads {P0, P2}   D1 reads {P1, P3}
```

> \[!note\]
>
> Key simplifying assumption: per side, dcp_size is either 1 (fully replicated, unchanged pre-existing behavior) or tp_size (fully sharded, disjoint 1/dcp_size slice per rank) — never partial (1 < dcp_size < tp_size).

This means DCP sharding and the old “multiple TP replicas, pick one canonically” logic never have to compose on the same side, which is what keeps the routing logic to three flat cases (both replicated / one-replicated-one-sharded / both-sharded) instead of a general partial-overlap solver.

Two supporting assumptions that follow from it:

- DCP sizes across sides always divide one another (real deployments use power-of-two/highly-composite degrees) — avoids needing gcd/lcm logic, plain modulo suffices.
- DCP shards at block granularity when a KV connector is attached (`cp_kv_cache_interleave_size` pinned to block_size) — this is quite fundamental. However I had to update a few attn backends to make sure they get the updated value.

## Sweep

gsm8k eval sweep, ran at block_size=64 (128 needs this fix [vllm-project/vllm#51031](https://github.com/vllm-project/vllm/pull/51031)) on `deepseek-ai/DeepSeek-V2-Lite-Chat`

| Configuration           | Status | Strict match       |
| ----------------------- | ------ | ------------------ |
| P TP4/DCP1 → D TP4/DCP1 | PASS   | 0.6535253980288097 |
| P TP2/DCP1 → D TP4/DCP1 | PASS   | 0.6489764973464746 |
| P TP4/DCP4 → D TP4/DCP4 | PASS   | 0.6588324488248674 |
| P TP4/DCP1 → D TP4/DCP4 | PASS   | 0.6648976497346475 |
| P TP4/DCP4 → D TP4/DCP1 | PASS   | 0.6595905989385898 |
| P TP2/DCP2 → D TP4/DCP4 | PASS   | 0.6573161485974223 |
| P TP2/DCP1 → D TP4/DCP4 | PASS   | 0.6542835481425322 |

---

## comments

**mergify** (2026-08-04T01:18:54Z):

This pull request has merge conflicts that must be resolved before it can be
merged. Please rebase the PR, @NickLucche.

<https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/syncing-a-fork>

**NickLucche** (2026-08-06T10:22:32Z):

/ci run

**github-actions** (2026-08-06T10:22:50Z):

✅ Triggered [Buildkite CI #82667](https://buildkite.com/vllm/ci/builds/82667) for commit `e4d32ec81fe9`.

**NickLucche** (2026-08-06T12:35:27Z):

/ci retry

**github-actions** (2026-08-06T12:35:45Z):

✅ The previous CI build is still running: <https://buildkite.com/vllm/ci/builds/82667>

**mergify** (2026-08-07T10:17:31Z):

This pull request has merge conflicts that must be resolved before it can be
merged. Please rebase the PR, @NickLucche.

<https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/syncing-a-fork>

**LucasWilkinson** (2026-08-26T13:58:12Z):

/ci run

**github-actions** (2026-08-26T13:58:30Z):

✅ Triggered [Buildkite CI #85659](https://buildkite.com/vllm/ci/builds/85659) for commit `a76bcd0ac6ed`.

**NickLucche** (2026-08-26T17:39:51Z):

/ci run

**LucasWilkinson** (2026-08-26T20:20:10Z):

/ci run

**github-actions** (2026-08-26T20:20:28Z):

✅ Triggered [Buildkite CI #85691](https://buildkite.com/vllm/ci/builds/85691) for commit `16b6bcb5596c`.

**NickLucche** (2026-08-27T08:00:18Z):

/ci run

**github-actions** (2026-08-27T08:00:36Z):

✅ Triggered [Buildkite CI #85773](https://buildkite.com/vllm/ci/builds/85773) for commit `fed0b74c3500`.

**NickLucche** (2026-08-27T09:25:23Z):

/ci retry

**NickLucche** (2026-08-27T11:10:13Z):

/ci run

**github-actions** (2026-08-27T11:10:31Z):

✅ Triggered [Buildkite CI #85801](https://buildkite.com/vllm/ci/builds/85801) for commit `f7b41f6e3732`.

**mergify** (2026-08-27T20:28:26Z):

This pull request has merge conflicts that must be resolved before it can be
merged. Please rebase the PR, @NickLucche.

<https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/syncing-a-fork>

**mergify** (2026-08-28T12:40:44Z):

Hi @NickLucche, the pre-commit checks have failed. Please run:



```bash
uv pip install pre-commit>=4.5.1
pre-commit install
pre-commit run --all-files
```

Then, commit the changes and push to your branch.

For future commits, `pre-commit` will run automatically on changed files before each commit.

**NickLucche** (2026-08-28T14:52:52Z):

/ci run

**NickLucche** (2026-08-28T15:18:50Z):

/ci run

**github-actions** (2026-08-28T15:19:14Z):

✅ Triggered [Buildkite CI #85980](https://buildkite.com/vllm/ci/builds/85980) for commit `7a79d4664f70`.

**NickLucche** (2026-08-28T16:01:57Z):

/ci retry

**github-actions** (2026-08-28T16:02:58Z):

✅ The previous CI build is still running: <https://buildkite.com/vllm/ci/builds/85980>

**NickLucche** (2026-08-29T14:09:19Z):

/ci run

**github-actions** (2026-08-29T14:09:37Z):

✅ Triggered [Buildkite CI #86145](https://buildkite.com/vllm/ci/builds/86145) for commit `86244e386bed`.

**ywang96** (2026-09-22T03:40:35Z):

@NickLucche

It seems that this change breaks all multimodal models, in particular, this line is crashing at start



```
    assert self.model_config.use_mla or dcp_size == 1
```

