---
title: '59997'
pageLayout: default
slug: thoughts/inference/vllm-prs/59997
permalink: https://aarnphm.xyz/thoughts/inference/vllm-prs/59997.md
generator:
  quartz: v4.6.0
  hostedProvider: Cloudflare
  baseUrl: aarnphm.xyz
full: https://aarnphm.xyz/llms-full.txt
---
# [#59997](https://github.com/aarnphm/aarnphm.github.io/issues/59997) \[Bugfix\]\[KV Connector\]\[NIXL\] Transfer the whole PLE short-conv page in disaggregated P/D

[vllm-project/vllm#59997](https://github.com/vllm-project/vllm/pull/59997) · aijanai · MERGED · created 2026-10-05T01:53:11Z · merged 2026-10-08T19:58:14Z

## Purpose

Fixes [#59642](https://github.com/aarnphm/aarnphm.github.io/issues/59642) (Qwen3.8-Flash-Next: 0% MTP acceptance in disaggregated P/D serving with `NixlConnector`).

The NIXL worker folds every KV cache layer that reports the same base address into one region (`register_kv_caches`, `if base_addr in seen_base_addresses`). In the block-outer HMA layout every cache group places its first page at byte 0 of the shared allocation, so on Qwen3.8-Flash-Next the PLE short-conv state (`Qwen4ExpPLELayer`, a tp-replicated `MambaSpec`, page = `channels × (conv_state_len + num_spec_tokens)` = 245760 B) lands in the same region as layer-0 GDN, the layer-3 QSA ring and the layer-3 MLA indexer cache, and that region keeps the MLA page length (204800 B). The PLE descriptors built from `block_len_per_layer[_ple_region_index]` therefore transfer only 204800/245760 bytes of each PLE slot: 8534 of 10240 channels arrive on the decode worker, the rest is whatever the block held before (never zeroed: the block is exempt from zeroing because an async load is expected to overwrite it). With `num_speculative_tokens >= 2` the spec-decode short conv reads those stale columns, the layer-1 hidden state becomes NaN, the target logits are NaN and the rejection sampler rejects every draft (the generated tokens are garbage too, not just slow).

This PR records the PLE page length at registration (`_ple_block_len`) and uses it for the PLE descriptors on both sides of the transfer (`_build_mamba_local` / `_build_mamba_remote`), exchanging it through `NixlAgentMetadata.ple_block_len` (optional, backward compatible) so the existing P/D geometry check still compares PLE pages. Region discovery is unchanged: the PLE keeps sharing its region with the pages it aliases, as `test_csa_linear_registration_discovers_shared_regions` expects; only the transfer length of its pages changes.

## Test Plan

- New unit test `tests/v1/kv_connector/unit/test_nixl_ple_region.py`: a CSA-linear-like config (MLA + GDN + ring + PLE sharing byte 0 of one block-outer tensor, PLE page larger than the MLA page); asserts the PLE descriptors cover the whole PLE page and that the page length is advertised in the handshake metadata. Before the fix the PLE descriptors carry the MLA page length.
- `tests/v1/kv_connector/unit/test_nixl_connector_hma.py` + `test_nixl_desc_geometry.py` run inside the serving image (vLLM `109b5363fc`) with the patch hot-patched: 307 passed before and after (one HF-download test deselected).
- End-to-end on GB200 (vLLM `109b5363fc` + this patch hot-patched into the image, Dynamo + `NixlConnector`, 1P/1D, `nvidia/Qwen3.8-Flash-Next-NVFP4`, MTP k=3, 200 requests of 64/64 tokens): mean acceptance length goes from 1.00 on 20/21 windows (zero accepted drafts) to 2.7–3.5 on every window, no NaN; the same configuration in aggregate mode gives \~2.1–2.4.

## Test Result

See above; details and the full investigation (per-request probes of the PLE conv state on prefill and decode, per-column comparison of the transferred slot) are in [#59642](https://github.com/aarnphm/aarnphm.github.io/issues/59642).

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---

## comments

**hmellor** (2026-10-06T14:16:29Z):

@claude review

**vadiklyutiy** (2026-10-06T14:43:40Z):

@claude review

**vadiklyutiy** (2026-10-06T14:44:25Z):

/ci run

**github-actions** (2026-10-06T14:45:02Z):

✅ Triggered [Buildkite CI #93103](https://buildkite.com/vllm/ci/builds/93103) for commit `2528cf240ed8`.

**vllm-agent** (2026-10-06T15:02:00Z):

<!-- ci-selector-shadow -->

### CI selector (shadow): 14 test steps (18 jobs) instead of 19 (25 jobs)

Shadow mode: this changes nothing about what CI runs. It shows what the evidence-based selector would pick for this PR, next to today’s rules. [How it works](https://github.com/vllm-project/ci-infra/tree/main/buildkite/ci_selector).

**Feedback welcome:** reply here if it would skip a step this change needs, or runs something unrelated.

| steps (jobs)           | Today’s rules | Selector | Would skip | Would add |
| ---------------------- | ------------- | -------- | ---------- | --------- |
| NVIDIA, CPU and others | 19 (25)       | 14 (18)  | 6 (8)      | 1 (1)     |
| AMD mirrors            | 28 (31)       | 13 (16)  | 16 (16)    | 1 (1)     |

<details><summary>Selector would run (14)</summary>

- `crosslayer-kv-layout-distributed-nixlconnector-pd-accuracy-tests-4-gpus`
- `distributed-flashinfer-nixlconnector-pd-accuracy-4-gpus`
- `distributed-nixlconnector-pd-accuracy-4-gpus`
- `dp-ep-distributed-nixlconnector-pd-accuracy-tests-4-gpus`
- `engine`
- `hybrid-ssm-nixlconnector-pd-accuracy-tests-4-gpus`
- `hybrid-ssm-nixlconnector-pd-prefix-cache-2-gpus`
- `kimi-k3-prefix-cache-4xb200` ×2
- `multiconnector-nixl-offloading-pd-accuracy-2-gpus`
- `multiconnector-nixl-offloading-pd-edge-cases-2-gpus`
- `nixlconnector-pd-edge-cases-2-gpus`
- `push-nixlconnector-pp-prefill-pd-accuracy-4-gpus`
- `v1-kv-connectors` ×4
- `v1-others-cpu`

</details>

<details><summary>Would skip (today's rules run them) (6)</summary>

- `ascend-npu-test`
- `cpu-language-generation-and-pooling-model-tests` ×3
- `v1-core`
- `v1-executor-worker`
- `v1-kv-offload`
- `v1-metrics-lmeval`

</details>

<details><summary>Would add (today's rules do not run them) (1)</summary>

- `engine` (code map)

</details>

<details><summary>AMD mirrors: would skip (16)</summary>

- `extract-hidden-states-integration-2-gpus`
- `fault-tolerance-e2e-2xh100`
- `fusion-and-compile-unit-tests-2xb200`
- `fusion-e2e-tp2-ar-rms-config-sweep-h100`
- `fusion-e2e-tp2-asynctp-config-sweep-h100`
- `fusion-e2e-tp2-b200`
- `fusion-e2e-tp2-quick-h100`
- `gemm-rs-ar-2xb200`
- `kernels-fusedmoe-layer-test-2-b200s`
- `kernels-fusedmoe-layer-test-2-h100s`
- `kernels-minimax-reduce-rms-test-2-gpus`
- `platform-tests`
- `v1-core`
- `v1-executor-worker`
- `v1-kv-offload`
- `v1-metrics-lmeval`

</details>

<details><summary>AMD mirrors: would add (1)</summary>

- `engine` (code map)

</details>

<sub>3 changed files · base `8bd7737a03` · head `2528cf240e` · Python record: build 93039 at `1e5d0ea888` · kernel record: table [`1e5d0ea`](https://github.com/aarnphm/aarnphm.github.io/commit/1e5d0ea888) (build 93039), map [`1e5d0ea`](https://github.com/aarnphm/aarnphm.github.io/commit/1e5d0ea888) · not counted: 10 build steps, 5 A100 steps the generator no longer emits, 11 optional steps the selector would also run</sub>

**vadiklyutiy** (2026-10-06T17:09:15Z):

/ci retry

**github-actions** (2026-10-06T17:09:36Z):

✅ Queued 1 failed job(s) for retry in [Buildkite CI #93103](https://buildkite.com/vllm/ci/builds/93103).

**vadiklyutiy** (2026-10-06T20:28:13Z):

/ci cancel

**github-actions** (2026-10-06T20:28:34Z):

✅ No cancelable CI build is running for branch `amassidda/fix-ple-conv-state-pd`.

**vadiklyutiy** (2026-10-06T20:29:13Z):

/ci retry

**github-actions** (2026-10-06T20:30:00Z):

✅ Queued 2 failed job(s) for retry in [Buildkite CI #93103](https://buildkite.com/vllm/ci/builds/93103).

**aijanai** (2026-10-07T06:36:58Z):

CI failures seem due to infra instability. All the others are green.

**aijanai** (2026-10-07T14:28:28Z):

@arpera it should be fine now

**arpera** (2026-10-08T16:07:54Z):

@NickLucche, please, have a look this PR if you have some bandwidth

**vadiklyutiy** (2026-10-08T16:09:44Z):

/ci run

**github-actions** (2026-10-08T16:11:59Z):

❌ This PR is 99 commits behind upstream `main`. Your branch must contain every commit currently on upstream `main`. No new CI build was started. Merge or rebase onto the latest `main`, then rerun `/ci run`. To test this branch at your own risk, use `/ci run --allow-stale`.

<!-- vllm-ci-command:6064064716 -->

**vadiklyutiy** (2026-10-08T16:12:40Z):

/ci run

**github-actions** (2026-10-08T16:13:04Z):

✅ Triggered [Buildkite CI #93641](https://buildkite.com/vllm/ci/builds/93641) for commit `891705ee328b`.

**arpera** (2026-10-08T16:52:30Z):

/ci run

**github-actions** (2026-10-08T16:52:59Z):

❌ This PR is 2 commits behind upstream `main`. Your branch must contain every commit currently on upstream `main`. No new CI build was started. Merge or rebase onto the latest `main`, then rerun `/ci run`. To test this branch at your own risk, use `/ci run --allow-stale`.

<!-- vllm-ci-command:6064808446 -->

**arpera** (2026-10-08T16:53:17Z):

/ci run

**github-actions** (2026-10-08T16:54:01Z):

✅ Triggered [Buildkite CI #93657](https://buildkite.com/vllm/ci/builds/93657) for commit `f1b1896e58ee`.

