---
title: '53664'
pageLayout: default
slug: thoughts/inference/vllm-prs/53664
permalink: https://aarnphm.xyz/thoughts/inference/vllm-prs/53664.md
generator:
  quartz: v4.6.0
  hostedProvider: Cloudflare
  baseUrl: aarnphm.xyz
full: https://aarnphm.xyz/llms-full.txt
---
# [#53664](https://github.com/aarnphm/aarnphm.github.io/issues/53664) \[Rocm\]\[Kimi-k3\] Fix pipeline_parallel support for the kimik3 DCP mode

[vllm-project/vllm#53664](https://github.com/vllm-project/vllm/pull/53664) · haic0 · MERGED · created 2026-08-25T02:48:32Z · merged 2026-09-09T14:50:25Z

## Summary

- Assert pipeline-parallel capability for both Kimi-K3 model architectures.
- Derive DCP-local MLA decode sequence lengths when `CommonAttentionMetadata` does not carry them, which keeps PP + DCP decode metadata complete.

## Scope cleanup

This revision removes the shared per-group cache geometry changes and tests that landed in [#53598](https://github.com/aarnphm/aarnphm.github.io/issues/53598). It also drops the older ROCm AITER MLA backend changes superseded by [#51705](https://github.com/aarnphm/aarnphm.github.io/issues/51705). The PR now contains only two PP-specific files and 12 added lines.

Related open work is disjoint: [#54347](https://github.com/aarnphm/aarnphm.github.io/issues/54347) enables sequence parallelism across PP stages, and [#54635](https://github.com/aarnphm/aarnphm.github.io/issues/54635) adds PP capability to MTP draft classes.

## Validation

- `uvx --from pre-commit>=4.5.1 pre-commit run --files tests/models/test_registry.py vllm/model_executor/layers/attention/mla_attention.py` — passed, including Ruff, mypy, forbidden-import, SPDX, and configuration checks.
- `git diff --check upstream/main...HEAD` — passed.
- The earlier runtime patch set was validated with PP2 x TP4 x DCP4 serving and GSM8K 5-shot (`0.9545` flexible exact match, `0.9530` strict exact match). The hardware model evaluation was not rerun for this scope-only rebase.

AI assistance was used for overlap analysis, rebasing, and validation orchestration. The submitter must review and understand every retained line before merge.

---

## comments

**mergify** (2026-08-25T07:04:05Z):

Hi @haic0, the pre-commit checks have failed. Please run:



```bash
uv pip install pre-commit>=4.5.1
pre-commit install
pre-commit run --all-files
```

Then, commit the changes and push to your branch.

For future commits, `pre-commit` will run automatically on changed files before each commit.

**mergify** (2026-09-07T07:40:45Z):

Hi @haic0, the pre-commit checks have failed. Please run:



```bash
uv pip install pre-commit>=4.5.1
pre-commit install
pre-commit run --all-files
```

Then, commit the changes and push to your branch.

For future commits, `pre-commit` will run automatically on changed files before each commit.

**haic0** (2026-09-07T07:42:11Z):

@GirasoleY
Addressed the scope feedback. I rebased onto current `main` and removed the shared cache geometry changes and tests now provided by [#53598](https://github.com/aarnphm/aarnphm.github.io/issues/53598) (`kv_cache_utils.py`, `kv_cache_coordinator.py`, `test_kv_cache_utils.py`, and `test_prefix_caching.py`). I also dropped the old ROCm AITER MLA backend patch superseded by [#51705](https://github.com/aarnphm/aarnphm.github.io/issues/51705) and the unrelated GDN contiguity tweak.

The PR is now limited to two PP-specific changes: Kimi-K3 PP capability coverage and the missing DCP-local MLA sequence-length fallback.

**coderabbitai** (2026-09-07T07:42:28Z):

<!-- This is an auto-generated comment: summarize by coderabbit.ai -->

<!-- review_stack_entry_start -->

[![Review Change Stack](https://storage.googleapis.com/coderabbit_public_assets/review-stack-in-coderabbit-ui.svg)](https://app.coderabbit.ai/change-stack/vllm-project/vllm/pull/53664)

<!-- review_stack_entry_end -->

<!-- recent_review_start -->

No actionable comments were generated in the recent review. 🎉

<details>
<summary>ℹ️ Recent review info</summary>

<details>
<summary>⚙️ Run configuration</summary>

**Configuration used**: Repository UI

**Review profile**: CHILL

**Plan**: Team

**Run ID**: `91d12baf-18d8-4c1d-9ff2-2394d4ba39fe`

</details>

<details>
<summary>📥 Commits</summary>

Reviewing files that changed from the base of the PR and between [`5893426`](https://github.com/aarnphm/aarnphm.github.io/commit/5893426b88f7b3cd21101d194eb1c6f0a6f0e27b) and [`e0a7e5a`](https://github.com/aarnphm/aarnphm.github.io/commit/e0a7e5a9f8fbf9b0d1ce8f31e10bfc5081b6e9a5).

</details>

<details>
<summary>📒 Files selected for processing (2)</summary>

- `tests/models/test_registry.py`
- `vllm/model_executor/layers/attention/mla_attention.py`

</details>

**Included review availability:** Your plan provides up to 10 included reviews per hour; 9 remain after this review.

</details>

---

<!-- recent_review_end -->

<!-- walkthrough_start -->

## Walkthrough

The changes add a DCP decode fallback for local sequence lengths and extend pipeline-parallel registry coverage to two Kimi model classes.

### Changes

**DCP metadata handling**

| Layer / File(s)                                                                                     | Summary                                                                                                                                                                          |
| --------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| **DCP local sequence-length fallback** <br> `vllm/model_executor/layers/attention/mla_attention.py` | DCP decode construction now asserts that `seq_lens` exists and derives per-rank local sequence lengths with `get_dcp_local_seq_lens` when common metadata does not provide them. |

**Pipeline-parallel registry coverage**

| Layer / File(s)                                                               | Summary                                                                                                                                         |
| ----------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------- |
| **Kimi pipeline-parallel test coverage** <br> `tests/models/test_registry.py` | The parametrized registry test now covers `KimiLinearForCausalLM` and `KimiK3ForConditionalGeneration` with `is_pp=True` and `init_cuda=False`. |

**Estimated code review effort:** 2 (Simple) | \~10 minutes

<!-- final_review_risk_start -->

**Merge Risk:** _⚪ Minimal_ · up to `e0a7e`

<!-- final_review_risk_coverage:{"sourceCommitId":"e0a7e5a9f8fbf9b0d1ce8f31e10bfc5081b6e9a5","coveredCommitId":"e0a7e5a9f8fbf9b0d1ce8f31e10bfc5081b6e9a5","kind":"reviewed"} -->

This change enables Kimi-K3 pipeline-parallel coverage and preserves DCP decode metadata when local lengths are absent. The available checks and runtime validation indicate no remaining merge-blocking risk.

<!-- final_review_risk_end -->

**Suggested reviewers:** `isotr0py`, `zjy0516`

<!-- walkthrough_end -->

<!-- pre_merge_checks_walkthrough_start -->

<details>
<summary>🚥 Pre-merge checks | ✅ 4 | ❌ 1</summary>

### ❌ Failed checks (1 warning)

|     Check name     | Status     | Explanation                                                                                                                                                                               | Resolution                                                                         |
| :----------------: | :--------- | :---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | :--------------------------------------------------------------------------------- |
| Docstring Coverage | ⚠️ Warning | Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 1 functions across 2 files. | Write docstrings for the functions missing them to satisfy the coverage threshold. |

<details>
<summary>✅ Passed checks (4 passed)</summary>

|         Check name         | Status   | Explanation                                                                                                                              |
| :------------------------: | :------- | :--------------------------------------------------------------------------------------------------------------------------------------- |
|     Linked Issues check    | ✅ Passed | Check skipped because no linked issues were found for this pull request.                                                                 |
| Out of Scope Changes check | ✅ Passed | Check skipped because no linked issues were found for this pull request.                                                                 |
|         Title check        | ✅ Passed | The title clearly identifies the main change: adding pipeline-parallel support for the Kimi-K3 model.                                    |
|      Description check     | ✅ Passed | The description accurately covers the pipeline-parallel capability assertions and DCP-local sequence-length derivation in the changeset. |

</details>

</details>

<!-- pre_merge_checks_walkthrough_end -->

<!-- tips_start -->

---

Thanks for using [CodeRabbit](https://coderabbit.ai?utm_source=oss\&utm_medium=github\&utm_campaign=vllm-project/vllm\&utm_content=53664)! It’s free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

<details>
<summary>❤️ Share</summary>

- [X](https://twitter.com/intent/tweet?text=I%20just%20used%20%40coderabbitai%20for%20my%20code%20review%2C%20and%20it%27s%20fantastic%21%20It%27s%20free%20for%20OSS%20and%20offers%20a%20free%20trial%20for%20the%20proprietary%20code.%20Check%20it%20out%3A\&url=https%3A//coderabbit.ai)
- [Mastodon](https://mastodon.social/share?text=I%20just%20used%20%40coderabbitai%20for%20my%20code%20review%2C%20and%20it%27s%20fantastic%21%20It%27s%20free%20for%20OSS%20and%20offers%20a%20free%20trial%20for%20the%20proprietary%20code.%20Check%20it%20out%3A%20https%3A%2F%2Fcoderabbit.ai)
- [Reddit](https://www.reddit.com/submit?title=Great%20tool%20for%20code%20review%20-%20CodeRabbit\&text=I%20just%20used%20CodeRabbit%20for%20my%20code%20review%2C%20and%20it%27s%20fantastic%21%20It%27s%20free%20for%20OSS%20and%20offers%20a%20free%20trial%20for%20proprietary%20code.%20Check%20it%20out%3A%20https%3A//coderabbit.ai)
- [LinkedIn](https://www.linkedin.com/sharing/share-offsite/?url=https%3A%2F%2Fcoderabbit.ai\&mini=true\&title=Great%20tool%20for%20code%20review%20-%20CodeRabbit\&summary=I%20just%20used%20CodeRabbit%20for%20my%20code%20review%2C%20and%20it%27s%20fantastic%21%20It%27s%20free%20for%20OSS%20and%20offers%20a%20free%20trial%20for%20proprietary%20code)

</details>

<sub>Comment `@coderabbitai help` to get the list of available commands.</sub>

<!-- tips_end -->

**shen-shanshan** (2026-09-08T09:21:52Z):

/ci run

**github-actions** (2026-09-08T09:23:57Z):

✅ Triggered [Buildkite CI #87663](https://buildkite.com/vllm/ci/builds/87663) for commit `e0a7e5a9f8fb`.

**github-actions** (2026-09-08T09:24:06Z):

✅ @haic0, CI is now available for this PR.

- `/ci run` starts upstream CI; `/amd-ci run` starts AMD CI only.
- `/ci retry` retries failed jobs in the CI build for the current PR head. If the current head has no CI build, it starts a new CI build for the current head containing only jobs that failed in the latest earlier CI build for this PR.
- `/amd-ci retry` retries failed jobs in AMD CI for the current PR head. Use `/amd-ci run` when the current head has no AMD CI build.
- `/ci cancel` cancels scheduled or running CI builds for this PR branch; `/amd-ci cancel` does the same for AMD CI only.

<!-- vllm-ci-authorized -->

**shen-shanshan** (2026-09-08T15:08:41Z):

/ci run

**github-actions** (2026-09-08T15:09:01Z):

✅ Triggered [Buildkite CI #87718](https://buildkite.com/vllm/ci/builds/87718) for commit `284f5277e223`.

**shen-shanshan** (2026-09-09T00:59:15Z):

/ci retry

**github-actions** (2026-09-09T00:59:33Z):

✅ Queued 4 failed job(s) for retry in [Buildkite CI #87718](https://buildkite.com/vllm/ci/builds/87718).

**haic0** (2026-09-09T06:38:15Z):

/ci retry

**github-actions** (2026-09-09T06:38:34Z):

✅ Triggered [Buildkite CI #87851](https://buildkite.com/vllm/ci/builds/87851) for commit `f0dddeae11de`, running 3 failed step(s) from [Buildkite CI #87718](https://buildkite.com/vllm/ci/builds/87718).

**haic0** (2026-09-09T06:54:34Z):

/ci run

**github-actions** (2026-09-09T06:54:54Z):

✅ Triggered [Buildkite CI #87855](https://buildkite.com/vllm/ci/builds/87855) for commit `f0dddeae11de`.

