---
title: '55027'
pageLayout: default
slug: thoughts/inference/vllm-prs/55027
permalink: https://aarnphm.xyz/thoughts/inference/vllm-prs/55027.md
generator:
  quartz: v4.6.0
  hostedProvider: Cloudflare
  baseUrl: aarnphm.xyz
full: https://aarnphm.xyz/llms-full.txt
---
# [#55027](https://github.com/aarnphm/aarnphm.github.io/issues/55027) \[Bugfix\]\[KV Connector\] MooncakeStore: exclude non-prefix-cacheable (QSA ring) groups; fix align-mode check

[vllm-project/vllm#55027](https://github.com/vllm-project/vllm/pull/55027) · zhewenl · MERGED · created 2026-09-02T22:12:22Z · merged 2026-09-14T02:18:16Z

## Purpose

Fix [#54726](https://github.com/aarnphm/aarnphm.github.io/issues/54726): Qwen3.8-Flash-Next with prefix caching and `MooncakeStoreConnector` fails because its QSA ring scratch cache is transferable, but is not valid hash-addressed prefix data.

The QSA cache returns a `CircularBufferSpec`; `CircularBufferSpec.prefix_cacheable` is explicitly `False`. Its capacity is 4 without speculative tokens and 8 with MTP=3, while aligned attention/Mamba groups use block size 800. MooncakeStore previously built its hash/store/lookup layout from every `enable_kv_transfer=True` group, so the ring reached a divisibility check intended only for storable prefix groups.

A second issue was hidden behind that failure: MooncakeStore inferred Mamba align mode by comparing each Mamba block size with the globally rewritten `cache_config.block_size`. With heterogeneous groups, that global value can differ from the resolved Mamba group size even when `mamba_cache_mode` is correctly `"align"`.

## Fix

- Add `KVCacheConfig.prefix_cacheable_group_ids` and `prefix_cacheable_groups` beside the existing transfer projection.
- Extend `KVCacheBlocks.get_block_ids(group_ids=...)` so scheduler consumers can request group projections without adding a generic selector to `KVCacheConfig`.
- Use only the prefix-cacheable transfer projection in MooncakeStore validation, coordinator, scheduler, and worker tensor-payload registration.
- Preserve scheduler-to-store group remapping so non-storable groups such as the QSA ring are skipped without shifting store group IDs.
- Validate Mamba support directly with `spec.mamba_cache_mode == "align"`.

This deliberately does **not** change `transfer_group_ids`, `transfer_groups`, or `select_transfer_block_ids`. NIXL and other P/D connectors continue to transfer all `enable_kv_transfer=True` groups, including the QSA ring scratch state.

## Duplicate-work check

Checked issue [#54726](https://github.com/aarnphm/aarnphm.github.io/issues/54726) and its comments, plus open PR searches for `54726 in:body` and `Mooncake QSA prefix cacheable`. The only matching open PR is this PR ([#55027](https://github.com/aarnphm/aarnphm.github.io/issues/55027)).

## Tests



```bash
.venv/bin/python -m pytest \
  tests/v1/core/test_kv_cache_utils.py \
  tests/v1/kv_connector/unit/test_mooncake_store_connector.py \
  tests/v1/kv_connector/unit/test_mooncake_store_coordinator.py \
  tests/v1/kv_connector/unit/test_mooncake_store_hma_e2e.py \
  tests/v1/kv_connector/unit/test_mooncake_store_scheduler.py \
  tests/v1/kv_connector/unit/test_mooncake_store_worker.py -q
```

Result on the current head: **412 passed, 14 warnings in 146.17s**.



```bash
.venv/bin/python -m pytest \
  tests/v1/kv_connector/unit/test_nixl_connector_hma.py -q
```

Result: **77 passed**. The remaining GPU integration case failed during connector initialization because NIXL is unavailable in the test environment; it did not reach block selection.



```bash
pre-commit run --files <all changed Python files>
```

Result: all applicable hooks passed, including Ruff and mypy.

## Model evaluation

- GSM8K 5-shot, thinking disabled, temperature 0, concurrency 512

Results:

| Pass                           | flexible-extract | strict-match |
| ------------------------------ | ---------------: | -----------: |
| First                          |           0.9682 |       0.9689 |
| After local prefix-cache reset |           0.9697 |       0.9704 |

All 2638 requests returned a non-empty answer, all finished with `stop`, and there were no API errors. After the reset, Mooncake performed 1416 Get calls covering 7092 keys / 72.59 GB, with zero failed keys and zero transfer errors. The original block-size assertion did not occur.

## AI assistance

AI coding agents (OpenAI Codex and Claude) assisted with analysis, implementation, testing, and review. The human submitter reviewed the changes. The commit includes DCO sign-off and AI co-author trailers.

---

## comments

**coderabbitai** (2026-09-02T22:12:31Z):

<!-- This is an auto-generated comment: summarize by coderabbit.ai -->

<!-- This is an auto-generated comment: skip review by coderabbit.ai -->

> \[!tip\]
>
> ## Draft PR not reviewed
>
> Draft PRs are not automatically reviewed by default.
>
> - [ ] <!-- {"checkboxId":"e9bb8d72-00e8-4f67-9cb2-caf3b22574fe"} --> Trigger a manual review
>
> To automatically review draft PRs, update your CodeRabbit configuration:
>
>
>
> ```yaml
> reviews:
>   auto_review:
>     drafts: true
> ```

<!-- end of auto-generated comment: skip review by coderabbit.ai -->

<!-- tips_start -->

---

Thanks for using [CodeRabbit](https://coderabbit.ai?utm_source=oss\&utm_medium=github\&utm_campaign=vllm-project/vllm\&utm_content=55027)! It’s free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

<details>
<summary>❤️ Share</summary>

- [X](https://twitter.com/intent/tweet?text=I%20just%20used%20%40coderabbitai%20for%20my%20code%20review%2C%20and%20it%27s%20fantastic%21%20It%27s%20free%20for%20OSS%20and%20offers%20a%20free%20trial%20for%20the%20proprietary%20code.%20Check%20it%20out%3A\&url=https%3A//coderabbit.ai)
- [Mastodon](https://mastodon.social/share?text=I%20just%20used%20%40coderabbitai%20for%20my%20code%20review%2C%20and%20it%27s%20fantastic%21%20It%27s%20free%20for%20OSS%20and%20offers%20a%20free%20trial%20for%20the%20proprietary%20code.%20Check%20it%20out%3A%20https%3A%2F%2Fcoderabbit.ai)
- [Reddit](https://www.reddit.com/submit?title=Great%20tool%20for%20code%20review%20-%20CodeRabbit\&text=I%20just%20used%20CodeRabbit%20for%20my%20code%20review%2C%20and%20it%27s%20fantastic%21%20It%27s%20free%20for%20OSS%20and%20offers%20a%20free%20trial%20for%20proprietary%20code.%20Check%20it%20out%3A%20https%3A//coderabbit.ai)
- [LinkedIn](https://www.linkedin.com/sharing/share-offsite/?url=https%3A%2F%2Fcoderabbit.ai\&mini=true\&title=Great%20tool%20for%20code%20review%20-%20CodeRabbit\&summary=I%20just%20used%20CodeRabbit%20for%20my%20code%20review%2C%20and%20it%27s%20fantastic%21%20It%27s%20free%20for%20OSS%20and%20offers%20a%20free%20trial%20for%20proprietary%20code)

</details>

<sub>Comment `@coderabbitai help` to get the list of available commands.</sub>

<!-- tips_end -->

**chatgpt-codex-connector** (2026-09-03T00:01:45Z):

<!-- codex-pull-request-review-summary -->

## Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

| Review             | Status                                                                                                            | Commit    | Review trigger     |
| ------------------ | ----------------------------------------------------------------------------------------------------------------- | --------- | ------------------ |
| 📝 **Code Review** | ✅ **Completed** <relative-time datetime="2026-09-03T00:05:40.294673Z">2026-09-03T00:05:40.294673Z</relative-time> | `f6db171` | Draft marked ready |

<details> <summary>ℹ️ About Codex in GitHub</summary>
<br/>

[Your team has set up Codex to review pull requests in this repo](https://chatgpt.com/codex/cloud/settings/general). Reviews are triggered when you

- Open a pull request for review
- Mark a draft as ready
- Comment “@codex review” or “@codex security review”.

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

</details>

**mergify** (2026-09-07T08:16:41Z):

This pull request has merge conflicts that must be resolved before it can be
merged. Please rebase the PR, @zhewenl.

<https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/syncing-a-fork>

**mergify** (2026-09-12T05:59:21Z):

This pull request has merge conflicts that must be resolved before it can be
merged. Please rebase the PR, @zhewenl.

<https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/syncing-a-fork>

**zhewenl** (2026-09-13T05:51:05Z):

/ci run

**github-actions** (2026-09-13T05:51:23Z):

✅ Triggered [Buildkite CI #88610](https://buildkite.com/vllm/ci/builds/88610) for commit `a6916cc7728d`.

**zhewenl** (2026-09-13T21:19:30Z):

/ci run

**github-actions** (2026-09-13T21:19:48Z):

✅ Triggered [Buildkite CI #88686](https://buildkite.com/vllm/ci/builds/88686) for commit `59f605639a69`.

