---
title: '58997'
pageLayout: default
slug: thoughts/inference/vllm-prs/58997
permalink: https://aarnphm.xyz/thoughts/inference/vllm-prs/58997.md
generator:
  quartz: v4.6.0
  hostedProvider: Cloudflare
  baseUrl: aarnphm.xyz
full: https://aarnphm.xyz/llms-full.txt
---
# [#58997](https://github.com/aarnphm/aarnphm.github.io/issues/58997) \[Core\] Remove deprecated mamba_cache_mode “all”

[vllm-project/vllm#58997](https://github.com/vllm-project/vllm/pull/58997) · ZJY0516 · MERGED · created 2026-09-28T08:49:52Z · merged 2026-10-01T09:59:38Z

**plan to merge this after v0.31 branch cut**

## Purpose

Remove `mamba_cache_mode="all"`, deprecated in [#55041](https://github.com/aarnphm/aarnphm.github.io/issues/55041) (v0.30.0). It is unsupported in MRV2 and never auto-selected (prefix caching defaults to `"align"`). `--mamba-cache-mode all` now fails validation.

This drops the `"all"`-only paths in config/platform, per-model guards, Mamba metadata builders, `MambaMixer`/`MambaMixer2`, the GPU model runner, `MambaSpec` sizing, and the NIXL/MoRIIO positional SSM transfer. `"none"` and `"align"` are unchanged.

Follow-ups (out of scope): the now-unused optional `block_idx_*` kernel args, and the `supports_mamba_prefix_caching` flag.

No existing PR removes this mode. Open PRs building on `"all"` ([#54637](https://github.com/aarnphm/aarnphm.github.io/issues/54637), [#50172](https://github.com/aarnphm/aarnphm.github.io/issues/50172), [#55260](https://github.com/aarnphm/aarnphm.github.io/issues/55260), [#54612](https://github.com/aarnphm/aarnphm.github.io/issues/54612), [#46048](https://github.com/aarnphm/aarnphm.github.io/issues/46048)) would no longer apply.

## Test Plan

- `pre-commit`, mypy
- Unit tests for the touched config, KV cache manager, scheduler, Mamba metadata, worker, and connector suites
- `tests/models/language/generation/test_hybrid.py -k apc`
- gsm8k with prefix caching, main vs this branch

`"all"`-only tests are removed; the two hybrid scheduler tests are ported to `"align"` with unchanged expectations; the NIXL Mamba prefix-cache script now uses `align`.

## Test Result

- pre-commit, mypy: pass
- Unit tests for the touched suites: 2365 passed. The only failures are 2 `test_cpu_offloading` Llama-3.2-1B (FLASH_ATTN / FLASHINFER) cases on a CPU-vs-cold timing assertion, which fail identically on the base commit.
- `test_hybrid.py -k apc` (Jamba, falcon-mamba): 7 passed

gsm8k, 5-shot, `--limit 250`, `enable_prefix_caching=True` (align), GB200:

| Model                        | main (strict / flexible) | this PR (strict / flexible) |
| ---------------------------- | ------------------------ | --------------------------- |
| granite-4.0-h-small (Mamba2) | 0.868 / 0.868            | 0.852 / 0.852               |
| Qwen3.5-35B-A3B-FP8 (GDN)    | 0.744 / 0.760            | 0.808 / 0.824               |

Differences are within run-to-run noise (stderr \~0.02-0.03).

AI assistance (Claude Code) was used; I have reviewed the changes and run the tests above.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---

## comments

**mergify** (2026-09-28T18:26:59Z):

This pull request has merge conflicts that must be resolved before it can be
merged. Please rebase the PR, @ZJY0516.

<https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/syncing-a-fork>

**tdoublep** (2026-09-29T14:54:56Z):

/ci run

**github-actions** (2026-09-29T14:55:17Z):

❌ This PR is 47 commits behind upstream `main`. Your branch must contain every commit currently on upstream `main`. No new CI build was started. Merge or rebase onto the latest `main`, then rerun `/ci run`. To test this branch at your own risk, use `/ci run --allow-stale`.

<!-- vllm-ci-command:5892763090 -->

**tdoublep** (2026-09-29T14:56:39Z):

/ci run

**github-actions** (2026-09-29T14:57:04Z):

✅ Triggered [Buildkite CI #91874](https://buildkite.com/vllm/ci/builds/91874) for commit `b210532a4297`.

**mergify** (2026-09-29T20:20:54Z):

This pull request has merge conflicts that must be resolved before it can be
merged. Please rebase the PR, @ZJY0516.

<https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/syncing-a-fork>

**ZJY0516** (2026-09-30T14:20:22Z):

/ci run —allow-stale

**github-actions** (2026-09-30T14:21:55Z):

✅ Triggered [Buildkite CI #92167](https://buildkite.com/vllm/ci/builds/92167) for commit `6b3cff969a44`.

**vllm-agent** (2026-09-30T15:09:40Z):

<!-- ci-selector-shadow -->

### CI selector (shadow): 130 test steps (205 jobs) instead of 111 (156 jobs)

Shadow mode: this changes nothing about what CI runs. It shows what the evidence-based selector would pick for this PR, next to today’s rules. [How it works](https://github.com/vllm-project/ci-infra/tree/main/buildkite/ci_selector).

**Feedback welcome:** reply here if it would skip a step this change needs, or runs something unrelated.

| steps (jobs)           | Today’s rules | Selector  | Would skip | Would add |
| ---------------------- | ------------- | --------- | ---------- | --------- |
| NVIDIA, CPU and others | 111 (156)     | 130 (205) | 26 (26)    | 45 (75)   |
| AMD mirrors            | 113 (165)     | 157 (224) | 0 (0)      | 44 (59)   |

<details><summary>Selector would run (130)</summary>

- `amd-fp8-moe-kernels-mi355`
- `amd-kernels-mi355`
- `amd-lm-eval-small-models-harness`
- `amd-native-quantization-kernels-mi355`
- `arm-cpu-test` ×3
- `async-engine-inputs-utils-worker`
- `basic-correctness` ×2
- `basic-correctness-cpu-offload`
- `basic-correctness-sleep-mode`
- `basic-models-test-other-cpu`
- `basic-models-tests-extra-initialization` ×14
- `basic-models-tests-initialization`
- `basic-models-tests-other`
- `batch-invariance-b200`
- `batch-invariance-h100`
- `cpu-kernel-tests` ×2
- `cpu-language-generation-and-pooling-model-tests` ×3
- `cpu-multi-modal-model-tests-n` ×4
- `cpu-multimodal-config`
- `cpu-params-env-tokenizers-parser`
- `cpu-quantization-model-tests`
- `cpu-qwen2-5-vl-multimodal-tests`
- `cpu-reasoning-renderers`
- `cpu-spec-decode-tests`
- `cpu-tool-parsers`
- `crosslayer-kv-layout-distributed-nixlconnector-pd-accuracy-tests-4-gpus`
- `cudagraph`
- `deepseek-v4-kernel-test-b200`
- `distributed-comm-ops`
- `distributed-compile-comm-4-gpus`
- `distributed-dp-tests-2-gpus`
- `distributed-dp-tests-4-gpus`
- `distributed-flashinfer-nixlconnector-pd-accuracy-4-gpus`
- `distributed-model-tests-2-gpus` ×3
- `distributed-mooncakeconnector-pd-accuracy-4-gpus`
- `distributed-nixlconnector-pd-accuracy-4-gpus`
- `distributed-torchrun-examples-4-gpus`
- `distributed-torchrun-shutdown-tests-2-gpus`
- `dp-ep-distributed-nixlconnector-pd-accuracy-tests-4-gpus`
- `e2e-core-1-gpu`
- `e2e-core-large-memory`
- `e2e-scheduling-1-gpu`
- `elastic-ep-scaling-test`
- `engine`
- `engine-1-gpu`
- `entrypoints-integration-api-server` ×4
- `entrypoints-integration-api-server-openai-completion`
- `entrypoints-integration-pooling`
- `entrypoints-integration-speech_to_text`
- `eplb-algorithm`
- `eplb-execution`
- `examples`
- `extract-hidden-states-integration`
- `extract-hidden-states-integration-2-gpus`
- `fault-tolerance-e2e-2xh100`
- `fusion-and-compile-unit-tests-2xb200`
- `glm5next-unit-tests`
- `hybrid-ssm-nixlconnector-pd-accuracy-tests-4-gpus`
- `hybrid-ssm-nixlconnector-pd-prefix-cache-2-gpus`
- `inkling-unit-tests-b200`
- `kernels-attention-test` ×7
- `kernels-b200` ×3
- `kernels-core-operation-test` ×3
- `kernels-deepgemm-test-h100`
- `kernels-fla-ops-test-b200`
- `kernels-flashmla-test-h100`
- `kernels-helion-test` ×5
- `kernels-mamba-test`
- `kernels-moe-test` ×5
- `kernels-quantization-test` ×6
- `kernels-root-misc-test-b200`
- `kimi-k3-unit-tests-b200`
- `kv-offload-large`
- `kv-offload-medium`
- `kv-offload-small`
- `language-models-tests-extra-standard` ×2
- `language-models-tests-granite-l4-compatibility`
- `language-models-tests-hybrid` ×2
- `language-models-tests-standard`
- `lm-eval-watermarking`
- `lora` ×4
- `lora-tp-distributed` ×4
- `metrics-tracing-2-gpus`
- `model-executor`
- `mrcr-eval-small-models`
- `multi-modal-models-standard-1-qwen2`
- `multi-modal-models-standard-2-qwen3-gemma`
- `multi-modal-models-standard-3-llava-qwen2-vl`
- `multi-modal-models-standard-4-other-whisper`
- `multi-modal-processor` ×4
- `multi-modal-processor-cpu` ×4
- `multiconnector-nixl-offloading-pd-accuracy-2-gpus`
- `multiconnector-nixl-offloading-pd-edge-cases-2-gpus`
- `nixlconnector-pd-edge-cases-2-gpus`
- `pipeline-context-parallelism-4-gpus`
- `platform-tests`
- `plugin-tests-2-gpus`
- `push-nixlconnector-pp-prefill-pd-accuracy-4-gpus`
- `pytorch-compilation-passes-unit-tests`
- `pytorch-compilation-unit-tests`
- `pytorch-fullgraph-cudagraph-l4-compatibility`
- `pytorch-fullgraph-test`
- `quantization` ×4
- `quantized-models-test`
- `quantized-moe-test-b200`
- `qwen4-exp-unit-tests`
- `qwen4-exp-unit-tests-cpu`
- `rayexecutorv2-4-gpus`
- `rust-frontend-distributed`
- `rust-frontend-openai-coverage`
- `sharded-rdt-weight-transfer`
- `spec-decode-draft-model` ×4
- `spec-decode-eagle-1-deepseek-qwen`
- `spec-decode-eagle-2-llama3-qwen-vl-other`
- `spec-decode-mtp-deepseek-mimo`
- `spec-decode-mtp-gemma4`
- `spec-decode-mtp-qwen3-5`
- `spec-decode-ngram-suffix`
- `spec-decode-speculators`
- `v1-attention-b200` ×2
- `v1-attention-h100-mi300` ×2
- `v1-core`
- `v1-executor-worker`
- `v1-kv-connectors` ×4
- `v1-kv-offload`
- `v1-logits-oracle`
- `v1-metrics-lmeval`
- `v1-others-cpu`
- `v1-sample`
- `v1-spec-decode`

</details>

<details><summary>Would skip (today's rules run them) (26)</summary>

- `ascend-npu-test`
- `basic-correctness-cumem`
- `basic-correctness-prefetch-offload`
- `benchmarks-cli-test`
- `cpu-distributed-tests-dp-tp`
- `cpu-distributed-tests-pp-tp`
- `distributed-compile-rpc-tests-2-gpus`
- `distributed-compile-unit-tests-2xh100`
- `entrypoints-integration-api-server-generate`
- `entrypoints-integration-api-server-openai-chat_completion`
- `entrypoints-integration-llm`
- `entrypoints-integration-multimodal`
- `entrypoints-integration-responses-api`
- `fusion-e2e-quick-h100`
- `fusion-e2e-tp2-b200`
- `fusion-e2e-tp2-quick-h100`
- `kernels-fusedmoe-layer-test-2-b200s`
- `kernels-fusedmoe-layer-test-2-h100s`
- `kernels-mhc-test-b200`
- `lm-eval-dspark-watermark-2xh100`
- `mooncake-ec-tcp-e2e-2-gpus`
- `pytorch-compilation-dynamic-shapes`
- `pytorch-compilation-unit-tests-h100`
- `regression`
- `samplers-multimodal-beam-search`
- `samplers-test`

</details>

<details><summary>Would add (today's rules do not run them) (45)</summary>

- `amd-fp8-moe-kernels-mi355` (code map)
- `amd-kernels-mi355` (code map)
- `amd-native-quantization-kernels-mi355` (code map)
- `arm-cpu-test` ×3 (code map)
- `cpu-multi-modal-model-tests-n` ×4 (code map)
- `cpu-quantization-model-tests` (code map)
- `cpu-qwen2-5-vl-multimodal-tests` (code map)
- `cudagraph` (code map)
- `deepseek-v4-kernel-test-b200` (code map)
- `distributed-comm-ops` (code map)
- `distributed-compile-comm-4-gpus` (code map)
- `distributed-dp-tests-4-gpus` (code map)
- `distributed-mooncakeconnector-pd-accuracy-4-gpus` (code map)
- `distributed-torchrun-examples-4-gpus` (code map)
- `elastic-ep-scaling-test` (code map)
- `engine-1-gpu` (code map)
- `eplb-algorithm` (code map)
- `eplb-execution` (code map)
- `examples` (code map)
- `extract-hidden-states-integration` (code map)
- `extract-hidden-states-integration-2-gpus` (code map)
- `fusion-and-compile-unit-tests-2xb200` (code map)
- `glm5next-unit-tests` (code map)
- `inkling-unit-tests-b200` (code map)
- `kernels-b200` ×3 (code map)
- `kernels-core-operation-test` ×3 (code map)
- `kernels-deepgemm-test-h100` (code map)
- `kernels-flashmla-test-h100` (code map)
- `kernels-helion-test` ×5 (code map)
- `kernels-quantization-test` ×6 (code map)
- `lm-eval-watermarking` (code map)
- `lora` ×4 (code map)
- `lora-tp-distributed` ×4 (code map)
- `mrcr-eval-small-models` (code map)
- `plugin-tests-2-gpus` (code map)
- `quantization` ×4 (code map)
- `quantized-models-test` (code map)
- `quantized-moe-test-b200` (code map)
- `rayexecutorv2-4-gpus` (code map)
- `rust-frontend-openai-coverage` (code map)
- `sharded-rdt-weight-transfer` (code map)
- `spec-decode-draft-model` ×4 (code map)
- `spec-decode-eagle-1-deepseek-qwen` (code map)
- `spec-decode-eagle-2-llama3-qwen-vl-other` (code map)
- `spec-decode-ngram-suffix` (code map)

</details>

<details><summary>AMD mirrors: would skip (0)</summary>

none

</details>

<details><summary>AMD mirrors: would add (44)</summary>

- `cudagraph` (code map)
- `deepseek-v4-kernel-test-h100` (code map)
- `distributed-mooncakeconnector-pd-accuracy-4-gpus` (code map)
- `distributed-tests-8xh100` (code map)
- `distributed-torchrun-examples-4-gpus` (code map)
- `docker-build-metadata` (code map)
- `engine-1-gpu` (code map)
- `entrypoints-unit-tests` (code map)
- `eplb-algorithm` (code map)
- `eplb-execution` (code map)
- `examples` (code map)
- `extract-hidden-states-integration` (code map)
- `fusion-e2e-config-sweep-h100` (code map)
- `glm5next-unit-tests` (code map)
- `inkling-unit-tests-b200` (code map)
- `jit-monitor-no-runtime-jit` (code map)
- `kernels-attention-diffkv-test-h100` (code map)
- `kernels-core-operation-test` ×3 (code map)
- `kernels-flashmla-test-h100` (code map)
- `kernels-helion-test` ×5 (code map)
- `lm-eval-turboquant-k3v4nc` (code map)
- `lm-eval-turboquant-k8v4` (code map)
- `lm-eval-turboquant-t3nc` (code map)
- `lm-eval-turboquant-t4nc` (code map)
- `lm-eval-watermarking` (code map)
- `lora` ×4 (code map)
- `lora-tp-distributed` ×4 (code map)
- `multi-modal-accuracy-eval-small-models` (code map)
- `plugin-tests-2-gpus` (code map)
- `pytorch-nightly-dependency-override-check` (code map)
- `quantization` ×4 (code map)
- `quantized-fusions` (code map)
- `quantized-models-test` (code map)
- `ray-dependency-compatibility-check` (code map)
- `rust-frontend-cargo-style-clippy` (code map)
- `rust-frontend-cargo-tests` (code map)
- `rust-frontend-core-correctness` (code map)
- `rust-frontend-openai-coverage` (code map)
- `rust-frontend-serve-admin-coverage` (code map)
- `rust-frontend-tool-use` (code map)
- `scale-out-ec-e2e-2-gpus` (code map)
- `sharded-rdt-weight-transfer` (code map)
- `torch-stable-abi-audit` (code map)
- `vllm-ir-tests` (code map)

</details>

#### CI results (2026-09-30 15:54 UTC)

279 passed, 1 failed, 3 pending.
CI is still running; the picture below is not final.

No misses: the selector would have run every failed job.

- `amd-mi355-dpx-entrypoints-integration-api-serve`: selector runs it

<sub>44 changed files · base `2e52a558f0` · head `6b3cff969a` · Python record: not used (/tmp/ci-infra-selector/buildkite/ci_selector/coverage-data/table.json.gz is table version 5, expected 7; re-merge it from the raw recordings) · kernel record: table [`866fa13`](https://github.com/aarnphm/aarnphm.github.io/commit/866fa130fa) (build 92059), map [`866fa13`](https://github.com/aarnphm/aarnphm.github.io/commit/866fa130fa) · not counted: 10 build steps, 5 A100 steps the generator no longer emits, 91 optional steps the selector would also run</sub>

**mergify** (2026-09-30T22:46:29Z):

This pull request has merge conflicts that must be resolved before it can be
merged. Please rebase the PR, @ZJY0516.

<https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/syncing-a-fork>

**ZJY0516** (2026-10-01T04:00:21Z):

/ci run —allow-stale

**github-actions** (2026-10-01T04:00:40Z):

✅ Triggered [Buildkite CI #92333](https://buildkite.com/vllm/ci/builds/92333) for commit `99c74238515d`.

⚠️ This PR is 1 commit behind upstream `main`. Running CI at your own risk because `--allow-stale` was requested; outdated CI configuration may cause failures. Before merging, merge or rebase onto the latest `main`, then rerun `/ci run` on the latest PR commit.

**ZJY0516** (2026-10-01T06:36:14Z):

/ci run —allow-stale

**github-actions** (2026-10-01T06:36:34Z):

✅ Triggered [Buildkite CI #92359](https://buildkite.com/vllm/ci/builds/92359) for commit `9d760affe9f9`.

