---
title: '44456'
pageLayout: default
slug: thoughts/inference/vllm-prs/44456
permalink: https://aarnphm.xyz/thoughts/inference/vllm-prs/44456.md
generator:
  quartz: v4.6.0
  hostedProvider: Cloudflare
  baseUrl: aarnphm.xyz
full: https://aarnphm.xyz/llms-full.txt
---
# [#44456](https://github.com/aarnphm/aarnphm.github.io/issues/44456) \[3/N\]\[KV-Cache Layout Refactor\] Standardize Mamba cache; drop `get_transfer_cache_regions`

[vllm-project/vllm#44456](https://github.com/vllm-project/vllm/pull/44456) · LucasWilkinson · MERGED · created 2026-06-03T22:22:37Z · merged 2026-07-21T09:16:16Z

PR [#42374](https://github.com/aarnphm/aarnphm.github.io/issues/42374) (first part of RFC [#42082](https://github.com/aarnphm/aarnphm.github.io/issues/42082)) has been split into 4 PRs:

[#44454](https://github.com/aarnphm/aarnphm.github.io/issues/44454) \[1/N\]\[KV-Cache Layout Refactor\] Refactor DSV4 KV cache config
[#44455](https://github.com/aarnphm/aarnphm.github.io/issues/44455) \[2/N\]\[KV-Cache Layout Refactor\] Pack K/V into the content dim across attention backends
<span>&rarr;</span> [#44456](https://github.com/aarnphm/aarnphm.github.io/issues/44456) \[3/N\]\[KV-Cache Layout Refactor\] Standardize Mamba cache; drop `get_transfer_cache_regions`
[#44458](https://github.com/aarnphm/aarnphm.github.io/issues/44458) \[4/N\]\[KV-Cache Layout Refactor\] Standardize KV cache layout

## Summary

Move Mamba (and conv/ssm) reshaping into a new `bind_kv_cache` hook; drop the now-unnecessary `TransferTopology.get_transfer_cache_regions` from the connector. This is an intermediate step toward [#42374](https://github.com/aarnphm/aarnphm.github.io/issues/42374) that does **not** yet introduce the cross-layer `L` dim.

Stacked on top of [#44455](https://github.com/aarnphm/aarnphm.github.io/issues/44455) (pack K/V into the content dim).

## Testing

Mamba `bind_kv_cache` unpacking and the single-tensor NIXL/mooncake Mamba registration require GPU + multi-node P/D setups to exercise (hybrid attention+Mamba models hit both bind paths). The submitter will validate these on appropriate hardware before merge.

## AI assistance

This PR was prepared with AI assistance (Claude).

---

## comments

**mergify** (2026-06-05T15:49:17Z):

This pull request has merge conflicts that must be resolved before it can be
merged. Please rebase the PR, @LucasWilkinson.

<https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/syncing-a-fork>

**mergify** (2026-06-12T22:00:33Z):

This pull request has merge conflicts that must be resolved before it can be
merged. Please rebase the PR, @LucasWilkinson.

<https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/syncing-a-fork>

**mergify** (2026-06-22T19:34:55Z):

This pull request has merge conflicts that must be resolved before it can be
merged. Please rebase the PR, @LucasWilkinson.

<https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/syncing-a-fork>

**mergify** (2026-07-12T02:50:14Z):

Hi @LucasWilkinson, the pre-commit checks have failed. Please run:



```bash
uv pip install pre-commit>=4.5.1
pre-commit install
pre-commit run --all-files
```

Then, commit the changes and push to your branch.

For future commits, `pre-commit` will run automatically on changed files before each commit.

**mergify** (2026-07-12T04:31:38Z):

Hi @LucasWilkinson, the pre-commit checks have failed. Please run:



```bash
uv pip install pre-commit>=4.5.1
pre-commit install
pre-commit run --all-files
```

Then, commit the changes and push to your branch.

For future commits, `pre-commit` will run automatically on changed files before each commit.

**mergify** (2026-07-16T05:18:04Z):

This pull request has merge conflicts that must be resolved before it can be
merged. Please rebase the PR, @LucasWilkinson.

<https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/syncing-a-fork>

