---
title: '49612'
pageLayout: default
slug: thoughts/inference/vllm-prs/49612
permalink: https://aarnphm.xyz/thoughts/inference/vllm-prs/49612.md
generator:
  quartz: v4.6.0
  hostedProvider: Cloudflare
  baseUrl: aarnphm.xyz
full: https://aarnphm.xyz/llms-full.txt
---
# [#49612](https://github.com/aarnphm/aarnphm.github.io/issues/49612) \[KV Connector\] Support NIXL heterogeneous P/D block sizes for hybrid models

[vllm-project/vllm#49612](https://github.com/vllm-project/vllm/pull/49612) · njhill · MERGED · created 2026-07-23T16:06:20Z · merged 2026-07-28T13:45:03Z

Hybrid (mamba) models previously asserted out heterogeneous block sizes entirely — yet they are the models where P/D block sizes most readily diverge, since the mamba-padded attention block size varies with TP sharding ([#41037](https://github.com/aarnphm/aarnphm.github.io/issues/41037)). Lift the restriction:

- Mamba state blocks are indivisible, so their descriptors always use local page geometry and their desc ids are never ratio-expanded; attention descriptors remain expanded to remote-block granularity.
- A shared per-group mapping (pull READ and push WRITE) expands attention groups to remote granularity and clips to the transferred coverage, while mamba groups map 1:1.
- The receive post-process permutes only attention-layer caches (a lazily cached list) and zeroes everything past the transferred coverage — the untransferred token tail of the last partially covered block plus any blocks beyond it — whose zeroing the scheduler skipped for the load. The clip happens at remote-block granularity with a block-size ratio, and at kernel-block granularity when equal kernel pages meet differing logical block sizes, so mamba hybrids need this zeroing even when the ratio is 1 (including MLA).
- The multi-read hetero-TP path (P_TP > D_TP with sharded SSM state, [#49297](https://github.com/aarnphm/aarnphm.github.io/issues/49297)) builds its per-source split handles from the remote- granularity descriptors, keyed by (tp_ratio, remote_block_size); replicated and single-source attention descriptors pass through whole, and SSM chunking is unaffected by the ratio.

Still unsupported, now rejected loudly at handshake time: head-sharded attention reads combined with a block-size mismatch, and host-buffer mode with a block-size ratio.

Note this is an alternative to [vllm-project/vllm#45575](https://github.com/vllm-project/vllm/pull/45575), which inflates the block-size on one side to make it common between P and D.

Fixes [vllm-project/vllm#41037](https://github.com/vllm-project/vllm/issues/41037).

---

## comments

**mergify** (2026-07-27T18:20:58Z):

This pull request has merge conflicts that must be resolved before it can be
merged. Please rebase the PR, @njhill.

<https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/syncing-a-fork>

