---
cited_by:
  - '[[thoughts/inference/kv-layout#3-what-each-layout-costs-the-kernels|reading-map §3]]'
  - '[[thoughts/inference/kv-transfer#5-reading-order|reading-map §5]]'
date: '2026-11-11'
kind: pr
local:
  - file:///Users/aarnphm/workspace/vllm/vllm/v1/attention/backends/flashinfer.py
  - file:///Users/aarnphm/workspace/vllm/vllm/v1/attention/backends/triton_attn.py
  - file:///Users/aarnphm/workspace/vllm/vllm/distributed/kv_transfer/kv_connector/utils.py
  - file:///Users/aarnphm/workspace/vllm/vllm/utils/torch_utils.py
  - file:///Users/aarnphm/workspace/vllm/vllm/v1/attention/backends/flash_attn_diffkv.py
merged: '2026-07-11'
number: 44455
source:
  - https://github.com/vllm-project/vllm/blob/51878e5b6e4cf27352c427b8508b4f338ba39c9c/vllm/v1/attention/backends/flashinfer.py
  - https://github.com/vllm-project/vllm/blob/51878e5b6e4cf27352c427b8508b4f338ba39c9c/vllm/distributed/kv_transfer/kv_connector/v1/nixl/base_worker.py
  - https://github.com/vllm-project/vllm/blob/51878e5b6e4cf27352c427b8508b4f338ba39c9c/vllm/v1/attention/backends/triton_attn.py
  - https://github.com/vllm-project/vllm/blob/51878e5b6e4cf27352c427b8508b4f338ba39c9c/vllm/distributed/kv_transfer/kv_connector/utils.py
  - https://github.com/vllm-project/vllm/blob/51878e5b6e4cf27352c427b8508b4f338ba39c9c/csrc/libtorch_stable/fused_minimax_m3_qknorm_rope_kv_insert_kernel.cu
  - https://github.com/vllm-project/vllm/blob/51878e5b6e4cf27352c427b8508b4f338ba39c9c/vllm/utils/torch_utils.py
  - https://github.com/vllm-project/vllm/blob/51878e5b6e4cf27352c427b8508b4f338ba39c9c/vllm/v1/attention/backends/triton_attn_diffkv.py
  - https://github.com/vllm-project/vllm/blob/51878e5b6e4cf27352c427b8508b4f338ba39c9c/vllm/v1/attention/backends/flash_attn_diffkv.py
state: merged
tags:
  - vllm
  - vllm-pr
  - hetero-tp-kv-layout
title: '#44455 [2/N][KV-Cache Layout Refactor] Pack K/V into the content dim across attention backends'
topic: hetero-tp-kv-layout
url: https://github.com/vllm-project/vllm/pull/44455
created: '2026-11-11'
modified: '2026-11-11'
published: '2026-11-11'
pageLayout: default
slug: hinterland/papers/maps/prs/44455
permalink: https://aarnphm.xyz/hinterland/papers/maps/prs/44455.md
generator:
  quartz: v4.6.0
  hostedProvider: Cloudflare
  baseUrl: aarnphm.xyz
full: https://aarnphm.xyz/llms-full.txt
---
Packs K and V into the content dimension across attention backends, changing the KV cache shape from `(num_blocks, 2, block_size, num_kv_heads, head_size)` (separate leading K/V dim) to \`(num_blocks, num_kv_heads, block_size… · full dump: [[thoughts/inference/vllm-prs/44455|#44455]]

