---
title: '33241'
pageLayout: default
slug: thoughts/inference/vllm-prs/33241
permalink: https://aarnphm.xyz/thoughts/inference/vllm-prs/33241.md
generator:
  quartz: v4.6.0
  hostedProvider: Cloudflare
  baseUrl: aarnphm.xyz
full: https://aarnphm.xyz/llms-full.txt
---
# [#33241](https://github.com/aarnphm/aarnphm.github.io/issues/33241) Revert “Enable Cross layers KV cache layout at NIXL Connector ([#30207](https://github.com/aarnphm/aarnphm.github.io/issues/30207))”

[vllm-project/vllm#33241](https://github.com/vllm-project/vllm/pull/33241) · orozery · MERGED · created 2026-01-28T09:52:09Z · merged 2026-01-28T12:36:00Z

Fully reverts [#30207](https://github.com/aarnphm/aarnphm.github.io/issues/30207) and its [#33052](https://github.com/aarnphm/aarnphm.github.io/issues/33052) follow-up.

---

## comments

**mergify** (2026-01-28T09:52:52Z):

Documentation preview: [https://vllm—33241.org.readthedocs.build/en/33241/](https://vllm--33241.org.readthedocs.build/en/33241/)

**vadiklyutiy** (2026-03-14T13:13:32Z):

@orozery @liranschour don’t we need cross layers kv-cache for performance reasons?

Seems [#34158](https://github.com/aarnphm/aarnphm.github.io/issues/34158) and [#36867](https://github.com/aarnphm/aarnphm.github.io/issues/36867) will solve the original issue.

**orozery** (2026-03-15T09:57:05Z):

> @orozery @liranschour don’t we need cross layers kv-cache for performance reasons?
>
> Seems [#34158](https://github.com/aarnphm/aarnphm.github.io/issues/34158) and [#36867](https://github.com/aarnphm/aarnphm.github.io/issues/36867) will solve the original issue.

[#33339](https://github.com/aarnphm/aarnphm.github.io/issues/33339) re-introduced it, but this time it is off by default.
It can be tested using:



```
--kv-transfer-config '{..., "kv_connector_extra_config": {"enable_cross_layers_blocks": "True"}}'
```

**vadiklyutiy** (2026-03-15T11:38:36Z):

> [#33339](https://github.com/aarnphm/aarnphm.github.io/issues/33339) re-introduced it, but this time it is off by default. It can be tested using:
>
>
>
> ```
> --kv-transfer-config '{..., "kv_connector_extra_config": {"enable_cross_layers_blocks": "True"}}'
> ```

don’t we want to enable it by default?

**vadiklyutiy** (2026-03-15T11:45:09Z):

and I see the following code



```python
@property
def prefer_cross_layer_blocks(self) -> bool:
  backend = get_current_attn_backend(self._vllm_config)
  if backend.get_name() not in ('FLASH_ATTN', 'FLASHINFER'):
    return False
```

Don’t know what problem with `FLASH_ATTN`. But we can enable it for `FLASHINFER` after 2 PRs mentioned above.

**orozery** (2026-03-15T12:04:21Z):

> don’t we want to enable it by default?

Eventually yes.
I think the plan is to to let users test it for a while before making it the default.

**vadiklyutiy** (2026-03-15T13:09:29Z):

> > don’t we want to enable it by default?
>
> Eventually yes. I think the plan is to to let users test it for a while before making it the default.

Just wondering, the change was merged 5 weeks ago. Did you get any positive or negative feedback?

**orozery** (2026-03-15T13:19:34Z):

> Just wondering, the change was merged 5 weeks ago. Did you get any positive or negative feedback?

Good point.
@NickLucche we should make sure users are aware of it so we can get feedback.

**vadiklyutiy** (2026-03-15T13:34:06Z):

My concern about this approach

> I think the plan is to to let users test it for a while before making it the default.

is that users aren’t aware about the functionality and even if we tell about it to users who we are working directly, we will cover only minority of users who can get benefits from the functionality.

