---
cited_by:
  - '[[thoughts/inference/qwen3.8-runtime#2-delta-vs-predecessors|qwen4next §2]]'
date: '2026-11-11'
kind: pr
local:
  - file:///Users/aarnphm/workspace/vllm/vllm/model_executor/models/qwen3_vl.py
  - file:///Users/aarnphm/workspace/vllm/vllm/model_executor/models/qwen3.py
  - file:///Users/aarnphm/workspace/vllm/vllm/model_executor/layers/rotary_embedding/mrope.py
  - file:///Users/aarnphm/workspace/vllm/vllm/config/model.py
  - file:///Users/aarnphm/workspace/vllm/vllm/model_executor/layers/rotary_embedding/__init__.py
  - file:///Users/aarnphm/workspace/vllm/vllm/model_executor/models/qwen3_moe.py
  - file:///Users/aarnphm/workspace/vllm/vllm/model_executor/models/qwen3_5.py
  - file:///Users/aarnphm/workspace/vllm/vllm/model_executor/models/qwen3_next.py
merged: '2026-10-10'
number: 59853
source:
  - https://github.com/vllm-project/vllm/blob/a98247ab4db686ee03c66d5feb3c761e52a2f8ab/vllm/model_executor/models/qwen3_vl.py
  - https://github.com/vllm-project/vllm/blob/a98247ab4db686ee03c66d5feb3c761e52a2f8ab/vllm/model_executor/models/qwen3.py
  - https://github.com/vllm-project/vllm/blob/a98247ab4db686ee03c66d5feb3c761e52a2f8ab/vllm/model_executor/layers/rotary_embedding/mrope.py
  - https://github.com/vllm-project/vllm/blob/a98247ab4db686ee03c66d5feb3c761e52a2f8ab/vllm/config/model.py
  - https://github.com/vllm-project/vllm/blob/a98247ab4db686ee03c66d5feb3c761e52a2f8ab/vllm/model_executor/layers/rotary_embedding/__init__.py
  - https://github.com/vllm-project/vllm/blob/a98247ab4db686ee03c66d5feb3c761e52a2f8ab/vllm/model_executor/models/qwen3_moe.py
  - https://github.com/vllm-project/vllm/blob/a98247ab4db686ee03c66d5feb3c761e52a2f8ab/vllm/model_executor/models/qwen3_5.py
  - https://github.com/vllm-project/vllm/blob/a98247ab4db686ee03c66d5feb3c761e52a2f8ab/vllm/model_executor/models/qwen3_next.py
state: merged
tags:
  - vllm
  - vllm-pr
  - new-architectures
title: '#59853 [Model] Drop the 4x M-RoPE cache headroom for Qwen3-VL, Qwen3.5 and Qwen3.8'
topic: new-architectures
url: https://github.com/vllm-project/vllm/pull/59853
created: '2026-11-11'
modified: '2026-11-11'
published: '2026-11-11'
pageLayout: default
slug: hinterland/papers/maps/prs/59853
permalink: https://aarnphm.xyz/hinterland/papers/maps/prs/59853.md
generator:
  quartz: v4.6.0
  hostedProvider: Cloudflare
  baseUrl: aarnphm.xyz
full: https://aarnphm.xyz/llms-full.txt
---
Qwen3-VL, Qwen3.5 and Qwen3.8 build their M-RoPE cos/sin cache at 4x `max_position_embeddings`. That headroom exists for Qwen2.5-VL video positions, which these models do not produce, so this sizes the cache like plain RoPE unless video… · full dump: [[thoughts/inference/vllm-prs/59853|#59853]]

