---
cited_by:
  - '[[thoughts/papers/xie2026mhcmanifoldconstrainedhyperconnections#16-mhc-manifold-constrained-hyper-connections-with-v41s-delayedsingle-pass-pre|dsv41 §1.6]]'
  - '[[thoughts/inference/deepseek-v4.1-serving#4-pr-reading-order|dsv41 §4]]'
date: '2026-11-11'
kind: pr
local:
  - file:///Users/aarnphm/workspace/vllm/CMakeLists.txt
merged: '2026-09-21'
number: 57643
source:
  - https://github.com/vllm-project/vllm/blob/9a70c233cd5d3ef3b14d79c582a6f9ddfa03cf80/csrc/libtorch_stable/all_reduce_mhc.cu
  - https://github.com/vllm-project/vllm/blob/9a70c233cd5d3ef3b14d79c582a6f9ddfa03cf80/vllm/models/deepseek_v41/nvidia/ops/mhc.py
  - https://github.com/vllm-project/vllm/blob/9a70c233cd5d3ef3b14d79c582a6f9ddfa03cf80/vllm/models/deepseek_v41/nvidia/ops/mega_mhc.py
  - https://github.com/vllm-project/vllm/blob/9a70c233cd5d3ef3b14d79c582a6f9ddfa03cf80/vllm/models/deepseek_v41/nvidia/model.py
  - https://github.com/vllm-project/vllm/blob/9a70c233cd5d3ef3b14d79c582a6f9ddfa03cf80/vllm/models/deepseek_v4/nvidia/model.py
  - https://github.com/vllm-project/vllm/blob/9a70c233cd5d3ef3b14d79c582a6f9ddfa03cf80/CMakeLists.txt
state: merged
tags:
  - vllm
  - vllm-pr
  - new-architectures
title: '#57643 [Perf][DSV4.1] Fuse TP all-reduce with mHC input preparation'
topic: new-architectures
url: https://github.com/vllm-project/vllm/pull/57643
created: '2026-11-11'
modified: '2026-11-11'
published: '2026-11-11'
pageLayout: default
slug: hinterland/papers/maps/prs/57643
permalink: https://aarnphm.xyz/hinterland/papers/maps/prs/57643.md
generator:
  quartz: v4.6.0
  hostedProvider: Cloudflare
  baseUrl: aarnphm.xyz
full: https://aarnphm.xyz/llms-full.txt
---
Fuse TP all-reduce, mHC post mixing, delayed collapse, and RMSNorm at DSV4.1 decoder boundaries. This replaces three launches with one, preserves the BF16 boundaries, and retains coefficient overlap. The kernel reuses the existing MNNVL… · full dump: [[thoughts/inference/vllm-prs/57643|#57643]]

