---
cited_by:
  - '[[thoughts/inference/kimi-k3#5-sharp-edges-each-fix-shows-an-invariant|kimik3 §5]]'
date: '2026-11-11'
kind: pr
local: []
merged: '2026-08-02'
number: 50761
source:
  - https://github.com/vllm-project/vllm/blob/c6668106760e1f8222abd19a6ee9e93fc060cb19/vllm/models/kimi_k3/amd/linear.py
state: merged
tags:
  - vllm
  - vllm-pr
  - new-architectures
title: '#50761 [ROCm][Bugfix][Kimi-K3] Preserve MoE correction bias in FP32'
topic: new-architectures
url: https://github.com/vllm-project/vllm/pull/50761
created: '2026-11-11'
modified: '2026-11-11'
published: '2026-11-11'
pageLayout: default
slug: hinterland/papers/maps/prs/50761
permalink: https://aarnphm.xyz/hinterland/papers/maps/prs/50761.md
generator:
  quartz: v4.6.0
  hostedProvider: Cloudflare
  baseUrl: aarnphm.xyz
full: https://aarnphm.xyz/llms-full.txt
---
Kimi-K3 checkpoints store `e_score_correction_bias` in FP32 and the AMD router already emits FP32 logits. The AMD model currently constructs the correction-bias parameter under the ambient BF16 model dtype, so the standard weight loader… · full dump: [[thoughts/inference/vllm-prs/50761|#50761]]

