---
title: '50500'
pageLayout: default
slug: thoughts/inference/vllm-prs/50500
permalink: https://aarnphm.xyz/thoughts/inference/vllm-prs/50500.md
generator:
  quartz: v4.6.0
  hostedProvider: Cloudflare
  baseUrl: aarnphm.xyz
full: https://aarnphm.xyz/llms-full.txt
---
# [#50500](https://github.com/aarnphm/aarnphm.github.io/issues/50500) \[Compressed-Tensors\] Support Kimi-K3 quantized models

[vllm-project/vllm#50500](https://github.com/vllm-project/vllm/pull/50500) · kylesayrs · MERGED · created 2026-07-31T01:30:57Z · merged 2026-07-31T17:06:30Z

## Purpose

- Support CT configs for Kimi-K3

## Changes

- Add `SupportsQuant` and `packed_modules_mapping` to `KimiLinearModel`. `SupportsQuant` reads the `packed_modules_mapping` and updates the quantization config accordingly so that fused modules (fused gate up modules of shared expert) are properly ignored.

## Testing

- Validated serve and eval against `RedHatAI/Kimi-K3-NVFP4`
- Validated serve and sanity against `moonshotai/Kimi-K3`

---

## comments

