---
title: '53132'
pageLayout: default
slug: thoughts/inference/vllm-prs/53132
permalink: https://aarnphm.xyz/thoughts/inference/vllm-prs/53132.md
generator:
  quartz: v4.6.0
  hostedProvider: Cloudflare
  baseUrl: aarnphm.xyz
full: https://aarnphm.xyz/llms-full.txt
---
# [#53132](https://github.com/aarnphm/aarnphm.github.io/issues/53132) Support kimi k3 nvfp4 checkpoint

[vllm-project/vllm#53132](https://github.com/vllm-project/vllm/pull/53132) · wzhao18 · MERGED · created 2026-08-20T16:17:23Z · merged 2026-08-21T05:59:45Z

## Purpose

This PR supports running kimi k3 nvfp4 checkpoint `nvidia/Kimi-K3-NVFP4`

## Test Plan

- Kimi K3 GSM8k on 8 x B300

## Test Result

TP8:



```
vllm serve nvidia/Kimi-K3-NVFP4 \
  -tp 8 \
  --load-format fastsafetensors \
  --no-enable-flashinfer-autotune \
  --trust-remote-code \
  --language-model-only \
  --attention-config '{"mla_prefill_backend":"TRTLLM_RAGGED","use_prefill_query_quantization":true}' \
  --kv-cache-dtype fp8 \
  --speculative-config '{"model":"Inferact/Kimi-K3-DSpark","method":"dspark","num_speculative_tokens":4,"rejection_sample_method":"standard"}' \
  --enable-prompt-tokens-details \
  --prefix-match-unit 128

|Tasks|Version|     Filter     |n-shot|  Metric   |   |Value |   |Stderr|
|-----|------:|----------------|-----:|-----------|---|-----:|---|-----:|
|gsm8k|      3|flexible-extract|     5|exact_match|↑  |0.9591|±  |0.0055|
|     |       |strict-match    |     5|exact_match|↑  |0.9591|±  |0.0055|
```

TEP8:



```
|Tasks|Version|     Filter     |n-shot|  Metric   |   |Value |   |Stderr|
|-----|------:|----------------|-----:|-----------|---|-----:|---|-----:|
|gsm8k|      3|flexible-extract|     5|exact_match|↑  |0.9606|±  |0.0054|
|     |       |strict-match    |     5|exact_match|↑  |0.9598|±  |0.0054|
```

DCP8:



```
|Tasks|Version|     Filter     |n-shot|  Metric   |   |Value |   |Stderr|
|-----|------:|----------------|-----:|-----------|---|-----:|---|-----:|
|gsm8k|      3|flexible-extract|     5|exact_match|↑  |0.9575|±  |0.0056|
|     |       |strict-match    |     5|exact_match|↑  |0.9575|±  |0.0056|
```

---

<details>
<summary> Essential Elements of an Effective PR Description Checklist </summary>

- [ ] The purpose of the PR, such as “Fix some issue (link existing issues this PR will resolve)”.
- [ ] The test plan, such as providing test command.
- [ ] The test results, such as pasting the results comparison before and after, or e2e results
- [ ] (Optional) The necessary documentation update, such as updating `supported_models.md` and `examples` for a new model.

</details>

---

## comments

**zyongye** (2026-08-21T04:07:19Z):

/ci run

**github-actions** (2026-08-21T04:07:37Z):

✅ Triggered [Buildkite CI #84955](https://buildkite.com/vllm/ci/builds/84955) for commit `dd72b3df39b2`.

**DamirLisak** (2026-08-21T12:29:11Z):

@wzhao18 support for deepseek v4 nvfp4 checkpoint [#52447](https://github.com/aarnphm/aarnphm.github.io/issues/52447)? 🙏☺️

