---
title: '52188'
pageLayout: default
slug: thoughts/inference/vllm-prs/52188
permalink: https://aarnphm.xyz/thoughts/inference/vllm-prs/52188.md
generator:
  quartz: v4.6.0
  hostedProvider: Cloudflare
  baseUrl: aarnphm.xyz
full: https://aarnphm.xyz/llms-full.txt
---
# [#52188](https://github.com/aarnphm/aarnphm.github.io/issues/52188) \[Spec decode\] Support Kimi-K3 DCP with DSpark

[vllm-project/vllm#52188](https://github.com/vllm-project/vllm/pull/52188) · wzhao18 · MERGED · created 2026-08-13T16:10:03Z · merged 2026-08-17T20:08:46Z

## Purpose

This PR adds support for running Kimi-K3 decode context parallel with DSpark with FlashinferMLA and Tokenspeed as target causal attention backend and Tokenspeed as the draft non-causal backend.

## Test Plan

Kimi K3 GSM8k with the different backend combination.

## Test Result

Default (no backend specified):



```
vllm serve moonshotai/Kimi-K3 \
  --tensor-parallel-size 8 \
  -dcp 8 \
  --load-format fastsafetensors \
  --no-enable-flashinfer-autotune \
  --trust-remote-code \
  --language-model-only \
  --attention-config '{"mla_prefill_backend":"TRTLLM_RAGGED","use_prefill_query_quantization":true}' \
  --kv-cache-dtype fp8 \
  --speculative-config '{"model":"Inferact/Kimi-K3-DSpark","method":"dspark","num_speculative_tokens":4,"rejection_sample_method":"standard"}'

|Tasks|Version|     Filter     |n-shot|  Metric   |   |Value |   |Stderr|
|-----|------:|----------------|-----:|-----------|---|-----:|---|-----:|
|gsm8k|      3|flexible-extract|     5|exact_match|↑  |0.9606|±  |0.0054|
|     |       |strict-match    |     5|exact_match|↑  |0.9606|±  |0.0054|
```

FlashinferMLA target + Tokenspeed draft:



```
|Tasks|Version|     Filter     |n-shot|  Metric   |   |Value |   |Stderr|
|-----|------:|----------------|-----:|-----------|---|-----:|---|-----:|
|gsm8k|      3|flexible-extract|     5|exact_match|↑  |0.9613|±  |0.0053|
|     |       |strict-match    |     5|exact_match|↑  |0.9613|±  |0.0053|
```

Tokenspeed target + Tokenspeed draft:



```
|Tasks|Version|     Filter     |n-shot|  Metric   |   |Value |   |Stderr|
|-----|------:|----------------|-----:|-----------|---|-----:|---|-----:|
|gsm8k|      3|flexible-extract|     5|exact_match|↑  |0.9621|±  |0.0053|
|     |       |strict-match    |     5|exact_match|↑  |0.9621|±  |0.0053|
```

---

<details>
<summary> Essential Elements of an Effective PR Description Checklist </summary>

- [ ] The purpose of the PR, such as “Fix some issue (link existing issues this PR will resolve)”.
- [ ] The test plan, such as providing test command.
- [ ] The test results, such as pasting the results comparison before and after, or e2e results
- [ ] (Optional) The necessary documentation update, such as updating `supported_models.md` and `examples` for a new model.

</details>

---

## comments

**github-actions** (2026-08-14T22:03:34Z):

✅ @wzhao18, CI is now available for this PR.

- `/ci run` starts a CI build.
- `/ci retry` retries failed jobs in the CI build for the current PR head. If the current head has no CI build, it starts a new CI build for the current head containing only jobs that failed in the latest earlier CI build for this PR.
- `/ci cancel` cancels scheduled or running CI builds for this PR branch.

<!-- vllm-ci-authorized -->

**wzhao18** (2026-08-14T22:05:36Z):

/ci run

**github-actions** (2026-08-14T22:05:53Z):

✅ Triggered [Buildkite CI #83978](https://buildkite.com/vllm/ci/builds/83978) for commit `2814ebfc4f8d`.

**wzhao18** (2026-08-15T02:35:44Z):

/ci retry

**github-actions** (2026-08-15T02:36:01Z):

✅ Queued 1 failed job(s) for retry in [Buildkite CI #83978](https://buildkite.com/vllm/ci/builds/83978).

**mergify** (2026-08-15T14:34:26Z):

This pull request has merge conflicts that must be resolved before it can be
merged. Please rebase the PR, @wzhao18.

<https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/syncing-a-fork>

**wzhao18** (2026-08-15T17:36:46Z):

/ci run

**github-actions** (2026-08-15T17:37:03Z):

✅ Triggered [Buildkite CI #84034](https://buildkite.com/vllm/ci/builds/84034) for commit `3f7e125943ee`.

**mergify** (2026-08-15T17:43:20Z):

Hi @wzhao18, the pre-commit checks have failed. Please run:



```bash
uv pip install pre-commit>=4.5.1
pre-commit install
pre-commit run --all-files
```

Then, commit the changes and push to your branch.

For future commits, `pre-commit` will run automatically on changed files before each commit.

**wzhao18** (2026-08-15T19:35:58Z):

/ci retry

**github-actions** (2026-08-15T19:36:14Z):

✅ Queued 1 failed job(s) for retry in [Buildkite CI #84034](https://buildkite.com/vllm/ci/builds/84034).

**wzhao18** (2026-08-15T23:48:56Z):

/ci run

**github-actions** (2026-08-15T23:49:12Z):

✅ CI is already running for this commit: <https://buildkite.com/vllm/ci/builds/84034>

**wzhao18** (2026-08-15T23:54:34Z):

/ci retry

**github-actions** (2026-08-15T23:54:51Z):

✅ Queued 1 failed job(s) for retry in [Buildkite CI #84034](https://buildkite.com/vllm/ci/builds/84034).

**wzhao18** (2026-08-16T23:27:52Z):

/ci retry

**github-actions** (2026-08-16T23:28:08Z):

✅ Queued 2 failed job(s) for retry in [Buildkite CI #84034](https://buildkite.com/vllm/ci/builds/84034).

**wzhao18** (2026-08-17T04:00:10Z):

/ci run

**github-actions** (2026-08-17T04:00:29Z):

✅ Triggered [Buildkite CI #84149](https://buildkite.com/vllm/ci/builds/84149) for commit `6cf05e8e06ff`.

**wzhao18** (2026-08-17T05:10:14Z):

/ci retry

**github-actions** (2026-08-17T05:10:30Z):

✅ Queued 2 failed job(s) for retry in [Buildkite CI #84149](https://buildkite.com/vllm/ci/builds/84149).

**wzhao18** (2026-08-17T14:21:00Z):

/ci run

**github-actions** (2026-08-17T14:21:19Z):

✅ Triggered [Buildkite CI #84207](https://buildkite.com/vllm/ci/builds/84207) for commit `6d6cb7d53df8`.

**wzhao18** (2026-08-17T16:18:28Z):

/ci run

**github-actions** (2026-08-17T16:18:48Z):

✅ Triggered [Buildkite CI #84210](https://buildkite.com/vllm/ci/builds/84210) for commit `073f76abcee4`.

**wzhao18** (2026-08-17T17:22:30Z):

/ci retry

**wzhao18** (2026-08-17T19:03:05Z):

/ci retry

**github-actions** (2026-08-17T19:03:23Z):

✅ Queued 1 failed job(s) for retry in [Buildkite CI #84210](https://buildkite.com/vllm/ci/builds/84210).

