---
title: '58065'
pageLayout: default
slug: thoughts/inference/vllm-prs/58065
permalink: https://aarnphm.xyz/thoughts/inference/vllm-prs/58065.md
generator:
  quartz: v4.6.0
  hostedProvider: Cloudflare
  baseUrl: aarnphm.xyz
full: https://aarnphm.xyz/llms-full.txt
---
# [#58065](https://github.com/aarnphm/aarnphm.github.io/issues/58065) \[Spec Decode\] Enable async scheduling for DFlash

[vllm-project/vllm#58065](https://github.com/vllm-project/vllm/pull/58065) · majunze2001 · MERGED · created 2026-09-22T03:27:03Z · merged 2026-09-22T05:05:45Z

## Scope

Adds `dflash` to the async-scheduling allowlist in `VllmConfig`, on both the explicit `--async-scheduling` path (previously a hard `ValueError`) and the default auto-enable path (previously force-disabled with a warning). The V2 model runner’s async machinery (async output copies, GPU-side draft-token buffers) is method-agnostic, and the DFlash speculator consumes the same GPU tensors (`num_sampled`, `num_rejected`, `last_sampled`) as the DSpark one, so no engine changes beyond the allowlist are needed.

## Configuration

No new configuration. With default settings, async scheduling now stays enabled for `--speculative-config '{"method":"dflash", ...}'` instead of being silently disabled. `--no-async-scheduling` still forces it off.

## Launch command

<details>
<summary>vllm serve command used for validation (GB200, TP4)</summary>



```bash
vllm serve Qwen/Qwen3.8-27B \
  --host 0.0.0.0 --port 8000 -tp 4 \
  --max-num-seqs 32 --max-num-batched-tokens 8192 \
  --language-model-only \
  --compilation-config '{"pass_config": {"fuse_allreduce_rms": false}}' \
  --speculative-config '{"method":"dflash","model":"z-lab/Qwen3.8-27B-DFlash2","num_speculative_tokens":7,"attention_backend":"TRITON_ATTN"}'
```

(The `fuse_allreduce_rms` workaround is unrelated to this PR: the DFlash2 candidate selector trips a piecewise-compile assert when the allreduce-RMS fusion splits compile ranges at TP4. The TRITON_ATTN draft backend is used because the FLASH_ATTN path is broken for this draft/target pair — FA4-hd256 page-size clash with the hybrid target’s mamba-forced KV block size.)

</details>

## Validation

- New unit test `test_dflash_allows_async_scheduling` in `tests/test_config.py`: builds a minimal DFlash2 draft config and asserts async scheduling stays enabled (previously force-disabled). Run: `.venv/bin/python -m pytest tests/test_config.py -k "dflash or async_scheduling" -q` → 5 passed. `pre-commit run --files vllm/config/vllm.py tests/test_config.py` → all passed (ruff check + format included).
- E2E correctness on 4xGB200 TP4, Qwen/Qwen3.8-27B + z-lab/Qwen3.8-27B-DFlash2 draft (7 speculative tokens, standard rejection, greedy draft, async scheduling on): GSM8K 5-shot flexible-extract 0.9708 / strict-match 0.9704, matching the no-spec baseline 0.9719/0.9716 and the DSpark draft 0.9708/0.9712. Measured acceptance length 5.96 on GSM8K traffic. Repeated with prefix caching enabled (default): 0.9735 / 0.9735 — the hybrid-mamba + prefix-cache + spec-decode path is also correct.

## Performance

Same-build A/B, bs=1 pure-decode (ISL=1, OSL=1000, 2 warmup + 4 requests, `vllm bench serve`, 4xGB200 TP4, DFlash2 with TRITON_ATTN draft backend fully CUDA-graphed, synthetic acceptance length 6):

| async scheduling              | TPOT (mean) | decode tok/s |
| ----------------------------- | ----------- | ------------ |
| off (`--no-async-scheduling`) | 2.127 ms    | 470.2        |
| on (default with this PR)     | 1.510 ms    | 662.3        |

**+41% decode throughput (1.41x lower TPOT) at bs=1** with DFlash2 enabled.

## Notes

AI assistance was used for this change (code, tests, and validation); all quoted results were produced by actually running the commands above.

Duplicate-work check: `gh pr list --search "dflash async"` and `gh search prs "async scheduling" "dflash"` over vllm-project/vllm return no open or merged PR enabling async scheduling for DFlash; the allowlist in `vllm/config/vllm.py` on master still excludes it. The sibling DSpark was allowlisted in its original PR ([#46995](https://github.com/aarnphm/aarnphm.github.io/issues/46995)); nothing since covers DFlash.

---

## comments

