---
date: '2026-11-11'
description: DFlash2 adds a candidate-lattice selector and short convolution to block-diffusion drafting.
tags:
  - inference
  - specdec
title: DFlash2
created: '2026-11-11'
modified: '2026-11-11'
published: '2026-11-11'
pageLayout: default
slug: thoughts/inference/dflash2
permalink: https://aarnphm.xyz/thoughts/inference/dflash2.md
generator:
  quartz: v4.6.0
  hostedProvider: Cloudflare
  baseUrl: aarnphm.xyz
full: https://aarnphm.xyz/llms-full.txt
---
## mechanism (Inco AI / z-lab)

two independent additions on top of DFlash:

**(a) pairwise path selector.** keep top-K=16 candidates per slot. score every adjacent pair:

S_t(a, b) = U_t(b) + ⟨A(a) ⊙ H(h_t), B(b)⟩

- U_t(b): DFlash’s own logit for b (the unary term; this matters, see LiLiCorr)
- A, B: **per-token codebooks**, vocab × rank. A(a) depends only on the predecessor’s token id
- H(h_t): rank-dim gate projected from the drafter hidden state at slot t

all K×K edge scores for all slots come out of one einsum. then a greedy (or Gumbel-sampled) walk follows the best successor. serial work is B−1 argmaxes over 16 scalars.

**(b) two-tap dynamic depthwise conv** before and after every attention and MLP sublayer: Conv(x)_t = k_\{t,0\}⊙&#x78;_&#x74; + k_\{t,1\}⊙x\_\{t−1\}, kernel = learned base + hidden-dependent correction shared per 16-channel group. slot 0 reads the last verified token. lineage: Canon layers ([2512.17351](https://arxiv.org/abs/2512.17351) \[@allenzhu2026physicslanguagemodels41\] [2606.03825](https://arxiv.org/abs/2606.03825) \[@sieberling2026dynamicshortconvolutionsimprove\] [2606.03825](https://arxiv.org/abs/2606.03825)). the blog’s ablation: 5 layers + conv reaches Recall\@1 77.6% at position 6 vs 78.7% for 15 layers, at 0.7% latency vs 15.2%. one conv ≈ ten extra layers.

## reported results

**numbers**: Qwen3.5-4B mean τ: MTP 4.54, DFlash 4.92, DSpark 5.49, DFlash2 5.97 (+21% over DFlash at +1.3% cycle latency). Qwen3.8-27B (2B drafter, block 8, H200, SGLang): 3.43× at c=1 on GSM8K, collapsing to 1.01× on MT-Bench at c=32. oracle over the top-16 lattice is 6.79 τ on GSM8K vs 4.61 for the selector, so the lattice holds \~1.5× more acceptance than the walk extracts.

## vLLM implementation

**in vLLM**: merged [#52816](https://github.com/vllm-project/vllm/pull/52816) (Zihan Zhang, 2026-08-20). no new method string; `method: "dflash"` dispatches on architecture `DFlash2DraftModel` in [`spec_decode/__init__.py`](https://github.com/vllm-project/vllm/blob/c059bee812842fb76c2b045e240c02f37c0d498d/vllm/v1/worker/gpu/spec_decode/__init__.py#L28-L42). model: [`qwen3_dflash2.py`](https://github.com/vllm-project/vllm/blob/c059bee812842fb76c2b045e240c02f37c0d498d/vllm/model_executor/models/qwen3_dflash2.py) (`CandidateSelector`, `_score_edges`, `DFlashGroupedConv`). speculator + the shared walk kernel: [`dflash2/speculator.py`](https://github.com/vllm-project/vllm/blob/c059bee812842fb76c2b045e240c02f37c0d498d/vllm/v1/worker/gpu/spec_decode/dflash2/speculator.py). follow-ups: fused grouped conv [#55960](https://github.com/aarnphm/aarnphm.github.io/issues/55960), speculators-format loading [#53797](https://github.com/aarnphm/aarnphm.github.io/issues/53797), GLM-5.3-Flash drafters [#56983](https://github.com/aarnphm/aarnphm.github.io/issues/56983).

