---
title: '57934'
pageLayout: default
slug: thoughts/inference/vllm-prs/57934
permalink: https://aarnphm.xyz/thoughts/inference/vllm-prs/57934.md
generator:
  quartz: v4.6.0
  hostedProvider: Cloudflare
  baseUrl: aarnphm.xyz
full: https://aarnphm.xyz/llms-full.txt
---
# [#57934](https://github.com/aarnphm/aarnphm.github.io/issues/57934) \[SpecDecode\] Add LiLiCorr drafter

[vllm-project/vllm#57934](https://github.com/vllm-project/vllm/pull/57934) · askliar · MERGED · created 2026-09-21T11:43:20Z · merged 2026-09-25T14:08:29Z

## What changes

Add `LiLiCorrDraftModel` ([2608.20530](https://arxiv.org/abs/2608.20530) \[@rusanovsky2026lilicorrlightweightlikelihoodcorrelation\] ) to the existing MRV2 DFlash path. The exported SGLang head reranks global candidate lattices using target embeddings, full-vocabulary log probabilities, draft hidden states and the normalized last committed target context. Both plain and grouped-convolution checkpoints are supported.

The port reuses DFlash2 convolution layers and extracts its existing candidate walk/cache orchestration into a shared helper. Greedy and probabilistic drafting use the current vLLM sampling and rejection machinery. No new scheduler, attention backend or verification algorithm is introduced. Correlator and convolution projections use the draft quantization configuration except for lattice QKV and the split-factor/fused-edge projections. Those retain floating-point weights and `F.linear`; `factor_input_proj`, `out_head` and `in_head` must be excluded during export. Reduced draft vocabularies are outside this scope.

ModelOpt exclusions use checkpoint-local draft layer numbers, while vLLM assigns global runtime layer numbers to draft attention layers. The quantization config now retains its exported exclusions and adds runtime-index aliases before constructing the draft layers. This lets mixed BF16/NVFP4 variants allocate the correct linear method without changing checkpoint names or attention/KV layer identity.

References: SGLang [sgl-project/sglang#37462](https://github.com/sgl-project/sglang/pull/37462), DSpark [vllm-project/vllm#46995](https://github.com/vllm-project/vllm/pull/46995), DFlash2 [vllm-project/vllm#52816](https://github.com/vllm-project/vllm/pull/52816).

### Validation status

PR has been tested on valid LiLiCorr checkpoint using 4xGB200 GPUs. It has achieved identical AL.

---

## comments

**mergify** (2026-09-21T11:43:56Z):

Documentation preview: [https://vllm—57934.org.readthedocs.build/en/57934/](https://vllm--57934.org.readthedocs.build/en/57934/)

**benchislett** (2026-09-24T14:34:28Z):

Ideally we’ll go back and optimize/simplify the model code, but I’m okay to merge for now

**benchislett** (2026-09-24T14:35:03Z):

/ci run

**github-actions** (2026-09-24T14:36:01Z):

✅ Triggered [Buildkite CI #90951](https://buildkite.com/vllm/ci/builds/90951) for commit `5c95b0b3b3dd`.

**github-actions** (2026-09-24T14:36:12Z):

✅ @askliar, CI is now available for this PR.

- `/ci run` starts upstream CI; `/amd-ci run` starts AMD CI only.
- Your branch must contain every commit currently on its upstream target branch. Merge or rebase onto the latest target branch, then rerun the command. Append `--allow-stale` to a run command to test an outdated branch at your own risk.
- `/ci retry` retries failed jobs in the CI build for the current PR head. If the current head has no CI build, it starts a new CI build for the current head containing only jobs that failed in the latest earlier CI build for this PR.
- `/amd-ci retry` retries failed jobs in AMD CI for the current PR head. Use `/amd-ci run` when the current head has no AMD CI build.
- `/ci cancel` cancels scheduled or running CI builds for this PR branch; `/amd-ci cancel` does the same for AMD CI only.

<!-- vllm-ci-authorized -->

**benchislett** (2026-09-24T17:32:20Z):

seems like there was an issue on main, retrying after rebase

**benchislett** (2026-09-24T17:32:26Z):

/ci run

**github-actions** (2026-09-24T17:32:47Z):

✅ Triggered [Buildkite CI #91001](https://buildkite.com/vllm/ci/builds/91001) for commit `9ffeaa725068`.

**mergify** (2026-09-24T22:37:34Z):

This pull request has merge conflicts that must be resolved before it can be
merged. Please rebase the PR, @askliar.

<https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/syncing-a-fork>

**askliar** (2026-09-24T23:22:35Z):

/ci retry

**github-actions** (2026-09-24T23:22:53Z):

✅ Triggered [Buildkite CI #91088](https://buildkite.com/vllm/ci/builds/91088) for commit `a1f327c2836f`, running 6 failed step(s) from [Buildkite CI #91001](https://buildkite.com/vllm/ci/builds/91001).

**HF-001** (2026-09-28T09:31:08Z):

@askliar Hi, we are testing LiLiCorr. Could you share a BF16 LiLiCorr checkpoint for Qwen3-8B, including the complete config and trained correlator weights?

