---
title: '52148'
pageLayout: default
slug: thoughts/inference/vllm-prs/52148
permalink: https://aarnphm.xyz/thoughts/inference/vllm-prs/52148.md
generator:
  quartz: v4.6.0
  hostedProvider: Cloudflare
  baseUrl: aarnphm.xyz
full: https://aarnphm.xyz/llms-full.txt
---
# [#52148](https://github.com/aarnphm/aarnphm.github.io/issues/52148) \[Attention\] Fix FlashInfer SM12x prefill with sinks

[vllm-project/vllm#52148](https://github.com/vllm-project/vllm/pull/52148) · askliar · MERGED · created 2026-08-13T10:33:31Z · merged 2026-08-13T21:25:10Z

## Summary

Use FlashInfer’s sink-aware paged prefill wrapper on SM12x when attention sinks are enabled. The generic FA2 prefill path accepts a `sinks` argument but does not apply it, so [#49718](https://github.com/aarnphm/aarnphm.github.io/issues/49718) can use XQA for decode while producing incorrect prefill output.

The wrapper is specialized with the active dtypes, head dimensions, sliding window, and softmax scale. DCP, NVFP4, SM90/SM100, and sink-free paths are unchanged.

This is not a duplicate of the automatic revert [#51987](https://github.com/aarnphm/aarnphm.github.io/issues/51987): that PR removes SM12x XQA support, while this change preserves XQA and fixes the prefill path for sink-attention models on SM12x .

## Validation

- ` pytest tests/v1/attention/test_attention_backends.py::test_flashinfer_native_prefill_with_sinks -q` passing
- SM12x GPT-OSS model eval fixed and passing
- Nemotron and Qwen models both working as expected

---

## comments

**mgoin** (2026-08-13T15:13:26Z):

/ci run

**github-actions** (2026-08-13T15:14:06Z):

✅ Triggered [Buildkite CI #83752](https://buildkite.com/vllm/ci/builds/83752) for commit `c2fb5a90ae0c`.

**stecasta** (2026-08-13T15:44:01Z):

Validated on DGX Spark (GB10, sm121), same GPU and job, only the patch or backend flag changed.

**`gpt-oss-20b`**, GSM8K 5-shot, limit 200:

| arm           | flexible-extract | Harmony non-terminal errors |
| ------------- | ---------------- | --------------------------- |
| stock         | 0.000            | 196                         |
| **+ this PR** | **0.875**        | **0**                       |
| `TRITON_ATTN` | 0.880            | 0                           |

**Non-sink controls, no regression.**

`Qwen3-8B`: 0.87 flexible / 0.82 strict, identical stock and patched.

Nemotron 3.5 Lightning NVFP4 with a DSpark-3 drafter (hybrid mamba plus attention), 200 items:

| arm           | flexible-extract | acceptance length (unforced) |
| ------------- | ---------------- | ---------------------------- |
| stock run 1   | 0.830            | 3.29 to 3.34                 |
| **+ this PR** | **0.840**        | 3.23 to 3.31                 |

**mgoin** (2026-08-13T21:25:00Z):

dgx spark test passed, merging <https://buildkite.com/vllm/ci/builds/83752#019ffbb2-b776-476f-bdd8-128988ec7298>

