---
title: '48735'
pageLayout: default
slug: thoughts/inference/vllm-prs/48735
permalink: https://aarnphm.xyz/thoughts/inference/vllm-prs/48735.md
generator:
  quartz: v4.6.0
  hostedProvider: Cloudflare
  baseUrl: aarnphm.xyz
full: https://aarnphm.xyz/llms-full.txt
---
# [#48735](https://github.com/aarnphm/aarnphm.github.io/issues/48735) \[Perf\] Improve `--linear-backend` filtering

[vllm-project/vllm#48735](https://github.com/vllm-project/vllm/pull/48735) · askliar · MERGED · created 2026-07-15T12:32:48Z · merged 2026-08-07T22:15:55Z

## Purpose

The `--linear-backend <name>` currently raises at startup for any linear layer type the backend doesn’t cover, so single-scheme backends are unusable. `flashinfer_b12x` is doubly blocked: its only kernel is absent from the NVFP4 auto-selection list, so the filter comes back empty even for NVFP4 layers and the documented opt-in never worked.

## Changes

1. **Replace the five duplicated filter blocks** with one `_resolve_backend_kernels()` helper that logs and falls back to normal selection for uncovered layer types.

2. **List FlashInfer-B12x × NvFp4-linearKernel in `POSITIVE_FILTER_HINTS`** after the fix kernel – unless both are supported, but the cuDNN filter can now find the kernel. It stays behind `CUTLASS_EPILOGUE` deliberately: the b12x GEMM needs FlashInfer autotuning (opt-in) to win; its heuristic fallback tactics are up to 3.6× slower at prefill shapes. Its previous exclusion reason (`NVIDIA-cutlass-3.4.5.1`) is fixed in nvidia/cutlass-3.4.5.1.

## Test

**GB10 (SM121)**, Nemotron-3-Super-120B-A12B-NVFP4, ISL=150k/USL=4k, 10 requests, autotuning enabled: `--linear-backend flashinfer_b12x` previously crashed at startup; now selects the b12x GEMM and improves TTFT 12% over FlashInfer-cutlass (70.1s → 69.4s), 17.2 → 17.7 GB/s, `--linear-backend auto` selection is unchanged.

---

## comments

**askliar** (2026-07-27T14:24:24Z):

@mgoin any chance you could take a look, please? Really important for our internal work that we have to publish this week already.

**mergify** (2026-07-31T23:57:21Z):

This pull request has merge conflicts that must be resolved before it can be
merged. Please rebase the PR, @askliar.

<https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/syncing-a-fork>

**faradawn** (2026-08-04T16:29:10Z):

Hi @mgoin, do you need any additional logs or info? Hope to merge before Friday

**askliar** (2026-08-06T09:18:24Z):

@mgoin thank you for your feedback - I’ve simplified code greatly and it should be ready for getting merged now.

**github-actions** (2026-08-06T12:42:19Z):

✅ @askliar, CI is now available for this PR.

- `/ci run` starts a CI build.
- `/ci retry` retries failed jobs in the CI build for the current PR head. If the current head has no CI build, it starts a new CI build for the current head containing only jobs that failed in the latest earlier CI build for this PR.

<!-- vllm-ci-authorized -->

**mgoin** (2026-08-07T20:23:34Z):

/ci run

**github-actions** (2026-08-07T20:23:54Z):

✅ Triggered [Buildkite CI #82918](https://buildkite.com/vllm/ci/builds/82918) for commit `cf59b20e1bd5`.

**mgoin** (2026-08-07T22:15:24Z):

Nit: we need to update the behavior for MoE backend as well!

