---
title: '43328'
pageLayout: default
slug: thoughts/inference/vllm-prs/43328
permalink: https://aarnphm.xyz/thoughts/inference/vllm-prs/43328.md
generator:
  quartz: v4.6.0
  hostedProvider: Cloudflare
  baseUrl: aarnphm.xyz
full: https://aarnphm.xyz/llms-full.txt
---
# [#43328](https://github.com/aarnphm/aarnphm.github.io/issues/43328) Enable B12x backend for non-gated MoEs (like Nemotron)

[vllm-project/vllm#43328](https://github.com/vllm-project/vllm/pull/43328) · askliar · MERGED · created 2026-05-21T15:41:53Z · merged 2026-07-06T19:40:07Z

## Summary

Stacked on top of [#40082](https://github.com/aarnphm/aarnphm.github.io/issues/40082).

This PR refines the FlashInfer B12x MoE integration by switching the SM12x MoE path to FlashInfer’s `B12xMoEWrapper` API and adding ReLU2 / non-gated MoE coverage.

Key changes:

- Use `B12xMoEWrapper` for `FlashInferB12xExperts`
- Keep BF16 hidden states as unquantized inputs; B12x handles FP4 activation quantization internally
- Support both SiLU gated MoE and ReLU2 non-gated MoE
- Add ReLU2 test coverage
- Update the test helper to allow non-default activation and `is_act_and_mul`

## Duplicate-work check

This is intentionally not a duplicate of [#40082](https://github.com/aarnphm/aarnphm.github.io/issues/40082): it is an incremental stacked change on top of [#40082](https://github.com/aarnphm/aarnphm.github.io/issues/40082).

---

## comments

**mergify** (2026-05-21T15:51:54Z):

Hi @askliar, the pre-commit checks have failed. Please run:



```bash
uv pip install pre-commit>=4.5.1
pre-commit install
pre-commit run --all-files
```

Then, commit the changes and push to your branch.

For future commits, `pre-commit` will run automatically on changed files before each commit.

> \[!tip\]
>
> <details>
> <summary>Is <code>mypy</code> failing?</summary>
> <br/>
> <code>mypy</code> is run differently in CI. If the failure is related to this check, please use the following command to run it locally:
>
>
>
> ```bash
> # For mypy (substitute "3.10" with the failing version if needed)
> pre-commit run --hook-stage manual mypy-3.10
> ```
>
> </details>

**mergify** (2026-06-04T19:02:35Z):

This pull request has merge conflicts that must be resolved before it can be
merged. Please rebase the PR, @askliar.

<https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/syncing-a-fork>

**askliar** (2026-06-22T09:29:12Z):

@pavanimajety @AndreasKaratzas can you please take a look

**mergify** (2026-07-02T14:50:20Z):

Hi @askliar, the pre-commit checks have failed. Please run:



```bash
uv pip install pre-commit>=4.5.1
pre-commit install
pre-commit run --all-files
```

Then, commit the changes and push to your branch.

For future commits, `pre-commit` will run automatically on changed files before each commit.

