---
title: '54722'
pageLayout: default
slug: thoughts/inference/vllm-prs/54722
permalink: https://aarnphm.xyz/thoughts/inference/vllm-prs/54722.md
generator:
  quartz: v4.6.0
  hostedProvider: Cloudflare
  baseUrl: aarnphm.xyz
full: https://aarnphm.xyz/llms-full.txt
---
# [#54722](https://github.com/aarnphm/aarnphm.github.io/issues/54722) \[Qwen4\] validate FP8 PLE weight scale after loading

[vllm-project/vllm#54722](https://github.com/vllm-project/vllm/pull/54722) · peakcrosser7 · MERGED · created 2026-09-01T07:24:13Z · merged 2026-09-02T04:30:59Z

## Purpose

Fix a fail-open risk in the Qwen3.8 Flash Next FP8 PLE weight loading path.

If `ngram_embedding.weight_scale` is missing or incorrectly mapped, the registered parameter could remain uninitialized without being detected. This PR:

- Stores the BF16 PLE global scale as FP32 so the existing sentinel initialization applies.
- Validates the sentinel in `process_weights_after_loading()` after all checkpoint shards have been processed.

Thanks to @Dev-Jahn for identifying and reporting this issue.

## Not a duplicate

Searched the open PRs for [Qwen3.8 PLE weight_scale](https://github.com/vllm-project/vllm/pulls?q=is%3Apr+is%3Aopen+Qwen3.8+PLE+weight_scale) and [FP8 PLE scale](https://github.com/vllm-project/vllm/pulls?q=is%3Apr+is%3Aopen+FP8+PLE+scale). I did not find an open PR addressing this missing post-load validation.

## Test Plan



```bash
python -m pytest -q tests/models/qwen4_exp/test_ple.py

pre-commit run --files \
  vllm/models/qwen4_exp/nvidia/ple_layer.py \
  tests/models/qwen4_exp/test_ple.py

git diff --check
```

## Test Result

- Unit tests: 11 passed.
- Pre-commit hooks: all passed.
- git diff —check: passed.
- Missing scale now raises ValueError during post-load processing.

> AI assistance was used for code analysis, implementation, and test drafting. I reviewed the resulting changes and test results.

---

<details>
<summary> Essential Elements of an Effective PR Description Checklist </summary>

- [x] The purpose of the PR, such as “Fix some issue (link existing issues this PR will resolve)”.
- [x] The test plan, such as providing test command.
- \[ x The test results, such as pasting the results comparison before and after, or e2e results
- [ ] (Optional) The necessary documentation update, such as updating `supported_models.md` and `examples` for a new model.

</details>

---

## comments

**Dev-Jahn** (2026-09-01T07:51:20Z):

Looks right to me, thanks for turning it around quickly. Storing the scale as FP32 puts it on the path where `create_fp8_scale_parameter` applies its `finfo.min` sentinel, and the `process_weights_after_loading` check turns the old silent case into a load-time error; numerics are unchanged since the BF16 checkpoint value round-trips exactly through FP32 and the dequant already upcasts. The `load_scale=False` test pins the failure case.

One forward-looking note for the [#53899](https://github.com/aarnphm/aarnphm.github.io/issues/53899) rebase: under `VLLM_PLE_CPU_OFFLOAD` the GPU process skips this method’s `create_weights`, so this check will run only in the offload worker. The GPU-side retention path (`load_weights` keeping `_offload_weight_scale`) fails closed today through the dequant-time RuntimeError, so nothing is wrong, but adding the same post-load check to that branch would move the failure to startup there too.

**Isotr0py** (2026-09-02T02:52:29Z):

/ci run

**github-actions** (2026-09-02T02:53:05Z):

✅ Triggered [Buildkite CI #86761](https://buildkite.com/vllm/ci/builds/86761) for commit `0d084422a50e`.

