---
title: '39091'
pageLayout: default
slug: thoughts/inference/vllm-prs/39091
permalink: https://aarnphm.xyz/thoughts/inference/vllm-prs/39091.md
generator:
  quartz: v4.6.0
  hostedProvider: Cloudflare
  baseUrl: aarnphm.xyz
full: https://aarnphm.xyz/llms-full.txt
---
# [#39091](https://github.com/aarnphm/aarnphm.github.io/issues/39091) \[Bugfix\]\[Reasoning\] Nemotron V3: surface reasoning as content when thinking is unterminated

[vllm-project/vllm#39091](https://github.com/vllm-project/vllm/pull/39091) · askliar · MERGED · created 2026-04-06T17:14:24Z · merged 2026-06-10T12:58:19Z

## Problem

With Nemotron V3 and `enable_thinking=False` / `force_nonempty_content=True`, the model sometimes streams its whole answer inside an **unterminated** `<think>` block (it never emits `</think>`). Clients that read only `content` (not `reasoning`) then get `null`.

## Fix

**`vllm/reasoning/nemotron_v3_reasoning_parser.py`**

- `_should_force_content(request)` — centralizes the opt-in check (`request.chat_template_kwargs`: `enable_thinking is False` or `force_nonempty_content is True`).
- `get_streaming_fallback_content(text, request)` (new, opt-in) — returns the reasoning extracted from `text`, or `None` when not opted in.

**`vllm/parser/abstract_parser.py`**

- `finalize_generation` (terminal streaming delta) — when reasoning never terminated, **duplicates** the already-streamed reasoning into `content`, reading `state.previous_text` (still the full trace, because the tool phase only resets it after reasoning ends). Then flushes any unstreamed tool-call args (unchanged behavior). `parse_delta`’s finish path calls it.

## Behavior (opt-in on)

| case                                | `reasoning` | `content`          |
| ----------------------------------- | ----------- | ------------------ |
| streaming, `<think>…` never closed  | trace       | trace (duplicated) |
| streaming, `<think>…</think>answer` | trace       | answer (no dup)    |

## Scope / non-goals

- Streaming promotion fires **only when the model never closed `<think>`**. A closed `</think>` followed by empty content is left to main in streaming (non-streaming still moves it) — streaming can’t retract already-sent `reasoning`, so it’s a duplicate by nature.

## Testing

- `pytest tests/reasoning/test_nemotron_v3_reasoning_parser.py tests/parser/test_streaming.py tests/parser/test_parse.py` → pass.
- `pre-commit` (ruff, ruff-format, mypy-3.10, typos, SPDX) → pass.
- Verified on `NVIDIA-Nemotron-3-Nano-30B-A3B-NVFP4` (`--reasoning-parser nemotron_v3 --tool-call-parser qwen3_coder`).

## Notes

Not a duplicate of an existing PR. AI assistance was used; changes were reviewed line-by-line.

---

## comments

**mergify** (2026-04-14T08:21:46Z):

This pull request has merge conflicts that must be resolved before it can be
merged. Please rebase the PR, @askliar.

<https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/syncing-a-fork>

**mergify** (2026-05-15T11:02:27Z):

This pull request has merge conflicts that must be resolved before it can be
merged. Please rebase the PR, @askliar.

<https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/syncing-a-fork>

**mergify** (2026-06-05T15:46:25Z):

Hi @askliar, the pre-commit checks have failed. Please run:



```bash
uv pip install pre-commit>=4.5.1
pre-commit install
pre-commit run --all-files
```

Then, commit the changes and push to your branch.

For future commits, `pre-commit` will run automatically on changed files before each commit.

**mergify** (2026-06-09T12:35:04Z):

Hi @askliar, the pre-commit checks have failed. Please run:



```bash
uv pip install pre-commit>=4.5.1
pre-commit install
pre-commit run --all-files
```

Then, commit the changes and push to your branch.

For future commits, `pre-commit` will run automatically on changed files before each commit.

**bbrowning** (2026-06-10T14:13:00Z):

What was the “why” of this change? The model generated only reasoning, so why should that get moved into the content field instead of being properly parsed as reasoning?

**bbrowning** (2026-06-18T17:46:45Z):

Ok, I figured out the reason for this change on my own and made sure the behavior was preserved gated behind the same `force_nonempty_content` chat template kwarg as before in the streaming parser engine rewrite of [#45755](https://github.com/aarnphm/aarnphm.github.io/issues/45755)

