---
title: '19056'
pageLayout: default
slug: thoughts/inference/vllm-prs/19056
permalink: https://aarnphm.xyz/thoughts/inference/vllm-prs/19056.md
generator:
  quartz: v4.6.0
  hostedProvider: Cloudflare
  baseUrl: aarnphm.xyz
full: https://aarnphm.xyz/llms-full.txt
---
# [#19056](https://github.com/aarnphm/aarnphm.github.io/issues/19056) \[Bug\]: Hermes tool parser stream output error in Qwen3 case

[vllm-project/vllm#19056](https://github.com/vllm-project/vllm/issues/19056) · LiuLi1998 · CLOSED · created 2025-06-03T06:26:37Z

### Your current environment

<details><summary>Details</summary>
<p>

DEBUG 06-03 06:00:03.465 \[**init**.py:28\] No plugins for group vllm.platform_plugins found.
DEBUG 06-03 06:00:03.465 \[**init**.py:34\] Checking if TPU platform is available.
DEBUG 06-03 06:00:03.465 \[**init**.py:44\] TPU platform is not available because: No module named ‘libtpu’
DEBUG 06-03 06:00:03.465 \[**init**.py:52\] Checking if CUDA platform is available.
DEBUG 06-03 06:00:03.484 \[**init**.py:72\] Confirmed CUDA platform is available.
DEBUG 06-03 06:00:03.485 \[**init**.py:100\] Checking if ROCm platform is available.
DEBUG 06-03 06:00:03.486 \[**init**.py:114\] ROCm platform is not available because: No module named ‘amdsmi’
DEBUG 06-03 06:00:03.486 \[**init**.py:122\] Checking if HPU platform is available.
DEBUG 06-03 06:00:03.486 \[**init**.py:129\] HPU platform is not available because habana_frameworks is not found.
DEBUG 06-03 06:00:03.486 \[**init**.py:140\] Checking if XPU platform is available.
DEBUG 06-03 06:00:03.486 \[**init**.py:150\] XPU platform is not available because: No module named ‘intel_extension_for_pytorch’
DEBUG 06-03 06:00:03.486 \[**init**.py:158\] Checking if CPU platform is available.
DEBUG 06-03 06:00:03.486 \[**init**.py:181\] Checking if Neuron platform is available.
DEBUG 06-03 06:00:03.487 \[**init**.py:52\] Checking if CUDA platform is available.
DEBUG 06-03 06:00:03.498 \[**init**.py:72\] Confirmed CUDA platform is available.
INFO 06-03 06:00:03.499 \[**init**.py:248\] Automatically detected platform cuda.
Collecting environment information…
/usr/local/lib/python3.10/dist-packages/\_distutils_hack/**init**.py:30: UserWarning: Setuptools is replacing distutils. Support for replacing an already imported distutils is deprecated. In the future, this condition will fail. Register concerns at <https://github.com/pypa/setuptools/issues/new?template=distutils-deprecation.yml>
warnings.warn(
Traceback (most recent call last):
File “/workspace/collect_env.py”, line 818, in <module>
main()
File “/workspace/collect_env.py”, line 797, in main
output = get_pretty_env_info()
File “/workspace/collect_env.py”, line 792, in get_pretty_env_info
return pretty_str(get_env_info())
File “/workspace/collect_env.py”, line 598, in get_env_info
vllm_version = get_vllm_version()
File “/workspace/collect_env.py”, line 285, in get_vllm_version
version_str = **version_tuple**\[-1\]
TypeError: ‘generator’ object is not subscriptable
root\@ll-qwen360-fc5fc5f95-zptzk:/workspace# python3 collect_env.py
DEBUG 06-03 06:01:15.959 \[**init**.py:28\] No plugins for group vllm.platform_plugins found.
DEBUG 06-03 06:01:15.959 \[**init**.py:34\] Checking if TPU platform is available.
DEBUG 06-03 06:01:15.959 \[**init**.py:44\] TPU platform is not available because: No module named ‘libtpu’
DEBUG 06-03 06:01:15.959 \[**init**.py:52\] Checking if CUDA platform is available.
DEBUG 06-03 06:01:15.977 \[**init**.py:72\] Confirmed CUDA platform is available.
DEBUG 06-03 06:01:15.978 \[**init**.py:100\] Checking if ROCm platform is available.
DEBUG 06-03 06:01:15.978 \[**init**.py:114\] ROCm platform is not available because: No module named ‘amdsmi’
DEBUG 06-03 06:01:15.978 \[**init**.py:122\] Checking if HPU platform is available.
DEBUG 06-03 06:01:15.978 \[**init**.py:129\] HPU platform is not available because habana_frameworks is not found.
DEBUG 06-03 06:01:15.978 \[**init**.py:140\] Checking if XPU platform is available.
DEBUG 06-03 06:01:15.978 \[**init**.py:150\] XPU platform is not available because: No module named ‘intel_extension_for_pytorch’
DEBUG 06-03 06:01:15.978 \[**init**.py:158\] Checking if CPU platform is available.
DEBUG 06-03 06:01:15.979 \[**init**.py:181\] Checking if Neuron platform is available.
DEBUG 06-03 06:01:15.979 \[**init**.py:52\] Checking if CUDA platform is available.
DEBUG 06-03 06:01:15.990 \[**init**.py:72\] Confirmed CUDA platform is available.
INFO 06-03 06:01:15.991 \[**init**.py:248\] Automatically detected platform cuda.
Collecting environment information…
/usr/local/lib/python3.10/dist-packages/\_distutils_hack/**init**.py:30: UserWarning: Setuptools is replacing distutils. Support for replacing an already imported distutils is deprecated. In the future, this condition will fail. Register concerns at <https://github.com/pypa/setuptools/issues/new?template=distutils-deprecation.yml>
warnings.warn(
Traceback (most recent call last):
File “/workspace/collect_env.py”, line 818, in <module>
main()
File “/workspace/collect_env.py”, line 797, in main
output = get_pretty_env_info()
File “/workspace/collect_env.py”, line 792, in get_pretty_env_info
return pretty_str(get_env_info())
File “/workspace/collect_env.py”, line 598, in get_env_info
vllm_version = get_vllm_version()
File “/workspace/collect_env.py”, line 285, in get_vllm_version
version_str = **version_tuple**\[-1\]
TypeError: ‘generator’ object is not subscriptable

</p>
</details>

### 🐛 Describe the bug

The extract_tool_calls_streaming component of the Hermes tool parser improperly handled a specific token during the parsing of streaming tool function outputs when the Qwen3 generates a specific token.

case: Qwen3 tokenizer vocab has a token `"}`, id 9207
if Qwen3 models output this token, hermes extract_tool_calls_streaming method will truncate this token to `"` in which case `}` will be discarded, further led to broken tool call’s json schema output

### LLM

Qwen3 series

<details><summary>vLLM serve log</summary>
<p>

\[2025-06-03 13:09:22\] DEBUG 06-03 05:09:22.878 \[hermes_tool_parser.py:127\] delta_text: ”\}
\[2025-06-03 13:09:22\] DEBUG 06-03 05:09:22.878 \[hermes_tool_parser.py:128\] delta_token_ids: \[9207\]
\[2025-06-03 13:09:22\] DEBUG 06-03 05:09:22.878 \[hermes_tool_parser.py:243\] Parsed tool call \{‘name’: ‘summary’, ‘arguments’: \{‘query’: ‘DeepSeek-R1-0528具体有哪些提升’, ‘title’: ‘DeepSeek-R1-0528版本升级亮点解析’, ‘style’: ‘研究报告’, ‘outline’: \[\{‘h2’: ‘一、版本迭代背景’, ‘guideline’: ‘阐述DeepSeek-R1系列研发历程及0528版本开发目标’\}, \{‘h2’: ‘二、核心技术突破’, ‘guideline’: ‘对比分析模型架构优化、训练数据升级、算法改进等核心提升点’\}, \{‘h2’: ‘三、性能指标提升’, ‘guideline’: ‘从推理速度、准确率、多语言支持度等维度量化展示提升幅度’\}, \{‘h2’: ‘四、应用场景拓展’, ‘guideline’: ‘说明在代码生成、内容创作、逻辑推理等场景的专项优化’\}, \{‘h2’: ‘五、用户使用体验’, ‘guideline’: ‘描述实际应用中的响应速度、交互流畅度等体验改善’\}\]\}\}
\[2025-06-03 13:09:22\] DEBUG 06-03 05:09:22.878 \[hermes_tool_parser.py:282\] Trying to parse current tool call with ID 0
\[2025-06-03 13:09:22\] DEBUG 06-03 05:09:22.878 \[hermes_tool_parser.py:296\] diffing old arguments: \{‘query’: ‘DeepSeek-R1-0528具体有哪些提升’, ‘title’: ‘DeepSeek-R1-0528版本升级亮点解析’, ‘style’: ‘研究报告’, ‘outline’: \[\{‘h2’: ‘一、版本迭代背景’, ‘guideline’: ‘阐述DeepSeek-R1系列研发历程及0528版本开发目标’\}, \{‘h2’: ‘二、核心技术突破’, ‘guideline’: ‘对比分析模型架构优化、训练数据升级、算法改进等核心提升点’\}, \{‘h2’: ‘三、性能指标提升’, ‘guideline’: ‘从推理速度、准确率、多语言支持度等维度量化展示提升幅度’\}, \{‘h2’: ‘四、应用场景拓展’, ‘guideline’: ‘说明在代码生成、内容创作、逻辑推理等场景的专项优化’\}, \{‘h2’: ‘五、用户使用体验’, ‘guideline’: ‘描述实际应用中的响应速度、交互流畅度等体验改善’\}\]\}
\[2025-06-03 13:09:22\] DEBUG 06-03 05:09:22.878 \[hermes_tool_parser.py:297\] against new ones: \{‘query’: ‘DeepSeek-R1-0528具体有哪些提升’, ‘title’: ‘DeepSeek-R1-0528版本升级亮点解析’, ‘style’: ‘研究报告’, ‘outline’: \[\{‘h2’: ‘一、版本迭代背景’, ‘guideline’: ‘阐述DeepSeek-R1系列研发历程及0528版本开发目标’\}, \{‘h2’: ‘二、核心技术突破’, ‘guideline’: ‘对比分析模型架构优化、训练数据升级、算法改进等核心提升点’\}, \{‘h2’: ‘三、性能指标提升’, ‘guideline’: ‘从推理速度、准确率、多语言支持度等维度量化展示提升幅度’\}, \{‘h2’: ‘四、应用场景拓展’, ‘guideline’: ‘说明在代码生成、内容创作、逻辑推理等场景的专项优化’\}, \{‘h2’: ‘五、用户使用体验’, ‘guideline’: ‘描述实际应用中的响应速度、交互流畅度等体验改善’\}\]\}
\[2025-06-03 13:09:22\] DEBUG 06-03 05:09:22.878 \[hermes_tool_parser.py:347\] got diff ”

</p>
</details>

**Client code**



```python
import openai

api_key = ''
api_base = ''
client = openai.OpenAI(api_key=api_key, base_url=api_base)

tools = [
  {
    'type': 'function',
    'function': {
      'name': 'summary',
      'description': '总结工具，用于在资料准备完成后生成回答，并展示给用户。',
      'parameters': {
        'type': 'object',
        'properties': {
          'query': {'description': '用户的问题', 'type': 'string'},
          'title': {
            'description': '回答的标题，与用户问题密切相关，不超过15个字。',
            'type': 'string',
          },
          'style': {
            'description': '回答风格。可能的回答风格包含: 自媒体文章，短视频脚本，媒体稿件，论文，研究报告等',
            'enum': [
              '自媒体文章',
              '短视频脚本',
              '媒体稿件',
              '论文',
              '研究报告',
            ],
            'type': 'string',
          },
          'outline': {
            'description': '回答的大纲',
            'type': 'array',
            'items': {
              'h2': {'type': 'string', 'description': '大纲的二级标题'},
              'guideline': {
                'type': 'string',
                'description': '如何填充该二级标题的指南',
              },
            },
          },
          'search_result_index_list': {
            'type': 'array',
            'items': {'type': 'string'},
          },
          'literature_paper_search_result': {
            'type': 'array',
            'items': {
              'type': 'object',
              'properties': {
                'title': {'type': 'string'},
                'paper_url': {
                  'type': 'string',
                  'description': '文献pdf/word URL',
                },
                'abstract': {
                  'type': 'string',
                  'description': '论文检索摘要原文，不要对文献摘要修改',
                },
                'content': {
                  'type': 'string',
                  'description': '文献与检索相关内容原文，不要对内容精简',
                },
              },
            },
          },
          'literature_image_list': {
            'type': 'array',
            'items': {
              'type': 'object',
              'properties': {
                'url': {'type': 'string'},
                'image_description': {'type': 'string'},
              },
            },
          },
        },
        'required': [
          'query',
          'title',
          'style',
          'outline',
          'search_result_index_list',
          'literature_paper_search_result',
        ],
      },
    },
  }
]
try:
  models = client.models.list()
  model = models.data[0].id
  stream = True
  response = client.chat.completions.create(
    model=model,
    messages=[
      {
        'role': 'user',
        'content': '充分理解用户问题并创建一个plan。 \n\n用户问题：DeepSeek-R1-0528具体有哪些提升，请生成一份详细的报告\n\nplan ID：plan_1748576400\n\n参考资料：\n无',
      }
    ],
    stream=stream,
    max_tokens=4096,
    tool_choice='auto',
    tools=tools,
    extra_body={'chat_template_kwargs': {'enable_thinking': True}},
  )
  if not stream:
    print(response.choices[0].message.content)
    print(response.choices[0].message.tool_calls)
  else:
    full_response = ''
    tool_calls_list = []
    for chunk in response:
      if chunk.choices[0].delta:
        if chunk.choices[0].delta.content:
          full_response += chunk.choices[0].delta.content
        if chunk.choices[0].delta.tool_calls:
          tool_calls_list.append(chunk.choices[0].delta.tool_calls)
    if tool_calls_list:
      merged_tool_call = {}
      for tool_call_chunk in tool_calls_list:
        for tool_call in tool_call_chunk:
          print(tool_call)
          merged_tool_call.setdefault(tool_call.index, {})
          merged_tool_call[tool_call.index].setdefault(
            'name', tool_call.function.name
          )
          merged_tool_call[tool_call.index].setdefault('arguments', '')

          if tool_call.function.arguments is not None:
            merged_tool_call[tool_call.index]['arguments'] = (
              merged_tool_call[tool_call.index]['arguments']
              + tool_call.function.arguments
            )

      print('Merged Tool Call:', merged_tool_call)
    print(full_response)

except Exception as e:
  print(f'请求失败: {e}')
```

<details><summary>Client log</summary>
<p>

ChoiceDeltaToolCall(index=0, id=‘chatcmpl-tool-ca0a10ac646744be850c62b1301ff39b’, function=ChoiceDeltaToolCallFunction(arguments=None, name=‘summary’), type=‘function’)
ChoiceDeltaToolCall(index=0, id=None, function=ChoiceDeltaToolCallFunction(arguments=’\{“query”: ”’, name=None), type=None)
ChoiceDeltaToolCall(index=0, id=None, function=ChoiceDeltaToolCallFunction(arguments=‘Deep’, name=None), type=None)
ChoiceDeltaToolCall(index=0, id=None, function=ChoiceDeltaToolCallFunction(arguments=‘Seek’, name=None), type=None)
ChoiceDeltaToolCall(index=0, id=None, function=ChoiceDeltaToolCallFunction(arguments=’-R’, name=None), type=None)
ChoiceDeltaToolCall(index=0, id=None, function=ChoiceDeltaToolCallFunction(arguments=‘1’, name=None), type=None)
ChoiceDeltaToolCall(index=0, id=None, function=ChoiceDeltaToolCallFunction(arguments=’-’, name=None), type=None)
ChoiceDeltaToolCall(index=0, id=None, function=ChoiceDeltaToolCallFunction(arguments=‘0’, name=None), type=None)
ChoiceDeltaToolCall(index=0, id=None, function=ChoiceDeltaToolCallFunction(arguments=‘5’, name=None), type=None)
ChoiceDeltaToolCall(index=0, id=None, function=ChoiceDeltaToolCallFunction(arguments=‘2’, name=None), type=None)
ChoiceDeltaToolCall(index=0, id=None, function=ChoiceDeltaToolCallFunction(arguments=‘8’, name=None), type=None)
ChoiceDeltaToolCall(index=0, id=None, function=ChoiceDeltaToolCallFunction(arguments=‘具体’, name=None), type=None)
ChoiceDeltaToolCall(index=0, id=None, function=ChoiceDeltaToolCallFunction(arguments=‘有哪些’, name=None), type=None)
ChoiceDeltaToolCall(index=0, id=None, function=ChoiceDeltaToolCallFunction(arguments=‘提升’, name=None), type=None)
ChoiceDeltaToolCall(index=0, id=None, function=ChoiceDeltaToolCallFunction(arguments=’”,’, name=None), type=None)
ChoiceDeltaToolCall(index=0, id=None, function=ChoiceDeltaToolCallFunction(arguments=’ ”’, name=None), type=None)
ChoiceDeltaToolCall(index=0, id=None, function=ChoiceDeltaToolCallFunction(arguments=‘title’, name=None), type=None)
ChoiceDeltaToolCall(index=0, id=None, function=ChoiceDeltaToolCallFunction(arguments=’”:’, name=None), type=None)
ChoiceDeltaToolCall(index=0, id=None, function=ChoiceDeltaToolCallFunction(arguments=’ ”’, name=None), type=None)
ChoiceDeltaToolCall(index=0, id=None, function=ChoiceDeltaToolCallFunction(arguments=‘Deep’, name=None), type=None)
ChoiceDeltaToolCall(index=0, id=None, function=ChoiceDeltaToolCallFunction(arguments=‘Seek’, name=None), type=None)
ChoiceDeltaToolCall(index=0, id=None, function=ChoiceDeltaToolCallFunction(arguments=’-R’, name=None), type=None)
ChoiceDeltaToolCall(index=0, id=None, function=ChoiceDeltaToolCallFunction(arguments=‘1’, name=None), type=None)
ChoiceDeltaToolCall(index=0, id=None, function=ChoiceDeltaToolCallFunction(arguments=’-’, name=None), type=None)
ChoiceDeltaToolCall(index=0, id=None, function=ChoiceDeltaToolCallFunction(arguments=‘0’, name=None), type=None)
ChoiceDeltaToolCall(index=0, id=None, function=ChoiceDeltaToolCallFunction(arguments=‘5’, name=None), type=None)
ChoiceDeltaToolCall(index=0, id=None, function=ChoiceDeltaToolCallFunction(arguments=‘2’, name=None), type=None)
ChoiceDeltaToolCall(index=0, id=None, function=ChoiceDeltaToolCallFunction(arguments=‘8’, name=None), type=None)
ChoiceDeltaToolCall(index=0, id=None, function=ChoiceDeltaToolCallFunction(arguments=‘版本’, name=None), type=None)
ChoiceDeltaToolCall(index=0, id=None, function=ChoiceDeltaToolCallFunction(arguments=‘升级’, name=None), type=None)
ChoiceDeltaToolCall(index=0, id=None, function=ChoiceDeltaToolCallFunction(arguments=‘亮点’, name=None), type=None)
ChoiceDeltaToolCall(index=0, id=None, function=ChoiceDeltaToolCallFunction(arguments=‘解析’, name=None), type=None)
ChoiceDeltaToolCall(index=0, id=None, function=ChoiceDeltaToolCallFunction(arguments=’”,’, name=None), type=None)
ChoiceDeltaToolCall(index=0, id=None, function=ChoiceDeltaToolCallFunction(arguments=’ ”’, name=None), type=None)
ChoiceDeltaToolCall(index=0, id=None, function=ChoiceDeltaToolCallFunction(arguments=‘style’, name=None), type=None)
ChoiceDeltaToolCall(index=0, id=None, function=ChoiceDeltaToolCallFunction(arguments=’”:’, name=None), type=None)
ChoiceDeltaToolCall(index=0, id=None, function=ChoiceDeltaToolCallFunction(arguments=’ ”’, name=None), type=None)
ChoiceDeltaToolCall(index=0, id=None, function=ChoiceDeltaToolCallFunction(arguments=‘研究报告’, name=None), type=None)
ChoiceDeltaToolCall(index=0, id=None, function=ChoiceDeltaToolCallFunction(arguments=’”,’, name=None), type=None)
ChoiceDeltaToolCall(index=0, id=None, function=ChoiceDeltaToolCallFunction(arguments=’ ”’, name=None), type=None)
ChoiceDeltaToolCall(index=0, id=None, function=ChoiceDeltaToolCallFunction(arguments=‘outline’, name=None), type=None)
ChoiceDeltaToolCall(index=0, id=None, function=ChoiceDeltaToolCallFunction(arguments=’”:’, name=None), type=None)
ChoiceDeltaToolCall(index=0, id=None, function=ChoiceDeltaToolCallFunction(arguments=’ \[\{”’, name=None), type=None)
ChoiceDeltaToolCall(index=0, id=None, function=ChoiceDeltaToolCallFunction(arguments=‘h’, name=None), type=None)
ChoiceDeltaToolCall(index=0, id=None, function=ChoiceDeltaToolCallFunction(arguments=‘2’, name=None), type=None)
ChoiceDeltaToolCall(index=0, id=None, function=ChoiceDeltaToolCallFunction(arguments=’”:’, name=None), type=None)
ChoiceDeltaToolCall(index=0, id=None, function=ChoiceDeltaToolCallFunction(arguments=’ ”’, name=None), type=None)
ChoiceDeltaToolCall(index=0, id=None, function=ChoiceDeltaToolCallFunction(arguments=‘一’, name=None), type=None)
ChoiceDeltaToolCall(index=0, id=None, function=ChoiceDeltaToolCallFunction(arguments=’、’, name=None), type=None)
ChoiceDeltaToolCall(index=0, id=None, function=ChoiceDeltaToolCallFunction(arguments=‘版本’, name=None), type=None)
ChoiceDeltaToolCall(index=0, id=None, function=ChoiceDeltaToolCallFunction(arguments=‘迭代’, name=None), type=None)
ChoiceDeltaToolCall(index=0, id=None, function=ChoiceDeltaToolCallFunction(arguments=‘背景’, name=None), type=None)
ChoiceDeltaToolCall(index=0, id=None, function=ChoiceDeltaToolCallFunction(arguments=’”,’, name=None), type=None)
ChoiceDeltaToolCall(index=0, id=None, function=ChoiceDeltaToolCallFunction(arguments=’ ”’, name=None), type=None)
ChoiceDeltaToolCall(index=0, id=None, function=ChoiceDeltaToolCallFunction(arguments=‘guid’, name=None), type=None)
ChoiceDeltaToolCall(index=0, id=None, function=ChoiceDeltaToolCallFunction(arguments=‘eline’, name=None), type=None)
ChoiceDeltaToolCall(index=0, id=None, function=ChoiceDeltaToolCallFunction(arguments=’”:’, name=None), type=None)
ChoiceDeltaToolCall(index=0, id=None, function=ChoiceDeltaToolCallFunction(arguments=’ ”’, name=None), type=None)
ChoiceDeltaToolCall(index=0, id=None, function=ChoiceDeltaToolCallFunction(arguments=‘阐述’, name=None), type=None)
ChoiceDeltaToolCall(index=0, id=None, function=ChoiceDeltaToolCallFunction(arguments=‘Deep’, name=None), type=None)
ChoiceDeltaToolCall(index=0, id=None, function=ChoiceDeltaToolCallFunction(arguments=‘Seek’, name=None), type=None)
ChoiceDeltaToolCall(index=0, id=None, function=ChoiceDeltaToolCallFunction(arguments=’-R’, name=None), type=None)
ChoiceDeltaToolCall(index=0, id=None, function=ChoiceDeltaToolCallFunction(arguments=‘1’, name=None), type=None)
ChoiceDeltaToolCall(index=0, id=None, function=ChoiceDeltaToolCallFunction(arguments=‘系列’, name=None), type=None)
ChoiceDeltaToolCall(index=0, id=None, function=ChoiceDeltaToolCallFunction(arguments=‘研发’, name=None), type=None)
ChoiceDeltaToolCall(index=0, id=None, function=ChoiceDeltaToolCallFunction(arguments=‘历程’, name=None), type=None)
ChoiceDeltaToolCall(index=0, id=None, function=ChoiceDeltaToolCallFunction(arguments=‘及’, name=None), type=None)
ChoiceDeltaToolCall(index=0, id=None, function=ChoiceDeltaToolCallFunction(arguments=‘0’, name=None), type=None)
ChoiceDeltaToolCall(index=0, id=None, function=ChoiceDeltaToolCallFunction(arguments=‘5’, name=None), type=None)
ChoiceDeltaToolCall(index=0, id=None, function=ChoiceDeltaToolCallFunction(arguments=‘2’, name=None), type=None)
ChoiceDeltaToolCall(index=0, id=None, function=ChoiceDeltaToolCallFunction(arguments=‘8’, name=None), type=None)
ChoiceDeltaToolCall(index=0, id=None, function=ChoiceDeltaToolCallFunction(arguments=‘版本’, name=None), type=None)
ChoiceDeltaToolCall(index=0, id=None, function=ChoiceDeltaToolCallFunction(arguments=‘开发’, name=None), type=None)
ChoiceDeltaToolCall(index=0, id=None, function=ChoiceDeltaToolCallFunction(arguments=‘目标’, name=None), type=None)
ChoiceDeltaToolCall(index=0, id=None, function=ChoiceDeltaToolCallFunction(arguments=’”\},’, name=None), type=None)
ChoiceDeltaToolCall(index=0, id=None, function=ChoiceDeltaToolCallFunction(arguments=’ \{”’, name=None), type=None)
ChoiceDeltaToolCall(index=0, id=None, function=ChoiceDeltaToolCallFunction(arguments=‘h’, name=None), type=None)
ChoiceDeltaToolCall(index=0, id=None, function=ChoiceDeltaToolCallFunction(arguments=‘2’, name=None), type=None)
ChoiceDeltaToolCall(index=0, id=None, function=ChoiceDeltaToolCallFunction(arguments=’”:’, name=None), type=None)
ChoiceDeltaToolCall(index=0, id=None, function=ChoiceDeltaToolCallFunction(arguments=’ ”’, name=None), type=None)
ChoiceDeltaToolCall(index=0, id=None, function=ChoiceDeltaToolCallFunction(arguments=‘二’, name=None), type=None)
ChoiceDeltaToolCall(index=0, id=None, function=ChoiceDeltaToolCallFunction(arguments=’、’, name=None), type=None)
ChoiceDeltaToolCall(index=0, id=None, function=ChoiceDeltaToolCallFunction(arguments=‘核心技术’, name=None), type=None)
ChoiceDeltaToolCall(index=0, id=None, function=ChoiceDeltaToolCallFunction(arguments=‘突破’, name=None), type=None)
ChoiceDeltaToolCall(index=0, id=None, function=ChoiceDeltaToolCallFunction(arguments=’”,’, name=None), type=None)
ChoiceDeltaToolCall(index=0, id=None, function=ChoiceDeltaToolCallFunction(arguments=’ ”’, name=None), type=None)
ChoiceDeltaToolCall(index=0, id=None, function=ChoiceDeltaToolCallFunction(arguments=‘guid’, name=None), type=None)
ChoiceDeltaToolCall(index=0, id=None, function=ChoiceDeltaToolCallFunction(arguments=‘eline’, name=None), type=None)
ChoiceDeltaToolCall(index=0, id=None, function=ChoiceDeltaToolCallFunction(arguments=’”:’, name=None), type=None)
ChoiceDeltaToolCall(index=0, id=None, function=ChoiceDeltaToolCallFunction(arguments=’ ”’, name=None), type=None)
ChoiceDeltaToolCall(index=0, id=None, function=ChoiceDeltaToolCallFunction(arguments=‘对比’, name=None), type=None)
ChoiceDeltaToolCall(index=0, id=None, function=ChoiceDeltaToolCallFunction(arguments=‘分析’, name=None), type=None)
ChoiceDeltaToolCall(index=0, id=None, function=ChoiceDeltaToolCallFunction(arguments=‘模型’, name=None), type=None)
ChoiceDeltaToolCall(index=0, id=None, function=ChoiceDeltaToolCallFunction(arguments=‘架构’, name=None), type=None)
ChoiceDeltaToolCall(index=0, id=None, function=ChoiceDeltaToolCallFunction(arguments=‘优化’, name=None), type=None)
ChoiceDeltaToolCall(index=0, id=None, function=ChoiceDeltaToolCallFunction(arguments=’、’, name=None), type=None)
ChoiceDeltaToolCall(index=0, id=None, function=ChoiceDeltaToolCallFunction(arguments=‘训练’, name=None), type=None)
ChoiceDeltaToolCall(index=0, id=None, function=ChoiceDeltaToolCallFunction(arguments=‘数据’, name=None), type=None)
ChoiceDeltaToolCall(index=0, id=None, function=ChoiceDeltaToolCallFunction(arguments=‘升级’, name=None), type=None)
ChoiceDeltaToolCall(index=0, id=None, function=ChoiceDeltaToolCallFunction(arguments=’、’, name=None), type=None)
ChoiceDeltaToolCall(index=0, id=None, function=ChoiceDeltaToolCallFunction(arguments=‘算法’, name=None), type=None)
ChoiceDeltaToolCall(index=0, id=None, function=ChoiceDeltaToolCallFunction(arguments=‘改进’, name=None), type=None)
ChoiceDeltaToolCall(index=0, id=None, function=ChoiceDeltaToolCallFunction(arguments=‘等’, name=None), type=None)
ChoiceDeltaToolCall(index=0, id=None, function=ChoiceDeltaToolCallFunction(arguments=‘核心’, name=None), type=None)
ChoiceDeltaToolCall(index=0, id=None, function=ChoiceDeltaToolCallFunction(arguments=‘提升’, name=None), type=None)
ChoiceDeltaToolCall(index=0, id=None, function=ChoiceDeltaToolCallFunction(arguments=‘点’, name=None), type=None)
ChoiceDeltaToolCall(index=0, id=None, function=ChoiceDeltaToolCallFunction(arguments=’”\},’, name=None), type=None)
ChoiceDeltaToolCall(index=0, id=None, function=ChoiceDeltaToolCallFunction(arguments=’ \{”’, name=None), type=None)
ChoiceDeltaToolCall(index=0, id=None, function=ChoiceDeltaToolCallFunction(arguments=‘h’, name=None), type=None)
ChoiceDeltaToolCall(index=0, id=None, function=ChoiceDeltaToolCallFunction(arguments=‘2’, name=None), type=None)
ChoiceDeltaToolCall(index=0, id=None, function=ChoiceDeltaToolCallFunction(arguments=’”:’, name=None), type=None)
ChoiceDeltaToolCall(index=0, id=None, function=ChoiceDeltaToolCallFunction(arguments=’ ”’, name=None), type=None)
ChoiceDeltaToolCall(index=0, id=None, function=ChoiceDeltaToolCallFunction(arguments=‘三’, name=None), type=None)
ChoiceDeltaToolCall(index=0, id=None, function=ChoiceDeltaToolCallFunction(arguments=’、’, name=None), type=None)
ChoiceDeltaToolCall(index=0, id=None, function=ChoiceDeltaToolCallFunction(arguments=‘性能’, name=None), type=None)
ChoiceDeltaToolCall(index=0, id=None, function=ChoiceDeltaToolCallFunction(arguments=‘指标’, name=None), type=None)
ChoiceDeltaToolCall(index=0, id=None, function=ChoiceDeltaToolCallFunction(arguments=‘提升’, name=None), type=None)
ChoiceDeltaToolCall(index=0, id=None, function=ChoiceDeltaToolCallFunction(arguments=’”,’, name=None), type=None)
ChoiceDeltaToolCall(index=0, id=None, function=ChoiceDeltaToolCallFunction(arguments=’ ”’, name=None), type=None)
ChoiceDeltaToolCall(index=0, id=None, function=ChoiceDeltaToolCallFunction(arguments=‘guid’, name=None), type=None)
ChoiceDeltaToolCall(index=0, id=None, function=ChoiceDeltaToolCallFunction(arguments=‘eline’, name=None), type=None)
ChoiceDeltaToolCall(index=0, id=None, function=ChoiceDeltaToolCallFunction(arguments=’”:’, name=None), type=None)
ChoiceDeltaToolCall(index=0, id=None, function=ChoiceDeltaToolCallFunction(arguments=’ ”’, name=None), type=None)
ChoiceDeltaToolCall(index=0, id=None, function=ChoiceDeltaToolCallFunction(arguments=‘从’, name=None), type=None)
ChoiceDeltaToolCall(index=0, id=None, function=ChoiceDeltaToolCallFunction(arguments=‘推理’, name=None), type=None)
ChoiceDeltaToolCall(index=0, id=None, function=ChoiceDeltaToolCallFunction(arguments=‘速度’, name=None), type=None)
ChoiceDeltaToolCall(index=0, id=None, function=ChoiceDeltaToolCallFunction(arguments=’、’, name=None), type=None)
ChoiceDeltaToolCall(index=0, id=None, function=ChoiceDeltaToolCallFunction(arguments=‘准确’, name=None), type=None)
ChoiceDeltaToolCall(index=0, id=None, function=ChoiceDeltaToolCallFunction(arguments=‘率’, name=None), type=None)
ChoiceDeltaToolCall(index=0, id=None, function=ChoiceDeltaToolCallFunction(arguments=’、’, name=None), type=None)
ChoiceDeltaToolCall(index=0, id=None, function=ChoiceDeltaToolCallFunction(arguments=‘多’, name=None), type=None)
ChoiceDeltaToolCall(index=0, id=None, function=ChoiceDeltaToolCallFunction(arguments=‘语言’, name=None), type=None)
ChoiceDeltaToolCall(index=0, id=None, function=ChoiceDeltaToolCallFunction(arguments=‘支持’, name=None), type=None)
ChoiceDeltaToolCall(index=0, id=None, function=ChoiceDeltaToolCallFunction(arguments=‘度’, name=None), type=None)
ChoiceDeltaToolCall(index=0, id=None, function=ChoiceDeltaToolCallFunction(arguments=‘等’, name=None), type=None)
ChoiceDeltaToolCall(index=0, id=None, function=ChoiceDeltaToolCallFunction(arguments=‘维度’, name=None), type=None)
ChoiceDeltaToolCall(index=0, id=None, function=ChoiceDeltaToolCallFunction(arguments=‘量化’, name=None), type=None)
ChoiceDeltaToolCall(index=0, id=None, function=ChoiceDeltaToolCallFunction(arguments=‘展示’, name=None), type=None)
ChoiceDeltaToolCall(index=0, id=None, function=ChoiceDeltaToolCallFunction(arguments=‘提升’, name=None), type=None)
ChoiceDeltaToolCall(index=0, id=None, function=ChoiceDeltaToolCallFunction(arguments=‘幅度’, name=None), type=None)
ChoiceDeltaToolCall(index=0, id=None, function=ChoiceDeltaToolCallFunction(arguments=’”\},’, name=None), type=None)
ChoiceDeltaToolCall(index=0, id=None, function=ChoiceDeltaToolCallFunction(arguments=’ \{”’, name=None), type=None)
ChoiceDeltaToolCall(index=0, id=None, function=ChoiceDeltaToolCallFunction(arguments=‘h’, name=None), type=None)
ChoiceDeltaToolCall(index=0, id=None, function=ChoiceDeltaToolCallFunction(arguments=‘2’, name=None), type=None)
ChoiceDeltaToolCall(index=0, id=None, function=ChoiceDeltaToolCallFunction(arguments=’”:’, name=None), type=None)
ChoiceDeltaToolCall(index=0, id=None, function=ChoiceDeltaToolCallFunction(arguments=’ ”’, name=None), type=None)
ChoiceDeltaToolCall(index=0, id=None, function=ChoiceDeltaToolCallFunction(arguments=‘四’, name=None), type=None)
ChoiceDeltaToolCall(index=0, id=None, function=ChoiceDeltaToolCallFunction(arguments=’、’, name=None), type=None)
ChoiceDeltaToolCall(index=0, id=None, function=ChoiceDeltaToolCallFunction(arguments=‘应用场景’, name=None), type=None)
ChoiceDeltaToolCall(index=0, id=None, function=ChoiceDeltaToolCallFunction(arguments=‘拓展’, name=None), type=None)
ChoiceDeltaToolCall(index=0, id=None, function=ChoiceDeltaToolCallFunction(arguments=’”,’, name=None), type=None)
ChoiceDeltaToolCall(index=0, id=None, function=ChoiceDeltaToolCallFunction(arguments=’ ”’, name=None), type=None)
ChoiceDeltaToolCall(index=0, id=None, function=ChoiceDeltaToolCallFunction(arguments=‘guid’, name=None), type=None)
ChoiceDeltaToolCall(index=0, id=None, function=ChoiceDeltaToolCallFunction(arguments=‘eline’, name=None), type=None)
ChoiceDeltaToolCall(index=0, id=None, function=ChoiceDeltaToolCallFunction(arguments=’”:’, name=None), type=None)
ChoiceDeltaToolCall(index=0, id=None, function=ChoiceDeltaToolCallFunction(arguments=’ ”’, name=None), type=None)
ChoiceDeltaToolCall(index=0, id=None, function=ChoiceDeltaToolCallFunction(arguments=‘说明’, name=None), type=None)
ChoiceDeltaToolCall(index=0, id=None, function=ChoiceDeltaToolCallFunction(arguments=‘在’, name=None), type=None)
ChoiceDeltaToolCall(index=0, id=None, function=ChoiceDeltaToolCallFunction(arguments=‘代码’, name=None), type=None)
ChoiceDeltaToolCall(index=0, id=None, function=ChoiceDeltaToolCallFunction(arguments=‘生成’, name=None), type=None)
ChoiceDeltaToolCall(index=0, id=None, function=ChoiceDeltaToolCallFunction(arguments=’、’, name=None), type=None)
ChoiceDeltaToolCall(index=0, id=None, function=ChoiceDeltaToolCallFunction(arguments=‘内容’, name=None), type=None)
ChoiceDeltaToolCall(index=0, id=None, function=ChoiceDeltaToolCallFunction(arguments=‘创作’, name=None), type=None)
ChoiceDeltaToolCall(index=0, id=None, function=ChoiceDeltaToolCallFunction(arguments=’、’, name=None), type=None)
ChoiceDeltaToolCall(index=0, id=None, function=ChoiceDeltaToolCallFunction(arguments=‘逻辑’, name=None), type=None)
ChoiceDeltaToolCall(index=0, id=None, function=ChoiceDeltaToolCallFunction(arguments=‘推理’, name=None), type=None)
ChoiceDeltaToolCall(index=0, id=None, function=ChoiceDeltaToolCallFunction(arguments=‘等’, name=None), type=None)
ChoiceDeltaToolCall(index=0, id=None, function=ChoiceDeltaToolCallFunction(arguments=‘场景’, name=None), type=None)
ChoiceDeltaToolCall(index=0, id=None, function=ChoiceDeltaToolCallFunction(arguments=‘的’, name=None), type=None)
ChoiceDeltaToolCall(index=0, id=None, function=ChoiceDeltaToolCallFunction(arguments=‘专项’, name=None), type=None)
ChoiceDeltaToolCall(index=0, id=None, function=ChoiceDeltaToolCallFunction(arguments=‘优化’, name=None), type=None)
ChoiceDeltaToolCall(index=0, id=None, function=ChoiceDeltaToolCallFunction(arguments=’”\},’, name=None), type=None)
ChoiceDeltaToolCall(index=0, id=None, function=ChoiceDeltaToolCallFunction(arguments=’ \{”’, name=None), type=None)
ChoiceDeltaToolCall(index=0, id=None, function=ChoiceDeltaToolCallFunction(arguments=‘h’, name=None), type=None)
ChoiceDeltaToolCall(index=0, id=None, function=ChoiceDeltaToolCallFunction(arguments=‘2’, name=None), type=None)
ChoiceDeltaToolCall(index=0, id=None, function=ChoiceDeltaToolCallFunction(arguments=’”:’, name=None), type=None)
ChoiceDeltaToolCall(index=0, id=None, function=ChoiceDeltaToolCallFunction(arguments=’ ”’, name=None), type=None)
ChoiceDeltaToolCall(index=0, id=None, function=ChoiceDeltaToolCallFunction(arguments=‘五’, name=None), type=None)
ChoiceDeltaToolCall(index=0, id=None, function=ChoiceDeltaToolCallFunction(arguments=’、’, name=None), type=None)
ChoiceDeltaToolCall(index=0, id=None, function=ChoiceDeltaToolCallFunction(arguments=‘用户’, name=None), type=None)
ChoiceDeltaToolCall(index=0, id=None, function=ChoiceDeltaToolCallFunction(arguments=‘使用’, name=None), type=None)
ChoiceDeltaToolCall(index=0, id=None, function=ChoiceDeltaToolCallFunction(arguments=‘体验’, name=None), type=None)
ChoiceDeltaToolCall(index=0, id=None, function=ChoiceDeltaToolCallFunction(arguments=’”,’, name=None), type=None)
ChoiceDeltaToolCall(index=0, id=None, function=ChoiceDeltaToolCallFunction(arguments=’ ”’, name=None), type=None)
ChoiceDeltaToolCall(index=0, id=None, function=ChoiceDeltaToolCallFunction(arguments=‘guid’, name=None), type=None)
ChoiceDeltaToolCall(index=0, id=None, function=ChoiceDeltaToolCallFunction(arguments=‘eline’, name=None), type=None)
ChoiceDeltaToolCall(index=0, id=None, function=ChoiceDeltaToolCallFunction(arguments=’”:’, name=None), type=None)
ChoiceDeltaToolCall(index=0, id=None, function=ChoiceDeltaToolCallFunction(arguments=’ ”’, name=None), type=None)
ChoiceDeltaToolCall(index=0, id=None, function=ChoiceDeltaToolCallFunction(arguments=‘描述’, name=None), type=None)
ChoiceDeltaToolCall(index=0, id=None, function=ChoiceDeltaToolCallFunction(arguments=‘实际’, name=None), type=None)
ChoiceDeltaToolCall(index=0, id=None, function=ChoiceDeltaToolCallFunction(arguments=‘应用’, name=None), type=None)
ChoiceDeltaToolCall(index=0, id=None, function=ChoiceDeltaToolCallFunction(arguments=‘中的’, name=None), type=None)
ChoiceDeltaToolCall(index=0, id=None, function=ChoiceDeltaToolCallFunction(arguments=‘响应’, name=None), type=None)
ChoiceDeltaToolCall(index=0, id=None, function=ChoiceDeltaToolCallFunction(arguments=‘速度’, name=None), type=None)
ChoiceDeltaToolCall(index=0, id=None, function=ChoiceDeltaToolCallFunction(arguments=’、’, name=None), type=None)
ChoiceDeltaToolCall(index=0, id=None, function=ChoiceDeltaToolCallFunction(arguments=‘交互’, name=None), type=None)
ChoiceDeltaToolCall(index=0, id=None, function=ChoiceDeltaToolCallFunction(arguments=‘流畅’, name=None), type=None)
ChoiceDeltaToolCall(index=0, id=None, function=ChoiceDeltaToolCallFunction(arguments=‘度’, name=None), type=None)
ChoiceDeltaToolCall(index=0, id=None, function=ChoiceDeltaToolCallFunction(arguments=‘等’, name=None), type=None)
ChoiceDeltaToolCall(index=0, id=None, function=ChoiceDeltaToolCallFunction(arguments=‘体验’, name=None), type=None)
ChoiceDeltaToolCall(index=0, id=None, function=ChoiceDeltaToolCallFunction(arguments=‘改善’, name=None), type=None)
ChoiceDeltaToolCall(index=0, id=None, function=ChoiceDeltaToolCallFunction(arguments=’”’, name=None), type=None)
ChoiceDeltaToolCall(index=0, id=None, function=ChoiceDeltaToolCallFunction(arguments=’\]’, name=None), type=None)
ChoiceDeltaToolCall(index=0, id=None, function=ChoiceDeltaToolCallFunction(arguments=’\}’, name=None), type=None)
Merged Tool Call: \{‘chatcmpl-tool-ca0a10ac646744be850c62b1301ff39b’: \{‘name’: ‘summary’, ‘arguments’: ”\}, None: \{‘name’: None, ‘arguments’: ’\{“query”: “DeepSeek-R1-0528具体有哪些提升”, “title”: “DeepSeek-R1-0528版本升级亮点解析”, “style”: “研究报告”, “outline”: \[\{“h2”: “一、版本迭代背景”, “guideline”: “阐述DeepSeek-R1系列研发历程及0528版本开发目标”\}, \{“h2”: “二、核心技术突破”, “guideline”: “对比分析模型架构优化、训练数据升级、算法改进等核心提升点”\}, \{“h2”: “三、性能指标提升”, “guideline”: “从推理速度、准确率、多语言支持度等维度量化展示提升幅度”\}, \{“h2”: “四、应用场景拓展”, “guideline”: “说明在代码生成、内容创作、逻辑推理等场景的专项优化”\}, \{“h2”: “五、用户使用体验”, “guideline”: “描述实际应用中的响应速度、交互流畅度等体验改善”\]\}’\}\}

</p>
</details>

### Before submitting a new issue…

- [x] Make sure you already searched for relevant issues, and asked the chatbot living at the bottom right corner of the [documentation page](https://docs.vllm.ai/en/latest/), which can answer lots of frequently asked questions.

---

## comments

**chaunceyjiang** (2025-06-03T09:18:09Z):

~~Can you test this PR [vllm-project/vllm#17506](https://github.com/vllm-project/vllm/pull/17506) to see if it resolves your issue?~~

**LiuLi1998** (2025-06-03T09:29:16Z):

> Can you test this PR [#17506](https://github.com/vllm-project/vllm/pull/17506) to see if it resolves your issue?

Thanks, I think when i set tool_choice=“auto”, tool_choice_function_name will be none, so the code will not implement the logic of this PR.

**LiuLi1998** (2025-06-03T09:33:17Z):

> Can you test this PR [#17506](https://github.com/vllm-project/vllm/pull/17506) to see if it resolves your issue?

My start cmd:
vllm
serve
/model
—host 0.0.0.0
—port “$SERVICE_PORT"     --gpu-memory-utilization 0.9
    -tp "$gpu_count”
—block-size 256
—enable-chunked-prefill
—max_num_batched_tokens 4096
—max-model-len 131072
—max_seq_len_to_capture 131072
—enable-auto-tool-choice
—tool-call-parser hermes

**chaunceyjiang** (2025-06-03T14:32:04Z):

Could you wrap your **client code** with \`\`\` \`\`\` so I can try to reproduce the issue locally?

**LiuLi1998** (2025-06-04T02:43:51Z):

> Could you wrap your **client code** with ` ` so I can try to reproduce the issue locally?

I have updated this issue

**vladrad** (2025-06-06T03:52:23Z):

> Can you test this PR [#17506](https://github.com/vllm-project/vllm/pull/17506) to see if it resolves your issue?

I tested this with thinking on/off and no go.

**LiuLi1998** (2025-06-06T07:32:20Z):

> > Can you test this PR [#17506](https://github.com/vllm-project/vllm/pull/17506) to see if it resolves your issue?
>
> I tested this with thinking on/off and no go.

you can test [vllm-project/vllm#19058](https://github.com/vllm-project/vllm/pull/19058), works for me

**vladrad** (2025-06-06T16:45:26Z):

I tested this morning. I applied your pr to the file. restarted server ( I don’t think I need a rebuild).



```
Choice(delta=ChoiceDelta(content='\n\n', function_call=None, refusal=None, role=None, tool_calls=[]), finish_reason=None, index=0, logprobs=None)
Choice(delta=ChoiceDelta(content=None, function_call=None, refusal=None, role=None, tool_calls=[ChoiceDeltaToolCall(index=0, id='chatcmpl-tool-436d347bb35549aa83758b8fe65c4fb6', function=ChoiceDeltaToolCallFunction(arguments=None, name='filesystem/directory_tree'), type='function')]), finish_reason=None, index=0, logprobs=None)
Choice(delta=ChoiceDelta(content=None, function_call=None, refusal=None, role=None, tool_calls=[ChoiceDeltaToolCall(index=0, id=None, function=ChoiceDeltaToolCallFunction(arguments='{"path": ".', name=None), type=None)]), finish_reason=None, index=0, logprobs=None)
Choice(delta=ChoiceDelta(content='', function_call=None, refusal=None, role=None, tool_calls=[]), finish_reason='tool_calls', index=0, logprobs=None)


{"path": ". #<---- incorrect
```

I am using openai sdk. this is what the last tokens look like.

I tried both reasoning on/off. I tried 2-3 different chat templates. ChoiceDeltaToolCallFunction(arguments=’\{“path”: ”.’, name=None).

All of them render the params incorrectly when doing tool calls.

**motocoder-cn** (2025-06-11T07:23:05Z):

@LiuLi1998 [#19058](https://github.com/aarnphm/aarnphm.github.io/issues/19058) not works for me, but hermes real has some problem in parse qwen3 tool

**cdhcrs** (2025-06-16T03:28:38Z):

Qwen2.5 also has the same issue.

