---
title: '50109'
pageLayout: default
slug: thoughts/inference/vllm-prs/50109
permalink: https://aarnphm.xyz/thoughts/inference/vllm-prs/50109.md
generator:
  quartz: v4.6.0
  hostedProvider: Cloudflare
  baseUrl: aarnphm.xyz
full: https://aarnphm.xyz/llms-full.txt
---
# [#50109](https://github.com/aarnphm/aarnphm.github.io/issues/50109) \[Kernel\]\[CI\] `--jit-monitor-mode error` e2e tests for kernel warmup infra

[vllm-project/vllm#50109](https://github.com/vllm-project/vllm/pull/50109) · NickLucche · MERGED · created 2026-07-28T08:09:50Z · merged 2026-07-31T12:53:18Z

This PR actions this comment [vllm-project/vllm#47456 (comment)](https://github.com/vllm-project/vllm/issues/47456#issuecomment-5068299757) ([vllm-project/vllm#47456](https://github.com/vllm-project/vllm/issues/47456)) as part of the broader effort of getting JITs under control with a proper warmup framework.

It does 3 things:

- centralizes existing jit tests into a separate directory
- separates tests into a new buildkite job for better parallelization as well as tracking in PRs
- adds an e2e test for specific models which runs with `--jit-monitor-mode error`

In particular the latter purposely **only loads a subset of dummy layers** of the whole model, to keep test fast and footprint minimal.
All we aim to test here is that the paths the various blocks repeateadly call in the forwad are correctly precompiled, so we don’t need to execute all blocks, but rather make sure different kernel groups are present (FA/MoE..).

The idea is that as we start grinding models to ensure no jits are present, we add a new line here



```
JIT_MONITOR_MODELS = [
    JitModel("Qwen/Qwen3-0.6B"),
    JitModel("deepseek-ai/DeepSeek-V2-Lite-Chat", trust_remote_code=True),
    JitModel("deepseek-ai/DeepSeek-V3", trust_remote_code=True),
    JitModel("ibm-granite/granite-4.0-tiny-preview"),
```

along with the PR that addresses it.

Ultimately the goal should be able to run `--jit-monitor-mode error ` confidently in nightlies.

PS tests are currently expected to fail ‘cause we’re still fighting jits for these models, so I am skipping them for now



```
(EngineCore pid=1644919) ERROR 07-28 08:43:14 [core.py:1347]     return hook(
(EngineCore pid=1644919) ERROR 07-28 08:43:14 [core.py:1347]            ^^^^^
(EngineCore pid=1644919) ERROR 07-28 08:43:14 [core.py:1347]   File "/mnt/vast/home/nicolo.lucchesi/upstream/vllm/vllm/utils/jit_monitor.py", line 249, in _on_jit_compile
(EngineCore pid=1644919) ERROR 07-28 08:43:14 [core.py:1347]     _log_triton_jit_compile(fn_name, kwargs)
(EngineCore pid=1644919) ERROR 07-28 08:43:14 [core.py:1347]   File "/mnt/vast/home/nicolo.lucchesi/upstream/vllm/vllm/utils/jit_monitor.py", line 225, in _log_triton_jit_compile
(EngineCore pid=1644919) ERROR 07-28 08:43:14 [core.py:1347]     _handle_jit_event(
(EngineCore pid=1644919) ERROR 07-28 08:43:14 [core.py:1347]   File "/mnt/vast/home/nicolo.lucchesi/upstream/vllm/vllm/utils/jit_monitor.py", line 129, in _handle_jit_event
(EngineCore pid=1644919) ERROR 07-28 08:43:14 [core.py:1347]     raise RuntimeError(message % args)
(EngineCore pid=1644919) ERROR 07-28 08:43:14 [core.py:1347] RuntimeError: Triton kernel JIT compilation during inference: mask_empty_context_kernel. This causes a latency spike; consider extending warmup to cover this shape/config.
```

cc @LopezCastroRoberto

---

## comments

**NickLucche** (2026-07-28T08:41:29Z):

related to [vllm-project/vllm#49349](https://github.com/vllm-project/vllm/issues/49349)

