---
title: '44800'
pageLayout: default
slug: thoughts/inference/vllm-prs/44800
permalink: https://aarnphm.xyz/thoughts/inference/vllm-prs/44800.md
generator:
  quartz: v4.6.0
  hostedProvider: Cloudflare
  baseUrl: aarnphm.xyz
full: https://aarnphm.xyz/llms-full.txt
---
# [#44800](https://github.com/aarnphm/aarnphm.github.io/issues/44800) \[Core\] Add `VLLM_GPU_SYNC_CHECK` env var

[vllm-project/vllm#44800](https://github.com/vllm-project/vllm/pull/44800) · njhill · MERGED · created 2026-06-07T18:10:03Z · merged 2026-06-26T15:24:33Z

vLLM now uses asynchronous scheduling by default and in the majority of cases. Performance relies on the absence of any gpu<span>&larr;</span>>cpu synchronizations on the main cuda stream, but such syncs can be opaque and it is easy for them to creep in accidentally.

This change adds a `VLLM_GPU_SYNC_CHECK` env var which enables `torch.cuda.set_sync_debug_mode` for the model forward pass and sampler, so that we can easily check for such syncs.

A new `gpu_sync_allowed()` context manager can be used to wrap known syncs (i.e. temporary or unavoidable).

This PR is a subset of [vllm-project/vllm#43107](https://github.com/vllm-project/vllm/pull/43107) and includes the changes just to add the env var and supporting machinery, without actually enabling it in CI yet.

I’ll update that other PR or open a new ones to address the remaining gpu<span>&larr;</span>>cpu syncs (in many places via use of `gpu_sync_allowed`).

Claude was used to help with this.

---

## comments

**mergify** (2026-06-08T16:35:50Z):

Hi @njhill, the pre-commit checks have failed. Please run:



```bash
uv pip install pre-commit>=4.5.1
pre-commit install
pre-commit run --all-files
```

Then, commit the changes and push to your branch.

For future commits, `pre-commit` will run automatically on changed files before each commit.

