---
title: '59119'
pageLayout: default
slug: thoughts/inference/vllm-prs/59119
permalink: https://aarnphm.xyz/thoughts/inference/vllm-prs/59119.md
generator:
  quartz: v4.6.0
  hostedProvider: Cloudflare
  baseUrl: aarnphm.xyz
full: https://aarnphm.xyz/llms-full.txt
---
# [#59119](https://github.com/aarnphm/aarnphm.github.io/issues/59119) \[DSv4.1\] Avoid runtime recompiles of \_ring_slot_mapping_kernel

[vllm-project/vllm#59119](https://github.com/vllm-project/vllm/pull/59119) · Juntian777 · MERGED · created 2026-09-29T02:25:30Z · merged 2026-09-29T07:31:30Z

## Summary

`_ring_slot_mapping_kernel` builds the compressor ring slot mapping in `CompressorMetadataBuilder.build()` and runs on every step. Its int args `num_tokens` and `num_actual_tokens` change from step to step, and Triton specializes int args by value (`== 1`, `% 16 == 0`, or neither), so each new combination compiles a new variant while serving. These compiles are not covered by startup warmup because the triggering combinations depend on the live batch (for example, a real token count that is a multiple of 16 padded to one that is not). Under DP/EP, a rank that stops to compile stalls every other rank at the next collective.

This PR marks `block_table_stride`, `num_actual_tokens` and `num_tokens` as `do_not_specialize`, so one variant is compiled at startup and reused for every batch. These args only feed address and mask arithmetic; dropping their divisibility hints narrows some loads/stores, but the kernel is latency-bound on its dependent loads and performance is unchanged (see below).

Evidence from a DSv4.1-Flash DEP4 AgentX run on GB200: a new variant of this kernel was compiled 46 min after the server was healthy, in the middle of a benchmark, and the Triton cache holds 6 variants of it.

This does not overlap with [#58560](https://github.com/aarnphm/aarnphm.github.io/issues/58560), which changes the kernel’s store mask (null block), not its specialization.

## Test Plan

- `pytest tests/kernels/test_compressor_kv_cache.py`
- Kernel microbenchmark: main (bare `@triton.jit`) vs this PR on GB200, FlashInfer `bench_gpu_time_with_cupti` (CUPTI), `block_table_stride=1` and `CAPACITY=8` (5 draft tokens) as in serving. 37 token counts from 1 to 8192, exact and padded; outputs match exactly.

## Test Result

- 136 passed, 13 skipped.
- Performance is unchanged. PR / main geomean over the 37 shapes is 1.005 with cold L2 (range 0.99–1.02) and 0.98 with warm L2 (noise at \~1.5 us). Compiled variants over the sweep: main 5, PR 1.

Cold L2, a subset of the shapes:

| real / padded tokens | main (us) | PR (us) |
| -------------------- | --------- | ------- |
| 1 / 1                | 2.11      | 2.14    |
| 42 / 48              | 1.98      | 2.02    |
| 256 / 256            | 2.02      | 2.02    |
| 366 / 368            | 1.70      | 1.73    |
| 1500 / 1504          | 1.86      | 1.89    |
| 4096 / 4096          | 2.27      | 2.24    |
| 8188 / 8192          | 2.24      | 2.24    |

AI assistance (Claude Code) was used; I reviewed the change and ran the tests and benchmark above.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---

## comments

**zyongye** (2026-09-29T06:15:52Z):

/ci run

**github-actions** (2026-09-29T06:16:59Z):

✅ Triggered [Buildkite CI #91761](https://buildkite.com/vllm/ci/builds/91761) for commit `6ff49a6fcf65`.

