---
title: '50090'
pageLayout: default
slug: thoughts/inference/vllm-prs/50090
permalink: https://aarnphm.xyz/thoughts/inference/vllm-prs/50090.md
generator:
  quartz: v4.6.0
  hostedProvider: Cloudflare
  baseUrl: aarnphm.xyz
full: https://aarnphm.xyz/llms-full.txt
---
# [#50090](https://github.com/aarnphm/aarnphm.github.io/issues/50090) \[Kimi-K3\] Add AttnRes kernels

[vllm-project/vllm#50090](https://github.com/vllm-project/vllm/pull/50090) · gau-nernst · MERGED · created 2026-07-28T06:05:12Z · merged 2026-07-28T07:22:09Z

## Purpose

[#50000](https://github.com/aarnphm/aarnphm.github.io/issues/50000). Add AttnRes kernels

- Specialized CUDA C++ kernel for NVIDIA sm100
- Triton for NVIDIA fallback and AMD

This PR also adds Buildkite CI for K3

## Test Plan



```bash
# NVIDIA
pytest -v tests/models/kimi_k3/test_attn_res.py

# AMD
pytest -v tests/models/kimi_k3/test_amd_attn_res.py
```

## Test Result

---

<details>
<summary> Essential Elements of an Effective PR Description Checklist </summary>

- [ ] The purpose of the PR, such as “Fix some issue (link existing issues this PR will resolve)”.
- [ ] The test plan, such as providing test command.
- [ ] The test results, such as pasting the results comparison before and after, or e2e results
- [ ] (Optional) The necessary documentation update, such as updating `supported_models.md` and `examples` for a new model.

</details>

---

## comments

**namgyu-yun** (2026-07-28T13:26:11Z):

Hi @gau-nernst, question on the SM100 path here.

The kernel indexes `layer_res/delta/output` with a hard-coded row stride of `H`, threading real strides only for blocks.

Nothing enforces that: the dispatch asserts `stride(-1) == 1` but not `stride(0) == hidden_size`, and there’s no `STD_TORCH_CHECK` either. On a B200 build, feeding a row-padded `prefix (stride(0)=7175, H=7168)` gives `max|out-ref| = 6.66` against the suite’s 8e-2 — no fault, no NaN, and the in-place prefix update is wrong too.

Is packed rows an intended precondition? If the K3 model never passes padded slices, I’ll send a `STD_TORCH_CHECK` so it fails loudly. If they’re reachable, the dispatch should decline them and fall back to Triton, which already threads the strides correctly. Happy to do either and hear your thoughts\~

**gau-nernst** (2026-07-28T13:33:55Z):

@namgyu-youn Good catch. I think just add `STD_TORCH_CHECK` is enough, we don’t expect the inputs to be strided.

Right now the team is trying to get [#50000](https://github.com/aarnphm/aarnphm.github.io/issues/50000) merged, so you can open a PR after that is merged.

