---
title: '58814'
pageLayout: default
slug: thoughts/inference/vllm-prs/58814
permalink: https://aarnphm.xyz/thoughts/inference/vllm-prs/58814.md
generator:
  quartz: v4.6.0
  hostedProvider: Cloudflare
  baseUrl: aarnphm.xyz
full: https://aarnphm.xyz/llms-full.txt
---
# [#58814](https://github.com/aarnphm/aarnphm.github.io/issues/58814) \[Bugfix\]\[Kimi-K3\] Refresh DSpark context KV cache pointers after the KV cache is re-bound

[vllm-project/vllm#58814](https://github.com/vllm-project/vllm/pull/58814) · okorzh-amd · MERGED · created 2026-09-26T02:12:19Z · merged 2026-09-28T16:05:11Z

\##Purpose

Fixes a GPU memory fault and silent memory corruption in the Kimi-K3 DSpark draft, introduced by [#57632](https://github.com/aarnphm/aarnphm.github.io/issues/57632).



```
21:05:36  JIT kernel warmup finished            (all 8 ranks)
          Warning: Queue error - HSA_STATUS_ERROR_MEMORY_FAULT
          Memory access fault by GPU node-14 ... on address 0x7ec3c95b0000. Reason: Unknown.
21:15:38  [Rank 0..7] Watchdog caught collective operation timeout: _ALLGATHER_BASE ... 600000 ms
          RuntimeError: Engine core initialization failed
```

K3DSparkModel caches the per-layer KV cache base pointers, and the block-layout flag, on first use and never rebuilds them. Since [#57632](https://github.com/aarnphm/aarnphm.github.io/issues/57632) captures the context K/V precompute in the draft CUDA graph, that first use now happens during the CUDA graph memory-profiling capture. That capture runs against a temporary KV cache that is freed afterwards. The real KV cache is then allocated at different addresses, but the draft keeps the stale pointers, so every later context K/V insert writes into freed memory.

This change keys both cached values on the live kv_cache.data_ptr() tuple and rebuilds them when it changes. The pointer array is updated in place, so captured graphs keep reading live pointers. The rebuild only runs on eager calls; an assert guards against it running during capture.

kimi_k3/nvidia/dspark_mla.py is the only K3 DSpark implementation and is used on every platform, so this also applies to CUDA. It was only tested on ROCm.

\##Test Plan

- 8x MI355X, Kimi-K3 TP8 + DSpark (ROCM_AITER_MLA, fp8 KV cache), nightly [`29468dd`](https://github.com/aarnphm/aarnphm.github.io/commit/29468dde) (which contains [#57632](https://github.com/aarnphm/aarnphm.github.io/issues/57632)).
- Recipe config for the conc-4 cell: —max-num-seqs 8, K=5, FULL_AND_PIECEWISE, 8k/1k random prompts.

\##Test Result

<img width="704" height="107" alt="image" src="https://github.com/user-attachments/assets/110d07c8-3f14-4272-849a-2bbe190ad27d" />

---

## comments

**github-actions** (2026-09-26T02:17:37Z):

✅ @okorzh-amd, CI is now available for this PR.

- `/ci run` starts upstream CI; `/amd-ci run` starts AMD CI only.
- Your branch must contain every commit currently on its upstream target branch. Merge or rebase onto the latest target branch, then rerun the command. Append `--allow-stale` to a run command to test an outdated branch at your own risk.
- `/ci retry` retries failed jobs in the CI build for the current PR head. If the current head has no CI build, it starts a new CI build for the current head containing only jobs that failed in the latest earlier CI build for this PR.
- `/amd-ci retry` retries failed jobs in AMD CI for the current PR head. Use `/amd-ci run` when the current head has no AMD CI build.
- `/ci cancel` cancels scheduled or running CI builds for this PR branch; `/amd-ci cancel` does the same for AMD CI only.

<!-- vllm-ci-authorized -->

**okorzh-amd** (2026-09-26T02:18:20Z):

/ci run

**github-actions** (2026-09-26T02:18:38Z):

❌ This PR is 1 commit behind upstream `main`. Your branch must contain every commit currently on upstream `main`. No new CI build was started. Merge or rebase onto the latest `main`, then rerun `/ci run`. To test this branch at your own risk, use `/ci run --allow-stale`.

<!-- vllm-ci-command:5842312070 -->

**okorzh-amd** (2026-09-26T02:24:35Z):

/ci run

**github-actions** (2026-09-26T02:24:54Z):

✅ Triggered [Buildkite CI #91368](https://buildkite.com/vllm/ci/builds/91368) for commit `92b6ac9a8d78`.

**okorzh-amd** (2026-09-26T02:28:10Z):

@Juntian777 could you please help to review?

**okorzh-amd** (2026-09-26T21:54:45Z):

/ci run

**github-actions** (2026-09-26T21:55:03Z):

❌ This PR is 16 commits behind upstream `main`. Your branch must contain every commit currently on upstream `main`. No new CI build was started. Merge or rebase onto the latest `main`, then rerun `/ci run`. To test this branch at your own risk, use `/ci run --allow-stale`.

<!-- vllm-ci-command:5850226337 -->

**okorzh-amd** (2026-09-26T21:55:57Z):

/ci run

**github-actions** (2026-09-26T21:56:16Z):

✅ Triggered [Buildkite CI #91426](https://buildkite.com/vllm/ci/builds/91426) for commit `73ae7a80dfe6`.

