---
title: '60235'
pageLayout: default
slug: thoughts/inference/vllm-prs/60235
permalink: https://aarnphm.xyz/thoughts/inference/vllm-prs/60235.md
generator:
  quartz: v4.6.0
  hostedProvider: Cloudflare
  baseUrl: aarnphm.xyz
full: https://aarnphm.xyz/llms-full.txt
---
# [#60235](https://github.com/aarnphm/aarnphm.github.io/issues/60235) \[Docs\] Run pr-checklist before opening a PR

[vllm-project/vllm#60235](https://github.com/vllm-project/vllm/pull/60235) · ZJY0516 · MERGED · created 2026-10-06T13:31:59Z · merged 2026-10-06T14:12:19Z

## Overview

Make agents run the `pr-checklist` skill before they open a PR, drafts included. Right now they often skip it or put it off until the PR is marked ready. Follow-up to [#57084](https://github.com/aarnphm/aarnphm.github.io/issues/57084) and [#59638](https://github.com/aarnphm/aarnphm.github.io/issues/59638).

## Claims

- `pr-checklist` skill description: says when to use the skill (before opening a PR, drafts included, or before pushing review fixes). This is the text Claude Code reads when it decides whether to invoke a skill.
- `AGENTS.md`: replaces the advisory “Use … before submitting a PR” with a concrete step that covers draft PRs. The line count is unchanged (159 lines).

## Validation

A/B test with Claude Code (Opus 5.5) in headless mode. Each run had its own clone and was given the same prompt: “the fix on this branch is committed; push it and open a draft PR to vllm-project/vllm”. The test branch was a real bugfix ([#60071](https://github.com/aarnphm/aarnphm.github.io/issues/60071)) cherry-picked onto its parent commit, with `gh` replaced by a stub that only logs calls. Each run used one of two docs variants, with the variant folded into the base commit:

| Docs variant | Runs | Used pr-checklist before `gh pr create` (or before stopping) |
| ------------ | ---- | ------------------------------------------------------------ |
| `main`       | 3    | 1/3                                                          |
| this PR      | 3    | 3/3 (2 via the Skill tool, 1 by reading `SKILL.md`)          |

In the two `main` runs that skipped the checklist, the agent still did the other pre-PR steps listed in `AGENTS.md`: the duplicate-PR search, pre-commit, and tests. In one run of this PR’s variant (new-1), the agent ran the checklist but then stopped without opening the PR, because it noticed the remote was not GitHub.



```bash
pre-commit run --files AGENTS.md .agents/skills/pr-checklist/SKILL.md  # passed
```

## Details

- Motivation: in practice, agents read “before submitting a PR” as “before marking ready for review”. They open a draft and defer the checklist, as in [#60210](https://github.com/aarnphm/aarnphm.github.io/issues/60210). The skill description also gave no trigger moment, so nothing linked the skill to `gh pr create`.
- Limitations: the sample is small (n=3 per variant) and used a single model and a single prompt. The runs also loaded the submitter’s user-level Claude Code config. Wording alone cannot guarantee the checklist runs; a hard gate would need tooling such as a hook, which is out of scope here.
- Not a duplicate: [#60111](https://github.com/aarnphm/aarnphm.github.io/issues/60111) edits section 4.3 of the same `SKILL.md` but not its frontmatter, so the two PRs do not conflict. No other open PR or issue covers this.

---

<details>
<summary> Pull Request Checklist </summary>

- [x] I used vLLM’s `/pr-checklist` skill. (Mandatory for agents, optional for humans).

- [x] AI assistance was used during the creation of this PR.

- [x] **Design Fit:** Minimizes impact on core components, reuses existing functionality, and justifies added complexity.

- [x] **Testing and Validation:** Validates the change and ensures any added tests are meaningful and reliable, with CI coverage or documented CI resource constraints and validation performed outside CI.

- [x] **Code Quality and Style:** Keeps code and comments clear and concise, and updates relevant documentation and examples.

- [x] **Pull Request Contents:** Includes a brief summary and relevant links, supports claims with evidence, explains root causes and implementation trade-offs, and follows the contributing guide.

</details>

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---

## comments

**benchislett** (2026-10-06T13:58:05Z):

/ci run

**github-actions** (2026-10-06T13:58:26Z):

❌ This PR is 102 commits behind upstream `main`. Your branch must contain every commit currently on upstream `main`. No new CI build was started. Merge or rebase onto the latest `main`, then rerun `/ci run`. To test this branch at your own risk, use `/ci run --allow-stale`.

<!-- vllm-ci-command:6017865498 -->

**ZJY0516** (2026-10-06T14:00:45Z):

/ci run —allow-stale

**github-actions** (2026-10-06T14:02:01Z):

✅ Triggered [Buildkite CI #93095](https://buildkite.com/vllm/ci/builds/93095) for commit `c092a7689138`.

