sgl-project/sglang
Naming conventions for SGLang speculative decoding identifiers.
npx skills add sgl-project/sglang --skill speculative-naming
Related neighbors and high-traction skills in the same topics — useful to compare before installing.
Add EAGLE-3 or draft-model speculative decoding to a Jetson vLLM server when TPOT is the bottle…
1.1K installsAccelerate LLM inference using speculative decoding, Medusa multiple heads, and lookahead decod…
750 installsAccelerate LLM inference using speculative decoding, Medusa multiple heads, and lookahead decod…
344 installsOther skills from sgl-project/sglang · top by installs.
npx skills add sgl-project/sglang
Declared targets from SKILL.md / docs. Unmarked agents are not listed — the skill may still install via the CLI.
main
Files included with this skill beyond the listing page.
SKILL.md
6,298 B
SUMMARY.md
324 B
Apply this skill when adding, renaming, or reviewing identifiers in speculative decoding code (anything under python/sglang/srt/speculative/, related attention backends, scheduler accumulators, IPC fields, observability metrics, or CLI flags).
-edUse the verb form accept everywhere. Don't use the past-participle form accepted.
| Don't | Do |
|---|---|
numacceptedtokens |
numaccepttokens |
accepted_indices |
accept_indices |
acceptedtokenids |
accept_tokens (also see Rule 3) |
bonustoken / bonustokensThe "+1" token that the target model always emits in addition to verifying drafts is the bonus token. Use bonustoken / bonustokens per Rule 7.
| Don't | Do |
|---|---|
verifiedid / verifiedids |
bonustoken / bonustokens |
outputid / outputids (when referring to the bonus) |
bonustoken / bonustokens |
req.output_ids (the full output history of a request) is unrelated and stays as is.
accept includes bonus; correct excludes bonusThe semantic distinction lives in the verb, not the noun. Don't enumerate noun pairs.
| Verb | Meaning |
|---|---|
**accept_*** |
Includes the bonus token |
**correct_*** |
Drafts only, no bonus |
Pair with whatever noun fits the data (tokens, drafts, indices, …). No required pairing, but preferred default nouns: accepttokens and correctdrafts — correct semantically describes drafts (what got verified), accept describes the resulting token sequence (incl. bonus).
| Form | Meaning |
|---|---|
accepttokens / acceptindices |
Include bonus |
correct_drafts |
Drafts only, no bonus |
numaccepttokens |
Count incl. bonus |
numcorrectdrafts |
Count excl. bonus |
acceptrate / acceptlength follow paper conventionThese two metric names are entrenched in the spec-decoding literature and in external-facing fields (meta_info, Prometheus). Their semantics are paper-defined, not Rule-3-defined:
| Name | Paper term | Bonus? | Definition |
|---|---|---|---|
accept_rate |
$\alpha$ (Leviathan 2023) | No | per-draft-token acceptance probability = correctdrafts / proposeddrafts |
accept_length |
$\tau$ (EAGLE) | Yes | avg tokens per verify step = completiontokens / verifyct |
Internal counters still follow Rule 3 strict semantics: numcorrectdrafts (no bonus), numaccepttokens (with bonus).
num for counts; ct for counters; _rate for rates; no prefix for IDsEach form has its own marker. Never mix (no numXct, no numacceptrate).
| Form | Pattern | Meaning | Examples |
|---|---|---|---|
| Count | num_X |
Snapshot quantity at one point in time (often a tensor or scalar) | numaccepttokens, numcorrectdrafts, numproposeddrafts |
| Counter | X_ct |
Monotonically incrementing accumulator over time | specverifyct, forward_ct |
| Rate / ratio | X_rate |
Fractional value in [0, 1] |
accept_rate |
| Tokens / content array | no prefix | The actual token data, not a count | accepttokens, correctdrafts, bonus_token |
tokenid / tokenids suffix in spec scopeid / ids and token / tokens are both fine. But don't combine — tokenid / tokenids is redundant inside spec decoding, because spec code only ever deals with vocab integers.
The semantic differs by scope:
| Scope | Example | What tokenid means |
|---|---|---|
| Framework / multimodal / tokenizer | imagetokenid, padtokenid, eostokenid, masktokenid, bostokenid |
A specific named/role token's vocab ID. The prefix names the role; tokenid says it's the integer ID for that role. Both halves carry information. |
| Spec decoding | acceptedtokenids, currtokenid, outtokenids |
Redundant. Spec only deals with vocab integers; id adds nothing beyond token. |
| Don't | Do |
|---|---|
acceptedtokenids |
accept_tokens (Rule 1 + 3) |
currtokenid |
current_token |
outtokenids |
out_tokens |
resolvespecoverlaptoken_ids |
resolvespecoverlaptokens |
Plural for any non-scalar tensor ([bs]-shaped, flat, or multi-dim); singular only for scalars (kernel tl.load results, single-int locals). Applies to all spec-decoding tensors (tokens, indices, etc.).
accept_tokens: torch.Tensor # [total_accepted] flat - plural
accept_indices: torch.Tensor # [bs, num_draft_tokens] - plural
draft_tokens: torch.Tensor # [bs * num_draft_tokens] flat - plural
bonus_tokens: torch.Tensor # [bs] - plural
accept_token = tl.load(...) # int32 scalar in a kernel iteration - singular
bonus_token = tl.load(...) # int32 scalar inside a kernel - singularThese rules apply to spec-decoding-specific identifiers. Pre-existing or framework-level names are kept.
seqlens, extendseqlens, cuseqlens_qimagetokenid, padtokenid, eostokenid, masktokenid, hottokenid, bostokenid, topk_idreq.inputids, req.outputids, req.origininputids, nexttokenids (model_runner.sample output)accepttokennum (sgl-kernel)reqid, gpuid, layerid, programidlen / lens names: numX is preferred for counts (Rule 4), but len / lens names are acceptable. Triton kernel params in particular often use lens / len to align with the PyTorch ecosystem (seqlens, cuseqlensq). Rule 1 still requires the -ed-less form (acceptlength OK, acceptedlength not).