Vllm Async Loading Debug

Overview

Trace the async KV loading path between vLLM scheduler and worker connectors, including prefetch and preemption edges.

Workflow

1) Confirm async-load symptom

• Record request IDs and statuses: WAITING, RUNNING, WAITINGFORREMOTE_KVS, PREEMPTED.
• Check async-load logs and whether prefetch is in progress.

2) Follow scheduler-side decisions

• python/pegaflow/connector/scheduler.py: prefetch query + LoadIntent creation.
• .project-plans/scheduler.py (private): loadkvasync gating and WAITINGFORREMOTE_KVS transitions.

3) Follow worker-side lifecycle

• python/pegaflow/connector/worker.py: startloadkv() calls engine_client.load() and tracks PyLoadState.
• getfinished() polls isready() and emits finished_recving.

4) Check preemption and retries

• .project-plans/scheduler.py (private): preemptrequest() and resetprefixcache() behavior.
• invalidblockids handling can trigger recompute or failure based on kvloadfailure_policy.

Reference Files

• Read references/async-loading.md for full call flow and log markers.