Installation

Basic install

uv sync

This installs all core dependencies defined in pyproject.toml.

All extras at once

The recommended way to install for most users:

uv sync --all-extras

This installs all optional extras (flash-attn, flash-attn-cute, etc.) in one go.

FP8 inference with deep-gemm

For certain models like GLM-5-FP8, you need deep-gemm. Install it via the fp8-inference dependency group:

uv sync --group fp8-inference

This installs the pre-built deep-gemm wheel. No CUDA build step is needed.

Dev dependencies

uv sync --group dev

Installs pytest, ruff, pre-commit, and other development tools.

Key files

• pyproject.toml — all dependencies, extras, and dependency groups
• uv.lock — pinned lockfile (update with uv sync --all-extras)