SKILL.md
MLflow Python Skill
Unified read/write MLflow operations via Python API with QuantStats integration for comprehensive trading metrics.
ADR: [2025-12-12-mlflow-python-skill](/docs/adr/2025-12-12-mlflow-python-skill.md)
Note: This skill uses Pandas (MLflow API requires it). The
mlflow-pythonpath is auto-skipped by the Polars preference hook.
Self-Evolving Skill: This skill improves through use. If instructions are wrong, parameters drifted, or a workaround was needed — fix this file immediately, don't defer. Only update for real, reproducible issues.
When to Use This Skill
CAN Do:
- Log backtest metrics (Sharpe, maxdrawdown, totalreturn, etc.)
- Log experiment parameters (strategy config, timeframes)
- Create and manage experiments
- Query runs with SQL-like filtering
- Calculate 70+ trading metrics via QuantStats
- Retrieve metric history (time-series data)
CANNOT Do:
- Direct database access to MLflow backend
- Artifact storage management (S3/GCS configuration)
- MLflow server administration
Prerequisites
Authentication Setup
MLflow uses separate environment variables for credentials (NOT embedded in URI):
# Option 1: mise + .env.local (recommended)
# Create .env.local in skill directory with:
MLFLOW_TRACKING_URI=http://mlflow.eonlabs.com:5000
MLFLOW_TRACKING_USERNAME=eonlabs
MLFLOW_TRACKING_PASSWORD=<password>
# Option 2: Direct environment variables
export MLFLOW_TRACKING_URI="http://mlflow.eonlabs.com:5000"
export MLFLOW_TRACKING_USERNAME="eonlabs"
export MLFLOW_TRACKING_PASSWORD="<password>"Verify Connection
/usr/bin/env bash << 'SKILL_SCRIPT_EOF'
ROOT="$(cc-plugin-root devops-tools)"
cd "$ROOT/skills/mlflow-python"
uv run scripts/query_experiments.py experiments
SKILL_SCRIPT_EOFQuick Start Workflows
A. Log Backtest Results (Primary Use Case)
/usr/bin/env bash << 'SKILL_SCRIPT_EOF_2'
ROOT="$(cc-plugin-root devops-tools)"
cd "$ROOT/skills/mlflow-python"
uv run scripts/log_backtest.py \
--experiment "crypto-backtests" \
--run-name "btc_momentum_v2" \
--returns path/to/returns.csv \
--params '{"strategy": "momentum", "timeframe": "1h"}'
SKILL_SCRIPT_EOF_2B. Search Experiments
uv run scripts/query_experiments.py experimentsC. Query Runs with Filter
uv run scripts/query_experiments.py runs \
--experiment "crypto-backtests" \
--filter "metrics.sharpe_ratio > 1.5" \
--order-by "metrics.sharpe_ratio DESC"D. Create New Experiment
uv run scripts/create_experiment.py \
--name "crypto-backtests-2025" \
--description "Q1 2025 cryptocurrency trading strategy backtests"E. Get Metric History
uv run scripts/get_metric_history.py \
--run-id abc123 \
--metrics sharpe_ratio,cumulative_returnQuantStats Metrics Available
The log_backtest.py script calculates 70+ metrics via QuantStats, including:
| Category | Metrics |
|---|---|
| Ratios | sharpe, sortino, calmar, omega, treynor |
| Returns | cagr, totalreturn, avgreturn, best, worst |
| Drawdown | maxdrawdown, avgdrawdown, drawdown_days |
| Trade | winrate, profitfactor, payoffratio, consecutivewins/losses |
| Risk | volatility, var, cvar, ulcerindex, serenityindex |
| Advanced | kellycriterion, recoveryfactor, riskofruin, information_ratio |
See [quantstats-metrics.md](./references/quantstats-metrics.md) for full list.
Bundled Scripts
| Script | Purpose |
|---|---|
log_backtest.py |
Log backtest returns with QuantStats metrics |
query_experiments.py |
Search experiments and runs (replaces CLI) |
create_experiment.py |
Create new experiment with metadata |
getmetrichistory.py |
Retrieve metric time-series data |
Configuration
The skill uses mise [env] pattern for configuration. See .mise.toml for defaults.
Create .env.local (gitignored) for credentials:
MLFLOW_TRACKING_URI=http://mlflow.eonlabs.com:5000
MLFLOW_TRACKING_USERNAME=eonlabs
MLFLOW_TRACKING_PASSWORD=<password>Reference Documentation
- [Authentication Patterns](./references/authentication.md) - Idiomatic MLflow auth
- [QuantStats Metrics](./references/quantstats-metrics.md) - Full list of 70+ metrics
- [Query Patterns](./references/query-patterns.md) - DataFrame operations
- [Migration from CLI](./references/migration-from-cli.md) - CLI to Python API mapping
Migration from mlflow-query
This skill replaces the CLI-based mlflow-query skill. Key differences:
| Feature | mlflow-query (old) | mlflow-python (new) |
|---|---|---|
| Log metrics | Not supported | mlflow.log_metrics() |
| Log params | Not supported | mlflow.log_params() |
| Query runs | CLI text parsing | DataFrame output |
| Metric history | Workaround only | Native support |
| Auth pattern | Embedded in URI | Separate env vars |
See [migration-from-cli.md](./references/migration-from-cli.md) for detailed mapping.
Troubleshooting
| Issue | Cause | Solution |
|---|---|---|
| Connection refused | MLflow server not running | Verify MLFLOWTRACKINGURI and server status |
| Authentication failed | Wrong credentials | Check MLFLOWTRACKINGUSERNAME and PASSWORD in .env |
| Experiment not found | Experiment name typo | Run query_experiments.py experiments to list all |
| QuantStats import error | Missing dependency | uv add quantstats in skill directory |
| Pandas import warning | Expected for this skill | Ignore - MLflow requires Pandas (hook-excluded) |
| Run creation fails | Experiment doesn't exist | Use create_experiment.py to create first |
| Metric history empty | Wrong run_id or metric name | Verify runid with queryexperiments.py runs |
| Returns CSV parse error | Wrong date format or columns | Check CSV has date index and returns column |
Post-Execution Reflection
After this skill completes, check before closing:
- Did the command succeed? — If not, fix the instruction or error table that caused the failure.
- Did parameters or output change? — If the underlying tool's interface drifted, update Usage examples and Parameters table to match.
- Was a workaround needed? — If you had to improvise (different flags, extra steps), update this SKILL.md so the next invocation doesn't need the same workaround.
Only update if the issue is real and reproducible — not speculative.