Data Source Priority - Alpaca API MANDATORY (v3.6.0)

CRITICAL POLICY

Alpaca API is the ONLY valid source for OHLCV (price/volume) data.

yfinance is NEVER a valid fallback for market data. Any code that falls back to yfinance for price or volume data is a BUG that must be fixed.

Permitted yfinance Usage (ONLY)

Why This Policy Exists

• yfinance crypto data has ~50% zero-volume bars
• yfinance equity data has gaps and unreliable volume
• Training on bad data produces bad models
• Silent fallbacks waste debugging time

Experiment Overview

Context

User reported ALL 18 crypto symbols failing selection filters:

BTCUSD: failed price (max_price $10k but BTC is $87k)
ETHUSD: failed data_quality (51% zero-volume bars)
Others: failed volume, trading_status

Root Cause: Alpaca API keys weren't configured, so DataFetcher fell back to yfinance. yfinance crypto data has:

• ~50% zero-volume bars (data gaps)
• Missing volume data on many bars
• Causes ALL filters to fail

v2.5.0 Solution: Fail Fast

OLD BEHAVIOR (v2.4.x): Warned about yfinance but continued anyway, wasting time debugging filter failures.

NEW BEHAVIOR (v2.5.0): Training notebook FAILS IMMEDIATELY if crypto is enabled without Alpaca API:

# In training notebook cell-16
alpaca_enabled = data_fetcher._use_alpaca_data

if not alpaca_enabled and selection_config.crypto.enabled:
    raise ValueError(
        'CRYPTO TRAINING REQUIRES ALPACA API. '
        'yfinance crypto data has ~50% missing volume. '
        'Set API keys or disable crypto training.'
    )

Clear Error Message

When crypto is enabled without Alpaca API:

======================================================================
ERROR: CRYPTO TRAINING REQUIRES ALPACA API
======================================================================

yfinance crypto data is unusable for training:
  - ~50% zero-volume bars
  - Causes all symbols to fail volume/data_quality filters
  - Training on bad data produces bad models

FIX OPTIONS:
  1. Set Alpaca API keys in Colab Secrets:
     - APCA_API_KEY_ID = your_key
     - APCA_API_SECRET_KEY = your_secret

  2. Set API_KEYS_FILE in previous cell:
     API_KEYS_FILE = "/content/Alpaca_trading/API_key_500Paper.txt"

  3. Disable crypto and train only equities:
     selection_config.crypto.enabled = False
     selection_config.equities.enabled = True

Failed Attempts (Critical)

Data Quality Comparison

API Key Configuration

For Google Colab (RECOMMENDED)

1. Go to Colab Secrets (key icon in left sidebar)
2. Add APCAAPIKEY_ID = your API key
3. Add APCAAPISECRET_KEY = your secret key
4. Enable access to notebook

For Training Notebook

# Cell 14: Option 1 - Environment variables (recommended)
# Keys are read from Colab Secrets automatically

# Cell 15: Option 2 - Keys file (after unzipping repo)
API_KEYS_FILE = '/content/Alpaca_trading/API_key_500Paper.txt'

For Local Development

# Add to ~/.bashrc or ~/.zshrc
export APCA_API_KEY_ID="your_key"
export APCA_API_SECRET_KEY="your_secret"

Diagnostic Check

from alpaca_trading.data.fetcher import DataFetcher
fetcher = DataFetcher(keys_file=API_KEYS_FILE)

# This is checked BEFORE selection in v2.5.0
if not fetcher._use_alpaca_data:
    print('Alpaca API: NOT AVAILABLE')
    # Notebook will fail here if crypto enabled
else:
    print('Alpaca API: ENABLED')

Key Insights

1. Alpaca API is MANDATORY - No exceptions, no fallbacks
2. yfinance is NOT a fallback - It's only for sector lookups on equity symbols
3. Fail fast, fail loud - Silent fallbacks waste debugging time and produce bad models
4. Environment variables are best - Work everywhere, no path issues
5. Check logs for "yfinance fetched" - This indicates a BUG that must be fixed
6. Any fallback code is a bug - If you see code falling back to yfinance for OHLCV, fix it

Files Modified (v2.5.0)

notebooks/training.ipynb:
  - Cell 16: Added fail-fast check before symbol selection
  - Cell 0: Updated header to v2.5.0 with changelog

alpaca_trading/selection/filters/hard_filters.py:
  - apply_hard_filters(): Simplified crypto path (skip yfinance-specific checks)

MCP Server Configuration

The project uses Alpaca's official MCP server for AI-assisted trading operations:

// .mcp.json
{
  "mcpServers": {
    "alpaca": {
      "type": "stdio",
      "command": "uvx",
      "args": ["--with", "pytz", "alpaca-mcp-server", "serve"],
      "env": {
        "ALPACA_API_KEY": "your_key",
        "ALPACA_SECRET_KEY": "your_secret",
        "ALPACA_PAPER_TRADE": "True"
      }
    }
  }
}

Note: The --with pytz flag is required to fix a missing dependency in the alpaca-mcp-server package.

References

• alpaca_trading/data/fetcher.py: DataFetcher implementation
• notebooks/training.ipynb: Training notebook with fail-fast check
• .mcp.json: MCP server configuration
• Alpaca API docs: https://docs.alpaca.markets/docs/
• Alpaca MCP Server: https://github.com/alpacahq/alpaca-mcp-server
• Skill: reward-function-hold-bias - Related v2.5.0 fix