markov-regime-features - Research Notes

Experiment Overview

Context

When visualizing RL observation features as a heatmap, the Markov regime features (volatility and trend probabilities) appeared as constant horizontal bands with values ~0.33, while all other features showed expected variability over time. This made the Markov features useless for regime detection.

Root Cause

The standalone buildinferenceobservation() function defaults to uniform priors when markovvolprobs and markovtrendprobs are not provided:

# From inference_obs_builder.py:391-401
if include_markov:
    if markov_vol_probs is not None:
        obs[:, feat_idx:feat_idx+3] = markov_vol_probs
    else:
        obs[:, feat_idx:feat_idx+3] = [0.33, 0.34, 0.33]  # Uniform prior

The 6 Markov features are:

Verified Workflow

Solution: Use InferenceObservationBuilder class

The InferenceObservationBuilder class maintains Markov state and updates probabilities based on price history:

from alpaca_trading.gpu.inference_obs_builder import InferenceObservationBuilder

# Create stateful builder with GPU Markov system
obs_builder = InferenceObservationBuilder(window=100, use_gpu_markov=True)

# Build observation - Markov states are updated from price history
obs = obs_builder.build(
    prices=prices,
    high=high,
    low=low,
)

# Access current regime estimates
print(f"Volatility: {obs_builder.vol_probs}")  # e.g., [0.15, 0.60, 0.25]
print(f"Trend: {obs_builder.trend_probs}")      # e.g., [0.20, 0.30, 0.50]

Visualizing Regime Evolution

To see how regimes change over time, build observations at multiple points:

eval_points = list(range(150, n_bars, 10))
vol_history = {'low': [], 'med': [], 'high': []}

for end_idx in eval_points:
    obs_builder.build(prices=prices[:end_idx], high=high[:end_idx], low=low[:end_idx])
    vol_history['low'].append(obs_builder.vol_probs[0])
    vol_history['med'].append(obs_builder.vol_probs[1])
    vol_history['high'].append(obs_builder.vol_probs[2])

plt.plot(eval_points, vol_history['high'], 'r-', label='P(High Vol)')
plt.plot(eval_points, vol_history['low'], 'g-', label='P(Low Vol)')

Failed Attempts (Critical)

Final Parameters

# Correct usage for dynamic Markov features
obs_builder = InferenceObservationBuilder(
    window=100,           # Must match training window
    use_gpu_markov=True   # Enable GPU-accelerated HMM
)

# For live trading - maintain single builder instance across bars
# For backtesting - create new builder per symbol, maintain across timesteps

Key Insights

• The standalone function is for single-shot inference where Markov state is passed externally
• The builder class is for stateful inference where Markov state evolves with price data
• GPU Markov system uses last 20 returns to estimate regime probabilities
• Uniform priors [0.33, 0.34, 0.33] indicate Markov system not receiving updates
• In heatmaps, constant horizontal bands at indices 19-24 are a red flag for this issue

References

• alpacatrading/gpu/inferenceobs_builder.py - Feature building implementation
• alpacatrading/gpu/harmonizedgpu.py - GPUMarkovSystem class
• notebooks/developbranchtesting.ipynb - Visualization examples