Production forecast architecture¶

This page documents the production forecast contract and how its result reaches the public display. It describes the scheduled path through btc_timesfm.ops.validated_entrypoints.run_forecast; local CLI runs and research experiments have different operational boundaries. Forecasts are experimental estimates, not guarantees.

System context¶

flowchart LR
  Scheduler[Dispatcher and workflow] -->|invoke| Runner[Validated entrypoint]
  Kraken[Kraken BTC USD candles] --> Validate[Provider validation]
  Runner --> Validate
  Validate --> Engine[Forecast engine]
  Release[(GitHub Release state)] -->|restore history| Runner
  Runner -->|results and diagnostics| Artifacts[Workflow artifacts]
  Runner -->|history and state| Release
  Runner -. optional .-> X[X publication]
  Release -->|forecast history| PagesBuild[Pages build]
  PagesBuild --> Static[Static forecast site]
  Static --> Browser[GitHub Pages display]
  SQLite[(SQLite history)] --> WSGI[Optional read-only API]
  APIClient[API client] --> WSGI

The static Pages site and the WSGI API are separate read paths. Pages rebuilds HTML and static JSON from the released SQLite history. The API is an optional authenticated service reading canonical SQLite; it is not the Pages backend. See the API contract and API service.

Production generation sequence¶

sequenceDiagram
  participant S as Scheduler
  participant V as Forecast runner
  participant D as Data validation
  participant H as SQLite history
  participant M as Models
  participant P as Postprocess
  participant R as GitHub Release
  participant X as X publisher
  S->>R: Restore history
  S->>V: Invoke forecast
  V->>D: Fetch BTC USD candles
  D-->>V: Validated market data
  V->>H: Read mature history
  V->>M: Run inference
  M-->>V: Forecasts and intervals
  V->>P: Calibrate and reconcile horizons
  P->>H: Persist forecast and outcomes
  P->>R: Publish durable history
  opt Optional health-gated publication
    P->>X: Publish forecast
  end
  R-->>S: Forecast history for future runs

The hourly dispatcher wakes at minute 7 and dispatches the forecast workflow without X enabled. The forecast workflow also has a two-hour cron at minute 37; its durable schedule guard avoids duplicate work and handles delayed runs. The Pages workflow has its own hourly fallback at minute 17 and also rebuilds after forecast workflow completion, on push to main, or manual dispatch. These are distinct schedules: a Pages rebuild does not generate a forecast.

Production data and model contract¶

1. run_forecast installs the production policy and instrumentation, then calls btc_timesfm.cli.btc_forecast.main().
2. fetch_redundant_hourly(512) selects and validates provider data, with fallback and provenance diagnostics. Production uses completed hourly BTC/USD candles; the Kraken engine uses XBTUSD. At least 64 completed candles are required and at least 513 closes are requested so the longest context has 512 hourly returns.
3. The forecast origin is the timestamp of the latest completed close. The rolling cache (.state/previous_forecast.json) is bootstrapped into the durable store when needed. Matured outcomes and exact target timestamps from durable history are used for scoring, calibration, and model weighting.
4. Derivatives (including funding/open interest), microstructure (including order-book signals), and cross-asset (including ETH) snapshots are collected as optional sources. Their health and retention are recorded. Their numeric fields are attached to the output's market_features after point inference for diagnostics/metadata; they are not covariates to TimesFM or the production baseline forecasts. Quarantined sources are excluded from feature attachment. The production model input remains BTC hourly log returns.
5. TimesFM is the timesfm[torch]==3.0.2 dependency from pyproject.toml and uv.lock, using checkpoint google/timesfm-3.0-pytorch at immutable revision 43046b85ec22d584a13f8098c2ed39c889e129c2. The evaluator runs on CPU with per-core batch size 1. Input is a one-dimensional sequence of hourly log returns, log(close[t] / close[t-1]); inference requests 16 steps and quantiles with symmetric averaging disabled.
6. Available TimesFM contexts are 168, 336, and 512 hourly returns (7, 14, and about 21 days). Each context is forecast separately. Published target horizons are 2, 4, 8, and 16 hours. Returned quantiles are the nine levels 0.1 through 0.9; the point forecast must match the median (0.5) quantile. Production retains Q10/Q50/Q90. For each horizon, return steps are accumulated and exponentiated from the latest BTC close to produce a price.
7. Persistence, seven-day drift, and AR(1) baselines forecast the same horizon set. Regime-aware priors are blended with leakage-guarded matured out-of-sample performance when sufficient history exists. Weighting has bounded contributions and persistence fallback behavior. The research ridge feature model is disabled by the production policy.
8. The ensemble is postprocessed with model disagreement, empirical Q10–Q90 coverage calibration, direction evidence, conditional calibration, abstention and threshold diagnostics, and cross-horizon coherence checks. The published ensemble Q10/Q50/Q90 are calibrated marginal price intervals; they are not a joint distribution over the entire path.
9. Forecast and matured outcomes are persisted in SQLite with write-once history semantics. The workflow verifies and exports SQLite and CSV, creates compressed durable Release assets, and preserves verified rollback/independent backup safeguards before replacing canonical assets. X publication is optional and gated: scheduled runs or a manual post_to_x request must pass health and preflight checks; publication uses durable idempotency state.

Quantiles and confidence are different¶

Q10/Q50/Q90 describe forecast price quantiles/interval bounds, subject to the calibration process. The confidence section is an evidence-quality assessment based on matured performance, calibration, sample depth, and drift; it is not the probability that a forecast is correct. Model agreement is also not a probability. Because per-hour marginal return quantiles are accumulated to price targets without a modeled joint dependence structure, accumulated marginal quantiles must not be described as true cumulative joint quantiles.

Execution boundaries¶

The legacy forecast walkthrough is retained as archived, implementation-era detail; this page is the production architecture and current contract reference.

Files and output contracts¶

Workflow-only state also includes observability and pipeline-health reports, X status/registry, and compressed shadow database. The canonical Release tag is forecast-history-v1; the forecast workflow uploads a run artifact with a seven-day retention period. For API payload details, use the linked API contract rather than treating static-site JSON and API responses as identical.

Implementation references¶

• src/btc_timesfm/ops/validated_entrypoints.py — production entrypoint, instrumentation, validation hooks.
• src/btc_timesfm/cli/btc_forecast.py — production orchestration and forecast.json assembly.
• src/btc_timesfm/forecasting/forecast_engine.py — return contexts, TimesFM contract, baselines, weighting, and ensemble.
• src/btc_timesfm/forecasting/forecast_policy.py — production/research policy boundary.
• .github/workflows/forecast.yml, hourly-forecast.yml, pages.yml — generation, schedule dispatch, durable release, optional publication, and Pages rebuild.
• src/btc_timesfm/web/static_site.py and src/btc_timesfm/api/forecast_service.py — distinct display/read paths.

For uncertainty interpretation, see forecast confidence and the public dashboard.