Skip to main content

Ground Truth

Commodity Forecasting System - Technical Documentation

UC Berkeley MIDS Capstone Project 2024

90%

Data Reduction

From 75,000 raw data points to 7,600 unified daily records

180x

Speedup

Evolution from V1 → V2 → V3 architecture

70%+

Accuracy

Rigorous statistical testing for model validation

93%

Compute Savings

"Fit many, publish few" deployment strategy

🔬 Research Agent

Data collection and ETL pipeline with 6 AWS Lambda functions. Bronze → Silver → Gold medallion architecture on Databricks.

  • 90% data reduction achievement
  • Continuous daily coverage
  • Zero null values

📈 Forecast Agent

Machine learning forecasting engine with 15+ models. Parallel Spark backfills and train-once architecture.

  • 180x speedup evolution
  • Probabilistic predictions
  • Statistical validation

💰 Trading Agent

Strategy optimization and execution with 9 trading strategies. Rolling Horizon MPC for dynamic decision-making.

  • 70%+ accuracy threshold
  • Statistical validation
  • Risk-adjusted returns