Delta Lake gives you time travel backward:
SELECT * FROM sales TIMESTAMP AS OF '2026-01-01'
But what about forward? What will your Delta table look like next month?
Nobody has built probabilistic future queries as a first-class Delta concept — until now.
Introducing Delta Foresight
GitHub: https://github.com/HarshalSant/delta-foresight Install: pip install delta-foresight
from delta_foresight import DeltaForesight
df = DeltaForesight(table="catalog.schema.daily_sales", time_column="sale_date", spark=spark) df.fit() forecast = df.predict(as_of="2026-09-01", confidence=0.90) df.materialize("catalog.delta_foresight.daily_sales_forecast")
Then query it with SQL: SELECT ds, revenue_forecast, revenue_lower_90, revenue_upper_90 FROM catalog.delta_foresight.daily_sales_forecast
What Makes It Different
The forecast IS a Delta table — not a report, not an export. Governed by Unity Catalog, queryable with SQL, shareable via Delta Sharing.
Mathematically valid prediction intervals — uses conformal prediction with proven coverage guarantees. Ask for 90%, get 90%.
Learns your table's temporal DNA — auto-detects frequency, trend, and seasonality from your own Delta history. No manual setup.
MLflow tracking built in — every forecast run logged automatically.
Works inside and outside Databricks — PySpark on Databricks, delta-rs locally, Parquet fallback for dev.
Use Cases
- Revenue planning: predict month-end close
- ML model health: forecast when AUC will breach threshold
- Inventory: predict stock depletion date
- Cost management: forecast DBU burn rate
- Data quality: forecast null rate trajectory
CLI
foresight predict --table catalog.schema.daily_sales --as-of 2026-09-01 foresight fingerprint --table catalog.schema.daily_sales foresight serve # REST API at localhost:8080/docs
Feedback welcome — which use case matters most to your team? GitHub Issues: https://github.com/HarshalSant/delta-foresight/issues
Also author of vigil-ml:
https://www.linkedin.com/in/harshalsant0
https://github.com/HarshalSant/
