Hi James,
This is a solid use case for the Lakehouse. Here's how I'd approach it based on patterns we use at Databricks.
Real-time vs batch ingestion
Batch (main path): Lakeflow Spark Declarative Pipelines (formerly Delta Live Tables / DLT) is a st...
I have had good results using Dash for apps that need charts and interactive UIs. It’s a solid option if you’re moving beyond Streamlit.
Why Dash works well for your use case:
Charts: Plotly (built into Dash) gives you interactive charts and good ...
