- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
11-16-2025 10:39 PM
How do you design a scalable, reliable pipeline that handles both fast/continuous data and slower bulk data in the same system?
- Labels:
-
Delta Lake
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
11-16-2025 11:52 PM
Very generic question 🙂 Here are general rules and best practices related to Databricks well-architected framework: https://docs.databricks.com/aws/en/lakehouse-architecture/well-architected Take a deeper look on operational excellence, reliability and performance efficiency. On the other hand, try to adopt a mediallion architecture to logically organize data https://www.databricks.com/glossary/medallion-architecture and usage of Unity catalog to centrally control and governance data.