- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
03-28-2022 01:17 AM
delta lake does not has levels. The bronze/silver/gold is a concept to structure the data lake.
You do not need a separate ADLS container, I would even advice against it. Why? Well, you have to ingest the data anyway, so it's best to do this only once (one container). Within a single container you can structure the data any way you want. It does not have to be bronze/silver/gold but can be anything that works for you/your company. I do suggest to foresee some kind of structure, it will help you in the long run.
For the reporting part: if you want to run reports, Databricks SQL is better than the classic spark clusters (which excel in engineering/ML). Databricks SQL supports concurrency ofc.