cancel
Showing results forย 
Search instead forย 
Did you mean:ย 
Data Engineering
Join discussions on data engineering best practices, architectures, and optimization strategies within the Databricks Community. Exchange insights and solutions with fellow data engineers.
cancel
Showing results forย 
Search instead forย 
Did you mean:ย 

Bronze silver gold layers

BeginnerBob
New Contributor III

Is there a best practise guide on setting up the delta lake for these 3 layers.

โ€‹

I'm looking for document or scripts to run that will assist me.

2 REPLIES 2

AmanSehgal
Honored Contributor III

Well the medallion architecture is not one fit for all use cases. It depends on your data landscape and how would you like to process data.

Bronze - Ingest your data from multiple sources.

Silver - Store clean and aggregated data

Gold - Store data to serve BI tools.

You need to design and implement your own pipeline for your own use case.

There are some small projects available on github which you can use to mimic the workflow. But their capability is very limited and they're far away from what real life data would look like.

This github repo and associated video can help you.

jose_gonzalez
Moderator
Moderator

hi @Lloyd Vickeryโ€‹ ,

I would highly recommend to use Databricks Delta Live Tables (DLT) docs here https://databricks.com/product/delta-live-tables

Connect with Databricks Users in Your Area

Join a Regional User Group to connect with local Databricks users. Events will be happening in your city, and you wonโ€™t want to miss the chance to attend and share knowledge.

If there isnโ€™t a group near you, start one and help create a community that brings people together.

Request a New Group