cancel
Showing results forย 
Search instead forย 
Did you mean:ย 
Data Engineering
cancel
Showing results forย 
Search instead forย 
Did you mean:ย 

Bronze silver gold layers

BeginnerBob
New Contributor III

Is there a best practise guide on setting up the delta lake for these 3 layers.

โ€‹

I'm looking for document or scripts to run that will assist me.

1 ACCEPTED SOLUTION

Accepted Solutions

Kaniz
Community Manager
Community Manager

Hi @Lloyd Vickeryโ€‹ , This link might be helpful for you. Please let me know if that helps.

View solution in original post

4 REPLIES 4

AmanSehgal
Honored Contributor III

Well the medallion architecture is not one fit for all use cases. It depends on your data landscape and how would you like to process data.

Bronze - Ingest your data from multiple sources.

Silver - Store clean and aggregated data

Gold - Store data to serve BI tools.

You need to design and implement your own pipeline for your own use case.

There are some small projects available on github which you can use to mimic the workflow. But their capability is very limited and they're far away from what real life data would look like.

This github repo and associated video can help you.

Kaniz
Community Manager
Community Manager

Hi @Lloyd Vickeryโ€‹ , This link might be helpful for you. Please let me know if that helps.

jose_gonzalez
Moderator
Moderator

hi @Lloyd Vickeryโ€‹ ,

I would highly recommend to use Databricks Delta Live Tables (DLT) docs here https://databricks.com/product/delta-live-tables

Kaniz
Community Manager
Community Manager

Hi @Lloyd Vickeryโ€‹ , Did you find any of the above answers helpful? Would you like to tell us if you solved it?

Welcome to Databricks Community: Lets learn, network and celebrate together

Join our fast-growing data practitioner and expert community of 80K+ members, ready to discover, help and collaborate together while making meaningful connections. 

Click here to register and join today! 

Engage in exciting technical discussions, join a group with your peers and meet our Featured Members.