cancel
Showing results for 
Search instead for 
Did you mean: 
Data Engineering
cancel
Showing results for 
Search instead for 
Did you mean: 

Forum Posts

KuldeepChitraka
by New Contributor III
  • 905 Views
  • 2 replies
  • 0 kudos

Lakehouse Architecture : How notebooks are organized and executed

We are implementing a lakehouse architecture and using Notebook to transform data from object storage. Most of the time our source is database for which there one folder per table in object storage.We have structure like below for various notebooksGO...

  • 905 Views
  • 2 replies
  • 0 kudos
Latest Reply
jose_gonzalez
Moderator
  • 0 kudos

Adding @Vidula Khanna​ and @Kaniz Fatma​ for visibility to help with your request

  • 0 kudos
1 More Replies
irfanijaz
by New Contributor
  • 403 Views
  • 0 replies
  • 0 kudos

Differently named storage accounts in different environments

Hi,I have a solution design question on which I am looking for some help. We have 2 environments in azure (dev and prod), each env has its own ADLS storage account with a different name of course. Within Databricks code we are NOT leveraging the mou...

  • 403 Views
  • 0 replies
  • 0 kudos
Constantine
by Contributor III
  • 882 Views
  • 2 replies
  • 4 kudos

Resolved! What's the best architecture for Structured Streaming and why?

I am building an ETL pipeline which reads data from a Kafka topic ( data is serialized in Thrift format) and writes it to Delta Table in databricks. I want to have two layersBronze Layer -> which has raw Kafka dataSilver Layer -> which has deserializ...

  • 882 Views
  • 2 replies
  • 4 kudos
Latest Reply
Kaniz
Community Manager
  • 4 kudos

Hi @John Constantine​ , This is just a close follow-up. Did @Hubert Dudek​ 's reply resolve your query?

  • 4 kudos
1 More Replies
Labels