data engineering discussion

[MO_Elsharawy]

Hey everyone!
Is anyone interested in discussing a hot topic that bridges both Data Engineering and Databricks? 🚀
I'm diving into areas like:

• DLT (Delta Live Tables) optimization

• Cluster & budget optimization

• Building scalable workflows

Would love to exchange insights, ideas, and best practices. Who’s in?

[nikhilj0421]

Hey @MO_Elsharawy, I would love to hear your insights on it.  

[szymon_dybczak]

Hi @MO_Elsharawy ,

Sounds like a cool idea. Do you have a concrete plan how to organise such thing?

[rafaelgodoy]

Hi! I'd love to participate.

We already have implemented some aspects of FinOps and observability at the level of all Databricks costs, budget policies for serverless operations, and policies across all domains. We have some best practices for scalable workflows and we can discuss them.