Databricks Community

Unity Catalog

Excited about the new features

Favourate talk at summit

Llm at production updates on mlflow 

My Databricks account

I like it.

Databricka 2023

What is your favorite tool for monitoring your pipelines?

Unity Catalog ABAC

Is there a roadmap for when ABAC (Attribute-based access control) wilö be available in Unity Catalog?

Data Engineering

What is a common problem you face when ingesting data in hot storage?

Delta Live Tables Best Practices

Delta Live Tables Best Practices session today was fantastic.

Key Note

Autloader processing semantics

Queue based Autoloader processes files in the order they are received only when the job is up and running. However when the job is down, the files that queue up are processed in lexical order once the job is up. Since autoloader jobs need to be stopp...

Does Databricks get cached result for a subquery?

If I run a query as "SELECT fare_amount FROM nyctaxi.trips where fare_amount > 1.5".  The query results will be cached for 24 hours.I then compose a second query using the previous query as a subquery "SELECT * FROM nyctaxi.trips WHERE fare_amount IN...

ETL TRAINING

Loved the delta live tables training

Databricks Conf - the DBT sesison eatlier today was 🔥

Loveing Databricks Summit

Loving the summit so far, awesome keynote speakers, great trainers and paid courses. Finished certification #databrickslearning

Migrating old solution to new optimal delta lake setup

Hi Databricks community!I have previsouly worked on a project that easily could be optimized with Databricks. It is currently running on Azure Synapse, but the premise is the same.Ill describe the scenario here:1. Data owners send a constant flow of ...