Databricks Community

@MandyR No, notifications for this training this time as well

UI shows all provisioned users, but REST/SQL only expose subsets depending on whether you query account vs workspace vs UC. To get a true overview, you need to combine account SCIM API + workspace SCIM API + UC system tables.

All docs are on Databricks official pages, I beleive below will be helpful to begin with.LakehouseIQ: AI Engine for Your Business — https://www.databricks.com/blog/introducing-lakehouseiq-ai-powered-engine-uniquely-understands-your-business - Officia...

To connect Databricks with web or mobile apps, most developers recommend exposing your data or models through a lightweight API layer. Use Databricks SQL Endpoints or MLflow model serving to generate secure REST endpoints your app can call directly. ...

Fantastic kickoff to the Databricks Advent Calendar 2025 , appreciate you steering the series, @Hubert-Dudek!

Hi @nanditakrishnan ,There's already something like that in databricks dashboards, but some conditions need to be fulfilled (i.e queries need to share same group by). One of dataset optimization techniques that databricks team implemented is doing fo...

Hi @gokkul ,Your app service principal needs to have a proper permission to write to UC table. You also need to use python databricks sdk to interact with UC object (i.e read/save a table).You can get some inspiration from following databricks cookbo...

Optimizing Databricks pipelines for large-scale workloads mostly comes down to smart architecture + efficient Spark practices.Key tips from real-world users:Use Delta Lake – for ACID transactions, incremental updates, and schema enforcement.Partition...

I have to process data for n number of devices which is sending data in every 5 seconds.I have a similar scenario where I have to take last 3 hours of data and process it for all the devices for some key parameters. Now if I am doing it sequentially ...

if you’re trying to mount EFS directly to an interactive cluster, you’ll usually need to handle it through init scripts since EFS requires the NFS client to be installed and mounted at cluster startup. One thing to double-check is whether your worksp...

Is this still valid answer in 2025 ? https://docs.databricks.com/aws/en/delta/tune-file-size#auto-compaction-for-delta-lake-on-databricks 

@szymon_dybczak - yes, as it relies on AI models, there are chances of missing few cases due to non-deterministic nature of it. I have used it with vast number of PDFs in anger and it has worked pretty well in all those cases. Have not tried with PNG...