Databricks Community

balajij8

Organizations solved the challenge of collecting, cleaning & governing structured data at scale via Delta Lake and Unity Catalog in Lakehouse. You have world class lineage, permissions, RBAC, ABAC and schemas as the nervous system. The nervous system...

balajij8

Combining SIGNAL statement with ATOMIC transactions in Databricks saves us from managing commits & rollbacks along with managing custom validations seamlessly - something that modern big data ETL frameworks struggle to deliver cleanly. They give the ...

balajij8

Databricks introduces multi-table transactions, allowing operations across multiple Delta tables to execute as a single atomic unit. Delta Lake has provided ACID guarantees at the table level, but ensuring atomicity across multiple tables previously ...

balajij8

Discussed the BI & Metrics Tax elimination using Databricks Metric Views here. Semantic Layer is a core component of the lakehouse with Metric Views. Modern stack is moving toward ai data experiences where organizations ask questions instead of build...

balajij8 · 02-26-2026

Discussed the BI & Metrics Tax elimination using Databricks Metric Views here. Organizations also face an older more persistent tax — the Ingestion Tax.To ingest data from a source like Salesforce or SQL Server into your Lakehouse, you typically stit...

balajij8

You can get the color scale only for Pivot tables that lets you add dimension fields to rows or columns to create cross tabulated views.

balajij8

AI tools generate code & pipelines that work functionally but ignore efficiency, scalability & cloud implications. I have bumped into belowCode generated by AI does a SQL cross join as its simple for most natural language queries that works but kills...

balajij8

You can useCREATE OR REPLACE VIEW newview AS SELECT t1.field1, item.field2, item.field3 FROM table1 AS t1 INNER JOIN table2 AS t2 ON t1.id = t2.id , LATERAL EXPLODE(t1.structure) AS structureitem(item)

balajij8

For the issue - Some of my cells/pipelines run for a very long time, and I want to automatically cancel/kill them when they exceed a certain time limit.You can use job notifications with Metric threshold (Duration Warning for notifications & Duration...

balajij8

You can use signal to do this if running in a notebook for code validation#Add in notebook import signal class TimeoutException(Exception): """Raised when a cell is run for very long time""" def timeout_handler(signum, frame): raise Timeout...

Databricks Community

User Stats

User Activity

Turning Lakehouse into Brainhouse via Knowledge Graphs

Databricks Transactions - Enforcing Business Validation Rules via SQL SIGNAL

Databricks Multi Table Transactions - All Data or Nothing

Databricks Metric Views - Moving Towards Business Semantics

Databricks Lake flow - Orchestration Layer is moving to where it belongs

Re: Not seeing color scale in Databricks table

Re: What's Your Biggest AI Pet Peeve?

Re: How to Convert a Lateral View to a Table Reference

Re: Kill/Cancel a Notebook Cell Running Too Long on an All-purpose Cluster

Re: Kill/Cancel a Notebook Cell Running Too Long on an All-purpose Cluster