Hi everyone,I built Intelligence Studio, an open-source workbench that lets you browse, test, analyse, and integrate with 640+ Databricks REST APIs -- all from one interface. No more juggling docs, curl commands, Postman collections, and multiple bro...
Hey Team, in my last post I asked how much AI has actually changed your day to day, and the responses were fantastic. But let's talk about the other side for a minute. I'll go first — I've started second-guessing almost everything I see on social med...
AI tools generate code & pipelines that work functionally but ignore efficiency, scalability & cloud implications. I have bumped into belowCode generated by AI does a SQL cross join as its simple for most natural language queries that works but kills...
If you've ever worked on an insurance data warehouse, or really any warehouse where data arrives from different systems at different times, you know the pain of keeping things in sync. I spent years building data warehouses for a property and casual...
In Part 1, we covered why multi-table transactions matter. Now let's build one. We'll create the tables from the claim wrap-up scenario, load sample P&C insurance data, and walk through what happens when the wrap-up succeeds, when it fails, and when...
Hi Guys, I have passed it already some time ago, but just recently have summarized all the materials which helped me to do it. Pay special attention to GitHub repository, which contains many great exercises prepared by Databricks teamhttps://youtu.be...
Cleared Databricks Data Engineering Associate recently. Practising real questions helped me a lot, and (Certs Topic) was a reliable resource.
If you've ever built a dashboard where you needed to track the same data across two different date dimensions, you know the frustration. You get the first chart working. You add the second. Then you realise cross-filtering just stopped working. I re...
Thanks for sharing great example with detailed explanation.
We needed job_id and run_id in a custom metrics Delta table so we could join to `system.lakeflow.job_run_timeline`. Tried four approaches before finding the one that works on serverless compute.What doesn't workspark.conf.get("spark.databricks.job.id...
PostgreSQL to Databricks made simpler with Lakeflow Connect (Public Preview).Databricks has introduced a PostgreSQL connector in Lakeflow Connect (Public Preview), enabling ingestion of PostgreSQL data into the Lakehouse using logical replication.Ins...
You can enable or disable previews using account console for account level previews and workspace for workspace level previews.More details here
Organizations solved the challenge of collecting, cleaning & governing structured data at scale via Delta Lake and Unity Catalog in Lakehouse. You have world class lineage, permissions, RBAC, ABAC and schemas as the nervous system. The nervous system...
A Data & AI–Driven Decision Engine for Modern Retail NetworksIntroductionIn modern retail, supply chains are no longer static networks — they are living, adaptive systems that must continuously respond to customer demand, fulfillment speed expectatio...
Great article! would love to know more as I have a very similar case with a concrete customer. Thanks, S.
We need to stop treating AI as a tool. It's time to treat it as a peer.I've been building a library of reusable skills for Claude — structured instructions that let AI agents handle complex, repetitive development workflows on Databricks and Azure AI...
Most construction teams don’t really have a data problem, at least not in the way we usually think about it. They already have dashboards everywhere. Finance has reports, project managers have schedule views, field teams have inspection logs. Everyon...
Combining SIGNAL statement with ATOMIC transactions in Databricks saves us from managing commits & rollbacks along with managing custom validations seamlessly - something that modern big data ETL frameworks struggle to deliver cleanly. They give the ...
How Digital Payment Lending Platforms Can Collaborate with Banks Without Exposing Sensitive Data1. Business Context & Regulatory RealityIn 2020, large Indian fintech platforms faced a unique regulatory constraint: NBFC‑led digital platforms were not ...
This is a solid breakdown of how secure data collaboration can be done without exposing sensitive information. The Clean Room approach really stands out because it shifts the model from data sharing to controlled computation, which is exactly what re...
I've spent years migrating SOC operations from traditional SIEM to Databricks. Not because it's trendy, but because SIEM has fundamental problems that no vendor update will fix: proprietary query languages that lock you in, no version control or test...
| User | Count |
|---|---|
| 85 | |
| 72 | |
| 50 | |
| 44 | |
| 42 |
