- 3315 Views
- 1 replies
- 3 kudos
ADF vs Databricks Workflows — When to Use Which
A practical guide from 12 years of building Azure data platforms.If you've spent any time working on Azure data platforms, you've probably hit this question at some point — should I build this in Azure Data Factory or just do it in Databricks?I've be...
- 3315 Views
- 1 replies
- 3 kudos
- 3 kudos
Great article clarifying the difference between ADF and Databricks Workflows!I have a design question for an architecture similar to your hybrid pattern: We use ADF to ingest raw data into the Bronze layer, while Databricks handles the downstream Bro...
- 3 kudos
- 291 Views
- 2 replies
- 2 kudos
Beyond Chat History: Designing Portable Memory and Decision Boundaries for Enterprise AI Agents
Persistent Memory Is Only the BeginningAs AI agents move from isolated conversations toward persistent workflows, memory is becoming part of enterprise AI infrastructure.An agent that remembers previous interactions can reduce repetitive instructions...
- 291 Views
- 2 replies
- 2 kudos
- 2 kudos
@ShenXu : I found this really interesting so did a bit of experimentation. My lab notes and a test notebook are available at https://medium.com/lab-notes/unity-catalog-functions-kept-ai-agents-from-acting-on-stale-memory-7d63e15f992b
- 2 kudos
- 68 Views
- 0 replies
- 1 kudos
Should I wait for the tram? Adding agents to a live Helsinki transit app on Databricks (part 2)
Hi everyone,Part 2 of my Helsinki live transit project is out. In part 1 I built a Databricks App that follows every tram and metro train in Helsinki in real time: a Lakeflow pipeline on HSL's live MQTT feed, Lakebase behind the live map, and a Genie...
- 68 Views
- 0 replies
- 1 kudos
- 109 Views
- 0 replies
- 1 kudos
Solution Accelerator Series | Genome-Wide Association Studies
Genome-wide association studies help identify genetic variations associated with a disease or trait. The Genome-Wide Association Studies Solution Accelerator builds on Glow, an open source project for genomic analysis, to provide a fast, scalable and...
- 109 Views
- 0 replies
- 1 kudos
- 143 Views
- 0 replies
- 0 kudos
Building an End-to-End FinOps Cost Attribution Model for Azure Databricks
When teams start looking at Databricks cost, the first instinct is usually to query system.billing.usage.That gives a good view of Databricks consumption, but it is only one part of the overall cost picture..For a more complete FinOps view on Azure, ...
- 143 Views
- 0 replies
- 0 kudos
- 182 Views
- 0 replies
- 2 kudos
Insurers Don’t Need Faster APIs. They Need Fewer Calls: Geospatial Scoring with H3 and Spatial SQL
Insurance · Reinsurance · Banking · Telecom · Retail · Agriculture — Per-Object API Calls to Per-Cell Scoring with H3 and Native Spatial SQLBy @Phani_sannala, in collaboration with @sridharplvYour lakehouse already holds the geometry. Why are you shi...
- 182 Views
- 0 replies
- 2 kudos
- 138 Views
- 0 replies
- 0 kudos
Insurers score millions of locations through an API. The API isn't slow, the unit of work is wrong.
Insurance · Reinsurance · Banking · Telecom · Retail · Agriculture — Per-Object API Calls to Per-Cell Scoring with H3 and Native Spatial SQL By @Phani_sannala, in collaboration with @sridharplvYour lakehouse already holds the geometry. Why are you s...
- 138 Views
- 0 replies
- 0 kudos
- 213 Views
- 0 replies
- 1 kudos
How to set budgets and alerts on SQL warehouses so self-service never surprises Finance
The quickest way to lose a self-service analytics programme is a bill nobody saw coming. In my last post I covered chargeback: working out, after the fact, which team spent what. That answers "whose cost is this?" It doesn't answer the question Finan...
- 213 Views
- 0 replies
- 1 kudos
- 337 Views
- 0 replies
- 1 kudos
Solution Accelerator Series | IOC Matching and Multicloud Query Federation
Cybersecurity in multicloud, multiregion environments can be complicated by fragmented security controls, data dispersion and compliance requirements. The IOC Matching and Multicloud Query Federation Solution Accelerator shows how a lakehouse platfor...
- 337 Views
- 0 replies
- 1 kudos
- 123 Views
- 0 replies
- 0 kudos
Turning P&C Claim Documents into Structured Data with Databricks AI Functions
A claim document may contain a policy number, loss date, estimated amount, and a description spread across paragraphs and form fields. Extracting those values is useful but we also need to know where they came from and whether they are suitable for d...
- 123 Views
- 0 replies
- 0 kudos
- 225 Views
- 0 replies
- 3 kudos
Built a live Helsinki tram tracker on Databricks with Lakeflow, Lakebase, Genie and Apps
Hi everyone! I built a near-real-time app that follows every tram and metro train in Helsinki using only Databricks: no Kafka, no extra servers, and the whole thing deploys from one repository.What the app doesA live map of every tram and metro train...
- 225 Views
- 0 replies
- 3 kudos
- 369 Views
- 2 replies
- 1 kudos
PARTITIONED BY, Liquid Clustering, and OPTIMIZE: when should you use each?
I put together a practical guide to Delta Lake table layouts, with SQL examples in Databricks:When to consider partitioning or Liquid Clustering.How OPTIMIZE works with partitions, ZORDER, and clustering.How to compare bytes scanned, query latency, a...
- 369 Views
- 2 replies
- 1 kudos
- 292 Views
- 1 replies
- 0 kudos
Row-level security for a RAG agent: what Unity Catalog enforces, and what you have to build
Two colleagues ask the same chatbot the same question. Should they get the same answer, ordifferent ones because they are allowed to see different things? Every organisation that puts an AIassistant on top of its own documents runs into this sooner o...
- 292 Views
- 1 replies
- 0 kudos
- 0 kudos
I'd love to be able to update this, because I pressed reply too early
- 0 kudos
- 210 Views
- 0 replies
- 0 kudos
The 32nd Column: Why Your Z-Order or Clustering Key May Be Doing Nothing
You cluster a table on a column, run the query, and nothing improves. The layout command completed successfully. The column is definitely in the filter. The file count in the query profile has not moved.Before assuming the clustering did not work, ch...
- 210 Views
- 0 replies
- 0 kudos
- 499 Views
- 0 replies
- 2 kudos
Solution Accelerator Series | Incident Investigation Using Graphistry
Cybersecurity investigations often require sifting through large volumes of log and telemetry data to uncover the patterns and relationships behind threat activity. The Incident Investigation Using Graphistry Solution Accelerator shows how to query i...
- 499 Views
- 0 replies
- 2 kudos
| User | Count |
|---|---|
| 85 | |
| 76 | |
| 73 | |
| 60 | |
| 43 |
