If you have a big Delta table that was partitioned years ago, you probably know the feeling. Half the partitions are tiny, a few are huge, and queries that filter on anything other than the partition column scan far more than they should. Liquid clus...
Why TetrisI wanted to know if one Databricks function can play Tetris, and what Unity Gateway sees when an AI agent hammers it move after move.It started with the ai_decide announcement by Jane Zhang, Matthew Ding and Archika Dogra. Their demo is Sna...
Sooner or later, a source team adds a column. Maybe it's a new field in a JSON event, or a CSV export that grew from 12 columns to 13. Nobody tells the data team, because to them it's a tiny change.What happens next depends on a single Auto Loader op...
This article focuses mainly on cost management which has become ever more important with advent of fast-pace AI. A rise in total spend tells a platform administrator that something changed, but not necessarily which workload caused it or what action ...
A document pipeline can fail in a way that looks a lot like success. The job finishes, the index status is green, the file count matches what you uploaded. Then someone asks a question and gets nothing back. No error, just an empty answer.It's a comm...
My entire LinkedIn feed is full of Claude dashboards. Everyone is talking about them like it’s the hottest topic right now, so I wanted to try it myself, and of course with our favourite platform, DatabricksShort version: I connected Claude to a Data...
Tips and Techniques for Ingesting Large JSON files with PySparkIntroductionSuppose you’ve ever struggled or grappled with consuming massive JSON files with PySpark. In that case, you are aware that insufficient data can always creep in and silently d...
Maintaining data quality with PySpark is essential for reliable analytics, and combining validation checks with consistent monitoring can help catch issues before they affect downstream workflows. Corn dogs, quesadillas, wraps, and burgers all share ...
Environment: one Azure Databricks workspace, pay-per-token foundation models, Enforce Unity Gateway off, requests sent with curl and a CLI-issued token.Symptom. A request naming a Unity Catalog model service, system.ai.claude-haiku-4-5, sent to https...
Thanks for the clear tests. I think the 404 is expected, and it is not linked to the enforcement setting.The three-part names under system.ai are Unity Gateway model services, not serving endpoints. The /serving-endpoints paths look up an endpoint by...
Most public GitHub repositories for Databricks are just dry code dumps. You get a massive YAML file, a basic README, and are expected to just figure it out.When building production-grade infrastructure, engineers need to see the architecture and unde...
This appears to be a truly useful resource for understanding 'Databricks Asset Bundles' and real-world 'IaC' workflows. I recently passed my exam; in my experience, when preparing for 'Databricks' certifications, combining hands-on practice with a hi...
Lakeshift is a Databricks App that I built for one job: moving legacy SQL and ETL code to Databricks in a way that a team can measure, check and trust. It sizes the code first, makes a rule-based draft with Databricks Labs Lakebridge, lets two AI mod...
Databricks Declarative Automation Bundles Abri um pull request que apagava um job. O plan mostrou “delete jobs.sample_job”, o check ficou verde e o merge liberou.Se eu clicasse, o job sumia de HML na hora e, depois da aprovação, de PRD também.O CLI ...
Responsible gaming initiatives are becoming increasingly important as regulation, gameplay experiences and the betting and gaming industry continue to expand. The Responsible Gaming Solution Accelerator, developed in partnership with AWS, shows how t...
excellent
Hi everyone,Part 2 of my Helsinki live transit project is out. In part 1 I built a Databricks App that follows every tram and metro train in Helsinki in real time: a Lakeflow pipeline on HSL's live MQTT feed, Lakebase behind the live map, and a Genie...
Hey @SumeshKashyap Thanks for writing this up, and for the question at the end. I found it interesting enough that I ran an experiment on it.I set up bike, walk or wait inputs like the ones your agent gets back from its tools (16 scenarios over six H...
Five controls to put in place before rolling out Databricks Genie Agents.The benchmarks passed. Business users are ready. But who checks failed questions, approves updates, watches query performance, and manages spending after go-live?For a Databrick...
Lakebase Search brings vector search and BM25 keyword search directly into Lakebase, Databricks’ managed, serverless PostgreSQL offering.It introduces two PostgreSQL extensions:lakebase_vector — Approximate nearest-neighbor (ANN) vector search, compa...
| User | Count |
|---|---|
| 85 | |
| 77 | |
| 73 | |
| 60 | |
| 43 |
