Thanks to the new column_object_metadata we have access to storage-level information such as MIME type, ETag, and other metadata. That functionality is for external volumes, as in some use cases that data was needed for ingestion. Just remember that ...
Unity Catalog secrets operations Create, Update, List, Read, or Reference are now in audit tables as well. I think that, aside from a clear permission model, it is the biggest benefit of storing secrets in UC. more news https://medium.com/databricksc...
How long does your business wait for a new report? A week? A month? Sometimes longer? The problem is often not Databricks, Spark, or compute. It's that every new dataset, metric, and Gold table has to go through the same Data Team. As the company gro...
x10 performance,x100 concurrency,price the same,impossible?not in databricks see benchmark https://www.sunnydata.ai/blog/databricks-rt-lakehouse-benchmark-results
Picture this: you’re sitting in front of Excel, staring at rows of numbers. You know the answers are in there somewhere—revenue trends, customer behavior, performance metrics—but pulling them out means writing queries, exporting data, or switching be...
Don’t overwrite your code — make it immutable! With every deployment, thanks to immutable_folder, all files are copied to a new, read-only folder. They are not overwritten. The biggest benefit is that jobs already underway will not fail or produce un...
19 is already here, but 18 is now LTS. In a new naming convention, LTS doesn’t have a minor version. New features, behavior changes, and fixes are now added incrementally to the same major Runtime version till it becomes LTS. In fact, that approach s...
Not yet an automatic update, but at least we can check whether our CLI version is up to date. #databricks My blog post with news: https://databrickster.medium.com/databricks-news-dabs-indexes-ltap-genie-last-update-25-july-ffac8533774f
Genie Spaces (now renamed to Genie Agents) are available now in DABs as … Genie Spaces. First, you need to run bundle generate to export your existing development Genie into a bundle. #databricks My blog post with news: https://databrickster.medium.c...
We can use Genie now, even in the CLI, with a simple ask command. #databricks My blog post with news: https://databrickster.medium.com/databricks-news-dabs-indexes-ltap-genie-last-update-25-july-ffac8533774f
If we set the REPLICA option in Postgres, we can then set CDF sync to Unity Catalog in the branch overview. #databricks more news and video https://medium.com/databrickscommunity/databricks-news-rt-lakehouse-reyden-lakebase-ttl-8416bdccf627
I hope soon it will be in DABS also LTAP is coming and is going to repalce that
One of the biggest challenges in modern data work isn’t just collecting information — it’s trusting it. Data moves through so many pipelines, transformations, and dashboards that by the time it reaches a business user, the question often becomes: whe...
I’ve been testing Genie Code in Databricks and wanted to understand not just the UX, but the actual cost behavior.My impression so far:for simple code edits / code assistance, it looks almost free,but if Genie Code starts doing things that involve re...
Great writeup - I'm also facing the same problem with Genie Code.1. This billing attribution gap isn't specific to Genie Code, it shows up anywhere an agent/orchestration layer spins up ad-hoc compute (serverless one) on your behalf. Eventhough you c...
As data engineers, we spend a significant amount of time writing SQL queries to ingest, transform, and analyse data. However, writing a query that returns the correct results is only part of the equation. Equally important is understanding how Spark ...
Azure Databricks now allows organisations to configure a custom URL at the account level, providing a unified and branded access point for all users.Instead of navigating multiple workspace-specific URLs, users can log in once using a single custom U...
I've also landed on the community page after trying to get the custom-url to work. In the accounts console itself the option is mentioned, but it seems it's unavailable for our account at the moment. Are there any prerequisites except for the ones in...
