For some Lakeflow connectors, we can pass a filter to limit which rows are loaded. It solves one big problem: initial, full load from tools like Google Analitycs can be almost impossible. Thanks to row_filter, we can limit ingestion and load, for exa...
In regions in which new Lakebase autoscaling is available, from Lakebase, you can access both autoscaling and older provisioned Lakebase instances. #databricks https://databrickster.medium.com/databricks-news-2026-week-2-12-january-2026-to-18-january...
Runtime 18, including Spark 4.1, is no longer in Beta. You can start migrating now. Runtime 18 is available now only for classic compute. Serverless, or SQL warehouse, still using older runtimes. Once 18 is everywhere, we will be able to use identifi...
All pivot table lovers can now export a pivot table from Dashboards to Excel #databricks More Databricks news: https://databrickster.medium.com/databricks-news-2026-week-2-5-january-2026-to-11-january-2026-0bfc6c592051
That's my favourite meme of all time
95% of GenAI projects fail. How to become part of the 5%? I tried to categorize the 5 most popular failure reasons #databricks https://www.sunnydata.ai/blog/why-95-percent-genai-projects-fail-databricks-agent-bricks
Accidentally dropping a table in production is every data engineer’s nightmare. One wrong command, one missing WHERE clause or worse, a rushed DROP TABLE and suddenly critical data is gone.Thankfully, Azure Databricks with Unity Catalog gives us a sa...
Excellent explanation of UNDROP in Unity Catalog. The examples and callouts around retention windows and edge cases make this very actionable. Thanks for putting this together. Cheers, Louis.
In Genie, there is a Deep Research mode (similar to ChatGPT Pro mode). It can now generate a report that we can save as a PDF. Really useful option to impress your management. #databricks More news https://medium.com/@databrickster/databricks-news-20...
Thank you for sharing this @Hubert-Dudek
Do you know that it is possible to extend the Assistant with agent skills? It is really straightforward and allows you, in fact, to extend the functionality of databricks. You can create templates for an assistant - I experimented with a template to ...
Databricks Assistant can also be used in databricks documentation without login to #databricks. Read and watch databricks news on:https://databrickster.medium.com/databricks-news-2026-week-2-5-january-2026-to-11-january-2026-0bfc6c592051
In this video article, I demonstrated how to automate resource creation in Azure Databricks using the Databricks SDK for Python. If you've ever found yourself manually creating catalogs, schemas, and volumes in the Databricks UI repeatedly, this vid...
As we move into 2026, the mandate for modern data and AI is clear. The industry has pivoted from evaluating isolated features to building foundational futures. Since the 2025 Data + AI Summit, I have consulted with numerous organizations across a ran...
If you are using a plan to deploy DABS, starting from 0.282, plan_version has been moved to 2. A new plan can have a different structure. Please keep in mind that inconsistencies in DABS versions can break your CI/CD. #databricks I wrote an article a...
Databricks MVP Conner Schiissler explores the new ai_parse_document() function, which replaces complex OCR integrations with a simple SQL command to extract structured markdown from unstructured files. Read here.
