Join us for the return of the Databricks Learning Festival (Virtual)! Mark your calendars from 15 January - 31 January 2025! Upskill today across data engineering, data analysis, machine learning, and generative AI. Join the thousands who have el...
The global economy in 2024 is a tale of two forces: optimism sparked by falling interest rates, and the uncertainty caused by geopolitical unrest. As companies focus on sustainable growth, such a volatile environment inevitably tests their resilience...
Meet Sujesh Menon, a valued member of our community! Sujesh is a Senior Data Engineer at Glanbia, Plc. He brings a wealth of knowledge and expertise to the group, and we are thrilled to have him here. We presented him with a range of questions, and...
The world of artificial intelligence (AI) and data analytics is about to get a significant boost, thanks to Databricks’ collaboration with NVIDIA. This work brings together the cutting-edge capabilities of Databricks’ Mosaic AI platform and NVIDIA AI...
This is part 2 of a two-part series on Structured Extraction with LLM on Databricks. Read here for part 1! Introduction In part 1 of this series, I demonstrated how to use a large language model (LL...
This blog talks about common ways in which Hive-style partitioning is used as a workaround for efficient data storage. Liquid Clustering improves partitioning and zorder techniques by simplifying data...
What is structured extraction? Structured extraction, sometimes referred to as “key information extraction,” “entity extraction,” or simply as “text-to-JSON,” is a process that transforms unstructu...
Throughout the dozens of engagements I’ve had since joining Databricks, I’ve found that customers often struggle to understand the scope and concept of Unity Catalog. Questions like “Does it store my ...