Databricks Community

As a machine learning practitioner, I’m incredibly excited about the release of Apache Spark 4.0 (officially announced May 23, 2025). This landmark update brings a bevy of new features and improvements that make distributed data science and AI worklo...

ai_query() is a built-in Databricks SQL function that lets you invoke a model serving endpoint from a SQL query. This enables data scientists to apply machine learning models (including traditional ML like XGBoost, not just LLMs) directly in dashboar...

Databricks ships some killer toys for large-language-model work: ai_query for in-warehouse inference Vector Search for lightning-fast retrieval Serving Endpoints for real-time chat Put them together, though, and you’ll trip over a few bo...

Hi, I’m Debu. I spend a lot of my day building and stress‑testing LLM‑powered systems, and one lesson keeps coming back: if you don’t measure your agent’s behavior over an entire conversation, you’re flying blind. Below is the exact notebook pattern ...

Over the years, I have collaborated closely with ML engineering leaders across various industries, guiding them on how to make the right chunking strategy decisions for their Retrieval-Augmented Generation (RAG) use cases. One of the biggest challeng...