Introduction Cost optimisation remains a pivotal challenge for customers dealing with processing large volumes of data and machine learning model training at scale in the cloud. Spot instances have re...
When setting up compute, there are many options and knobs to tweak and tune, and it can get quite overwhelming very quickly. To help you with optimally configuring your clusters, we have broken dow...
Authors: Andrey Mirskiy (@AndreyMirskiy) and Marco Scagliola (@MarcoScagliola) Welcome to the fourth part (#4) of our blog series on “Why Databricks SQL Serverless is the best fit for BI workloads”. I...
Databricks Model Serving provides a scalable, low-latency hosting service for AI models. It supports models ranging from small custom models to best-in-class large language models (LLMs). In this blog...
Unity Catalog (UC) is Databricks unified governance solution for all data and AI assets on the Data Intelligence Platform. UC is central to implementing MLOps on Databricks as it is where all your as...
In the world of data science, there is often a need to optimize or migrate legacy code. In this blog post, we address a common technical challenge faced by many data scientists and engineers - making ...
Inuktitut, the language of the Inuit, has 50 words for snow and ice. That’s - as they say - fake news, but the point made is metaphorical: When something is important to a people, their language finds...
Welcome to the fourth instalment of our blog series exploring Databricks Workflows, a powerful product for orchestrating data processing, machine learning, and analytics pipelines on the Databricks Da...
Authors: Ryuta Yoshimatsu, Michael Shtelma, Alex Miller Introduction Alignment of large language models (LLM) is a critical topic when building production-ready models for industrial use cases. An a...
MLflow MLflow stands out as the leading open source MLOps tool, and we strongly recommend its integration into your machine learning lifecycle. With its diverse components, MLflow significantly boo...
