MLOps Gym - Databricks Feature Store
This is the first of three articles about using the Databricks Feature Store. The first article will focus on using existing features to create your dataset and the basics of creating feature tables...
Featured topics
Blog Articles
How to enhance data protection in Azure Databricks with Confidential Computing
Proposed by Databricks in 2020, the Lakehouse architecture has increasingly been embraced by the industry in the past years. Databricks has since applied Generative AI and evolved the Databricks Lakeh...
Prepare Your Journey to Migrate from AWS Glue Data Catalog to Databricks Unity Catalog
In this blog, we will look at the migration from AWS Glue Data Catalog to Unity Catalog. We cover how to plan this migration as a step-by-step approach and emphasize meticulous planning, phased migrat...
Mastering the Spark UI
Almost anyone who uses Spark or Databricks is aware of the Spark UI, and we all know that it’s a super powerful tool in the right hands. It can reveal what’s going wrong and any inefficiencies in you...
Optimize costs for your Data and AI workloads with Azure and AWS Spot Instances
Introduction Cost optimisation remains a pivotal challenge for customers dealing with processing large volumes of data and machine learning model training at scale in the cloud. Spot instances have re...
MLOps Gym - Beginners Guide to Cluster Configuration for MLOps
When setting up compute, there are many options and knobs to tweak and tune, and it can get quite overwhelming very quickly. To help you with optimally configuring your clusters, we have broken dow...
Why Serverless Databricks SQL is the best for BI workloads: Part IV - Remote Query Result Cache
Authors: Andrey Mirskiy (@AndreyMirskiy) and Marco Scagliola (@MarcoScagliola) Welcome to the fourth part (#4) of our blog series on “Why Databricks SQL Serverless is the best fit for BI workloads”. I...
Attributing Costs in Databricks Model Serving
Databricks Model Serving provides a scalable, low-latency hosting service for AI models. It supports models ranging from small custom models to best-in-class large language models (LLMs). In this blog...
MLOps Gym - Unity Catalog Setup for MLOps
Unity Catalog (UC) is Databricks unified governance solution for all data and AI assets on the Data Intelligence Platform. UC is central to implementing MLOps on Databricks as it is where all your as...
Scaling Pandas with Databricks: Passing Parameters to Pandas UDFs
In the world of data science, there is often a need to optimize or migrate legacy code. In this blog post, we address a common technical challenge faced by many data scientists and engineers - making ...
- Erik on: Optimize costs for your Data and AI workloads with...
- quintenrosseel on: MLOps Gym - Unity Catalog Setup for MLOps
-
SepidehEb
on: MLOps Gym - Beginners Guide to MLflow
- Erik on: A Data Engineer's Guide to Optimized Streaming wit...
-
mattiazeni
- gatb27 on: Databricks cost analysis and cross charge with Pow...
-
SergeRielau
-
SergeRielau
- rantav on: How to implement Slowly Changing Dimensions when y...
- db_allrails on: Using dbt core with OAuth on Azure Databricks