Using dbt core with OAuth on Azure Databricks
By Jeroen Meulemans, Solutions Architect at Databricks Amsterdam, jeroen.meulemans@databricks.com, Thu 7 Sep 2023 Introduction This blog guides you through the process of configuring OAuth credential...
Blog Articles
Distributed Fine Tuning of LLMs on Databricks Lakehouse with Ray AI Runtime, Part 2
Authors: Anastasia Prokaieva and Puneet Jain In our first part, we have covered the main aspects of the data loading using Hugging Face integration with the Spark dataframes and how to use RayAIR to ...
How to build operational low-latency, stateful Spark Structured Streaming pipelines
Learn to build fast, stateful pipelines for operational workloads. Discover stateless vs. stateful streams, how to setup your cluster and more. Get hands-on building a pipeline with code snippets and ...
How to Sync nested Azure AD groups to Databricks
Authors: Abhishek Pratap (@aps) & Dipankar Kushari (@dkushari)In this blog, we explore how to synchronize nested groups in Databricks from your organization’s identity provider - Azure Active Director...
Top 10 query performance tuning tips for Databricks Serverless SQL
Databricks Serverless SQL (DBSQL) is the latest offering from Databricks to build data warehouses on the Lakehouse. It incorporates all the Lakehouse features like open format, unified analytics, and ...
How SSO works with Databricks and your Identity Provider: A walkthrough
What is Single Sign-On (SSO)?The rise of Software as a Service (Saas) has proliferated the adoption of specialized software in todays modern world. This resulted in enterprises adopting many Saas serv...
Distributed Fine Tuning of LLMs on Databricks Lakehouse with Ray AI Runtime, Part 1
Authors: Anastasia Prokaieva and Puneet Jain The aim of this blog is to show the end-to-end process of conversion from vanilla Hugging Face to Ray AIR on Databricks, without changing the training lo...
How to implement Slowly Changing Dimensions when you have duplicates - Part 1: What to look out for?
Maintaining Slowly Changing Dimensions (SCD) is a common practice in data warehousing to manage and track changes in your records over time. It enables businesses to make more informed and strategic d...
Beginners Guide to Databricks: Batch Processing and Streaming
In this blog, I would like to introduce to you the Databricks lakehouse platform and explain concepts like batch processing, streaming, apache spark at a high level and how it all ties together with s...
How to use Bokeh to build a shareable dashboard with Delta Lake and host it on Databricks
In order to gain valuable insights from large and complex data, it is necessary to use contemporary tools and technology. Organizations may enhance their performance by using data-driven choices and b...
- Ramana on: Processing LAS Well Log Files and Other Semi-Struc...
- -werners- on: AutoLoader: XML Data Ingestion with Built-in Data ...
-
thais_henrique on: Azure Databricks VM Inventory Dashboard
- Ramana on: Performance Showdown: withColumn vs withColumns in...
- Nidhig on: 💬 Chat with Your Data in Slack Using Databricks G...
- Fizzle on: How to Anonymize Vehicle Data with Databricks Delt...
- TheOC on: My First Experience at the Data + AI Summit: Highl...
- gjsau on: [PARTNER] Databricks Lakebase Costing: Compute, St...
- szymon_dybczak on: Ray on Spark: A Practical Architecture and Setup G...
- szymon_dybczak on: Routing Optimization on Databricks