June 15–18, 2026 Are you building the future with data and AI? Then this is your moment. The Call for Proposals for Data + AI Summit 2026 is officially open, and we want to hear from builders, practitioners, and innovators across the data and AI com...
- 106 Views
- 2 replies
- 1 kudos
yesterday
Your voice matters to us. We are planning the 2026 Data + AI Summit, and we’d love your input on what would make the experience even more valuable for you. Take a few minutes to share your feedback through our quick survey — your insights directly in...
- 104 Views
- 1 replies
- 3 kudos
yesterday
How to Register & Prepare If you're interested in advancing your skills with Databricks through a Specialist Session, here's a clear guide on how to register and what free courses you can take to prepare effectively. How to Begin Your Learning Path S...
- 2962 Views
- 2 replies
- 9 kudos
10-02-2025
Governing the Full Data Lifecycle: From Ingestion to Insight with Unity Catalog Date: Saturday, December 13 Time: 12:00 PM – 3:00 PM Location: Bengaluru, Karnataka (Register to see address) Registration: Approval required Register Here! About Thi...
- 746 Views
- 1 replies
- 3 kudos
2 weeks ago
Event Details Date: 19 December 2025Time: 6:00 PM – 7:30 PM ISTFormat: Virtual (Online) Marking RSVP on this page will not register your interest for the event. Please use the button below. Unlimited seats — but registration is recommended to recei...
- 691 Views
- 0 replies
- 6 kudos
2 weeks ago
Our Champion program has always celebrated the customers who go above and beyond to engage, help others, and uplift the Community. Recently, we have seen remarkable participation from Bricksters as well—and their impact deserves recognition too. Begi...
- 1068 Views
- 6 replies
- 13 kudos
4 weeks ago
As data volume and complexity increase, engineers are left figuring out how to manage, monitor and maintain fragile pipelines while also handling fragmented tools. The Big Book of Data Engineering equips you with cutting-edge methods for building pip...
- 1392 Views
- 5 replies
- 9 kudos
11-12-2025
We’re inching toward year-end, And the Community just dropped another week packed with sharp questions and standout wins.If you missed the action, here’s the pulse of everything that made the feed light up Week’s Power Players Spotlight on t...
- 201 Views
- 2 replies
- 1 kudos
Friday
Community Activity
- 2 Views
- 0 replies
- 0 kudos
Issues while running SYNC SCHEMA (HIVE-6384)
Hi All, We are in a process of migrating to UC from HMS and also changing the DBR from 11.3 to 16.4 LTS. When i am running sync schema from HMS to UC i am getting "Parquet does not support date. See HIVE-6384" error. Trying to resolve this, i have di...
- 2 Views
- 0 replies
- 0 kudos
- 1 Views
- 0 replies
- 0 kudos
Power BI refresh job task
I have tried Databricks job task to refresh power bi dataset and I have found 2 issues.1. I set up tables in Power BI Desktop using Import mode. After deploying the model to Power BI Service, I was able to download it as an Import mode model. However...
- 1 Views
- 0 replies
- 0 kudos
- 93 Views
- 2 replies
- 0 kudos
Jackson version conflict
Hi,I am trying to implement the Spark TableProvider api and i am experiencing a jar conflict (I am using the 17.3 runtime). com.fasterxml.jackson.databind.JsonMappingException: Scala module 2.15.2 requires Jackson Databind version >= 2.15.0 and < 2.1...
- 93 Views
- 2 replies
- 0 kudos
Latest Reply
- 0 kudos
For now we are trying to contact Databricks, In worst case scenario we were planning to shade the dependencies we need.would love to hear what has worked for you.Best,Shimon
- 0 kudos
- 7 Views
- 0 replies
- 0 kudos
How do you organize ML projects in Databricks workspaces?
How do you keep your machine-learning files, notebooks, and code properly organized in Databricks?
- 7 Views
- 0 replies
- 0 kudos
- 10 Views
- 0 replies
- 0 kudos
certification
i want to do certification in Databricks Certified Data Engineer Associate is there any discount ? and what will it cost in India .
- 10 Views
- 0 replies
- 0 kudos
- 46 Views
- 1 replies
- 1 kudos
Managing Spark Declarative Pipelines Permissions
We are currently managing our permissions via Terraform (including cluster creation, UC governance, etc.). We have a specific `data_engineer` role, and we need everyone with this role to be able to view and manage all of our SDPs.The Issue: Currently...
- 46 Views
- 1 replies
- 1 kudos
Latest Reply
- 1 kudos
This is a common governance challenge when teams move to Terraform-driven Databricks environments, trying to balance least-privilege access with operational efficiency. We see this frequently as platforms scale beyond a small admin group.At Kanerika,...
- 1 kudos
- 14 Views
- 0 replies
- 0 kudos
AI/BI Genie
I was working with genie and started using Research agent, and now I want to make the genie as a sharable Infrastructure as Code where I can manage all the changes through GitHub and so does anyone have any suggestions how to do this in a best optimi...
- 14 Views
- 0 replies
- 0 kudos
- 40 Views
- 1 replies
- 0 kudos
DLT File Level Deduplication
I want to create a DLT pipeline that incrementally processes csv files arriving daily. However, some of those files are duplicate - they have the same names and data but are in different directories. What is the best way to handle this? I'm assuming ...
- 40 Views
- 1 replies
- 0 kudos
Latest Reply
- 0 kudos
Hello @dgahram , Auto Loader tracks ingestion progress by persisting discovered file metadata in a RocksDB store within the checkpoint, which provides “exactly-once” processing for discovered files.Doc: https://docs.databricks.com/aws/en/ingestion...
- 0 kudos
- 81 Views
- 2 replies
- 2 kudos
AI/BI Dashboard embed issue in Databricks App
Hi everyone,I’ve created an AI/BI Dashboard in Azure Databricks and successfully published it, generated an embed link. My goal is to embed this dashboard inside a Databricks App (Streamlit) using an iframe.However, when I try to render the dashboard...
- 81 Views
- 2 replies
- 2 kudos
Latest Reply
- 2 kudos
Greetings @abhijit007 , I did a little resarch and have come up with some hints/guidance to help you: Hey @abhijit007 — I did a little digging on this and pulled together some concrete guidance that should help you unblock things. Thanks for sharing ...
- 2 kudos
- 106 Views
- 2 replies
- 3 kudos
🎤 Call for Presentations: Data + AI Summit 2026 is Open!
June 15–18, 2026 Are you building the future with data and AI? Then this is your moment. The Call for Proposals for Data + AI Summit 2026 is officially open, and we want to hear from builders, practitioners, and innovators across the data and AI com...
- 106 Views
- 2 replies
- 3 kudos
- 33 Views
- 0 replies
- 0 kudos
Show values as rows instead of columns in pivot table Databricks AI/BI
Is there a way to show values in columns instead of rows in pivot table function?I have fields I want to show as rows, and when I want to add multiple fields under values, they show as rows instead of columns.I want each values field to show as colum...
- 33 Views
- 0 replies
- 0 kudos
- 2629 Views
- 2 replies
- 7 kudos
Azure Databricks — Upgrade Managed Workspace to VNet Injected Workspace
Managed Workspace in Azure Databricks Azure Databricks, a first-party service on Microsoft Azure, seamlessly integrates with other Azure services and is easily deployed through the Azure portal with a few clicks. This simplified deployment wi...
- 2629 Views
- 2 replies
- 7 kudos
Latest Reply
- 7 kudos
does the redeployment delete or remove any of the catalogs or notebooks in the databricks environment?
- 7 kudos
- 35 Views
- 0 replies
- 0 kudos
Databricks Data Engineer Associate Exam Dec 2025
Hi, I am prepping for the Databricks DEA exam. I am seeing some conflicts with DLT/DP or declarative pipeline syntax. I am trying to see if the most up to date syntax is going to be on the exam as opposed to the legacy DLT. For example, current DP sy...
- 35 Views
- 0 replies
- 0 kudos
- 56 Views
- 0 replies
- 4 kudos
Training That Scales: How Databricks Adoption Really Takes Hold
I teach Databricks to all sorts of folks, coders, managers—everyone. What’s wild is how two companies with the same setup can have totally different experiences. Usually, it’s not the tech itself that’s the issue, but how people see it. Databricks tr...
- 56 Views
- 0 replies
- 4 kudos
- 56 Views
- 4 replies
- 0 kudos
Extract Snowflake data based on environment
Hi all, In the development workspace, I need to extract data from a table/view in Snowflake development environment. Example table is called as VD_DWH.SALES.SALES_DETAILWhen we deploy the code into production, it needs to extract data from a table/vi...
- 56 Views
- 4 replies
- 0 kudos
Latest Reply
- 0 kudos
Option 1 — Notebook widgets (simple & reliable)Use a Databricks widget to pass the environment (e.g., via Jobs/Workflows parameter), then build the table name dynamically.# Databricks notebook dbutils.widgets.dropdown("env", "dev", ["dev", "prod"], "...
- 0 kudos
Announcements
| User | Count |
|---|---|
| 2180 | |
| 1209 | |
| 1051 | |
| 949 | |
| 872 |
Latest from our Blog
Scaling XGBoost: How to Distribute Training with Ray and GPUs on Databricks
Problem Statement Technologies used: Ray, GPUs, Unity Catalog, MLflow, XGBoost For many data scientists, eXtreme Gradient Boosting (XGBoost) remains a popular algorithm for tackling regression and cla...
Best Practices: Building Compliance-Ready Agentic AI on the Databricks Platform
Financial institutions face a critical challenge: as member bases grow, how do you deliver personalized retirement advice at scale without proportionally increasing costs? More importantly, how do you...
How to perform Semantic Search in Databricks Lakebase
Introduction In today’s AI-native world, applications no longer rely on exact keyword matches—they understand meaning. This shift is powered by embeddings: numerical representations of text that captu...
Driving Revenue with Data: How Vehicle Service Notes Unlock Upsell Opportunities
The Hidden Story in Every Service Visit It’s a busy Tuesday morning at a dealership. A customer pulls in for what should be a simple oil change. The technician performs the inspection, then notices so...
