cancel
Showing results for 
Search instead for 
Did you mean: 
cancel
Showing results for 
Search instead for 
Did you mean: 
Data + AI Summit 2025 — registration now open!

Be part of a global movement! Connect with 22,000 data enthusiasts at 700+ sessions, keynotes, and hands-on training at this year’s Data + AI Summit. Whether you’re into data intelligence, governance, AI, or data warehousing, this is your chance to l...

  • 12957 Views
  • 3 replies
  • 5 kudos
02-19-2025
Upskill yourself and your teams at Data+AI Summit

Get 20% off paid training today using the code: TRNGDQ9TY. We are experiencing an unprecedented pace of technological innovation driven by AI and data. According to the World Economic Forum’s Future of Jobs report, six out of ten business leaders exp...

  • 487 Views
  • 0 replies
  • 3 kudos
Wednesday
[eBook] Migrate your legacy data warehouse to Databricks

Low-risk migration to a fast and open data warehouse Legacy enterprise data warehouses (EDWs) are becoming a bottleneck for businesses aiming to scale operations and adopt advanced analytics. Traditional EDWs struggle with: Scalability: Expensive har...

  • 480 Views
  • 0 replies
  • 1 kudos
Wednesday
Announcing updates to Databricks Query Profiles

At Databricks, we’re always working hard to make your queries run faster. Still, there are times when it is helpful to look a little deeper to see how your queries are turned into execution plans and distributed for parallel execution. That’s where Q...

  • 451 Views
  • 0 replies
  • 2 kudos
Wednesday
Databricks Community Champion - April 2025 - Brahma Reddy

This month, we are excited to spotlight our April Community Champion, Brahma Reddy (@Brahmareddy), Lead Data Engineer at Delta! Brahma is a passionate problem-solver who has made impactful contributions in the Databricks Community through knowledge ...

  • 685 Views
  • 3 replies
  • 4 kudos
2 weeks ago

Community Activity

Dharma25
by > New Contributor
  • 321 Views
  • 1 replies
  • 0 kudos

workflow not pickingup correct host value (While working with MLflow model registry URI)

Exception: mlflow.exceptions.MlflowException: An API request to https://canada.cloud.databricks.com/api/2.0/mlflow/model-versions/list-artifacts failed due to a timeout. The error message was: HTTPSConnectionPool(host='canada.cloud.databricks.com', p...

  • 321 Views
  • 1 replies
  • 0 kudos
Latest Reply
LRALVA
Honored Contributor
  • 0 kudos

@Dharma25 The error clearly shows that MLflow requests are being incorrectly routed to "canada.cloud.databricks.com" instead of your actual workspace URL, causing the timeout.This is a known issue that can occur with MLflow in Databricks, particularl...

  • 0 kudos
SKakarla
by > Visitor
  • 32 Views
  • 1 replies
  • 0 kudos

Notebooks owner shows 'Unknown'

Hi All,We are using CI/CD to deploy notebooks from GitHub and authenticating via Azure Service Principals (SPNs). Until last week, the notebook owner was correctly displayed as the SPN. However, over the past few days, the owner is now shown as "Unkn...

  • 32 Views
  • 1 replies
  • 0 kudos
Latest Reply
LRALVA
Honored Contributor
  • 0 kudos

@SKakarla The "Unknown" owner problem appearing suddenly across multiple environments does suggest a Databricks-side change rather than something in your configuration.This could be related to a recent update in how Databricks handles service princip...

  • 0 kudos
Chandinidcsk
by > Visitor
  • 20 Views
  • 1 replies
  • 0 kudos

Didn't recieve certificate for Data Engineer Professional

Hi @Advika_ @Advika @Cert-TeamOPS I successfully completed my Databricks Data Engineer Professional certification on 10th May 2025, but I haven't received my certificate until now. Can you please help me?Email Address: chandudcsk425@gmail.com Thanks ...

Capture.JPG
  • 20 Views
  • 1 replies
  • 0 kudos
Latest Reply
Cert-TeamOPS
Databricks Employee
  • 0 kudos

Hello @Chandinidcsk , Thank you for reaching out to Databricks Community support! We have shared you the certification email kindly check in your register email id. Apologies for the inconvenience caused. Best Wishes  Thanks & Regards @Cert_TeamOPS

  • 0 kudos
littlewat
by > New Contributor
  • 88 Views
  • 2 replies
  • 1 kudos

Why catalog API does not include the catalog ID in the response?

Hi!I'm using Terraform(TF) to manage the Databricks resources.I would like to rename the Unity catalog using TF, but I could not. (similar issues have been reported for this:- https://github.com/databricks/terraform-provider-databricks/issues?q=is%3A...

  • 88 Views
  • 2 replies
  • 1 kudos
Latest Reply
BigRoux
Databricks Employee
  • 1 kudos

The issue regarding renaming a Unity Catalog using Terraform is rooted in the way the Terraform databricks_catalog resource is implemented. Specifically, the catalog name is being used as the resource's ID, which is a direct consequence of limitation...

  • 1 kudos
1 More Replies
TamD
by > Contributor
  • 93 Views
  • 2 replies
  • 0 kudos

Cannot apply liquid clustering via DLT pipeline

I want to use liquid clustering on a materialised view created via a DLT pipeline, however, there doesn't appear to be a valid way to do this.Via table properties:@Dlt.table( name="<table name>, comment="<table description", table_propert...

  • 93 Views
  • 2 replies
  • 0 kudos
Latest Reply
BigRoux
Databricks Employee
  • 0 kudos

Based on the provided context, as of now, liquid clustering cannot be enabled directly on materialized views created via a Delta Live Tables (DLT) pipeline. Attempts to set table properties such as "delta.clusterBy" or "delta.liquidClustering.enabled...

  • 0 kudos
1 More Replies
Dave_Nithio
by > Contributor II
  • 916 Views
  • 1 replies
  • 0 kudos

Monitoring Databricks SQL Warehouse without Unity Catalog

I am looking to monitor my SQL Warehouse, especially the 'Running Clusters' metric that is available in the monitoring tab of the warehouse. This shows the start and shut down time as well as the number of running clusters:The issue I have run into i...

Dave_Nithio_0-1741034562647.png
  • 916 Views
  • 1 replies
  • 0 kudos
Latest Reply
Isi
Contributor III
  • 0 kudos

Hey @Dave_Nithio To monitor the “Running Clusters” metric for your SQL Warehouse, you can use the Databricks Cluster Events API. This API retrieves a list of events related to cluster activity, such as start and shutdown times, and provides paginated...

  • 0 kudos
barchiel33
by > New Contributor II
  • 355 Views
  • 3 replies
  • 1 kudos

Databricks SQL Python - Result fetching takes extremely long time

Hello All!I have a python script which utilizes the databricks SQL for python package in order to pull a databricks table into a pandas dataframe which is used to create a table in a Spotfire report. The table contains ~1.28 million rows, with 155 co...

  • 355 Views
  • 3 replies
  • 1 kudos
Latest Reply
Isi
Contributor III
  • 1 kudos

Hey @barchiel33 ,After reviewing your context further, I believe the most effective approach would be to set up an automated pipeline within Databricks that periodically extracts data based on the frequency you need (daily, weekly, hourly, etc.), cre...

  • 1 kudos
2 More Replies
CT123456
by > New Contributor
  • 33 Views
  • 2 replies
  • 0 kudos

Does Unity Catalog support Iceberg?

Hi, I came across this post on querying an AWS Glue table in S3 from Databricks using Unity Catalog: https://medium.com/@sidnahar13/aws-glue-meets-unity-catalog-effortless-data-governance-querying-in-databricks-5138f1e574a4. The post says support for...

  • 33 Views
  • 2 replies
  • 0 kudos
Latest Reply
BigRoux
Databricks Employee
  • 0 kudos

Databricks already supports querying AWS Glue-managed Iceberg tables stored in S3 by integrating with the AWS Glue Iceberg REST Catalog.Unity Catalog in Databricks supports Iceberg reads on Delta tables (UniForm), enhancing interoperability with Iceb...

  • 0 kudos
1 More Replies
oscarramosp
by > Visitor
  • 31 Views
  • 2 replies
  • 0 kudos

DLT Pipeline upsert question

Hello, I'm working on a DLT pipeline to build a what would be a Datawarehouse/Datamart. I'm facing issues trying to "update" my fact table when the dimensions that are outside the pipeline fail to be up to date at my processing time, so on the next r...

  • 31 Views
  • 2 replies
  • 0 kudos
Latest Reply
BigRoux
Databricks Employee
  • 0 kudos

The error encountered, "Cannot have multiple queries named catalog.schema.destination_fact for catalog.schema.destination_fact. Additional queries on that table must be named," arises because Delta Live Tables (DLT) disallows multiple unnamed queries...

  • 0 kudos
1 More Replies
amansatyawali
by > Visitor
  • 53 Views
  • 2 replies
  • 1 kudos

Resolved! Did not receive certification

Hi, I took my machine learning professional exam on 3rd May 2025, and received the mail for exam completion. In the mail it was written that I will receive the certificate on mail within 24-28 hours. But I have not received the certificate yet, can y...

  • 53 Views
  • 2 replies
  • 1 kudos
Latest Reply
amansatyawali
  • 1 kudos

I have received it. Thank you for the quick response.

  • 1 kudos
1 More Replies
thomas_berry
by > New Contributor
  • 27 Views
  • 1 replies
  • 0 kudos

federated queries on PostgreSQL - TimestampNTZ option

Hello,I am trying to migrate some spark reads away from JDBC into the federated queries based in unity catalog.Here is an example of the spark read command that I want to migrate:spark.read.format("jdbc").option("driver", "org.postgresql.Driver").opt...

  • 27 Views
  • 1 replies
  • 0 kudos
Latest Reply
LRALVA
Honored Contributor
  • 0 kudos

Hi @thomas_berry It's a common challenge when transitioning from Spark JDBC reads to Unity Catalog federated tables, especially when dealing with timestamp types.This option is specific to the JDBC data source and tells Spark to treat timestamp colum...

  • 0 kudos
Adrianj
by > New Contributor III
  • 13686 Views
  • 17 replies
  • 11 kudos

Databricks Bundles - How to select which jobs resources to deploy per target?

Hello, My team and I are experimenting with bundles, we follow the pattern of having one main file Databricks.yml and each job definition specified in a separate yaml for modularization. We wonder if it is possible to select from the main Databricks....

  • 13686 Views
  • 17 replies
  • 11 kudos
Latest Reply
Joao-Peixoto-DE
  • 11 kudos

Is there any update from databricks about this?

  • 11 kudos
16 More Replies
marcio_oliveira
by > Visitor
  • 65 Views
  • 3 replies
  • 2 kudos

Resolved! Job run failing to import modules

I have several notebooks that run code to ingest data from various APIs into our Data Warehouse. I have several modules that I reuse in multiple notebooks, things like redshift functions, string cleaning functions and json cleaning functions. Out of ...

marcio_oliveira_0-1747149522503.png
  • 65 Views
  • 3 replies
  • 2 kudos
Latest Reply
LRALVA
Honored Contributor
  • 2 kudos

Hi @marcio_oliveira Thanks for sharing the error and the context — this intermittent module import issue in Databricks Serverless jobs is a known behavior in some environments,and here’s what’s likely going wrong :Root Cause:A race condition or cold-...

  • 2 kudos
2 More Replies
Karthik_Karanm
by > New Contributor II
  • 129 Views
  • 3 replies
  • 2 kudos

Insufficient Permission Error When Serving RAG Model with Multiple Vector Search Indexes

Hi Community,I’m currently working on a Retrieval-Augmented Generation (RAG) use case in Databricks. I’ve successfully implemented and served a model that uses a single Vector Search index, and everything works as expected.However, when I try to serv...

  • 129 Views
  • 3 replies
  • 2 kudos
Latest Reply
LRALVA
Honored Contributor
  • 2 kudos

@Karthik_Karanm This is a known issue pattern when using multiple Unity Catalog (UC) vector search indexes inDatabricks Model Serving — especially under MLflow model serving endpoints with RAG architecture.Your model serving environment (i.e., the mo...

  • 2 kudos
2 More Replies
MauricioS
by > New Contributor II
  • 32 Views
  • 1 replies
  • 0 kudos

Need advice for a big source table DLT Pipeline

Hi all,I was hoping to get advice from someone with DLT Pipelines, I want to apologize in advance if this is a noob question, I'm really new into DLT, materialized views and streaming tablesI have the following scenario, my source is a big sales delt...

  • 32 Views
  • 1 replies
  • 0 kudos
Latest Reply
LRALVA
Honored Contributor
  • 0 kudos

Hi @MauricioS Absolutely not a noob question — you're touching on a common and important challenge in DLT pipelines,especially when dealing with large shared Delta tables and incremental ingestion from Unity Catalog sources.Let’s break it down so it’...

  • 0 kudos
Welcome to the Databricks Community!

Once you are logged in, you will be ready to post content, ask questions, participate in discussions, earn badges and more.

Spend a few minutes exploring Get Started Resources, Learning Paths, Certifications, and Platform Discussions.

Connect with peers through User Groups and stay updated by subscribing to Events. We are excited to see you engage!

Join Us as a Local Community Builder!

Passionate about hosting events and connecting people? Help us grow a vibrant local community—sign up today to get started!

Sign Up Now
Read Databricks Data Intelligence Platform reviews on G2

Latest from our Blog