- 240 Views
- 2 replies
- 0 kudos
Resolved! Custom and community connectors
Hi,The option to enable custom and community connectors does not seem to be available on the Previews page, how can this be enabled? Feature I'm referencing: Community connectors in Lakeflow Connect - Azure Databricks | Microsoft Learn
- 240 Views
- 2 replies
- 0 kudos
- 0 kudos
Hi @koen_hai, The Community Connectors feature is controlled from the workspace-level Previews page by a workspace admin. If you don’t see that option there, the workspace likely hasn’t been enrolled for the preview yet. In that case, please contact ...
- 0 kudos
- 2541 Views
- 4 replies
- 2 kudos
[Bug] Orphan storage location
Hello,I'm not able to re-create an external location after removing its owner from Databricks Account. I'm getting the following error:Input path url 'abfss://foo@bar.dfs.core.windows.net/' overlaps with an existing external location within 'CreateEx...
- 2541 Views
- 4 replies
- 2 kudos
- 2 kudos
Your metastore admin can list all external locationsYour metastore admin can then drop the external location
- 2 kudos
- 202 Views
- 1 replies
- 1 kudos
Resolved! Managing Default Start State for Continuous Streaming Jobs in Databricks Asset Bundles
’ve created a notebook that uses Spark Structured Streaming and runs continuously, so I’ve deployed the corresponding Databricks job using the continuous trigger mode.What I’d like is for this job to start automatically only in certain environments ...
- 202 Views
- 1 replies
- 1 kudos
- 1 kudos
I figured out that the continuous property has a pause_status aswell, not sure why I did not see this. So I think the above is solved!
- 1 kudos
- 247 Views
- 3 replies
- 0 kudos
Resolved! Best Compute Option for Near-Real-Time Databricks API Ingestion Pipeline
I’ve built an ingestion pipeline in Databricks consisting of two notebooks:The first notebook calls an external API every four minutes to retrieve the latest available data.Each API call returns approximately 109 rows.The API only exposes the most re...
- 247 Views
- 3 replies
- 0 kudos
- 0 kudos
Hi @mnissen1337 ,I would use serverless for that use case. It takes a time for job cluster to spin up (of course you can use pools, but given that your job needs to run every 5 minutes it doesn't make much sense), so serverless seems to be a great fi...
- 0 kudos
- 194 Views
- 1 replies
- 1 kudos
Regarding Databricks Certified Data Engineer Professional recent syllabus change
Dear Team,I am planning to appear Data Engineer Professional certification in June month end. Now I wanted to know will there be any change in syllabus or exam pattern. Because I heard syllabus has been changed for Data Engineer Associate exam.Kind r...
- 194 Views
- 1 replies
- 1 kudos
- 1 kudos
As far as I can see, the latest official syllabus/exam guide is available here,https://www.databricks.com/sites/default/files/2026-05/databricks-certified-data-engineer-associate-exam-guide-may-2026.pdfDatabricks updates the syllabus from time to tim...
- 1 kudos
- 88 Views
- 1 replies
- 0 kudos
Access denied to course
when I tried to access Data engineering + Generative AI course on Databricks account it says access denied contact your admin.. suggest what to do next?
- 88 Views
- 1 replies
- 0 kudos
- 0 kudos
Hi @AnandB ,Try following:1. Try the Academy portal directlyLog in directly at https://customer-academy.databricks.com/learn (or https://partner-academy.databricks.com/learn if you're a partner), clear your browser cache and cookies first, then searc...
- 0 kudos
- 115 Views
- 1 replies
- 0 kudos
Salesforce Connector - Lakeflow Connect 400 Error
HI All,I have been trying to setup Salesforce using Lakeflow Connect and followed instructions on the docshttps://docs.databricks.com/aws/en/connect/managed-ingestion#sfdcHowever I face into invalid_grant error However login history on salesforce sh...
- 115 Views
- 1 replies
- 0 kudos
- 0 kudos
Hi Vedanth,The invalid_grant error usually occurs due to authentication or OAuth configuration issues between Salesforce and Databricks Lakeflow Connect.Could you please verify the following points:Ensure the Salesforce user account is not locked and...
- 0 kudos
- 116 Views
- 2 replies
- 0 kudos
[DELTA_CREATE_EXTERNAL_TABLE_WITHOUT_TXN_LOG]
We are testing Delta writer in our environment to create bronze tables and recently, I just needed to add one table to the notebook code and rerun the whole notebook that failed because of this error : [DELTA_CREATE_EXTERNAL_TABLE_WITHOUT_TXN_LOG] Y...
- 116 Views
- 2 replies
- 0 kudos
- 0 kudos
@Yannick_B You are trying to register an external table pointing to a directory that does not contain required Delta transaction logs (_delta_log folder) and hence you see the error.When you run External Table Command, Databricks generally expects th...
- 0 kudos
- 130 Views
- 1 replies
- 0 kudos
Does enabling Change Data Feed on a Delta table affect OPTIMIZE and ZORDER performance?
Does enabling Change Data Feed on a Delta table affect OPTIMIZE and ZORDER performance?After enabling CDF on several large Delta tables, our OPTIMIZE jobs are taking noticeably longer. Is this expected, and are there any tuning parameters to minimize...
- 130 Views
- 1 replies
- 0 kudos
- 0 kudos
@GaneshI Yes, some overhead is expected. Databricks recommends predicate-based OPTIMIZE for large tables and Liquid Clustering over ZORDER.
- 0 kudos
- 148 Views
- 1 replies
- 0 kudos
Serverless Compute connectivity issues with .com.br domains vs. Classic Clusters Spark hangs
Hi everyone,I'm facing two specific issues in my Databricks Premium workspace (AWS - sa-east-1).Serverless Connectivity Issue: When using Serverless compute, I can successfully call APIs ending in .com, but calls to .com.br domains fail with connecti...
- 148 Views
- 1 replies
- 0 kudos
- 0 kudos
Hi there,Great breakdown of the symptoms — these are actually two distinct issues likely sharing a common root cause in your VPC/network configuration. Let me address both:Issue 1: Serverless Compute — .com.br DNS Resolution FailureRoot CauseServerle...
- 0 kudos
- 237 Views
- 2 replies
- 0 kudos
Lakebase not showing up
I am fairly new to Databricks and am learning it because a company I am working on is going to use it. One of the things they are going to use is Lakebase postgres so I thought I'd set it up on my personal account. First I don't see app switcher, sec...
- 237 Views
- 2 replies
- 0 kudos
- 0 kudos
If it is available, you would see at Compute->Lakebase and tabs for Provisioned and Autoscaling. This option Lakebase is next to Apps. There is another option dotted grid on the top right corner, the option just before your profile name, if you expan...
- 0 kudos
- 159 Views
- 1 replies
- 0 kudos
What is the recommended approach to enforce row-level security in Unity Catalog for external BI tool
We connect Tableau and Power BI to our Databricks SQL warehouse via OAuth tokens. Does Unity Catalog row filters apply at the SQL layer regardless of the BI tool, or do we need additional enforcement at the warehouse level?
- 159 Views
- 1 replies
- 0 kudos
- 0 kudos
Unity Catalog row filters apply at the SQL/query layer, so if Tableau or Power BI is querying a Databricks SQL warehouse, the filters are enforced there — you do not need a separate warehouse-level row-filter feature. Row filters and column masks are...
- 0 kudos
- 214 Views
- 2 replies
- 1 kudos
Best Practices: 1 job per 1 target table
We’re currently designing our Medallion Architecture pipelines using Lakeflow Jobs, and I wanted to get some opinions on orchestration best practices.Right now, our approach is essentially 1 job per target table (for example, each Bronze/Silver/Gold ...
- 214 Views
- 2 replies
- 1 kudos
- 1 kudos
We typically organize our workloads with one job per catalog, and then use one or more pipelines to load tables into the appropriate schemas. As our data engineers ingest raw data, this structure is primarily applied in the Silver and Gold layers of ...
- 1 kudos
- 1821 Views
- 3 replies
- 2 kudos
Resolved! Scheduling jobs with table update triggers
Hi all,Lately I've been experimenting with the newish feature of scheduling jobs on a table update trigger. There's one thing thats blokcing me from implementing it however and I was hoping someone found a solution to it.We occasionally perform a vac...
- 1821 Views
- 3 replies
- 2 kudos
- 2 kudos
Hi @Garybary, Quick clarification on how table update triggers actually behave, because this changes the answer significantly. Table update triggers fire on data-changing operations only (writes, merges, updates, deletes). A standalone VACUUM does NO...
- 2 kudos
- 6387 Views
- 11 replies
- 8 kudos
Resolved! Connect to azure data lake storage using databricks free edition
Hello guys, i'm using databricks free edition (serverless) and i am trying to connect to a azure data lake storage.The problem I'm having is that in the free edition we can't configure the cluster so I tried to make the connection via notebook using ...
- 6387 Views
- 11 replies
- 8 kudos
- 8 kudos
If you want to read from your Azure storage account using Databricks Free Edition, you can add a specific option when reading:spark.read.option("fs.azure.account.key.<storage-account-name>.dfs.core.windows.net", "your_storage_account...
- 8 kudos
| User | Count |
|---|---|
| 1644 | |
| 793 | |
| 568 | |
| 349 | |
| 287 |
