- 1081 Views
- 2 replies
- 0 kudos
Resolved! Azure Databricks - Exporting data frame to external volume
Hello,I am reading a Delta table and exporting it to an external volume. The Unity Catalog external volume points to an Azure Data Lake Storage container.When I run the code below, I encounter the error message shown below. (When I export the data to...
- 1081 Views
- 2 replies
- 0 kudos
- 0 kudos
Hi @lion_king_84 are we missing write volume permissions on this schema in UC?
- 0 kudos
- 1861 Views
- 3 replies
- 2 kudos
Resolved! Databricks Networking
Currently our workspace has almost reached its maximum IP capacity, and we would like to create additional compute resources to handle memory intensive tasks. In doing some research we have been running into some conflicting answers, so I wanted to a...
- 1861 Views
- 3 replies
- 2 kudos
- 2 kudos
Greeting @Thabang , I did some digging and there are potential mitigation paths, but they’re fairly involved and not something you’d want to tackle solo. The guidance consistently points to engaging Support—you’ll definitely need their help to do thi...
- 2 kudos
- 8929 Views
- 12 replies
- 4 kudos
Resolved! Deploy Workflow only to specific target (Databricks Asset Bundles)
I am using Databricks Asset Bundles to deploy Databricks workflows to all of my target environments (dev, staging, prod). However, I have one specific workflow that is supposed to be deployed only to the dev target environment.How can I implement tha...
- 8929 Views
- 12 replies
- 4 kudos
- 4 kudos
I know it's a bit old, but if someone is looking into a solution, then I was able to resolve the issue where I need to deploy some jobs only into the DEV target:https://github.com/databricks/bundle-examples/tree/main/knowledge_base/target_includes.Us...
- 4 kudos
- 817 Views
- 1 replies
- 2 kudos
Resolved! Databricks Asset Bundles Issue
Hi All,We are trying to update an existing job with some additional configurations. We are trying to deploy the new changes to the job using the DABs process. We have mentioned the job_id and job_name variables and defined them as well.Pipeline state...
- 817 Views
- 1 replies
- 2 kudos
- 2 kudos
Hi @Harish_Kumar_M ,Databricks Asset Bundles (DABs) require binding to existing workspace resources like manually created jobs before updates will reflect during deployment. https://docs.azure.cn/en-us/databricks/dev-tools/bundles/migrate-resourcesYo...
- 2 kudos
- 970 Views
- 2 replies
- 3 kudos
Resolved! timeout for sessions made with sdk
How do I determine the rules for when a session will timeout? From testing it seems to be related to inactivity. I am using JavaScript SDK and I want to test my logic for creating a new session when the current one times out based on this issue: http...
- 970 Views
- 2 replies
- 3 kudos
- 754 Views
- 2 replies
- 0 kudos
Resolved! Lineage & Query history table
Hi,I’m trying to perform a join between the Lineage and Query History system tables. Initially, I assumed statement_id could be used as the join key, but that didn’t seem correct. I then noticed that joining statement_id with entity_run_id also appea...
- 754 Views
- 2 replies
- 0 kudos
- 0 kudos
Hi @MadMax_71_2 ,Check ERD diagram. Here you should find what's the correct key:docs.databricks.com/aws/en/assets/images/system-tables-erd-690b80220d077015a023f90d59ff7560.svg
- 0 kudos
- 1203 Views
- 5 replies
- 3 kudos
Resolved! Questions regarding catalogs starting with __databricks_internal_catalog
Hello,I am a Data Engineer currently building a data platform in a Databricks-AWS environment.While querying system.information_schema.catalog_privileges, I noticed several catalogs starting with __databricks_internal_catalog. Specifically, there are...
- 1203 Views
- 5 replies
- 3 kudos
- 1422 Views
- 3 replies
- 2 kudos
Resolved! College Course Use - Sharing Data With Students
Hello Everyone,I am creating a new college course on Database design and SQL analytics and have decided to use Databricks as our platform in the course. We are going to be using the Free Edition so students do not need to pay for access. I'm wonder...
- 1422 Views
- 3 replies
- 2 kudos
- 2 kudos
Excellent information, thank you! The data generator was something I was not aware of so I will check that out.
- 2 kudos
- 1409 Views
- 2 replies
- 3 kudos
Resolved! Communication between two workspaces
Hi everyone, our team is implementing an application utilizing the Databricks App (in AzureDatabricks) to deploy a website in workspace A (sea). This web calls an API to the Databricks Dashboards in workspace B (jpe) to get a token to embed the dashb...
- 1409 Views
- 2 replies
- 3 kudos
- 3 kudos
The specific error message is a known symptom of Databricks network-policy enforcement for cross-workspace calls, not (only) an OAuth/service-principal problem.403 Cert validation failed. Both workspace comparison and snp system trusted checks did no...
- 3 kudos
- 517 Views
- 1 replies
- 2 kudos
Resolved! Company card / billing in Databricks express setup
For Databricks express setup only, is there a way to add company's VAT number, name and address during (or after) the "add a payment method" form, in order to receive a valid company's billing document from Databricks?
- 517 Views
- 1 replies
- 2 kudos
- 2 kudos
I dont thing this can be done. You can check with Tech support as well . https://help.databricks.com/
- 2 kudos
- 4696 Views
- 2 replies
- 0 kudos
How do I get rid of the GKE cluster?
hi!In our organisation we use databricks but I do not understand why this GKE cluster keeps getting created. We deploy workspaces and compute clusters through terraform and use the GCE tag"x-databricks-nextgen-cluster" = "true"From my understanding, ...
- 4696 Views
- 2 replies
- 0 kudos
- 0 kudos
Hello @Louis_Frolio I would like to confirm my understanding regarding cost attribution for workloads running on Next-Gen / Serverless / Ephemeral Compute that are tagged withx-databricks-nextgen-cluster = "true".Is it correct that, in this case, the...
- 0 kudos
- 880 Views
- 3 replies
- 1 kudos
Resolved! Enabling External Lineage on a free or trial account?
Hi,as part of a small OSS project I am doing, dbt-unity-lineage, I need to enable Bring your own data lineage (Public Preview as of December 2025). But it seems you can't enable that Preview in either free edition or Trial?I'd rather not use my emplo...
- 880 Views
- 3 replies
- 1 kudos
- 1 kudos
@fgeriksen , if you are satisfied with the response please "Accept as Solution" so that others will be informed as well. Cheers, Louis.
- 1 kudos
- 2382 Views
- 5 replies
- 3 kudos
Delta Sharing from Databricks to SAP BDC fails with invalid_client error
ContextWe are in the process of extracting data between SAP BDC Datasphere and Databricks (Brownfield Implementation).SAP Datasphere is hosted in AWS (eu10)Databricks is hosted in Azure (West Europe)The BDC Connect System is located in the same regio...
- 2382 Views
- 5 replies
- 3 kudos
- 3 kudos
The error DELTA_SHARING_INVALID_RECIPIENT_AUTH refers to an invalid authorization specification when accessing Delta Sharing resources. This maps to SQLSTATE code 28000 ("invalid authorization specification") and typically occurs when the recipient's...
- 3 kudos
- 658 Views
- 1 replies
- 1 kudos
Databricks certificate expired
Hello,I have a databricks workspace with sso authentication. the IDP is on azure.The client certificate expired and now, I can't log on to databricks to add the new one.How can I do? Any idea is welcomed.Thank you!!Best regards,daniela
- 658 Views
- 1 replies
- 1 kudos
- 1 kudos
This is an AWS Databricks workspace and your SSO is with EntraID? You'll need to create a Support Ticket and then Engineering can disable-SSO temporarily allowing you to login with user+OTP. The long term solution here is that you should: Set up Acco...
- 1 kudos
- 3586 Views
- 2 replies
- 2 kudos
Resolved! Databricks Apps - X-Forwarded-Access-Token not available
Hi all,I've recently been getting started with Databricks Apps, and have been trying to use on-behalf-of access. However, it seems that the X-Forwarded-Access-Token is not coming through in my app headers. I have lots of other X-Forwarded ones such a...
- 3586 Views
- 2 replies
- 2 kudos
- 2 kudos
Just to add from my experience, in case James’s solution doesn’t work for you: I tried fully stopping and restarting the app, but the user authentication page which asks for permission didn’t appear until I used incognito mode
- 2 kudos
