- 4288 Views
- 3 replies
- 5 kudos
Resolved! How do I use databricks-cli without manual configuration
I want to use databricks cli:databricks clusters listbut this requires a manual step that requires interactive work with the user:databricks configure --tokenIs there a way to use databricks cli without manual intervention so that you can run it as p...
- 4288 Views
- 3 replies
- 5 kudos
- 5 kudos
You can set two environment variables: DATABRICKS_HOST and DATABRICKS_TOKEN, and databricks-cli will use them. See the example of that in the DevOps pipelinesee the full list of environment variables at the end of the Authentication section of docume...
- 5 kudos
- 1022 Views
- 1 replies
- 1 kudos
Query related to Job cluster verses All Purpose Cluster
Currently if run the Notebook with Job Cluster it will show status as Pending for 2 minutes but All Purpose Cluster shows only 6seconds as pending and completes the run quickly. How to improve startup time of Job Cluster to mmatch with startup time ...
- 1022 Views
- 1 replies
- 1 kudos
- 1 kudos
Hi @Dileep_Karanki Thank you for reaching out and for your question. Here is a link for documentation about the available cluster types: https://docs.databricks.com/aws/en/compute#types-of-compute All-Purpose or Interactive Clusters: These are flex...
- 1 kudos
- 443 Views
- 1 replies
- 1 kudos
Schedule & Triggers
Hi guys, having some issue when I use Schedule and trigger in workflow when file arrives condition, if a place a file with same name it won't trigger is there any configuration I need to do in databricks? thanks
- 443 Views
- 1 replies
- 1 kudos
- 1 kudos
Hi @Leog Thank you for reaching out and for your sending your question. Per the documentation: https://docs.databricks.com/gcp/en/jobs/file-arrival-triggers Only new files trigger runs. Overwriting an existing file with a file of the same name does ...
- 1 kudos
- 1036 Views
- 4 replies
- 0 kudos
Unable to connect to the cluster
Container launch failure:An unexpected error was encountered while launching containers on worker instances for the cluster. Please retry and contact Databricks if the problem persists.Instance ID: i-02aadc9a7c3532af0Internal error message: Failed to...
- 1036 Views
- 4 replies
- 0 kudos
- 8584 Views
- 8 replies
- 3 kudos
[Azure Databricks] Create an External Location to Microsoft Fabric Lakehouse
Hi,I want to create an external location from Azure Databricks to a Microsoft Fabric Lakehouse, but seems I am missing something.What did I do:I created an "Access Connector for Azure Databricks" in Azure PortalI created a storage credential for the ...
- 8584 Views
- 8 replies
- 3 kudos
- 3 kudos
@stefanberreiter were you able to access Fabric onelake data from databricks unity catalog using Service prinicipal?
- 3 kudos
- 1317 Views
- 3 replies
- 4 kudos
Materialized view resources created after creation query failed
Hello, Im testing with Materialized Views and I ran the following query:create materialized view `xx`.`xx`.`stg_sales` PARTITIONED BY (id, external_id, tenant_id) TBLPROPERTIES ('cluster_by' = 'id') as with stg_accounts_events as ( select * from ...
- 1317 Views
- 3 replies
- 4 kudos
- 4 kudos
You are always welcome, Vitor. Do let me know for any queries. Good day.
- 4 kudos
- 911 Views
- 1 replies
- 0 kudos
How to Configure a Policy in Job Cluster Using Databricks Linked Service in ADF?
I am trying to configure a policy in a job cluster using Databricks Linked Service in Azure Data Factory (ADF). The goal is to restrict access to a specific group or individuals who can use this cluster.I want to know:How to define and enforce a clus...
- 911 Views
- 1 replies
- 0 kudos
- 0 kudos
In the Linked Services panel in the Azure Data Factory UI, find your Databricks Linked Service and click the squiggly braces {} to edit the service. From here, you'll see a definition in JSON. Create a new key: properties.typeProperties.policyId and ...
- 0 kudos
- 15271 Views
- 5 replies
- 5 kudos
Resolved! JDBC Driver support for OpenJDK 17
Connecting to Databricks using OpenJDK 17 I got the exception below. Are there any plans to fix the driver for OpenJDK17?java.sql.SQLException: [Databricks][DatabricksJDBCDriver](500540) Error caught in BackgroundFetcher. Foreground thread ID: 44. Ba...
- 15271 Views
- 5 replies
- 5 kudos
- 5 kudos
I still see the above error with databricks jdbc driver 2.6.33. Anyone aware of fix available either in driver or java?
- 5 kudos
- 654 Views
- 1 replies
- 0 kudos
Resolved! My databricks Exam still suspended request: 00635015
My exam has been suspended, and I received an email to schedule it. But I see that there is a delay of 10 to 15 days. Is it possible to schedule it for 19 March at 11 a.m.?because I have a deadline that I must not exceed to obtain this certificate.o...
- 654 Views
- 1 replies
- 0 kudos
- 0 kudos
Hello @saraM! Thank you for reaching out to us again and confirming.As you requested, the support team has rescheduled your exam for March 19, 2025, at 11:00 AM in Europe/Paris.Please log into your account and confirm. The rescheduled exam will be re...
- 0 kudos
- 1113 Views
- 2 replies
- 1 kudos
ERROR: Could not find a version that satisfies the requirement
I'm using Databricks workflows along with asset bundle to run my pipeline and recently we moved our pipeline from development to acceptance. We are also using JFrog as our package artifactory.Even though I do see the release candidate version under t...
- 1113 Views
- 2 replies
- 1 kudos
- 1 kudos
Thank you for the response. Apparently one of our internal library failed to install, since an init script was missing on the cluster. After adding that, this got resolved
- 1 kudos
- 1241 Views
- 1 replies
- 0 kudos
Lakehouse monitoring metrices tables not created automatically
Hello,I have an external table created in databricks unity catalog workspace and trying to "Create a monitor" for the same from quality tab.While creating the same the dashboard is getting created however the two metrices tables "profile" & "drift" a...
- 1241 Views
- 1 replies
- 0 kudos
- 0 kudos
Hi Nikhil,How are you doing today?, As per my understanding, It looks like the monitoring setup isn't fully completing, likely due to permissions, external table limitations, or system settings. First, check if your Unity Catalog schema has the right...
- 0 kudos
- 504 Views
- 1 replies
- 0 kudos
Liquid Clustering with Derived / Generated Fields
I'm having issues with liquid clustering not working when creating derived/generated fields in an asset bundle. Are there any known limitations here?
- 504 Views
- 1 replies
- 0 kudos
- 0 kudos
Hi Lottiejanedev,How are you doing today?, As per my understanding, Yes, Liquid Clustering works best with directly stored columns, and it might not behave as expected when clustering on derived or generated fields. One possible issue is that generat...
- 0 kudos
- 2203 Views
- 2 replies
- 2 kudos
Capturing notebook return codes in databricks jobs
Hi, I currently am running a number of notebook jobs from Azure Data Factory. A new requirement has come up where I need to capture a return code in ADF that has been generated from the note. I tried using dbutils.notebook.exit(json.dumps({"return_v...
- 2203 Views
- 2 replies
- 2 kudos
- 2 kudos
I tried dbutils.notebook.exit(<value>) where value is 0 and 1. I also tried sys.exit(<value>). None of this produces a difference in the output of the databricks CLI commanddatabricks jobs run-now --json '{"job_id": <JOB_ID>, "job_parameters": {"will...
- 2 kudos
- 5682 Views
- 4 replies
- 1 kudos
Databricks CLI how to start a job and pass the parameters?
I try to start a job ID 85218616788189 and pass one parameters 'demo' in Windows Shell.This works:databricks jobs run-now 85218616788189 If I try this one,databricks jobs run-now --json '{"job_id":85218616788189,"notebook_params": {"demo":"parameter...
- 5682 Views
- 4 replies
- 1 kudos
- 1 kudos
If the job itself has parameters you have to use 'job_parameters' instead. No need for escaping as of version 0.243.0databricks jobs run-now --json '{"job_id": <JOB_ID>, "job_parameters": {"will_exit": "yes"}}'
- 1 kudos
- 429 Views
- 1 replies
- 0 kudos
My databricks exam got suspened on 18th march and it is still in the suspened state.
My databricks exam got suspened on 18th march and it is still in the suspened state. I have raised a support request using the below link.https://help.databricks.com/s/contact-us?ReqType=trainingand my request id : #00635015 I would really like to ge...
- 429 Views
- 1 replies
- 0 kudos
- 0 kudos
Hello @saraM! It looks like this post duplicates the one you recently posted. The Original post has already been answered. I recommend continuing the discussion there to keep the conversation focused and organised.
- 0 kudos
| User | Count |
|---|---|
| 1619 | |
| 790 | |
| 493 | |
| 349 | |
| 287 |
