- 3542 Views
- 5 replies
- 11 kudos
- 3542 Views
- 5 replies
- 11 kudos
- 11 kudos
Thanks @Kaniz Fatma​ for selecting this as the best answer, Keep adding questions by that we can put our views and people get guidance And the databricks community can grow more.
- 11 kudos
- 15004 Views
- 6 replies
- 15 kudos
Job cluster vs All purpose cluster
Environment: AzureI've a workflow that takes approximately a minute to execute and I want to run the job every 2 minutes.. All purpose cluster:On attaching all purpose cluster to the job, it takes approx. 60 seconds to execute.Using job cluster:On at...
- 15004 Views
- 6 replies
- 15 kudos
- 3611 Views
- 4 replies
- 6 kudos
Resolved! Error When Starting the Cluster
I am having this error when running my cluster, any idea why?
- 3611 Views
- 4 replies
- 6 kudos
- 6 kudos
@Werner Stinckens​ , I checked again, you cannot change them after your workspace is deployed. The only way right now is to recreate the workspace and migrate. It’s not possible to update CIDR range right now without migration.
- 6 kudos
- 1380 Views
- 2 replies
- 0 kudos
Not able to merge Databricks cluster delta tables(location Abfs) with PowerBi for analytics Purpose
Hi everyone .. i have an issue while connecting azure Databricks cluster delta tables(location Abfs) with PowerBi for analytics Purpose. but after succesfully linking cluster using Server Hostname and HTTP Path with powerbi is showing an error:-Data...
- 1380 Views
- 2 replies
- 0 kudos
- 11905 Views
- 2 replies
- 1 kudos
Is it possible to have Cluster with pre-installed dependencies?
I run some jobs in the Databricks environment where some resources need authentication. I do this (and I need to) through the vault-cli in the init-script.However, every time in the init-script I need to install vault-cli and other libraries. Is ther...
- 11905 Views
- 2 replies
- 1 kudos
- 1 kudos
@João Victor Albuquerque​ :Yes, there are a few ways to pre-install libraries and tools in the Databricks environment:Cluster-scoped init scripts: You can specify a shell script to be run when a cluster is created or restarted. This script can includ...
- 1 kudos
- 5999 Views
- 2 replies
- 0 kudos
Azure Databricks cluster driver config
Hi there,I am trying to set up databricks storage account access in Global init script. according to Azure Databricks document on creating cluster with driver config for all clusters (https://learn.microsoft.com/en-us/azure/databricks/archive/compute...
- 5999 Views
- 2 replies
- 0 kudos
- 0 kudos
Hi @Shelly Xiao​, Thank you for reaching out regarding your issue with setting up a Databricks storage account access in the Global init script. It seems that the error is related to the format of your configuration file and the way you reference the...
- 0 kudos
- 1059 Views
- 2 replies
- 2 kudos
workflow cluster was create error
I set the workflow to run at 12:00 every day in the workflow, but the workflow failed with the error message below, and I don't know why.Run result unavailable: run failed with error message Unexpected failure while waiting for the cluster (0506-0233...
- 1059 Views
- 2 replies
- 2 kudos
- 2 kudos
Hello @Sangwoo Lee​ ,As mentioned by vignesh, it seems like an infra related issue. > Does the user (which executes the job) has access to start a cluster?> Incase if it is not an access issue and Incase if you are starting a lot of workflow jobs tog...
- 2 kudos
- 5095 Views
- 9 replies
- 1 kudos
Unable to launch notebook
I have created a 12.2 LTS cluster and trying to launch notebook attached to this cluster. But unable to launch, it is not giving any error instead it is still showing the same home page.
- 5095 Views
- 9 replies
- 1 kudos
- 1 kudos
Hi @Muskan Bansal​ Thank you for posting your question in our community! We are happy to assist you.To help us provide you with the most accurate information, could you please take a moment to review the responses and select the one that best answers...
- 1 kudos
- 3723 Views
- 3 replies
- 1 kudos
databricks cluster creation error
Hi Databricks Community,My Databricks workspace created on Azure pay-as-go subscription.I am facing two sided challenges.First>> I am not able to create Delta-Live-table pipeline or any other all-purpose multi-node cluster as it is throwing below e...
- 3723 Views
- 3 replies
- 1 kudos
- 1 kudos
HI All, Thanks for your reply. Just to update you, I am now able to to create DLT pipeline as well as all purpose multi-node cluster with minimum resources. This is due to Quota limit and I was able to increase it. But observation is, if I try to us...
- 1 kudos
- 2292 Views
- 4 replies
- 1 kudos
Create a Dockerfile from Cluster
Is there a way to create a Dockerfile from Workspace A's cluster configurations and deploy that on a different different cluster in Workspace B?
- 2292 Views
- 4 replies
- 1 kudos
- 1 kudos
Hi @Kevin Kim​ Hope everything is going great.Just wanted to check in if you were able to resolve your issue. If yes, would you be happy to mark an answer as best so that other members can find the solution more quickly? If not, please tell us so we ...
- 1 kudos
- 2705 Views
- 2 replies
- 0 kudos
pip install in cluster using web UI and extra index
In an init script or a notebook, we can:pip install --index-url=<our private pypi url> --extra-index-url=https://pypi.org/simple <a module>In the cluster web UI (libraries -> install library), we can give only the url of our private repository, but n...
- 2705 Views
- 2 replies
- 0 kudos
- 0 kudos
Hi @Philippe CRAVE​ Thank you for posting your question in our community! We are happy to assist you.To help us provide you with the most accurate information, could you please take a moment to review the responses and select the one that best answer...
- 0 kudos
- 6168 Views
- 8 replies
- 0 kudos
Not able to connect to On-Prem Oracle from Databricks cluster
Hi Everyone,I was trying to connect to Oracle Instance from Databricks cluster and it is giving below error:java.sql.SQLTimeoutException: ORA-12170: Cannot connect. TCP connect timeout of 30000ms for host xx.x.x.*** port 1521. (CONNECTION_ID=CgM7V7UB...
- 6168 Views
- 8 replies
- 0 kudos
- 0 kudos
@Satya89:The error message you received indicates that the TCP connection to the Oracle database timed out. This could be caused by a number of factors such as network issues, firewall restrictions, or the database being overloaded.Here are a few ste...
- 0 kudos
- 1338 Views
- 2 replies
- 4 kudos
Specifying cluster on running a job
Hi,Let's say that I am starting jobs with different parameters at a certain time each day in the following manner:response = requests.post( "https://%s/api/2.0/jobs/run-now" % (DOMAIN), headers={"Authorization": "Bearer %s" % TOKEN}, json={ ...
- 1338 Views
- 2 replies
- 4 kudos
- 4 kudos
@Tjadi Peeters​ You can select option Autoscaling/Enhanced Scaling in workflows which will scale based on workload
- 4 kudos
- 2839 Views
- 3 replies
- 0 kudos
Resolved! Why does my Single Node cluster automatically resize num_workers?
Hi community,We have setup a Databricks cluster as Single node with num_workers=0 . Sometimes the cluster automatically resizes to e.g. 10 workers. When I edit the cluster subsequently it gives an error that num_workers is not allowed for Single node...
- 2839 Views
- 3 replies
- 0 kudos
- 0 kudos
I think the issue is solved, at least it didn't occur in the last month. We monitored this via Azure Log Analytics. Maybe it was solved due to some patch/update, thanks anyway!
- 0 kudos
- 2498 Views
- 4 replies
- 1 kudos
Resolved! How can I cluster-install a c-Python library (pyRFC)?
If possible, how can one go about installing a Python library with SDK dependencies like pyRFC? (https://github.com/SAP/PyRFC)The SDK dependencies depend on the type of OS, and since we're running Databricks out of AWS, I assume one would have to mat...
- 2498 Views
- 4 replies
- 1 kudos
- 1 kudos
Hi @Wonseok Choi​ Thank you for your question! To assist you better, please take a moment to review the answer and let me know if it best fits your needs.Please help us select the best solution by clicking on "Select As Best" if it does.Your feedback...
- 1 kudos