- 928 Views
- 1 replies
- 0 kudos
multitask in Databricks
Hi Team is there any way we can utilize same cluster to run multiple dependent jobs in multi-task, starting cluster for every jobs take time
- 928 Views
- 1 replies
- 0 kudos
- 0 kudos
At this time it is not possible
- 0 kudos
- 6202 Views
- 1 replies
- 0 kudos
Resolved! When should I choose a different driver type on my cluster vs the worker type?
When creating a cluster the driver type defaults to choose the same type as the workers, and this is what I usually choose. But in what of situation would I want to choose a different driver type?
- 6202 Views
- 1 replies
- 0 kudos
- 0 kudos
Using the same instance type is a fine default. If you know that you need very large workers, but little happens on the driver, maybe you can save money with a smaller driver. Conversely, you may know that some parts of your notebook involve a lot of...
- 0 kudos
- 1852 Views
- 1 replies
- 0 kudos
Resolved! How does cluster autoscaling work?
What determines when the cluster autoscaling activates to add and remove workers? Also, can it be adjusted?
- 1852 Views
- 1 replies
- 0 kudos
- 0 kudos
> What determines when the cluster autoscaling activates to add and remove workersDuring scale-down, the service removes a worker only if it is idle and does not contain any shuffle data. This allows aggressive resizing without killing tasks or recom...
- 0 kudos
- 5231 Views
- 1 replies
- 0 kudos
Resolved! Scheduling cluster start and stop time
I want to schedule cluster to start in the morning and shut down by evening. How can I achieve that?
- 5231 Views
- 1 replies
- 0 kudos
- 0 kudos
You can call the REST API to schedule cluster starts and stops from a scheduler.See https://docs.databricks.com/dev-tools/api/latest/clusters.htmlPRO Tip: Use code generation tools within Postman to generate scripts in the language of your choice.
- 0 kudos
- 8759 Views
- 2 replies
- 1 kudos
- 8759 Views
- 2 replies
- 1 kudos
- 1 kudos
Databricks pool concept may help reduce cluster start up time. more details are here
- 1 kudos
- 1041 Views
- 1 replies
- 0 kudos
Unable to start cluster Error :- Defunct Resource Detected
Hi AllI am getting this error for some jobs. Can you please let me know what could be the reasonRun result unavailable: job failed with an error message -Run result unavailable: job failed with error messageUnexpected failure while waiting for the cl...
- 1041 Views
- 1 replies
- 0 kudos
- 0 kudos
This is an issue on the cloud level so try to put retries in the job as it happens not for all cluster start , it may fails once but will start after retry,Also, raise a databricks ticket , they will provide permanent solution
- 0 kudos
- 873 Views
- 1 replies
- 0 kudos
How to solve Hive connectivity issues?
I can see connectivity issues in my driver logs. How to solve this issue?
- 873 Views
- 1 replies
- 0 kudos
- 0 kudos
Can you give us some more error please, I hope you will get more error in logs, whether it is a connection issue because of JDbc URL or host name or password,something like this
- 0 kudos
- 4598 Views
- 1 replies
- 0 kudos
How do I see the java version being used on the cluster?
Environment Tab in the Spark UI
- 4598 Views
- 1 replies
- 0 kudos
- 0 kudos
In spark UI - > Environment Tab
- 0 kudos
- 1572 Views
- 1 replies
- 0 kudos
- 1572 Views
- 1 replies
- 0 kudos
- 0 kudos
There are two types of auto scaling in Databricks: Standard and Optimized. In both scenarios when tasks are submitted the cluster will begin scaling to execute as many of them in parallel immediately.Scaling down is different. In optimized autoscalin...
- 0 kudos
- 1049 Views
- 2 replies
- 0 kudos
Cluster goes unresponsive after installing a library
Right after I install a library in my cluster, my cluster goes unresponsive and nothing runs. How to solve this issue?
- 1049 Views
- 2 replies
- 0 kudos
- 0 kudos
it is a standard cluster. It is happening for all libraries. is there a way to debug or show the errors messages if any?
- 0 kudos
- 1145 Views
- 1 replies
- 0 kudos
Databricks notebook command gets cancelled:Generally when cluster is having init scripts or lib issues while starting cluster. Exact error can be look...
Databricks notebook command gets cancelled:Generally when cluster is having init scripts or lib issues while starting cluster. Exact error can be looked into driver logs.
- 1145 Views
- 1 replies
- 0 kudos
- 0 kudos
Awsome Knowledge
- 0 kudos
- 2037 Views
- 2 replies
- 2 kudos
- 2037 Views
- 2 replies
- 2 kudos
- 2 kudos
You can tag your cluster and that tags get propagated to the billing management and there you can see it the cost
- 2 kudos
- 897 Views
- 1 replies
- 0 kudos
How do you deploy Container Services to a Cluster Pool?
To speed up cluster provisioning in the case of use Container Services, how do you pre-install it to a pool.
- 897 Views
- 1 replies
- 0 kudos
- 0 kudos
Using Pools APIs.
- 0 kudos
- 2132 Views
- 1 replies
- 0 kudos
- 2132 Views
- 1 replies
- 0 kudos
- 0 kudos
Databricks starts to charge for DBUs once the virtual machine is up and the Spark context is initialized, which may include a portion of start up costs, but not all. Init scripts are loaded before the Spark context is initialized, which therefore wou...
- 0 kudos
- 32760 Views
- 5 replies
- 3 kudos
What's the difference between Interactive Clusters and Job Cluster?
I am new to databricks. I would like to know what is the difference between Interactive Clusters and Job Cluster? There are no official document now.
- 32760 Views
- 5 replies
- 3 kudos
- 3 kudos
Sports news Football news International football news Football news Thai football news, Thai football Follow news, know sports news at Siamsportnews
- 3 kudos