cancel
Showing results for 
Search instead for 
Did you mean: 
Data Engineering
cancel
Showing results for 
Search instead for 
Did you mean: 

Forum Posts

Leodatabricks
by Contributor
  • 3180 Views
  • 14 replies
  • 23 kudos

How to secure all clusters and then start running the code

When there are slow nodes, sometimes a job needs to resize its number of clusters to reach the required number of nodes. Is there any way to make sure no code is running before all nodes are secured? Thank you!

  • 3180 Views
  • 14 replies
  • 23 kudos
Latest Reply
Anonymous
Not applicable
  • 23 kudos

Hi @Leo Bao​ Hope everything is going great.Just wanted to check in if you were able to resolve your issue. If yes, would you be happy to mark an answer as best so that other members can find the solution more quickly? If not, please tell us so we ca...

  • 23 kudos
13 More Replies
uzairm
by New Contributor III
  • 4882 Views
  • 2 replies
  • 2 kudos

Resolved! ThreadPoolExecutor in Databricks

I am using a threadpool executor and running notebooks in parallel. However, these parallel notebooks are not using executors at all and all the load is going towards the driver node resulting in running out of memory for the driver node and eventual...

  • 4882 Views
  • 2 replies
  • 2 kudos
Latest Reply
Anonymous
Not applicable
  • 2 kudos

Hi @uzair mustafa​ Thank you for your question! To assist you better, please take a moment to review the answer and let me know if it best fits your needs.Please help us select the best solution by clicking on "Select As Best" if it does.Your feedbac...

  • 2 kudos
1 More Replies
MoJaMa
by Valued Contributor II
  • 1734 Views
  • 3 replies
  • 1 kudos
  • 1734 Views
  • 3 replies
  • 1 kudos
Latest Reply
User16783853906
Contributor III
  • 1 kudos

Databricks does not charge DBUs while instances are idle in the pool. Instance provider billing does apply.Please refer here for more information - https://docs.databricks.com/clusters/instance-pools/index.html

  • 1 kudos
2 More Replies
Labels