cancel
Showing results for 
Search instead for 
Did you mean: 
Data Engineering
Join discussions on data engineering best practices, architectures, and optimization strategies within the Databricks Community. Exchange insights and solutions with fellow data engineers.
cancel
Showing results for 
Search instead for 
Did you mean: 

Forum Posts

Aviral-Bhardwaj
by Esteemed Contributor III
  • 1128 Views
  • 0 replies
  • 31 kudos

Understanding Cluster Pools Sometimes we want to run our databricks code without any delay as reports are very emergency like the upstream team wants ...

Understanding Cluster PoolsSometimes we want to run our databricks code without any delay as reports are very emergency like the upstream team wants to save as much time as they can save in the starting cluster.That time we can use the pool of cluste...

  • 1128 Views
  • 0 replies
  • 31 kudos
PaulP
by New Contributor II
  • 2145 Views
  • 4 replies
  • 6 kudos

What is the best expected starting time for a cluster when using a pool?

Hi! I'm doing some tests to get an idea of how much time could be saved starting a cluster by using a pool and was wondering if the results I get are what should be expected.We're using AWS Databricks and used i3.xlarge as instance type (if that matt...

  • 2145 Views
  • 4 replies
  • 6 kudos
Latest Reply
Anonymous
Not applicable
  • 6 kudos

Hi @Paul Pelletier​ Hope all is well! Just wanted to check in if you were able to resolve your issue and would you be happy to share the solution or mark an answer as best? Else please let us know if you need more help. We'd love to hear from you.Tha...

  • 6 kudos
3 More Replies
Stephanraj
by New Contributor III
  • 5106 Views
  • 7 replies
  • 7 kudos

Resolved! Spark eventlog for Cluster pools

Hi,I would want to setup the cluster logging (to capture eventlogs to /dbfs/cluster-logs dir) in my cluster pool configuration? is it possible?If I create cluster manually, I could able to setup the cluster logging as mentioned here: https://docs.mic...

  • 5106 Views
  • 7 replies
  • 7 kudos
Latest Reply
Prabakar
Esteemed Contributor III
  • 7 kudos

Hi @Stephanraj C​ instance pool is to reduce cluster start and auto-scaling times for a cluster. Are you using any API to create clusters? If so could you please share the API request?

  • 7 kudos
6 More Replies
Labels