cancel
Showing results for 
Search instead for 
Did you mean: 
Data Engineering
Join discussions on data engineering best practices, architectures, and optimization strategies within the Databricks Community. Exchange insights and solutions with fellow data engineers.
cancel
Showing results for 
Search instead for 
Did you mean: 

Forum Posts

JonD
by New Contributor III
  • 1916 Views
  • 3 replies
  • 0 kudos

Resolved! Why does my Single Node cluster automatically resize num_workers?

Hi community,We have setup a Databricks cluster as Single node with num_workers=0 . Sometimes the cluster automatically resizes to e.g. 10 workers. When I edit the cluster subsequently it gives an error that num_workers is not allowed for Single node...

  • 1916 Views
  • 3 replies
  • 0 kudos
Latest Reply
JonD
New Contributor III
  • 0 kudos

I think the issue is solved, at least it didn't occur in the last month. We monitored this via Azure Log Analytics. Maybe it was solved due to some patch/update, thanks anyway!

  • 0 kudos
2 More Replies
nemesis
by New Contributor
  • 933 Views
  • 2 replies
  • 0 kudos

Same Public ip of two machines in workspace.

Is there any way to change public ip of driver node? when I try to launch two different instance (single node), I see those two machine have the same public ip.I'm writing this command into notebook of each instance. I got the same ip.from requests i...

  • 933 Views
  • 2 replies
  • 0 kudos
Latest Reply
Vidula
Honored Contributor
  • 0 kudos

Hi @Hakan Tekbas​ Hope all is well! Just wanted to check in if you were able to resolve your issue and would you be happy to share the solution or mark an answer as best? Else please let us know if you need more help. We'd love to hear from you.Thank...

  • 0 kudos
1 More Replies
Dileep_Vidyadar
by New Contributor III
  • 2356 Views
  • 7 replies
  • 5 kudos

Not Able to create Cluster on Community Edition for 3-4 days.

I am learning Pyspark on Community edition for a like month. It's been great until I am facing issues while creating a cluster for 3-4 Days.Sometimes it is taking 30 minutes to 60 minutes to create a Cluster and sometimes it is not even creating a Cl...

  • 2356 Views
  • 7 replies
  • 5 kudos
Latest Reply
Anonymous
Not applicable
  • 5 kudos

@Dileep Vidyadara​  - If your question was fully answered by @Hubert Dudek​, would you be happy to mark his answer as best?

  • 5 kudos
6 More Replies
User16783855534
by New Contributor III
  • 778 Views
  • 1 replies
  • 1 kudos

Can I have a Databricks Cluster that is only 1 node?

Yes you can create a "Single Node" Cluster, https://docs.databricks.com/clusters/single-node.html . It is currently not recommended to use "Single Node" cluster for streaming workloads

  • 778 Views
  • 1 replies
  • 1 kudos
Latest Reply
User16869510359
Esteemed Contributor
  • 1 kudos

Single Node clusters should not be used for production workloads involving streaming queries, or complex computations. The intention here is to bring up the Spark cluster for all kinds of workloads

  • 1 kudos
User16826992666
by Valued Contributor
  • 3898 Views
  • 1 replies
  • 0 kudos

Resolved! When should I use single node clusters vs standard?

I see that single node is a cluster mode option that I have when creating clusters. When should I use this compared to the standard mode?

  • 3898 Views
  • 1 replies
  • 0 kudos
Latest Reply
sean_owen
Honored Contributor II
  • 0 kudos

Single-node, like the name implies, is a single machine. It still has Spark, just a local cluster. This is a good choice if you are running a workload that does not use Spark, or only needs it for data access. One good example is a small deep learnin...

  • 0 kudos
Labels