cancel
Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
Showing results for 
Search instead for 
Did you mean: 
Help
Data Engineering
Join discussions on data engineering best practices, architectures, and optimization strategies within the Databricks Community. Exchange insights and solutions with fellow data engineers.
cancel
Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
Showing results for 
Search instead for 
Did you mean: 

Forum Posts

Start a conversation
JonD
by • New Contributor III
  • 3262 Views
  • 3 replies
  • 0 kudos

Resolved! Why does my Single Node cluster automatically resize num_workers?

Hi community,We have setup a Databricks cluster as Single node with num_workers=0 . Sometimes the cluster automatically resizes to e.g. 10 workers. When I edit the cluster subsequently it gives an error that num_workers is not allowed for Single node...

Data Engineering
Reply
  • 3262 Views
  • 3 replies
  • 0 kudos
View Replies
Latest Reply
JonD
New Contributor III
  • 0 kudos

I think the issue is solved, at least it didn't occur in the last month. We monitored this via Azure Log Analytics. Maybe it was solved due to some patch/update, thanks anyway!

  • 0 kudos
2 More Replies
nemesis
by • New Contributor
  • 1366 Views
  • 2 replies
  • 0 kudos

Same Public ip of two machines in workspace.

Is there any way to change public ip of driver node? when I try to launch two different instance (single node), I see those two machine have the same public ip.I'm writing this command into notebook of each instance. I got the same ip.from requests i...

Data Engineering
Reply
  • 1366 Views
  • 2 replies
  • 0 kudos
View Replies
Latest Reply
Vidula
Honored Contributor
  • 0 kudos

Hi @Hakan Tekbas​ Hope all is well! Just wanted to check in if you were able to resolve your issue and would you be happy to share the solution or mark an answer as best? Else please let us know if you need more help. We'd love to hear from you.Thank...

  • 0 kudos
1 More Replies
Dileep_Vidyadar
by • New Contributor III
  • 4662 Views
  • 6 replies
  • 4 kudos

Not Able to create Cluster on Community Edition for 3-4 days.

I am learning Pyspark on Community edition for a like month. It's been great until I am facing issues while creating a cluster for 3-4 Days.Sometimes it is taking 30 minutes to 60 minutes to create a Cluster and sometimes it is not even creating a Cl...

Data Engineering
Reply
  • 4662 Views
  • 6 replies
  • 4 kudos
View Replies
Latest Reply
Anonymous
Not applicable
  • 4 kudos

@Dileep Vidyadara​  - If your question was fully answered by @Hubert Dudek​, would you be happy to mark his answer as best?

  • 4 kudos
5 More Replies
User16783855534
by • New Contributor III
  • 1229 Views
  • 1 replies
  • 1 kudos

Can I have a Databricks Cluster that is only 1 node?

Yes you can create a "Single Node" Cluster, https://docs.databricks.com/clusters/single-node.html . It is currently not recommended to use "Single Node" cluster for streaming workloads

Data Engineering
Reply
  • 1229 Views
  • 1 replies
  • 1 kudos
View Replies
Latest Reply
brickster_2018
Databricks Employee
  • 1 kudos

Single Node clusters should not be used for production workloads involving streaming queries, or complex computations. The intention here is to bring up the Spark cluster for all kinds of workloads

  • 1 kudos
User16826992666
by • Valued Contributor
  • 6571 Views
  • 1 replies
  • 0 kudos

Resolved! When should I use single node clusters vs standard?

I see that single node is a cluster mode option that I have when creating clusters. When should I use this compared to the standard mode?

Data Engineering
Reply
  • 6571 Views
  • 1 replies
  • 0 kudos
View Replies
Latest Reply
sean_owen
Databricks Employee
  • 0 kudos

Single-node, like the name implies, is a single machine. It still has Spark, just a local cluster. This is a good choice if you are running a workload that does not use Spark, or only needs it for data access. One good example is a small deep learnin...

  • 0 kudos
Labels