cancel
Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
Showing results for 
Search instead for 
Did you mean: 
Help
Data Engineering
Join discussions on data engineering best practices, architectures, and optimization strategies within the Databricks Community. Exchange insights and solutions with fellow data engineers.
cancel
Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
Showing results for 
Search instead for 
Did you mean: 

Forum Posts

Start a conversation
sunil_smile
by • Contributor
  • 7977 Views
  • 8 replies
  • 10 kudos

Resolved! How i can add ADLS Gen2 - OAuth 2.0 as Cluster scope for my High concurrency Shared Cluster (without unity catalog)?

Hi All,Kindly help me , how i can add the ADLS gen2 OAuth 2.0 authentication to my high concurrency shared cluster. I want to scope this authentication to entire cluster not for particular notebook.Currently i have added them as spark configuration o...

image.png image
Data Engineering
Reply
  • 7977 Views
  • 8 replies
  • 10 kudos
View Replies
Latest Reply
Hubert-Dudek
Esteemed Contributor III
  • 10 kudos

error is because of missing default settings (create new cluster and do not remove them),the warning is because secrets should be put in secret scope, and then you should reference secrets in settings

  • 10 kudos
7 More Replies
Pat
by • Honored Contributor III
  • 10075 Views
  • 7 replies
  • 18 kudos

Resolved! Cluster Modes - High Concurrency

It took me quite some time to find the option to create a cluster in High Concurrency mode. It was hidden in the new UI.What should be the way to access the data with TAC?What is the equivalent mode to work with TAC ?Does it mean that we are being pu...

image image.png image
Data Engineering
Reply
  • 10075 Views
  • 7 replies
  • 18 kudos
View Replies
Latest Reply
Prabakar
Databricks Employee
  • 18 kudos

Thanks. Always happy to help.

  • 18 kudos
6 More Replies
Anonymous
by • Not applicable
  • 829 Views
  • 1 replies
  • 2 kudos

Why doesn't Databricks support Scala on high concurrency cluster? Is it on the roadmap?

Data Engineering
Reply
  • 829 Views
  • 1 replies
  • 2 kudos
View Replies
Latest Reply
User16826994223
Honored Contributor III
  • 2 kudos

Scala Use JVM to run its code, Scala cannot run different applications at a time with complete isolation of each task inside single jvm , that is the reason Scala doesn't support high concurrency cluster, I don't think it is on road map

  • 2 kudos
Anonymous
by • Not applicable
  • 10960 Views
  • 1 replies
  • 0 kudos

Resolved! When to use High Concurrency clusters? What are the benefits?

Data Engineering
Reply
  • 10960 Views
  • 1 replies
  • 0 kudos
View Replies
Latest Reply
Ryan_Chynoweth
Esteemed Contributor
  • 0 kudos

The key benefits of High Concurrency clusters are that they provide fine-grained sharing for maximum resource utilization and minimum query latencies.Note that a Standard cluster is recommended for a single user. Standard clusters can run workloads d...

  • 0 kudos
User16826992666
by • Valued Contributor
  • 2147 Views
  • 2 replies
  • 0 kudos

Resolved! When should I set the cluster mode to High Concurrency vs Standard?

How do I know which mode I should be using when creating a cluster?

Data Engineering
Reply
  • 2147 Views
  • 2 replies
  • 0 kudos
View Replies
Latest Reply
sajith_appukutt
Honored Contributor II
  • 0 kudos

High Concurrency clusters are ideal for groups of users who need to share resources or run ad-hoc jobs - for example data scientists sharing a cluster. They come with Query Watchdog, a process which keeps disruptive queries in check by automatically ...

  • 0 kudos
1 More Replies
Labels