cancel
Showing results for 
Search instead for 
Did you mean: 
Data Engineering
cancel
Showing results for 
Search instead for 
Did you mean: 

Forum Posts

AvijitDey
by New Contributor III
  • 2630 Views
  • 3 replies
  • 4 kudos

Resolved! Azure Databrick SQL bulk insert to AZ SQL

Env: Azure Databrick :version : 9.1 LTS (includes Apache Spark 3.1.2, Scala 2.12)Work Type : 56 GB Memory 2-8 node ( standard D13_V2)No of rows : 2470350 and 115 Column Size : 2.2 GBTime taken approx. 9 min Python Code .What will be best approach for...

  • 2630 Views
  • 3 replies
  • 4 kudos
Latest Reply
AvijitDey
New Contributor III
  • 4 kudos

Any further suggestion

  • 4 kudos
2 More Replies
User16776431030
by New Contributor III
  • 1057 Views
  • 1 replies
  • 0 kudos

Resolved! How can I make a cluster start up in the availability-zone (AZ) with the most available IPs?

I see the default in the UI is to always create clusters in a single AZ (e.g. us-west-2a), but want to distribute workloads across all available AZs.

  • 1057 Views
  • 1 replies
  • 0 kudos
Latest Reply
User16776431030
New Contributor III
  • 0 kudos

Found the answer - not available in the UI, but via API, you can submit the cluster definition with "aws_attributes": { "zone_id": "auto" },This is documented in the Cluster API: https://docs.databricks.com/dev-tools/api/latest/clusters.html#aw...

  • 0 kudos
Labels