cancel
Showing results for 
Search instead for 
Did you mean: 
Data Engineering
Join discussions on data engineering best practices, architectures, and optimization strategies within the Databricks Community. Exchange insights and solutions with fellow data engineers.
cancel
Showing results for 
Search instead for 
Did you mean: 

Forum Posts

weldermartins
by Honored Contributor
  • 2916 Views
  • 2 replies
  • 1 kudos

Resolved! How to make spark-submit work on windows?

I have Jupyter Notebook installed on my machine working normally. I tested running a Spark application by running the spark-submit command and it returns the message that the file was not found. What do you need to do to make it work?Below is a file ...

image
  • 2916 Views
  • 2 replies
  • 1 kudos
Latest Reply
Debayan
Esteemed Contributor III
  • 1 kudos

Hi, yet this is not tested in my lab, but could you please check and confirm if this works: https://stackoverflow.com/questions/37861469/how-to-submit-spark-application-on-cmd

  • 1 kudos
1 More Replies
Andriy_Shevchen
by New Contributor
  • 1783 Views
  • 4 replies
  • 3 kudos

Resolved! yarn.nodemanager.resource.memory-mb parameter update

I am currently working on determining proper cluster size for my Spark application and I have a question regarding Hadoop configuration parameter yarn.nodemanager.resource.memory-mb. From what I see, this parameter is responsible for setting the phys...

  • 1783 Views
  • 4 replies
  • 3 kudos
Latest Reply
Kaniz
Community Manager
  • 3 kudos

Hi @Andriy Shevchenko​, Just a friendly follow-up. Do you still need help, or does the above response help you to find the solution? Please let us know.

  • 3 kudos
3 More Replies
Ashish
by New Contributor II
  • 4131 Views
  • 5 replies
  • 3 kudos

Resolved! Cost of individual jobs running on a shared Databricks cluster

Hi All,I am working on a requirement where I need to calculate the cost of each spark job individually on a shared Azure/AWS Databricks cluster. There can be multiple jobs running on the cluster parallelly.Cost needs to be calculated after job comple...

  • 4131 Views
  • 5 replies
  • 3 kudos
Latest Reply
Kaniz
Community Manager
  • 3 kudos

Hi @Ashish Kardam​  (Customer)​ , Does @[werners] (Customer)​ 's or @Alex Ott​ 's replies answer your question?

  • 3 kudos
4 More Replies
Labels