cancel
Showing results for 
Search instead for 
Did you mean: 
Data Engineering
Join discussions on data engineering best practices, architectures, and optimization strategies within the Databricks Community. Exchange insights and solutions with fellow data engineers.
cancel
Showing results for 
Search instead for 
Did you mean: 

Forum Posts

Praveen
by New Contributor II
  • 6751 Views
  • 10 replies
  • 1 kudos

Resolved! Pass Typesafe config file to the Spark Submit Job

Hello everyone ! I am trying to pass a Typesafe config file to the spark submit task and print the details in the config file. Code: import org.slf4j.{Logger, LoggerFactory} import com.typesafe.config.{Config, ConfigFactory} import org.apache.spa...

  • 6751 Views
  • 10 replies
  • 1 kudos
Latest Reply
source2sea
Contributor
  • 1 kudos

I've experenced similar issues; please help to answer how to get this working;I've tried using below to be either /dbfs/mnt/blah path or dbfs:/mnt/blah pathin either spark_submit_task or spark_jar_task (via cluster spark_conf for java optinos); no su...

  • 1 kudos
9 More Replies
User16783855534
by New Contributor III
  • 6081 Views
  • 6 replies
  • 5 kudos
  • 6081 Views
  • 6 replies
  • 5 kudos
Latest Reply
Kaniz
Community Manager
  • 5 kudos

Hi @Neil Patel​ ​ , Just a friendly follow-up. Do you still need help, or do the above responses help you find the solution? Please let us know.

  • 5 kudos
5 More Replies
pjp94
by Contributor
  • 1326 Views
  • 4 replies
  • 9 kudos

Databrick Job - Notebook Execution

Question - When you set a reoccuring job to simply update a notebook, does databricks clear the state of the notebook prior to executing the notebook? If not, can I configure it to make sure it clears the state before running?

  • 1326 Views
  • 4 replies
  • 9 kudos
Latest Reply
Anonymous
Not applicable
  • 9 kudos

@Paras Patel​ - Would you be happy to mark Hubert's answer as best so that other members can find the solution more easily?Thanks!

  • 9 kudos
3 More Replies
Anonymous
by Not applicable
  • 1294 Views
  • 2 replies
  • 4 kudos

Multi-task Job Run starting point

Hi community!I would like to know if it is possible to start a Multi-task Job Run from and specific task. The use case is as follows:I have a 17 tasks JobA task in the middle, let's say a task after 2 dependencies, failsI found the error and now it i...

  • 1294 Views
  • 2 replies
  • 4 kudos
Latest Reply
BilalAslamDbrx
Honored Contributor III
  • 4 kudos

+1 to what @Dan Zafar​  said. We're working **** ** this. Looking forward to bring this to you in the near future.

  • 4 kudos
1 More Replies
Labels