cancel
Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
Showing results for 
Search instead for 
Did you mean: 
Help
Data Engineering
Join discussions on data engineering best practices, architectures, and optimization strategies within the Databricks Community. Exchange insights and solutions with fellow data engineers.
cancel
Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
Showing results for 
Search instead for 
Did you mean: 

Forum Posts

Start a conversation
User16790091296
by • Contributor II
  • 2627 Views
  • 1 replies
  • 0 kudos

How to create a databricks job with parameters via CLI?

I'm creating a new job in databricks using the databricks-cli:databricks jobs create --json-file ./deploy/databricks/config/job.config.jsonWith the following json:{ "name": "Job Name", "new_cluster": { "spark_version": "4.1.x-scala2.1...

Data Engineering
Reply
  • 2627 Views
  • 1 replies
  • 0 kudos
View Replies
Latest Reply
matthew_m
New Contributor III
  • 0 kudos

This is an old post but still relevant for future readers, so will answer how it is done. You need to add base_parameters flag in the notebook_task config, like the following.   "notebook_task": { "notebook_path": "...", "base_parameters": { ...

  • 0 kudos
shan_chandra
by • Esteemed Contributor
  • 2152 Views
  • 1 replies
  • 1 kudos

Resolved! Adding spark_conf tag on Jobs API

using Jobs API, when we create a new job to run on an interactive cluster, can we add spark_conf tag and specify spark config tuning parameters?

Data Engineering
Reply
  • 2152 Views
  • 1 replies
  • 1 kudos
View Replies
Latest Reply
shan_chandra
Esteemed Contributor
  • 1 kudos

 spark_conf needs to be set prior to the start of the cluster or have to restart the existing cluster. Hence, the spark_conf tag is available only on the job_cluster. you may have to set the configs manually on the interactive cluster prior to using ...

  • 1 kudos
F29
by • New Contributor
  • 1942 Views
  • 3 replies
  • 0 kudos

is possible duplicated a new job in another stage Databricks? Devops or some way?

I need duplicated a new job create in stage A in another stage, automatically. is posible?

Data Engineering
Reply
  • 1942 Views
  • 3 replies
  • 0 kudos
View Replies
Latest Reply
Atanu
Esteemed Contributor
  • 0 kudos

you may try to get the job details from our job api https://docs.databricks.com/dev-tools/api/latest/jobs.html#operation/JobsGet and get the response to duplicate it.,

  • 0 kudos
2 More Replies
Robbie
by • New Contributor III
  • 2599 Views
  • 4 replies
  • 5 kudos

Resolved! Why can't I create new jobs? ("You are not entitled to run this type of task...")

This morning I encountered an issue when trying to create a new job using the Workflows UI (in browser). Never had this issue before.The error message that appears is:"You are not entitled to run this type of task, please contact your Databricks admi...

Screenshot including the error message
Data Engineering
Reply
  • 2599 Views
  • 4 replies
  • 5 kudos
View Replies
Latest Reply
Kaniz_Fatma
Community Manager
  • 5 kudos

Hi @Robbie Capps​, I'm glad we could help you. Thank you for marking the best answer for us.

  • 5 kudos
3 More Replies
Labels