cancel
Showing results for 
Search instead for 
Did you mean: 
Machine Learning
Dive into the world of machine learning on the Databricks platform. Explore discussions on algorithms, model training, deployment, and more. Connect with ML enthusiasts and experts.
cancel
Showing results for 
Search instead for 
Did you mean: 

Forum Posts

Maverick1
by Valued Contributor II
  • 4848 Views
  • 2 replies
  • 4 kudos

Dynamic variable and multi-instance tasks.

1. How to pass dynamic variable values like "sysdate" to a job parameters, so that it will automatically take the updated values on the fly.2. How to run multiple instance of set of tasksin a job (for different parameters). For e.g the same pipeline ...

  • 4848 Views
  • 2 replies
  • 4 kudos
Latest Reply
gyapar
New Contributor II
  • 4 kudos

Hey Maverick1,Did you find a solution for your second question?I have also same approach. In databricks, it has workflows, job clusters, tasks etc.I'm trying to do creating one job cluster with one configuration or specification which has a workflow ...

  • 4 kudos
1 More Replies
yo1
by New Contributor II
  • 7190 Views
  • 6 replies
  • 2 kudos

Run one workflow dynamically with different parameter and schedule time.

Can we run one workflow for different parameters and different schedule time. so that only one workflow can executed for different parameters we do not have to create that workflow again and again. or we can say Is there any possibility to drive work...

  • 7190 Views
  • 6 replies
  • 2 kudos
Latest Reply
DBXC
Contributor
  • 2 kudos

Update / Solved: Using CLI on Linux/MacOS: Send in the sample json with job_id in it. databricks jobs run-now --json '{   "job_id":<job-ID>,  "notebook_params": {    <key>:<value>,    <key>:<value>  }}' Using CLI on Windows: Send in the sample json w...

  • 2 kudos
5 More Replies
GGG_P
by New Contributor III
  • 1674 Views
  • 1 replies
  • 2 kudos

How to proper use Databricks MLFlow Managed tracker/register with Databricks Workflow

Hey.I'm building a DevOps/MLOps pipeline to train/register simple scikit learn model.I created a simple Databricks Workflow to execute training and register task on specific .git branch. (Workflow is setup with Databricks Repo on specifc branch, with...

  • 1674 Views
  • 1 replies
  • 2 kudos
Latest Reply
GGG_P
New Contributor III
  • 2 kudos

It's working just by setting experiment on specific path mlflow.set_experiment(f"/Users/${username}/my_exp")

  • 2 kudos
Mirko
by Contributor
  • 1377 Views
  • 4 replies
  • 0 kudos

Why is there a limit in /2.1/jobs/list?

I detected that there ist a limit of 25 in /2.1/jobs/list. While from what i know /2.0/jobs/list had no limit? Why is this the case? Is it planned to increase the limit at some point?I know that the offset concept exist, but from my standpoint that i...

  • 1377 Views
  • 4 replies
  • 0 kudos
Latest Reply
User16873043099
Contributor
  • 0 kudos

Jobs API 2.1 jobs list responses will be capped at a limit of 25. With the introduction of pagination in Jobs API 2.1, and to stay in-line with providing increased stability, a limit was introduced on the amount Jobs API 2.1 jobslist responses.

  • 0 kudos
3 More Replies
Wayne
by New Contributor III
  • 962 Views
  • 2 replies
  • 3 kudos
  • 962 Views
  • 2 replies
  • 3 kudos
Latest Reply
Wayne
New Contributor III
  • 3 kudos

No error, just seeing the EXPAND DISK in cluster event logs. This is just a regular spark application. I am not sure if the cloud storage matters - a spark application uses it as input and output.

  • 3 kudos
1 More Replies
Hubert-Dudek
by Esteemed Contributor III
  • 2627 Views
  • 2 replies
  • 32 kudos

Databricks Roadmap Azure There are a lot of excitement new features coming in 2022. I tried to put them all on one list: Unity catalog (seems that it ...

Databricks Roadmap AzureThere are a lot of excitement new features coming in 2022. I tried to put them all on one list:Unity catalog (seems that it will exists next to hive metastore and it will be possible to migrate)Control metastore, unity creatio...

  • 2627 Views
  • 2 replies
  • 32 kudos
Latest Reply
Kaniz
Community Manager
  • 32 kudos

Hi @Hubert Dudek​ , We know the world is full of choices. Thank you for choosing us! We would like to thank excellent customers like you for your support. We couldn’t do it without you! 

  • 32 kudos
1 More Replies
Labels