cancel
Showing results for 
Search instead for 
Did you mean: 
Machine Learning
Dive into the world of machine learning on the Databricks platform. Explore discussions on algorithms, model training, deployment, and more. Connect with ML enthusiasts and experts.
cancel
Showing results for 
Search instead for 
Did you mean: 

Forum Posts

obiamaka
by New Contributor III
  • 2429 Views
  • 6 replies
  • 2 kudos

Resolved! Unable to view jobs in Databricks Workflow

This has been happening for about 2 days now. Any ideas on why this occurs and how it can be resolved. I attached a screenshot.

  • 2429 Views
  • 6 replies
  • 2 kudos
Latest Reply
obiamaka
New Contributor III
  • 2 kudos

This issue got resolved on it's own, but not sure what the problem was, probably a bug from a software update?

  • 2 kudos
5 More Replies
User16752245312
by New Contributor III
  • 2435 Views
  • 5 replies
  • 2 kudos

When running structured streaming jobs in production, what are the general best practices to reduce cost?

Consider a basic structured streaming use case of aggregating the data, perform some basic data cleaning transformation, and merge into a historical aggregate dataset.

  • 2435 Views
  • 5 replies
  • 2 kudos
Latest Reply
lawrence009
Contributor
  • 2 kudos

I second the recommendations: auto load with trigger, batch processing instead of continuous streaming where use case permits. In addition, test with a small batch firstfavor fewer larger workers over more smaller workersadjust your job cluster over...

  • 2 kudos
4 More Replies
KrishZ
by Contributor
  • 5534 Views
  • 4 replies
  • 4 kudos

How to use Parallel processing using Concurrent Jobs in Databricks ?

QuestionIt would be great if you could recommend how I go about solving the below problem. I haven't been able to find much help online. A. Background:A1. I have to text manipulation using python (like concatenation , convert to spacy doc , get verbs...

  • 5534 Views
  • 4 replies
  • 4 kudos
Latest Reply
Anonymous
Not applicable
  • 4 kudos

Hi @Krishna Zanwar​ Hope all is well! Just wanted to check in if you were able to resolve your issue and would you be happy to share the solution or mark an answer as best? Else please let us know if you need more help. We'd love to hear from you.Tha...

  • 4 kudos
3 More Replies
Mirko
by Contributor
  • 1368 Views
  • 4 replies
  • 0 kudos

Why is there a limit in /2.1/jobs/list?

I detected that there ist a limit of 25 in /2.1/jobs/list. While from what i know /2.0/jobs/list had no limit? Why is this the case? Is it planned to increase the limit at some point?I know that the offset concept exist, but from my standpoint that i...

  • 1368 Views
  • 4 replies
  • 0 kudos
Latest Reply
User16873043099
Contributor
  • 0 kudos

Jobs API 2.1 jobs list responses will be capped at a limit of 25. With the introduction of pagination in Jobs API 2.1, and to stay in-line with providing increased stability, a limit was introduced on the amount Jobs API 2.1 jobslist responses.

  • 0 kudos
3 More Replies
User16753724663
by Valued Contributor
  • 1417 Views
  • 1 replies
  • 0 kudos
  • 1417 Views
  • 1 replies
  • 0 kudos
Latest Reply
User16753724663
Valued Contributor
  • 0 kudos

We can use the below api to list out the jobs and then use the delete job api:https://docs.databricks.com/dev-tools/api/latest/jobs.html#listListEndpoint HTTP Method2.0/jobs/list GETOnce we list out the jobs, then we can use below API to delete them:...

  • 0 kudos
Labels