- 2166 Views
- 2 replies
- 3 kudos
Question about "foreachBatch" to remove duplicate records when streaming data
Hi,I am practicing with Databricks sample notebook published here:https://github.com/databricks-academy/advanced-data-engineering-with-databricksIn one of the notebooks (ADE 3.1 - Streaming Deduplication) (URL), there is a sample code to remove dupli...
- 2166 Views
- 2 replies
- 3 kudos
- 3 kudos
Hi @Mohammad Saber​ Great to meet you, and thanks for your question! Let's see if your peers in the community have an answer to your question first. Or else bricksters will get back to you soon. Thanks
- 3 kudos
- 1039 Views
- 1 replies
- 0 kudos
Question on single job with multi task
Say, I have a job with 10 parallel tasks. I had to cancel one of the tasks to fix something and I unable to restart just that task. Is this by design? Should I restart the job in this case.Q2) If one of the tasks fails, will it auto recover just tha...
- 1039 Views
- 1 replies
- 0 kudos
- 0 kudos
Hi @Jin Kim​, Please enable "Task Orchestration in Jobs" in your Admin Console, and then you can add as many tasks to your job. You can also specify the dependency of your task.
- 0 kudos