Databricks Community

tramtran · 08-01-2024

Hi everyone,I have a job with 2 tasks running independently. If one of them fails, the remaining task continues to run. I would like the job to fail if any task fails.Is there any way to do that?Thank you!

tramtran · 07-28-2024

Hi everyone,I have a streaming job with 29 notebooks that runs continuously. Initially, I allocated 28 GB of memory to the driver, but the job failed with a "Driver Out of Memory" error after 4 hours of execution.To address this, I increased the driv...

tramtran · 06-15-2024

Could you please provide guidance on the correct way to dynamically import a Python module from a user-specific path in Databricks Repos? Any advice on resolving the ModuleNotFoundError would be greatly appreciated.udf_check_table_exists notebook:fro...

tramtran · 08-02-2024

Actually, my job has multiple streaming tasks running independently. I would like to make the whole job stop if one of the streaming tasks fails.For example, one of the tasks failed after 8 minutes of execution, but the others continued running for 3...

tramtran · 07-29-2024

Thanks @xorbix_rshiva for your reply,There was deduplication logic in my code. Could I use _source_cdc_time for the watermark in this case?def merge_stream(microBatchDF, i): microBatchDF.createOrReplaceTempView("vw_delta") microBatchDF._jdf.spa...

tramtran · 06-17-2024

Thank you all again

tramtran · 06-17-2024

It works for me now  by refer this solution: [Errno 95] Operation not supported · Issue #113823 · MicrosoftDocs/azure-docs · GitHubcreating a "file" instead of a "notebook" and moving the code from the notebook into the file, I was able to use the "...

tramtran · 06-17-2024

Hi there,Have we found a solution for this problem? I'm facing the same issue as you 

Databricks Community

User Stats

User Activity

Make the job fail if a task fail

Driver: Out of Memory

How to import a function to another notebook?

Re: Make the job fail if a task fail

Re: Driver: Out of Memory

Re: How to import a function to another notebook?

Re: How to import a function to another notebook?

Re: Help With OSError: [Errno 95] Operation not supported: '/Workspace/Repos/Connectors....