Databricks Community

We have created databricks jobs and each has multiple tasks. Each task is 24/7 running streaming with checkpoint enabled. We want it to be stateful when cancel and run the job but it seems like, when we cancel the job run it kill the parent process a...

Hi,We have created databricks jobs and each has multiple tasks. Each task is 24/7 running streaming with checkpoint enabled. We want it to be stateful when cancel and run the job but it seems like, when we cancel the job run it kill the parent proces...

We have production streaming jobs running on Job Clusters. We face cluster related errors now and then, one such example is below error. Run failed with error message Cluster became unreachable during run Cause: Got invalid response: 404 /ERR_NGROK_3...

We have enabled predictive optimization at account level and metastore level. The enables check box can be seen in catalog details and table details. When i query the system.storage.predictive_optimization_operations_history table, it is still empty....

Hi,We upgraded the GKE cluster to GCE cluster as per the databricks documentation. It works fine with one or two notebooks in job. Our production job has more than 40 notebooks and each notebook access the secret api and seems like secret api server ...