Hi Have you tried configuring "cloudFiles.backfillInterval" ? Default value: NoneBut you can set it accordingly based on your use case.Please note this is only available in Databricks Runtime 8.4 and above. Please file a support case for us to inves...
Hi Atul,You are correct. Such errors generally denote that service you are communicating is under heavy load. It is also possible that few of those requests are being throttled. Please file a support case if this becomes too frequent. We will look cl...
We can use the below steps to push Cluster Logs to Elastic Search:1. Download the log4j-elasticsearch-java-api repo and build the jar file:git clone https://github.com/Downfy/log4j-elasticsearch-java-api.git
cd log4j-elasticsearch-java-api/
mvn clean...
Since the SparkContext is already up and running, it requires a restart. Technically, it might be possible to kill the JVM process and restart it but we do not recommend that approach. In this case, we recommend restarting the cluster so that the Sp...
