Databricks write to Azure Data Explorer writes suddenly become slower

RengarLee
Contributor

Now, I write to Azure Data explorer using Spark streaming. one day, writes suddenly become slower. restart is no effect.

I have a questions about Spark Streaming to Azure Data explorer.

Q1: What should I do to get performance to reply?

Figure 1 shows the performance of writing in the current table.

Figure 2 the performance of writing in the now table.

Figure 3 the performance of writing in the current table, but the checkpoint location is new.

Could it be that checkpoint location caused it?