Hubert-Dudek
Databricks MVP

Can you try:

  • remove megeSchema
  • remove repartition
  • analyze details about task 199
  • try maxBytesPerTrigger in readStream


My blog: https://databrickster.medium.com/