Hi,
we have our Databricks Jobs deployed via DABs, and they have been running fine for a while now (approximately 1 month since we migrated from ADF). However, since yesterday, we are getting a weird issue while writing. See error below:
[STREAM_FAILED] Query [id = query_id, runId = run_id] terminated with exception: Do not know how to convert JArray(List(JString(some_key_string))) into class java.lang.String SQLSTATE: XXKST
For context, we use AutoLoader File-Notification mode to fetch the data from ADLS, and store it into our bronze layer. Nothing changed - infrastructure wise - and I cannot find the reason why this is happening. It started in a couple of tasks - some working some not - and after a while, my whole bronze layer is not working anymore.
I would appreciate if you can nudge me in the right direction.
