Hi Pawan,
How are you doing today? As per my understanding, This error likely indicates an internal issue with Databricks, Delta Live Tables (DLT), or a Spark plugin. First, check if there are any known issues in Databricks release notes or community forums. If it’s a transient issue, try restarting the cluster and rerunning the query. Ensure you're querying the correct event log table with SELECT * FROM event_log WHERE pipeline_id = '<your_pipeline_id>'. If the issue persists, there might be corrupt data, so try filtering older records or limiting the query. Also, upgrading to the latest Databricks LTS (Long-Term Support) runtime might help with compatibility issues. If nothing works, raising a Databricks support ticket with the full stack trace is the best option. Hope this helps!
Give a try and ley me know.
Regards,
Brahma
