Facing issue while accessing DLT pipeline event logs.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
02-19-2025 10:23 PM
while accessing event logs created by dlt pipeline, i am getting below error.
[INTERNAL_ERROR] The Spark SQL phase analysis failed with an internal error. You hit a bug in Spark or the Spark plugins you use. Please, report this bug to the corresponding communities or vendors, and provide the full stack trace. SQLSTATE: XX000
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
02-24-2025 11:17 AM
Hi Pawan,
How are you doing today? As per my understanding, This error likely indicates an internal issue with Databricks, Delta Live Tables (DLT), or a Spark plugin. First, check if there are any known issues in Databricks release notes or community forums. If it’s a transient issue, try restarting the cluster and rerunning the query. Ensure you're querying the correct event log table with SELECT * FROM event_log WHERE pipeline_id = '<your_pipeline_id>'. If the issue persists, there might be corrupt data, so try filtering older records or limiting the query. Also, upgrading to the latest Databricks LTS (Long-Term Support) runtime might help with compatibility issues. If nothing works, raising a Databricks support ticket with the full stack trace is the best option. Hope this helps!
Give a try and ley me know.
Regards,
Brahma
