06-05-2023 10:53 PM
Hello,
I have some nested columns with hyphen i.e. sample-1 in struct column, recently DLT pipeline has started throwing synatx error. Before May 24, 2023, this was working fine.
Is this a new bug in May 2023 release?
After clearing table and table's data when it's running for first time, then no error is there. But next runs are throwing syntax error.
Note: We are creating live table with SCD-1 using APPLY_CHANGES.
06-14-2023 02:34 AM
Hi @Vidula Khanna ,
Issue got resolved automatically post DLT release 2023.21.
Strangely in release 2023.19, we got this hyphen issue i.e. "syntax error". This was working fine with prior releases also.
If you have more details on this, please do let me know.
Thanks for your support! @Kaniz Fatma @Vidula Khanna
06-06-2023 07:49 AM
Thank you for your response @Kaniz Fatma .
We have configured DLT pipeline channel to 'current', so it is using DBR 11.3 where it is having Delta Lake 2.1.0. Above mentioned link is for Delta Lake 2.3.0.
We have observed that default DBR was updated in DLT as per below to 11.3.11 from 11.3.10 in Release 2023.19.
https://docs.databricks.com/release-notes/delta-live-tables/2023/19/index.html
06-07-2023 06:15 AM
Yes, syntax error started appearing after May 24, that is when release 2023.19 happened.
06-14-2023 12:06 AM
Hi @Rishabh Tomar
We haven't heard from you since the last response from @Kaniz Fatma . Kindly share the information with us, and in return, we will provide you with the necessary solution.
Thanks and Regards
06-14-2023 02:34 AM
Hi @Vidula Khanna ,
Issue got resolved automatically post DLT release 2023.21.
Strangely in release 2023.19, we got this hyphen issue i.e. "syntax error". This was working fine with prior releases also.
If you have more details on this, please do let me know.
Thanks for your support! @Kaniz Fatma @Vidula Khanna
Join a Regional User Group to connect with local Databricks users. Events will be happening in your city, and you won’t want to miss the chance to attend and share knowledge.
If there isn’t a group near you, start one and help create a community that brings people together.
Request a New Group