โ06-05-2023 10:53 PM
Hello,
I have some nested columns with hyphen i.e. sample-1 in struct column, recently DLT pipeline has started throwing synatx error. Before May 24, 2023, this was working fine.
Is this a new bug in May 2023 release?
After clearing table and table's data when it's running for first time, then no error is there. But next runs are throwing syntax error.
Note: We are creating live table with SCD-1 using APPLY_CHANGES.
โ06-14-2023 02:34 AM
Hi @Vidula Khannaโ ,
Issue got resolved automatically post DLT release 2023.21.
Strangely in release 2023.19, we got this hyphen issue i.e. "syntax error". This was working fine with prior releases also.
If you have more details on this, please do let me know.
Thanks for your support! @Kaniz Fatmaโ @Vidula Khannaโ
โ06-06-2023 07:49 AM
Thank you for your response @Kaniz Fatmaโ .
We have configured DLT pipeline channel to 'current', so it is using DBR 11.3 where it is having Delta Lake 2.1.0. Above mentioned link is for Delta Lake 2.3.0.
We have observed that default DBR was updated in DLT as per below to 11.3.11 from 11.3.10 in Release 2023.19.
https://docs.databricks.com/release-notes/delta-live-tables/2023/19/index.html
โ06-07-2023 06:15 AM
Yes, syntax error started appearing after May 24, that is when release 2023.19 happened.
โ06-14-2023 12:06 AM
Hi @Rishabh Tomarโ
We haven't heard from you since the last response from @Kaniz Fatmaโ . Kindly share the information with us, and in return, we will provide you with the necessary solution.
Thanks and Regards
โ06-14-2023 02:34 AM
Hi @Vidula Khannaโ ,
Issue got resolved automatically post DLT release 2023.21.
Strangely in release 2023.19, we got this hyphen issue i.e. "syntax error". This was working fine with prior releases also.
If you have more details on this, please do let me know.
Thanks for your support! @Kaniz Fatmaโ @Vidula Khannaโ
Join a Regional User Group to connect with local Databricks users. Events will be happening in your city, and you wonโt want to miss the chance to attend and share knowledge.
If there isnโt a group near you, start one and help create a community that brings people together.
Request a New Group