topic Re: delta sharing issue after enable predictive optimization in Administration & Architecture

delta sharing issue after enable predictive optimization

lizou8 — Fri, 26 Jul 2024 22:48:57 GMT

Some of our delta sharing tables are not working

May be related to this, or maybe not, we enabled predictive optimization on all tables a few days ago

es not working any more

but any new tables created works fine after setting this

SET TBLPROPERTIES (delta.enableDeletionVectors=false);

anyone experience same issue? thanks

Re: delta sharing issue after enable predictive optimization

lizou8 — Sat, 27 Jul 2024 00:38:07 GMT

error

JsonParseException: Unexpected character ('k' (code 107)): was expecting comma to separate Object entries at [Source: (String)"{"type":"struct","fields":[{"name":"UserKey","type":"long","nullable":false,"metadata":{"comment":"{"key":"primary_key","is_identity":"true"}"}},{"name":"SystemDate","type":"date","nullable":false,"metadata":{}},{"name":"UserId","type":"string","nullable":false,"metadata":{}},{"name":"Email","type":"string","nullable":true,"metadata":{}},{"name":"UserName","type":"string","nullable":true,"metadata":{}},{"name":"Active","type":"boolean","nullable":true,"metadata":{}},{"name":"FullName","type":"st"[truncated 713 chars]; line: 1, column: 103]

at com.fasterxml.jackson.core.JsonParser._constructError(JsonParser.java:2477)

Re: delta sharing issue after enable predictive optimization

lizou8 — Sat, 27 Jul 2024 20:32:03 GMT

after some debugging, I find out a very unique cause if we used a JSON string in column comment, and it make sense that a JSON string in column comment breaks delta sharing

example: column COMMENT {"key": "primary_key", "is_identity": "true"}

The error said Unexpected character ('k' (code 107)): k comes from the "key" in JSON string leads me to find a solution is to set comment as blank

but