<?xml version="1.0" encoding="UTF-8"?>
<rss xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#" xmlns:taxo="http://purl.org/rss/1.0/modules/taxonomy/" version="2.0">
  <channel>
    <title>topic Optimization failed for timestampNtz in Data Engineering</title>
    <link>https://community.databricks.com/t5/data-engineering/optimization-failed-for-timestampntz/m-p/66251#M33064</link>
    <description>&lt;P&gt;We have a table using timestampNtz type for timestamp, which is also a cluster key for this table using liquid clustering. I ran OPTIMIZE &amp;lt;table-name&amp;gt;, it failed with error&lt;/P&gt;&lt;LI-CODE lang="markup"&gt;Unsupported datatype 'TimestampNTZType'&lt;/LI-CODE&gt;&lt;P&gt;&amp;nbsp;But the failed optmization also broke the query that filters on this field. Query as below will fail&lt;/P&gt;&lt;LI-CODE lang="markup"&gt;select * from table-name
WHERE field_name&amp;gt;'2024-01-02'&lt;/LI-CODE&gt;&lt;P&gt;&amp;nbsp;with error&lt;/P&gt;&lt;LI-CODE lang="markup"&gt;TimestampNTZType (of class org.apache.spark.sql.types.TimestampNTZType$)&lt;/LI-CODE&gt;&lt;P&gt;&amp;nbsp;How am I suppose to solve this issue?&lt;/P&gt;</description>
    <pubDate>Mon, 15 Apr 2024 10:11:06 GMT</pubDate>
    <dc:creator>Jennifer</dc:creator>
    <dc:date>2024-04-15T10:11:06Z</dc:date>
    <item>
      <title>Optimization failed for timestampNtz</title>
      <link>https://community.databricks.com/t5/data-engineering/optimization-failed-for-timestampntz/m-p/66251#M33064</link>
      <description>&lt;P&gt;We have a table using timestampNtz type for timestamp, which is also a cluster key for this table using liquid clustering. I ran OPTIMIZE &amp;lt;table-name&amp;gt;, it failed with error&lt;/P&gt;&lt;LI-CODE lang="markup"&gt;Unsupported datatype 'TimestampNTZType'&lt;/LI-CODE&gt;&lt;P&gt;&amp;nbsp;But the failed optmization also broke the query that filters on this field. Query as below will fail&lt;/P&gt;&lt;LI-CODE lang="markup"&gt;select * from table-name
WHERE field_name&amp;gt;'2024-01-02'&lt;/LI-CODE&gt;&lt;P&gt;&amp;nbsp;with error&lt;/P&gt;&lt;LI-CODE lang="markup"&gt;TimestampNTZType (of class org.apache.spark.sql.types.TimestampNTZType$)&lt;/LI-CODE&gt;&lt;P&gt;&amp;nbsp;How am I suppose to solve this issue?&lt;/P&gt;</description>
      <pubDate>Mon, 15 Apr 2024 10:11:06 GMT</pubDate>
      <guid>https://community.databricks.com/t5/data-engineering/optimization-failed-for-timestampntz/m-p/66251#M33064</guid>
      <dc:creator>Jennifer</dc:creator>
      <dc:date>2024-04-15T10:11:06Z</dc:date>
    </item>
  </channel>
</rss>

