<?xml version="1.0" encoding="UTF-8"?>
<rss xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#" xmlns:taxo="http://purl.org/rss/1.0/modules/taxonomy/" version="2.0">
  <channel>
    <title>topic Re: Collation Change Planning &amp;amp; Fixing in Data Engineering</title>
    <link>https://community.databricks.com/t5/data-engineering/collation-change-planning-amp-fixing/m-p/164531#M55284</link>
    <description>&lt;P&gt;Hi&amp;nbsp;&lt;a href="https://community.databricks.com/t5/user/viewprofilepage/user-id/135017"&gt;@skipowder99&lt;/a&gt;&amp;nbsp;,&lt;/P&gt;&lt;P&gt;According to docs,&amp;nbsp; ALTER CATALOG ... DEFAULT COLLATION only affects newly created schemas, and ALTER SCHEMA ... DEFAULT COLLATION only affects newly created objects. Likewise, changing a table’s default collation only affects new STRING columns; existing columns retain their original collation until altered individually.&lt;BR /&gt;&lt;BR /&gt;Make sure that your objects have expected collation. Another thing that it's worth checking is &lt;STRONG&gt;session collation.&lt;BR /&gt;&lt;/STRONG&gt;Queries use the session collation, not the catalog or schema collation. The system session default remains UTF8_BINARY.&amp;nbsp;&lt;BR /&gt;&lt;BR /&gt;&lt;/P&gt;&lt;P&gt;&lt;span class="lia-inline-image-display-wrapper lia-image-align-inline" image-alt="szymon_dybczak_0-1785424303217.png" style="width: 400px;"&gt;&lt;img src="https://community.databricks.com/t5/image/serverpage/image-id/29625iE39E92B57D672880/image-size/medium?v=v2&amp;amp;px=400" role="button" title="szymon_dybczak_0-1785424303217.png" alt="szymon_dybczak_0-1785424303217.png" /&gt;&lt;/span&gt;&lt;/P&gt;&lt;P&gt;So, that can cause the mismatch you experienced. Your target objects have&amp;nbsp;&lt;SPAN&gt;UTF8_LCASE collation, but still queries that are loading data to those objects use&amp;nbsp; UTF8_BINARY collation.&lt;BR /&gt;&lt;BR /&gt;&lt;STRONG&gt;If my answer was helpful, please consider marking it as accepted solution to the thread.&lt;/STRONG&gt;&lt;/SPAN&gt;&lt;/P&gt;</description>
    <pubDate>Thu, 30 Jul 2026 15:13:38 GMT</pubDate>
    <dc:creator>szymon_dybczak</dc:creator>
    <dc:date>2026-07-30T15:13:38Z</dc:date>
    <item>
      <title>Collation Change Planning &amp; Fixing</title>
      <link>https://community.databricks.com/t5/data-engineering/collation-change-planning-amp-fixing/m-p/164525#M55283</link>
      <description>&lt;P&gt;We recently decided to change from the default collation of UTF8_BINARY to UTF8_LCASE and there are many issues.&amp;nbsp; We attempted to cascade the collation defaults to all the Catalogs, Schemas, Tables, Views and Columns, but we still seem to have expression errors or data type mismatch errors.&amp;nbsp; Anyone has some advice on how to get all our notebooks and views going again with out explicit collation commands ?&lt;/P&gt;</description>
      <pubDate>Thu, 30 Jul 2026 14:39:33 GMT</pubDate>
      <guid>https://community.databricks.com/t5/data-engineering/collation-change-planning-amp-fixing/m-p/164525#M55283</guid>
      <dc:creator>skipowder99</dc:creator>
      <dc:date>2026-07-30T14:39:33Z</dc:date>
    </item>
    <item>
      <title>Re: Collation Change Planning &amp; Fixing</title>
      <link>https://community.databricks.com/t5/data-engineering/collation-change-planning-amp-fixing/m-p/164531#M55284</link>
      <description>&lt;P&gt;Hi&amp;nbsp;&lt;a href="https://community.databricks.com/t5/user/viewprofilepage/user-id/135017"&gt;@skipowder99&lt;/a&gt;&amp;nbsp;,&lt;/P&gt;&lt;P&gt;According to docs,&amp;nbsp; ALTER CATALOG ... DEFAULT COLLATION only affects newly created schemas, and ALTER SCHEMA ... DEFAULT COLLATION only affects newly created objects. Likewise, changing a table’s default collation only affects new STRING columns; existing columns retain their original collation until altered individually.&lt;BR /&gt;&lt;BR /&gt;Make sure that your objects have expected collation. Another thing that it's worth checking is &lt;STRONG&gt;session collation.&lt;BR /&gt;&lt;/STRONG&gt;Queries use the session collation, not the catalog or schema collation. The system session default remains UTF8_BINARY.&amp;nbsp;&lt;BR /&gt;&lt;BR /&gt;&lt;/P&gt;&lt;P&gt;&lt;span class="lia-inline-image-display-wrapper lia-image-align-inline" image-alt="szymon_dybczak_0-1785424303217.png" style="width: 400px;"&gt;&lt;img src="https://community.databricks.com/t5/image/serverpage/image-id/29625iE39E92B57D672880/image-size/medium?v=v2&amp;amp;px=400" role="button" title="szymon_dybczak_0-1785424303217.png" alt="szymon_dybczak_0-1785424303217.png" /&gt;&lt;/span&gt;&lt;/P&gt;&lt;P&gt;So, that can cause the mismatch you experienced. Your target objects have&amp;nbsp;&lt;SPAN&gt;UTF8_LCASE collation, but still queries that are loading data to those objects use&amp;nbsp; UTF8_BINARY collation.&lt;BR /&gt;&lt;BR /&gt;&lt;STRONG&gt;If my answer was helpful, please consider marking it as accepted solution to the thread.&lt;/STRONG&gt;&lt;/SPAN&gt;&lt;/P&gt;</description>
      <pubDate>Thu, 30 Jul 2026 15:13:38 GMT</pubDate>
      <guid>https://community.databricks.com/t5/data-engineering/collation-change-planning-amp-fixing/m-p/164531#M55284</guid>
      <dc:creator>szymon_dybczak</dc:creator>
      <dc:date>2026-07-30T15:13:38Z</dc:date>
    </item>
    <item>
      <title>Re: Collation Change Planning &amp; Fixing</title>
      <link>https://community.databricks.com/t5/data-engineering/collation-change-planning-amp-fixing/m-p/164543#M55287</link>
      <description>&lt;P&gt;Hi skipowder99,&lt;/P&gt;&lt;P&gt;When you set a DEFAULT COLLATION at the catalog or schema level, it is forward looking. It does not retroactively apply to existing tables, columns or views. It leaves you with a split environment&amp;nbsp;the newly created objects default to UTF8_LCASE, while the existing ones are still with UTF8_BINARY.&lt;/P&gt;&lt;P&gt;The expression and data type mismatch errors you are seeing occur because the engine enforces collation rules during query planning. If you join an older UTF8_BINARY column with a newer UTF8_LCASE column or compare an existing table column to a string literal (which now inherits the new schema default), it throws an error.&lt;/P&gt;&lt;P&gt;Views are particularly stubborn here as they bake in the implicit collation of their underlying columns at the exact moment they are created.&lt;/P&gt;&lt;P&gt;You need to systematically update the existing metadata&amp;nbsp;to get your environment running again.&lt;/P&gt;&lt;P&gt;1. Recompiling Views - As&amp;nbsp;views lock in the collation at creation, they won't automatically inherit the new schema default. You must recreate them&amp;nbsp;provided the underlying tables have already been fixed.&lt;/P&gt;&lt;P&gt;2. Updating Base Table Columns&amp;nbsp;- For the base tables, you have two options depending on your compute version. You can alter the column collations in place in serverless warehouse.&lt;/P&gt;&lt;DIV&gt;&lt;DIV&gt;&lt;DIV&gt;&lt;PRE&gt;ALTER TABLE my_schema.my_table 
ALTER COLUMN my_string_column TYPE STRING COLLATE UTF8_LCASE;&lt;/PRE&gt;&lt;/DIV&gt;&lt;/DIV&gt;&lt;/DIV&gt;&lt;P&gt;If that feature isn't available on the runtime, you can recreate the tables using a CTAS operation. Since your schema default is now UTF8_LCASE, the new table will automatically pick it up.&lt;/P&gt;&lt;P&gt;3. Execution Order&amp;nbsp;- You should audit and execute these changes in a strict dependency order to avoid recursive errors. You can find the lagging columns by querying the information schema. Apply the fixes bottom-up from there.&lt;/P&gt;&lt;UL&gt;&lt;LI&gt;&lt;P&gt;Update or recreate base tables first.&lt;/P&gt;&lt;/LI&gt;&lt;LI&gt;&lt;P&gt;Recreate the views that query those base tables directly.&lt;/P&gt;&lt;/LI&gt;&lt;LI&gt;&lt;P&gt;Recreate downstream views (views querying views) moving from the leaf nodes up to the root.&lt;/P&gt;&lt;/LI&gt;&lt;/UL&gt;</description>
      <pubDate>Thu, 30 Jul 2026 16:00:10 GMT</pubDate>
      <guid>https://community.databricks.com/t5/data-engineering/collation-change-planning-amp-fixing/m-p/164543#M55287</guid>
      <dc:creator>balajij8</dc:creator>
      <dc:date>2026-07-30T16:00:10Z</dc:date>
    </item>
  </channel>
</rss>

