Delta table partition folder names is getting changed

SakthiGanesh
New Contributor II

‎06-27-2025 01:56 AM

I am facing an issue where the expected date partition folder should be named in format like "campaign_created_date=2024-01-17", but instead it is writing as "ad", "8B" looks like a random folder names.

Usually it will be like below:
SakthiGanesh_0-1751013736357.png

Now it changed like below:

SakthiGanesh_1-1751013840570.png

When I searched on net I got some info like if MinWorkerVersion is greater than 2 will make this variation in folder name. But it is not clear for me. 

Can anyone let me know why this is happening suddenly? I would like to know how the MinWorkerVersion changes will affect the folder names and its Pros and Cons. What is the reason behind it.



 

0 Kudos

UmaMahesh1
Honored Contributor III

‎06-27-2025 05:24 AM

Can you attach the delta logs or check the partition column using desc extended once.

To me it looks like the data is partitioned on a diff column.

Uma Mahesh D
0 Kudos

Krishnamatta
Contributor

‎06-29-2025 08:00 PM

Hi Satish,

This is due to the column mapping enabled on the table

From Databricks Docs:

When you enable column mapping for a Delta table, random prefixes replace column names in partition directories for Hive-style partitioning. See Rename and drop columns with Delta Lake column mapping.

Reference:

https://docs.databricks.com/gcp/en/tables/partitions#do-delta-lake-and-parquet-share-partitioning-st...

 

0 Kudos