topic Re: Delta table partition folder names is getting changed in Data Engineering

Delta table partition folder names is getting changed

SakthiGanesh — Fri, 27 Jun 2025 08:56:21 GMT

I am facing an issue where the expected date partition folder should be named in format like "campaign_created_date=2024-01-17", but instead it is writing as "ad", "8B" looks like a random folder names.

Usually it will be like below:

Now it changed like below:

When I searched on net I got some info like if MinWorkerVersion is greater than 2 will make this variation in folder name. But it is not clear for me.

Can anyone let me know why this is happening suddenly? I would like to know how the MinWorkerVersion changes will affect the folder names and its Pros and Cons. What is the reason behind it.

Re: Delta table partition folder names is getting changed

UmaMahesh1 — Fri, 27 Jun 2025 12:24:23 GMT

Can you attach the delta logs or check the partition column using desc extended once.

To me it looks like the data is partitioned on a diff column.

Re: Delta table partition folder names is getting changed

Krishnamatta — Mon, 30 Jun 2025 03:00:07 GMT

Hi Satish,

This is due to the column mapping enabled on the table

From Databricks Docs:

When you enable column mapping for a Delta table, random prefixes replace column names in partition directories for Hive-style partitioning. See Rename and drop columns with Delta Lake column mapping.

Reference:

https://docs.databricks.com/gcp/en/tables/partitions#do-delta-lake-and-parquet-share-partitioning-strategies