Databricks upon inserting delta table data inserts into folders in Dev
Options
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
07-12-2024 03:51 PM
We have a Delta Table in Databricks. When we are inserting data into the Delta Table, in the storage account, it creates folders like: 05, 0H, 0F, 0O, 1T,1W, etc... and adds the parquet files there.
We have not defined any partitions. We are inserting data using following code:
df.write\
.mode('append')\
.option("mergeSchema", "true")\
.option("ignoreLeadingWhiteSpace","false") \
.option("ignoreTrailingWhiteSpace","false") \
.format('delta').save(prepped_path)
The Table Properties are:
delta.columnMapping.mode=name
delta.minReaderVersion=2
delta.minWriterVersion=6
delta.minReaderVersion=2
delta.minWriterVersion=6
delta.columnMapping.maxColumnId=18
