Please recommend other solutions if possible as above provided may not work due to below reasons.

Data History and Versioning:

  • To maintain history during the migration, ensure that your Delta table retains its version history. --> To maintain history, it has to keep those folder name with dot.
  • Delta Lake automatically tracks changes, so you’ll have access to historical data even after the migration. --> only if migration is possible which is not as folder name has dot.

Additional Considerations:

  • Verify that your workspace is enabled for Unity Catalog (if needed). --> Yes it is.
  • Confirm that your clusters are running Databricks Runtime 11.3 LTS or above to fully support Unity Catalog features. --> yes they are.