CONVERT TO DELTA fails to merge file schema

deployment_fail
New Contributor

I have a directory of Parquet files in Azure Data Lake Storage that I want to convert to a Delta Lake table. I run this:

CONVERT TO DELTA parquet.`abfss://container@storage_account.dfs.core.windows.net/directory_name`;

But it throws this error: "SparkException: [DELTA_FAILED_MERGE_SCHEMA_FILE] Failed to merge schema of file abfss://container@storage_account.dfs.core.windows.net/directory_name/file_name_123.parquet: ..."

I ran this in an all-purpose cluster with the `spark.databricks.delta.mergeSchema.enabled` config set to true.