Wrong FS: abfss://....., expected: dbfs:/ Error in DLT pipeline

harvey-c
New Contributor III

Dear Databricks community members:

Symptom

Received the error for a delta load, after a successful initial load with a  Unity Catalog Volume as a data source.

org.apache.spark.sql.streaming.StreamingQueryException: [STREAM_FAILED] Query [id = xxx, runId = xxx] terminated with exception: Wrong FS: abfss://container@storageaccount.dfs.core.windows.net/data.csv, expected: dbfs:/

Context

The Azure storage source was defined as a volume with external location. The DLT table was defined with dlt.readStream. and executed successfully. I changed the load path to next month in the same container, then the following load was failed with the above error.

Could you please advise what might go wrong and how to fix the issue? 

Thank you!