Hubert-Dudek
Databricks MVP

Pandas will not see dbfs storage this way. You need to change to pandas on spark or use the classic path.

import pyspark.pandas as pd
pd.read_csv("dbfs:/your_path/your_file.csv")
 
import pandas as pd
pd.read_csv("/dbfs/your_path/your_file.csv")


My blog: https://databrickster.medium.com/