I am trying to read a csv and do an activity from azure storage account using databricks shell script. I wanted to add this shell script into my big python code for other sources as well. I have created widgets for file path in python. I have created variables named file_datepath and get widgets data into it. I want to read csv from azure storage account using file_datepath variable in a shell command.

I tried below, its not working.

dbutils.widgets.text("source_storage_path", "datafolder/data/")

dbutils.widgets.text("file_datepath", "2022/12/")

source_storage_path = dbutils.widgets.get("source_storage_path")

file_datepath = dbutils.widgets.get("file_datepath")

%sh tr '\n' ' ' <'/dbfs/mnt/{source_storage_path}/{file_datepath}/*.csv' > '/dbfs/mnt/{source_storage_path}/{file_datepath}/*_new.csv'

but this is giving me error no such file or directry. I tried with $ and everything. Nothing seems working. Please help @Sherinus​ @Hubert Dudek​ @Werner Stinckens​