The issue was getting created because I was passing SAS Token for setting the configuration for Storage Account instead of Storage Access Key. It's working now without any issue..

spark.conf.set("fs.azure.account.key.datalakestorage.dfs.core.windows.net",dbutils.secrets.get(scope = "databricks-scope",key = "StorageAccessKey"))

I don't know why the error shows " The input is not a valid Base-64 string as it contains a non-base 64 character" but the issue was related to Storage Access Key.