Hi!
I'm experiencing different behaviours between two DBX Workspaces when trying to list file contents from an abfss: location.
In workspace A running len(dbutils.fs.ls('abfss://~~@~~~~.dfs.core.windows.net/~~/')) results in "Out[1]: 1551", while running the same command in workspace B results in "AnalysisException: MAX_LIST_SIZE_EXCEEDED: There are more than 1001 files. Please specify [LIMIT] to limit the number of files returned."
I can't seem to find any way to increase this limit and would prefer to work with dbutils.fs.ls as opposed to mounting the location.The only difference between these two Workspaces and the clusters I've tried is that Workspace B has Unity Catalog enabled.
