We used to have this problem, but worked around it by having the files in a UC external location, and using spark pandas instead of regular pandas, since it uses the spark access model and this works with UC grants. 

However, with the recent addition of UC Volumes, you can add the location as as a volume and access it as you would a file in a regular file system.