I encountered this same issue, and figured out a fix!

For some reason, it seems like only %sh cells can access the /tmp directory. So I just did...

%sh ch /tmp/<file> /dbfs/<desired-location> and then accessed it form there using Spark.