Great question; I've been trying to hunt that down also. `/local_disk0` looks like a good candidate, but it has restricted access and I can't confirm or use.

Would love to learn a solution someday. This is a big need for hybrid workflows & libraries common in my organization. Spark is great for some things, but we also use other frameworks extensively.