Using a Unity Catalog Volume to write and read back the DataFrame seems like a practical alternative to avoid mounting cloud storage. Just make sure to account for the extra I/O overhead and schema preservation when reloading the data.
Thanks for sharing the detailed reproducer. It looks like separating the AutoDependencies hook error from the serverless UDF dependency limitation is key to troubleshooting this issue. Testing on compatible classic compute seems like a good next step...
