I am wondering if it's possible to install and use GDAL on Databricks serverless compute. I couldn't manage to do that using pip install gdal, and I discovered that init scripts are not supported on serverless compute.
You can shallow clone the table to an external location after sync the result of the shallow clone to unity catalog. Here is an example https://medium.com/@wahidatoui/sync-delta-tables-stored-in-dbfs-managed-or-external-to-unity-catalog-1fe29a3ae61b
