Databricks Community

I’ve spent a good part of this year on Unity Catalog backups at scale: thousands of Delta tables, nightly DEEP CLONE into a separate backup catalog, and a backup window that kept getting longer. The obvious fix was “only clone tables that actually ch...

I needed PDAL on Databricks for a LiDAR processing workload. PDAL is a C++ library with a long native dependency chain (GDAL, PROJ, HDF5, a recent libstdc++), and the Python bindings have to match the interpreter exactly. The usual approach, an apt-g...

Tables are the easy part of a Unity Catalog backup, because you have DEEP CLONE. Volumes have no equivalent. When I started on a self-service backup tool this year, the first version backed volumes up the way I suspect most people do: list the files ...