Be cautious about managing production schemas/volumes through DABs. Keep the critical data lifecycle separate from bundle destroy, and test the destroy behavior in a non-prod workspace first.
Use a single Auto Loader stream and extract the objecttype from _metadata.file_name. Then route each batch to the appropriate table based on that value. This scales much better than repeatedly listing files and creating separate streams for each obje...
The _otel_logs and _otel_metrics tables may remain empty if your traces are stored primarily in the spans table. Check _otel_spans first and verify that the UC trace storage and permissions are configured correctly.For a new setup, use Unity Catalog ...
Could be an issue with the cluster’s driver or Spark session not initializing properly. Since even print("Hello") and SELECT 1 hang, check the driver/cluster logs and try the same code on a default cluster to isolate whether the custom m4.large confi...
