I completely agree with @lingareddy_Alva on the costing part. One small point I would like to mention is We should only enable SPOT instances (60-90% cost savings) in Development/non-critical(PROD) environment. This option works great and is indeed c...
cloudFiles.cleanSource.retentionDurationType: Interval StringAmount of time to wait before processed files become candidates for archival with cleanSource. Must be greater than 7 days for DELETE. No minimum restriction for MOVE.Available in Databrick...
@altafsagar please refer the following github url for the course code and notebook.GitHub - NhanNguyen001/Databricks-devops-essentials-for-data-engineering-2.0.1
