sajith_appukutt
Databricks Employee
Databricks Employee

You could use storage lifecycle policy for the s3 bucket used for storing intermediate results and configure expiration actions. This way temporary/intermediate results would be automatically cleaned up

View solution in original post