brockb
Databricks Employee
Databricks Employee

Hi,
Hopefully this question is related to testing and any production data would get persisted to a table but one example is:

df = (
spark.range(10)
.write
.format("delta")
.mode("append")
.save("file:/tmp/data")
)

ALTER TABLE delta.`file:/tmp/data` CLUSTER BY (id);

DESC DETAIL delta.`file:/tmp/data`
OPTIMIZE delta.`file:/tmp/data`;
 
Thanks.