Hi @Cheuk Hin Christophe Poon​ have you optimize your table anytime since it's creation? If not, then optimize may take some time depending on the no of underlying files.

Please try to run optimize manually as described in below document:

https://docs.databricks.com/sql/language-manual/delta-optimize.html

If this doesn't help, you can try disabling DBIO cache by setting below in your notebook:

spark.conf.set("spark.databricks.io.cache.enabled", "false")