Thanks @Retired_mod, I will look into ephemeral clusters.
Can you share the link for the official databricks on gcp documentation? I didn't find it in the post link you shared.

View solution in original post