temp tables in Databricks

[AMadan]

Hey Team,

We are in a migration phase from SQL server to Databricks.

In our sql server databricks, lot of times, #temp tables created and dropped like typical Datawarehousing sql.

In Databricks, we can do 2 things. I want to check which one is best approach and why?

1) create temp views in databricks and make use of views later in the query

2) create delta table with temp(prefix) and make use of temp table and at the end drop temp table.

I am looking forward for your response.

[berserkersap]

Just to add a bit more from experience

If you are doing to use the temp tables too many times, then using temp views is going to slow down your cluster . It is due to lazy evaluation unless you persist that view (dataframe cache or globaltempview).

Every time your temp view is called, it will execute the view definition/code. This also depends upon your cluster capacity, how complex the view definition is, how much data the temp view has.

Please correct me if I am wrong.