What is the difference between createTempView, createGlobalTempView and registerTempTable

pmezentsev — Sat, 01 Apr 2017 15:03:26 GMT

Hi, friends!

I have a question about difference between this three functions:

dataframe . createTempView
dataframe . createGlobalTempView
dataframe . registerTempTable

all of them create intermediate tables.

How to decide which I have to choose in computations?

Re: What is the difference between createTempView, createGlobalTempView and registerTempTable

KeshavP — Tue, 11 Apr 2017 22:39:49 GMT

From my understanding, createTempView (or more appropriately createOrReplaceTempView) has been introduced in Spark 2.0 to replace registerTempTable, which has been deprecated in 2.0. CreateTempView creates an in memory reference to the Dataframe in use. The lifetime for this is tied to the spark session in which the Dataframe was created in. createGlobalTempView (which is not present in 2.0 but is added in 2.1.0) on the other hand allows you to create the references that can be used across spark sessions. So depending upon whether you need to share data across sessions, you can use either of the methods. Ideally your notebooks in same cluster share the same spark session, but there is an option to setup clusters where each notebook has its own session. So all it boils down to is that where do you create the data frame and where do you want to access it.

topic What is the difference between createTempView, createGlobalTempView and registerTempTable in Data Engineering

What is the difference between createTempView, createGlobalTempView and registerTempTable

Re: What is the difference between createTempView, createGlobalTempView and registerTempTable