topic What's the difference between a Global view and a Temp view? in Data Engineering

What's the difference between a Global view and a Temp view?

User15787040559 — Tue, 22 Jun 2021 22:39:55 GMT

The difference between Global and Temp is how the lifetime of the view is tied to the application:

http://spark.apache.org/docs/latest/api/python/reference/api/pyspark.sql.DataFrame.createOrReplaceTempView.html?highlight=createorreplacetempview#pyspark.sql.DataFrame.createOrReplaceTempView

Creates or replaces a local temporary view with this DataFrame.

The lifetime of this temporary table is tied to the SparkSession that was used to create this DataFrame.

http://spark.apache.org/docs/latest/api/python/reference/api/pyspark.sql.DataFrame.createOrReplaceGlobalTempView.html?highlight=createorreplaceglobaltempview#pyspark.sql.DataFrame.createOrReplaceGlobalTempView

Creates or replaces a global temporary view using the given name.

The lifetime of this temporary view is tied to this Spark application.

Re: What's the difference between a Global view and a Temp view?

aladda — Wed, 23 Jun 2021 04:11:37 GMT

In the context of Databricks Notebooks and Clusters

A Temp View is available across the context of a Notebook and is a common way of sharing data across various language REPL - Ex:- Python to Scala.
A Global Temp View is available to all Notebooks running on that Databricks Cluster

Re: What's the difference between a Global view and a Temp view?

ScottSmithDB — Sat, 27 Apr 2024 01:27:18 GMT

Correct A Temp View is scoped to the SparkSession and dropped when that session closes. Each notebook runs in its own SparkSession. The Global Temp View is scoped to the cluster and dropped when the cluster re-starts or you drop it.

----------------------------------------------------------------------

TEMPORARY

TEMPORARY views are visible only to the session that created them and are dropped when the session ends.

GLOBAL TEMPORARY

Applies to: check marked yes Databricks Runtime

GLOBAL TEMPORARY views are tied to a system preserved temporary schema global_temp.

https://docs.databricks.com/en/sql/language-manual/sql-ref-syntax-ddl-create-view.html