by
Abhot
• New Contributor
- 568 Views
- 0 replies
- 0 kudos
Hello , 1 ) Which one is better during large data processing - Temp table vs Temporary view vs temp Table function . 2) How lazy evaluation better for processing ? and which one of the above helps in lazy evaluation
- 568 Views
- 0 replies
- 0 kudos
- 1543 Views
- 4 replies
- 1 kudos
var df2 = spark.read
.format("csv")
.option("sep", ",")
.option("header", "true")
.option("inferSchema", "true")
.load("src/main/resources/datasets/titanic.csv")
df2.createOrReplaceTempView("titanic")
spark.table("titanic").cach...
- 1543 Views
- 4 replies
- 1 kudos
Latest Reply
can you share what the *newtitanic* is I think that you would have done something similarspark.sql("create table newtitanic as select * from titanic")something like this works for me, but the issue is i first make a temp view then again create a tab...
3 More Replies
- 1035 Views
- 4 replies
- 4 kudos
How might I cache a temp table?The documentation suggests it is possible: https://docs.databricks.com/spark/latest/spark-sql/language-manual/delta-cache.htmlConsider the following on DBR 10.5 and Spark 3.2.1:```%pythondf.createOrReplaceTempView("chan...
- 1035 Views
- 4 replies
- 4 kudos
Latest Reply
Thank you! The newer documentation does indeed work for me.
3 More Replies