topic Re: Recursive view error while using spark 3.2.0 version in Data Engineering

Recursive view error while using spark 3.2.0 version

manasa — Mon, 14 Feb 2022 07:30:27 GMT

This happens while creating temp view using below code block

latest_data.createOrReplaceGlobalTempView("e_test")

ideally this command should replace the view if e_test already exists instead it is throwing

"Recursive view `global_temp`.`e_test` detected (cycle: `global_temp`.`e_test` -> `global_temp`.`e_test`)" error..Can anyone help me with this

-werners- — Mon, 14 Feb 2022 10:02:51 GMT

does it work when you don't use a global view?

manasa — Mon, 14 Feb 2022 10:16:39 GMT

Nope.Same error

User16763506477 — Wed, 09 Mar 2022 01:52:27 GMT

Hi @Manasa Kalluri Is "latest_data" referring to "e_test".

How latest_data is created ?

manasa — Thu, 10 Mar 2022 05:25:55 GMT

Hi @Gaurav Rupnar ,

latest_data=spark.read.format('csv').option("header","true").option("multiLine","true").load(file_path)

latest_data is a simple dataframe of a small csv file

shan_chandra — Thu, 31 Mar 2022 02:10:40 GMT

Hi, @Manasa, could you please check SPARK-38318 and use Spark 3.1.2, Spark 3.2.2, or Spark 3.3.0 to allow cyclic reference?

jose_gonzalez — Mon, 11 Apr 2022 18:36:25 GMT

Hi @Manasa Kalluri ,

Just a friendly follow-up. Do you still need help with this issue?

manasa — Wed, 27 Apr 2022 07:39:46 GMT

No, it was resolved.Thanks!