topic Re: SELECT from table saved under path in Data Engineering

SELECT from table saved under path

johnb1 — Fri, 27 Jan 2023 14:04:04 GMT

Hi!

I saved a dataframe as a delta table with the following syntax:

(test_df
  .write
  .format("delta")
  .mode("overwrite")
  .save(output_path)
)

How can I issue a SELECT statement on the table?

What do I need to insert into [table_name] below?

SELECT * FROM [table_name]

Ajay-Pandey — Fri, 27 Jan 2023 16:20:19 GMT

Hi @John B there is two way to access your delta table-

SELECT * FROM delta.`your_delta_table_path`

df.write.format("delta").mode("overwrite").option("path", "your_path").saveAsTable("table_name")

Now you can use your select query-

SELECT * FROM [table_name]

johnb1 — Fri, 27 Jan 2023 16:49:55 GMT

Hi @Ajay Pandey

When trying the first option I get:

Ajay-Pandey — Sat, 28 Jan 2023 04:49:59 GMT

Hi @John B in the above code instead of your path variable put your absolute path. Please refer to snapshot-

Ajay-Pandey — Sat, 28 Jan 2023 04:59:11 GMT