Databricks Community

MBV3 · 11-24-2022

What is the best way to delete files from the gcp bucket inside spark job?

MBV3 · 10-31-2022

Hi,I have data in parquet format in GCS buckets partitioned by name eg. gs://mybucket/name=ABCD/I am trying to create a table in Databaricks as followsDROP TABLE IF EXISTS name_test; CREATE TABLE name_testUSING parquetLOCATION "gs://mybucket/name=*/...

MBV3 · 11-01-2022

CREATE TABLE name_test ( id INT, other STRING ) PARTITIONED BY ( name STRING); COPY INTO name_test FROM "gs://<my_bucket>/" FILEFORMAT = parquet; This combination worked, after some trial and error.Thanks for your help Pat.

MBV3 · 11-01-2022

Hi PatThanks for your reply.I tried your approach but it didn't worked getting the following exceptionError in SQL statement: AnalysisException: Cannot use all columns for partition columns com.databricks.backend.common.rpc.DatabricksExceptions$SQLEx...

Databricks Community

User Stats

User Activity

Delete a file from GCS folder

External table from parquet partition

Re: External table from parquet partition

Re: External table from parquet partition