Unable to create a partitioned table on s3 data

[Constantine]

I write data to s3 like

data.write.format("delta").mode("append").option("mergeSchema", "true").save(s3_location)

and create a partitioned table like

CREATE TABLE IF NOT EXISTS demo_table
USING DELTA
PARTITIONED BY (column_a)
LOCATION {s3_location};

which throws an error

Error in SQL statement: AnalysisException: It is not allowed to specify partitioning when the table schema is not defined.

Is there a different way to write to s3 location so that I can solve this issue

[Hubert-Dudek · Accepted solution]

@John Constantine​ , In CREATE TABLE, you need to specify fields:

CREATE TABLE IF NOT EXISTS demo_table
    (column_a STRING,
    number INT)
USING DELTA PARTITIONED BY
     (column_a)
LOCATION 
   {s3_location};

and when you save data before creating a table, it should include a partition. I think it better first register the table and then write to it.

[Kaniz]

Hi @John Constantine​ , Did the above suggestions provided by @Hubert Dudek​ help your case?