How to Overwrite Using pyspark's JDBC without loosing constraints on table columns

Abeeya
New Contributor II

Hello,

My table has primary key constraint on a perticular column, Im loosing primary key constaint on that column each time I overwrite the table , What Can I do to preserve it? Any Heads up would be appreciated

Tried Below

df.write.option("truncate", "true").jdbc(url=DATABASE_URL, table=DATABASE_TABLE, mode="overwrite", properties=DATABASE_PROPERTIES)
 

But It errored out as below when a new column was added to dataframe which was not there in table

AnalysisException: Column "new_col" not found in schema Some

Hubert-Dudek
Databricks MVP

@Abeeya .​ , Mode "truncate", is correct to preserve the table. However, when you want to add a new column (mismatched schema), it wants to drop it anyway.


My blog: https://databrickster.medium.com/

View solution in original post