07-21-2022 11:51 PM
Hi ,
I am having a delta table and table contains data and I need to alter the datatype for a particular column.
For example :
Consider the table name is A and column name is Amount with datatype Decimal(9,4).
I need alter the Amount column datatype from Decimal(9,4) to Decimal(18,4).
Is there any ALTER command similar to what we have in SQL server to alter the column datatype.
Can anyone suggest any ideas on this.
07-19-2023 12:20 PM
Hi @HariharaSam
The following documents the info about how to alter a Delta table schema.
https://docs.databricks.com/delta/update-schema.html
07-25-2022 12:45 AM
At the moment you will have to rewrite the data as you cannot change the type of a column.
https://docs.delta.io/latest/delta-batch.html#change-column-type-or-name
But (others may correct me if I'm wrong here) I think type change is in the dev backlog.
08-30-2022 02:02 PM
Hi @Hariharan Sambath Please check following document: https://docs.databricks.com/spark/2.x/spark-sql/language-manual/alter-table-or-view.html#change-colu...
04-12-2023 09:13 AM
from pyspark.sql.functions import *
spark.read.table("schema.table") \
.withColumn("namecolumn",col("namecolumn").cast("string")) \
.write.format("delta").mode("overwrite").option("overwriteSchema",true).saveAsTable("schema.table")
07-17-2023 10:36 AM
.write.format("delta").mode("overwrite").option("overwriteSchema",true).saveAsTable("schema.table")
Need to add double quotes around true..
.option("overwriteSchema","true")
Key-value should be string, not Boolean
07-19-2023 12:11 PM
You will need to write the data to a new table. Read the data in spark and cast the data and write it to a new table.
07-19-2023 12:20 PM
Hi @HariharaSam
The following documents the info about how to alter a Delta table schema.
https://docs.databricks.com/delta/update-schema.html
Join a Regional User Group to connect with local Databricks users. Events will be happening in your city, and you won’t want to miss the chance to attend and share knowledge.
If there isn’t a group near you, start one and help create a community that brings people together.
Request a New Group