- 2874 Views
- 1 replies
- 1 kudos
Surrogate key using identity column.
I want to create a surrogate in the delta table And i used the identity column id-Generated as DefaultCan i insert rows into the delta table using only spark.sql like Insert query ? or i can also use write delta format options? If i use the df.write ...
- 2874 Views
- 1 replies
- 1 kudos
- 1 kudos
Hello @Menaka Murugesan​ ,If you are using the identity column, I believe you would have created the table as below, (starts with value 1 and step 1)CREATE TABLE my_table ( id INT IDENTITY (1, 1) PRIMARY KEY, value STRING )You can insert values i...
- 1 kudos
- 8283 Views
- 1 replies
- 0 kudos
How to create a surrogate key sequence which I can use in SCD cases?
Hi Community I would like to know if there is an option to create an integer sequence which persists even if the cluster is shut down. My target is to use this integer value as a surrogate key to join different tables or do Slowly changing dimensio...
- 8283 Views
- 1 replies
- 0 kudos
- 0 kudos
Hi @pascalvanbellen ,There is no concept of FK, PK, SK in Spark. But Databricks Delta automatically takes care of SCD type scenarios. https://docs.databricks.com/spark/latest/spark-sql/language-manual/merge-into.html#slowly-changing-data-scd-type-2 ...
- 0 kudos