cancel
Showing results for 
Search instead for 
Did you mean: 
Community Platform Discussions
Connect with fellow community members to discuss general topics related to the Databricks platform, industry trends, and best practices. Share experiences, ask questions, and foster collaboration within the community.
cancel
Showing results for 
Search instead for 
Did you mean: 

Identity column has null values

rachit-prodigal
New Contributor

I created a table in databricks using a dbt model pre hook 

 

CREATE TABLE IF NOT EXISTS accounts (
account_id BIGINT GENERATED ALWAYS AS IDENTITY,
description STRING
other columns
)

I use the same dbt model to merge values into this table in the post-hook

 

post_hook = [
"
MERGE INTO accounts AS target
USING incremental_accounts AS source
ON <join-condition>
WHEN MATCHED THEN UPDATE SET
target.description = source.description
WHEN NOT MATCHED BY target THEN
INSERT(description)
VALUES (source.description)
"

the model code itself is simple in the file incremental_accounts.sql

SELECT description from new_accounts

The table accounts was created by this model and only updated by this model by the post-hook queries.

The problem is that I noticed null values in account_id which should not be the case as it is auto-generated. 

I identified the same issue with 4 other tables which were built similarly using dbt models.

 

Can you please help me with this issue. 

 

0 REPLIES 0

Connect with Databricks Users in Your Area

Join a Regional User Group to connect with local Databricks users. Events will be happening in your city, and you won’t want to miss the chance to attend and share knowledge.

If there isn’t a group near you, start one and help create a community that brings people together.

Request a New Group