filipniziol
Esteemed Contributor

Hi @brickster ,

The error message in the screenshot indicates that there is an issue with casting sizeInBytes from STRING to BIGINT related to the SnapshotState in Delta Lake. This is not caused by the columns you are trying to create in your Delta table but rather relates to internal metadata managed by Delta Lake.

What it means is that most likely the metadata of Delta table is corrupt. For example you created the table, then you dropped it, but still there are some leftover files in the table location.

You need to clean-up the table location before recreating it to make sure there are no older files anymore.

Here are the steps:
1. Table creation (this is already done):

 

%sql
CREATE TABLE test_transact (transaction_id string, post_date date)

 

2. Check the table location

 

%sql
DESCRIBE DETAIL test_transact;

 

filipniziol_0-1725184349733.png

 -> this is what you want to clean-up before recreating the table

3. Drop the table -> if the metadata is corrupt, some files will not be removed

 

%sql
DROP TABLE IF EXISTS test_transact

 

4. Show the content of the location (copy-paste from point 2.)

 

display(dbutils.fs.ls("dbfs:/user/hive/warehouse/test_transact"))

 

If the metadata is corrupt, you will see some files there even after DROP TABLE:

filipniziol_1-1725184567975.png

5. Clean-up the TABLE location:

 

dbutils.fs.rm("dbfs:/user/hive/warehouse/test_transact", recurse=True)

 

6. Try now to recreate the table and insert the data.