topic How to add Current date as one of the column in Databricks in Data Engineering

How to add Current date as one of the column in Databricks

Neli — Fri, 20 Jan 2023 04:33:37 GMT

I am trying to create new column "Ingest_date" in table which should contain current date. I am getting error "Current date cannot be used in a generated column". Can you please review and suggest alternative to get the current date in delta table.

Re: How to add Current date as one of the column in Databricks

daniel_sahal — Fri, 20 Jan 2023 07:59:22 GMT

A generation expression can use any SQL functions in Spark that always return the same result when given the same argument values

Source: https://docs.delta.io/latest/delta-batch.html#use-generated-columns

It means that it's intended to not work.

You can still insert the value of current_date() in your ingestion pipeline.

Re: How to add Current date as one of the column in Databricks

Neli — Fri, 20 Jan 2023 19:39:55 GMT

Thank you Daniel. Appreciate your inputs.

This is the requirement which i am trying to achieve. I will create a table with 5 columns and 4 columns will be inserted by process . Ingest date column should be populated with current date when columns are inserted.