nikhilj0421
Databricks Employee
Databricks Employee

Hi @OmarSEKkOUTI , you can leverage DLT feature to do so. 

Please check: https://docs.databricks.com/aws/en/dlt/transform

https://docs.databricks.com/aws/en/dlt/stateful-processing

Here is the step-by-step tutorial: https://docs.databricks.com/aws/en/dlt/tutorials

You can create a table and partition it by date, and you can use the Medallion architecture of the DLT. 

Partitions can be like below:

nikhilj0421_0-1748177623308.png