Add data manually to DLT

[Gilg]

Hi Team,

Is there a way that we can add data manually to the tables that are generated by DLT?

We have done a PoC using DLT for Sep 15 to current data. Now, that they are happy, they wanted the previous data from Synapse and put into Databricks.

I can easily do this by pointing to the root of the source folder, but the files are group via batchid/year/month/date sub-folders.

The only way that I can think of and probably the easiest is to look directly to delta table created by Synapse and have a separate table in Databricks.

Ideally, I want to only have 1 table rather than 2 tables. i.e. DimRetailer (databricks) instead of DimRetailer_synapse and DimRetailer (databricks)

Cheers,

G

[Kaniz]

Hi @Gilg, Yes, you can add data manually to the tables generated by DLT (Delta Live Tables).

However, it would be best to be careful not to directly modify, add, or delete Parquet data files in a Delta table, as this can lead to lost data or table corruption.

If you want to add previous data from Synapse into Databricks, you can create a pipeline that reads the data from Synapse and writes it to the Delta table in Databricks.