โ06-06-2024 11:29 PM
I need guidance on migrating the Delta live table to another pipeline without a full refresh
โ06-07-2024 12:23 AM
Hi, Yes you can use the same checkpoint location in the new pipeline.
โ06-06-2024 11:45 PM
Hi delta live tables are not stored in the metastore they are stored in specified storage location, Changing the "Target" parameter in the new pipeline settings will allow you to re-register the tables in a new schema without reprocessing the data. The data will remain in the same location specified in the "Storage location" parameter, and only the table registration in the Hive Metastore will be updated to point to the new schema.
โ06-06-2024 11:49 PM
Can i migrate the checkpoint to a new pipeline?
โ06-07-2024 12:23 AM
Hi, Yes you can use the same checkpoint location in the new pipeline.
โ06-07-2024 02:11 AM
Thank you very much
โ06-07-2024 08:04 AM
Again please, I remove the table from the old pipeline. but the new pipeline was showing an error already from the old pipeline.
Join a Regional User Group to connect with local Databricks users. Events will be happening in your city, and you wonโt want to miss the chance to attend and share knowledge.
If there isnโt a group near you, start one and help create a community that brings people together.
Request a New Group