Recover a deleted DLT pipeline

[standup1]

Hello,

does anyone know how to recover a deleted dlt pipeline, or at least recover deleted tables that were managed by the dlt pipeline ? We have a pipeline that stopped working and throwing all kind of errors, so we decided to create a new one and deleted the bad one. However, all tables were deleted along with the pipeline. Luckily, this was just a testing pipeline, but we still can’t figure out how to recover those tables. Any information you can share is greatly appreciated. Thank you 

[Kaniz]

Hi @standup1, 

• Unfortunately, once a DLT pipeline is deleted, it cannot be directly restored. However, you can create a new pipeline to replace the old one.
• To avoid losing data, consider following these best practices:
  • Version Control: Maintain version control for your pipelines. This way, you can revert to a previous version if needed.
  • Backup Configuration: Regularly back up your pipeline configurations, including table definitions and transformations.
  • Pipeline Monitoring: Set up monitoring and alerts to catch issues early and prevent data loss.
  • The critical part is recovering the tables associated with the deleted pipeline. Here’s how you can approach this:
    • Refresh Failed Tables:
      • In Databricks, you can use the Refresh Failed Tables option to recover from the point of failure in DLT.
      • On the Pipeline details page, click Refresh failed tables.
      • If you want to update only specific tables, click next to the button and select the desired tables for refresh.
    • Refer to the official documentation for more details on how to perform this operation .

[standup1]

Thank you, Kanzi. Just to confirm that I understood you correctly. If the pipeline is deleted [like in our case] without having version control, backup configuration..etc already implemented. There's no way to recover those tables, not the pipeline. my concern is just the tables/data.