Hello @Vamsi_S 

Good day! Did you Preprocessing Table Names in CI/CD and Generate YAML Dynamically (Recommended for Dynamic, Automated Ingestion)

Did you contact your databricks account manager (incase if you working with a company) for feature request?

Hey @Khaja_Zaffer 

Thanks for sharing your thoughts. Yes, dynamic generation of YAML works in this case, but I wanted to know if there is any other workaround that is officially made available by Databricks.

I did not contact anyone from Databricks. Before reaching out I wanted to try all different possibilities. 
Thank you again!

I’ve been using the notebook style as there are some transformations that need to be done on destination_table names as the source_table names has special characters. My requirement is: for example, I have a schema with 5 tables, 10 individual tables from another schema. Since each table name needs to be transformed, I am getting all the tables and creating a list of objects. In the first run if the number of objects is 15 it creates pipeline with the name specified in the notebook. And by the second run of the pipeline, a new table gets created in the schema, we will have 16 objects which changes the ingestion definition of the pipeline. So, the second run creates a new pipeline with name “Managed Ingestion Pipeline - <original-pipeline_id>”. This has been the issue as we are not able to capture the history of the pipeline properly. I just want a way to update the ingestion definition of the pipeline but keep same pipeline name. Find the link to the documentation of notebook style Ingestoin Pipelines with Lakeflow Connect below.
SQL Server CDC Connector Setup

This is a code snippet I am working on:

Please advise if anyone has a solution to this issue.