I am designing a metadataโdriven ETL framework to migrate approximately 500 tables from Db2 to PostgreSQL.After reviewing multiple design patterns and blog posts, I am uncertain about the recommended approach for storing ETL metadata such as source schema, target schema, transformation logic, and test cases. Should this metadata be stored in database tables or in external configuration files such as YAML?In addition, what is the recommended orchestration pattern for this type of migration? Is it preferable to have a dedicated workflow or pipeline per target gold table, or a more generic workflow that loops over a metadata list of 500 tables and processes them in a parameterized way?
