Thanks but I found what I was looking for in THIS article. This shows how to access a Metadata Delta table which is maintained outside the pipeline and convert the data to a dictionary that can be used to do different processing within the pipeline.