topic Re: Intelligent source to target mapping in Data Engineering

Intelligent source to target mapping

ShankarM — Tue, 30 Jul 2024 04:40:59 GMT

I want to implement source to target mapping in such a way that source and target columns are auto mapped using intelligent AI mapping resulting in reduction of mapping efforts especially when there are 100+ columns in a table. Metadata information of source and target are provided by the user and the system maps the columns automatically and populates them.

How can we do this in Databricks?

Re: Intelligent source to target mapping

ShankarM — Thu, 01 Aug 2024 04:00:41 GMT

Hi @Retired_mod thanks for your reply.

Can you let me know how auto mapping of columns can be implemented using AI tools in Databricks. Do you have any data architecture explaining the flow. What information needs to be inputted so that suggestions can be done by the model and user can get a UI to approve the mapping. I need to implement this logic for the customer.

Re: Intelligent source to target mapping

ShankarM — Wed, 07 Aug 2024 04:01:14 GMT

Can you please reply to my latest follow up question?