Dataframe to update subset of fields in table...

plynton — Mon, 03 Oct 2022 20:23:55 GMT

I have a table that I'll update with multiple inputs (csv). Is there a simple way to update my target when the source fields won't be a 1:1 match? Another challenge I've run into is that my sources don't have a header field, though I guess I could rename the columns if this would be beneficial.

At this point, I've created a blank dataframe based on a schema and then tried to .copy the source columns into the empty df. However, when I do this I get a warning from DB telling me that this is inefficient.

Any help greatly appreciated.

Re: Dataframe to update subset of fields in table...

Hubert-Dudek — Tue, 11 Oct 2022 14:07:22 GMT

Read your CSV as a dataframe and than make update using Merge (upsert).

topic Re: Dataframe to update subset of fields in table... in Data Engineering

Dataframe to update subset of fields in table...

Re: Dataframe to update subset of fields in table...