Options
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
01-23-2024 08:08 AM - edited 01-23-2024 08:09 AM
Hi @ShlomoSQM - Please refer to the below steps and let us know if it works
1) Add a column(with column) for filename during readStream data from autoloader using input_file_name() function
2) split the dataframe (df to df1, df2) into two based on type1 and type2 (using file name contains type1.. say for example) using schema 1 and schema 2.
3) using merge statement, insert/update the data to the table1 and table2 with the respective dataframes for type1(df1), df2.