AutoLoader - process multiple files
Options
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
12-13-2022 07:27 AM
I need to process files of different schema coming to different folders in ADLS using Autoloader. Do I need to start a separate read stream for each file type / folder or can this be handled using a single stream ?
When I tried using a single stream, multiple files gets clubbed in the same read and its quite difficult to process the files as the schema is different.
Labels:
- Labels:
-
Autoloader
-
Different Schema