Databricks Autoloader streamReader does not include the partition column as part of output.
Options
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
02-11-2023 06:20 AM
I have folder structure at source such as
/transaction/date_=2023-01-20/hr_=02/tras01.csv
/transaction/date_=2023-01-20/hr_=03/tras02.csv
Where 'date_' and 'hr_' are my partitions and present in the dataset as well. But the streamReader does not read these columns as output. Whereas the same file if I read as CSV the column shows. Am I missing something here?
Please help!
Labels: