Options
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
04-03-2026 02:57 PM - edited 04-03-2026 03:01 PM
Hi ,
Bingooo!!!
.option("pathGlobFilter", "*.json") \ # <-- ONLY pick .json files WORKED FOR ME.
I read the documents throrughly and understood what ("cloudFiles.format", "json") it actually does.It tells autoloader
to parse the incoming file as json but pathGlobFilter picks up only specified format out of all formats (csv,xml etc).In the input directory I have .csv file with 33 records.
Now it inserts only 2 records (without nulls)
Thanks a lot for your time and efforts to solve this issue.