Options
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
04-24-2025 09:38 AM
Yes, Databricks will still open and parse the JSON files, even if you're only selecting _metadata.
It must infer schema and perform basic parsing, unless you explicitly avoid it.
So, even if you do:
.select("_metadata")
It doesn't skip reading the file contents — it still downloads, parses, and caches to process the data.
LR