Partitioning or Processing : Reading CSV file with size of 5 to 9 GB
Options
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
10-09-2023 05:12 AM
Hi Team,
Would you please guide me on
Instance with 28GB and 8 Cores
1. how data bricks reading 5 to 9GB files from BLOB storage ? ( directly loaded full file into one nodes memory )
2. howmany tasks will be created based on Core ? how many executors will be allocated ? How task designed per size of files.
Regards,Nantha