Partitioning or Processing : Reading CSV file with size of 5 to 9 GB

NanthakumarYoga
New Contributor III

Hi Team,

Would you please guide me on

Instance with 28GB and 8 Cores

1. how data bricks reading 5 to 9GB files from BLOB storage ? ( directly loaded full file into one nodes memory )

2. howmany tasks will be created based on Core ? how many executors will be allocated ? How task designed per size of files.

Regards,Nantha