Databricks Community

hi , i am running the below query in databricks , first job5 created with 10 partitions .and again job6 started where actual processing started.in job5 is it identifying schema , when schema check will be done for the new dataset. is it checked by dr...

Hi, as per spark internals, once count function executed in worker nodes , one of the worker node collect all the count of records and do summation ?or count of records from all worker nodes passed to driver node. and summation done driver node side....