Databricks Community

Hi Team,We have df has 70 M row count and we are calling an API for 6000 rows per set using df.repartition(rep_count).foreachpartition(func_name) so in func_name we are calling API post req for that partition but when we are trying to print/log somet...

Hi Team,I want to get realtime log for cluster executor and driver stderr/stdout log while performing data operations and save those log in catalog's volume