Hi Emma,
thx. My problem regarding CPU is: i have a workload that need much memory per task and there seem to be no compute (on azure) with more than 8GB per Core.
But i found a solution that indeed works: "spark.task.cpus 2" (so the settings work in general - my assumption was wrong)
this doubles the available ram per task - but halfs CPU utilization. but this is fine for the moment.
Regarding the log setting: i assume that this setting is also set by again my expectation about result is wrong: in run "output" i see all errors and i want to see my logs there - i know guess that it will always only show error logs or print and never user log regardless the setting.