If you are using API, you can set the logging using the below piece of information in your json body.
https://docs.microsoft.com/en-us/azure/databricks/dev-tools/api/latest/examples#cluster-log-example
"cluster_log_conf": {
"dbfs": {
"destination": "dbfs:/cluster-logs"
}
},
