topic Re: Logs from dlt-execution computes in Administration & Architecture

Logs from dlt-execution computes

lubiarzm1 — Thu, 16 Apr 2026 09:45:34 GMT

Hi guys!

I faced an issue with the permission model in Databricks.
Data engineers in my team are using a Pipeline that runs on serverless compute. The permissions for the pipeline are configured correctly, for example as follows:

resource "databricks_permissions" "pipelines_int_usage" { provider = databricks depends_on = [data.databricks_pipelines.pipeline, module.groups] for_each = { for id in local.all_pipeline_ids : id => id } pipeline_id = each.value access_control { group_name = module.groups.admins_name permission_level = "CAN_MANAGE" } access_control { group_name = module.groups.data_engineers_name permission_level = "CAN_RUN" } access_control { group_name = module.groups.data_analysts_name permission_level = "CAN_VIEW" } access_control { group_name = module.groups.deployers_name permission_level = "CAN_MANAGE" } access_control { group_name = module.groups.support_name permission_level = "CAN_MANAGE" } access_control { service_principal_name = data.databricks_service_principal.spn.application_id permission_level = "IS_OWNER" } }

And the whole configuration works correctly, but when it comes to reading the Driver Logs, the Data_Engineers group is experiencing permission issues — even the support group is unable to access them. Where should the appropriate permissions be applied to allow these users to read the logs?

Re: Logs from dlt-execution computes

Ashwin_DSA — Thu, 16 Apr 2026 10:46:51 GMT

Hi @lubiarzm1,

For Lakeflow Spark Declarative Serverless Pipelines, this isn’t controlled by the databricks_permissions block on the pipeline.

By default, only the pipeline owner and workspace admins can view the driver logs, even if other users have CAN_MANAGE / CAN_RUN on the pipeline. To let your Data_Engineers and support groups read the driver logs, you must relax the log ACL via Spark config. Check this page.

If this answer resolves your question, could you mark it as “Accept as Solution”? That helps other users quickly find the correct fix.

Re: Logs from dlt-execution computes

lubiarzm1 — Thu, 16 Apr 2026 12:24:16 GMT

Thanks a lot for help !