topic Re: Getting driver error for my job when migrating to Unity in Data Engineering

Getting driver error for my job when migrating to Unity

utkarshamone — Fri, 13 Mar 2026 08:31:53 GMT

I am in the process of migrating our jobs from the legacy hive metastore to Unity. I have modified my existing job to read and write from a different bucket as part of the migration. The only change I have made to my job config is to enable this setting -

data_security_mode = "USER_ISOLATION"

As per my understanding this enables access to Unity Catalog on the cluster
I am running the latest DBR

But I keep getting this error, which I cannot understand and solve
Any help is appreciated!

Thanks in advance!

Re: Getting driver error for my job when migrating to Unity

utkarshamone — Fri, 13 Mar 2026 12:16:38 GMT

Quick correction here - I am using DBR 16.4 LTS (Scala 2.12, Spark 3.5.2)

Re: Getting driver error for my job when migrating to Unity

utkarshamone — Thu, 19 Mar 2026 10:13:45 GMT

Hi all!
Who can help me with this? A bit stuck here

Re: Getting driver error for my job when migrating to Unity

balajij8 — Thu, 19 Mar 2026 14:42:53 GMT

You can validate using "SINGLE_USER"