topic AnalysisException: [UNRESOLVED_ROUTINE] Cannot resolve routine `=` in Data Engineering

AnalysisException: [UNRESOLVED_ROUTINE] Cannot resolve routine `=`

phanvy5403 — Fri, 17 Jul 2026 04:54:21 GMT

Hi all,

We are experiencing an AnalysisException when migrating our Databricks cluster runtime from 13.3 LTS to 16.4 LTS (Scala 2.13), and would appreciate your help in identifying the root cause.

**Error:**
```
AnalysisException: [UNRESOLVED_ROUTINE] Cannot resolve routine `=` on search path
[`system`.`builtin`, `system`.`session`, `spark_catalog`.`default`].
SQLSTATE: 42883
```

**Where it fails:**
The exception is thrown inside a Scala `object` class when calling `.where()` on a DataFrame — specifically at:
```
org.apache.spark.sql.Dataset.filter(Dataset.scala:1993)
org.apache.spark.sql.Dataset.where(Dataset.scala:2021)
```
The filter uses the Column DSL (e.g. `col("job_id") === lit(jobId)`), not raw SQL strings.

**What we have tried:**
- Replacing `.where("enabled = 1")` string predicates with Column DSL equivalents (`===`, `=!=`) — same error
- Confirming `SHOW FUNCTIONS LIKE '='` returns the function as expected
- Verifying `rawDF.printSchema()` succeeds — the DataFrame itself is healthy

**Questions:**
1. Is this a known regression or breaking change in the HybridAnalyzer between DBR 13.3 LTS and 16.4 LTS?
2. Is there a recommended way to ensure built-in operators are resolved correctly when the active catalog is not `spark_catalog`?
3. Is setting `USE CATALOG spark_catalog` at session start the intended workaround, or is there a runtime/cluster config that should handle this?

Thank you

Re: AnalysisException: [UNRESOLVED_ROUTINE] Cannot resolve routine `=`

balajij8 — Fri, 17 Jul 2026 16:50:21 GMT

You can add a cluster level Spark configuration to set the default catalog at startup avoiding per session configurations.

spark.databricks.sql.initial.catalog.name spark_catalog

You can use spark.sql.path.enabled = true and explicitly set PATH to include system builtin

spark.conf.set("spark.sql.path.enabled", "true") spark.sql("SET PATH = system.builtin, system.session, spark_catalog.default")

Re: AnalysisException: [UNRESOLVED_ROUTINE] Cannot resolve routine `=`

henry_collins — Fri, 17 Jul 2026 22:09:02 GMT

It doesn't sound like the === operator itself is the issue, since it should compile into a normal Catalyst expression rather than a SQL function call. The fact that Spark is trying to resolve = as a routine makes me suspect a catalog or analyzer behavior change in DBR 16.4. check whether a custom catalog, Unity Catalog setting, or Spark extension is intercepting analysis before assuming it's a regression. If you can share a minimal reproducible example and your catalog/session configuration, it'll be much easier to narrow down.

Re: AnalysisException: [UNRESOLVED_ROUTINE] Cannot resolve routine `=`

VaDim — Tue, 04 Aug 2026 14:55:42 GMT

Having a similar issue when moving from DBR 13 to DBR 16.4 in a multi-task Job. The first task works ok but the 2nd with identical code but different params fails. For me it fails with Cannot resolve function `and` (I assume from the .between(from_date, to_date)).

Anyone solved it? or knows of a workaround?