Louis_Frolio
Databricks Employee
Databricks Employee

Greetings @dplatform_user , I did some digging and found a few helpful hints/tips for you to consider. 

What's happening

You're hitting UNRESOLVED_ROUTINE: Cannot resolve routine isNotNull on DBR 16.4 during a DEEP CLONE. Same clone works on 13.3. Simpler tables are fine.

This is a known 16.x bug — not a missing function, not a UC permissions issue. On 16.x, Spark sessions can have their in-memory function registry cleared and then get reused. When Delta's internal clone path tries to invoke built-ins like isNotNull (for stats collection, constraint validation, etc.), the planner can't find them. Fixes are in 17.2+ with backports planned for 16.4 via a feature flag.

Rule out the simple stuff first

Run these on the same 16.4 cluster in a fresh notebook:

SELECT isnotnull(1);
SHOW FUNCTIONS LIKE 'isNotNull';

If isnotnull(1) works and the only result from SHOW FUNCTIONS is a system.builtin entry, you're in the known bug bucket.

Workarounds

  1. Run the clone from a fresh job cluster, not a long-lived all-purpose cluster that's been running other jobs or retries.

  2. Test on 17.2+ if available. If it works there and not on 16.4, that confirms the bug and strengthens your case for an ES ticket.

  3. If you need the copy now and don't need incremental refresh, use CTAS and manually add back constraints:

CREATE TABLE target AS SELECT * FROM source;

 

Hope this helps, Louis

View solution in original post