Materialized View backing pipeline retains old Unity Catalog after catalog rename (dev_cat → dap_catalog_dev) resulting in NO_SUCH_CATALOG_EXCEPTION

This issue blocks the creation and refresh of Materialized Views after a Unity Catalog rename.

Workspace Information

Pipeline Run ID

5337905b-2d28-45f0-8f40-b9279943436a

Cluster used during validation

0722-134910-ls3f4px7-v2n

Problem Summary

We renamed our Unity Catalog from:

dev_cat

to

dap_catalog_dev

using the Databricks UI.

Following the rename, existing Materialized Views became invalid.

Even after dropping the Materialized View and recreating it in the renamed catalog, Databricks continues referencing the old catalog (dev_cat) while validating the backing managed pipeline.

As a result, creation/validation of the Materialized View fails.

Error

Encountered an error with Unity Catalog while setting up the pipeline on cluster 0722-134910-ls3f4px7-v2n.

Ensure that your Unity Catalog configuration is correct, and that required resources (e.g., catalog, schema) exist and are accessible.

Also verify that the cluster has appropriate permissions to access Unity Catalog.

Details:

[NO_SUCH_CATALOG_EXCEPTION]
Catalog 'dev_cat' was not found.

SQLSTATE: 42704

Environment

• Unity Catalog enabled workspace
• Materialized Views
• Serverless Pipeline
• Managed Pipeline generated by Materialized View
• Pipeline uses Photon
• User has Workspace Admin privileges

Steps Performed

Step 1

Originally created Materialized View under

dev_cat

Everything functioned correctly.

Step 2

Renamed catalog

dev_cat

to

dap_catalog_dev

using the Databricks UI.

Step 3

Dropped the original Materialized View.

The SQL object was successfully removed.

However, the backing managed pipeline still exists.

Step 4

Attempted to recreate the Materialized View under

dap_catalog_dev

using

CREATE OR REPLACE MATERIALIZED VIEWdap_catalog_dev.caltrack.vw_item_17_electric_hourly_materializedAS...

Pipeline validation immediately fails.

Investigation Performed

We inspected the generated pipeline YAML.

The managed pipeline still contains references to the old catalog.

Pipeline Name

MV-dev_cat.caltrack.vw_item_17_electric_hourly_materialized

Pipeline configuration contains

catalog: dev_cat

Managed SQL definition

CREATE MATERIALIZED VIEWdev_cat.caltrack.vw_item_17_electric_hourly_materialized

However, the source dataset references the renamed catalog

FROMdap_catalog_dev.caltrack.vw_item_17_electric_hourly

Additionally,

dataset_name:
dap_catalog_dev.caltrack.vw_item_17_electric_hourly_materialized

This results in inconsistent metadata.

Summary of pipeline metadata:

Additional Observation

Running

DESCRIBE EXTENDED
dap_catalog_dev.caltrack.vw_item_17_electric_yearly_agg_materialized;

returns

TABLE_DOES_NOT_EXIST

Table

dev_cat.caltrack.__materialization_mat_...

does not exist

This indicates the hidden internal materialization table still references the old catalog.

Expected Behaviour

After renaming a Unity Catalog,

or after dropping and recreating the Materialized View,

the managed pipeline should be regenerated with

catalog = dap_catalog_dev

and all internal metadata should reference the renamed catalog.

Actual Behaviour

The managed pipeline continues using

catalog = dev_cat

which no longer exists.

Validation therefore fails with

NO_SUCH_CATALOG_EXCEPTION

Troubleshooting Already Performed

• Confirmed catalog rename completed successfully.
• Confirmed new catalog exists.
• Confirmed schemas exist.
• Confirmed user has administrative permissions.
• Dropped the Materialized View.
• Waited for cleanup.
• Recreated the Materialized View.
• Verified pipeline still exists.
• Inspected generated pipeline YAML.
• Verified stale catalog references.
• Verified hidden materialization table still references the old catalog.