cancel
Showing results for 
Search instead for 
Did you mean: 
Administration & Architecture
cancel
Showing results for 
Search instead for 
Did you mean: 

Consequences of removing a workspace from a metastore in Azure Databricks

m997al
Contributor

In the documentation (Enable a workspace for Unity Catalog - Azure Databricks | Microsoft Learn), it appears that I can remove a workspace from a metastore, and as long as the workspace has jobs that don't use tables, files, and models stored in any catalog, then after unlinking from the metastore all the jobs in the workspace should still run.

In my case the jobs are not using any local data in Databricks, they are reaching out to some cloud databases and performing operations there.

I'm also assuming that the only models that will disappear are the registered models in the metastore (in a catalog).  But models that mlflow has logged to dbfs will still be there after the unlinking.

Just trying to confirm this prior to unlinking.

Why unlink?  Because apparently we can only have one metastore per Azure region (regardless of Azure subscription).  So we are reconfiguring given that information and will create a new metastore in another Azure subscription, same Azure region, given our needs.

1 ACCEPTED SOLUTION

Accepted Solutions

Kaniz
Community Manager
Community Manager

Hi @m997al , Based on the information provided, if you unlink a workspace from a metastore, the jobs in the workspace that do not depend on tables, files, and models stored in any catalog should still run. This is because the jobs are not using any local data in Databricks, but are instead interacting with cloud databases. 

As for the models, only the registered models in the metastore (in a catalog) should disappear after unlinking. Models that MLflow has logged to dbfs should still be accessible. 

However, please note that changing the default catalog can break existing data operations that depend on it. So, if your jobs depend on the default catalog, they might fail after unlinking.

View solution in original post

1 REPLY 1

Kaniz
Community Manager
Community Manager

Hi @m997al , Based on the information provided, if you unlink a workspace from a metastore, the jobs in the workspace that do not depend on tables, files, and models stored in any catalog should still run. This is because the jobs are not using any local data in Databricks, but are instead interacting with cloud databases. 

As for the models, only the registered models in the metastore (in a catalog) should disappear after unlinking. Models that MLflow has logged to dbfs should still be accessible. 

However, please note that changing the default catalog can break existing data operations that depend on it. So, if your jobs depend on the default catalog, they might fail after unlinking.

Welcome to Databricks Community: Lets learn, network and celebrate together

Join our fast-growing data practitioner and expert community of 80K+ members, ready to discover, help and collaborate together while making meaningful connections. 

Click here to register and join today! 

Engage in exciting technical discussions, join a group with your peers and meet our Featured Members.