Questions regarding catalogs starting with __databricks_internal_catalog

[Seunghyun]

Hello,

I am a Data Engineer currently building a data platform in a Databricks-AWS environment.

While querying system.information_schema.catalog_privileges, I noticed several catalogs starting with __databricks_internal_catalog. Specifically, there are catalogs named in the format of __databricks_internal_catalog_lakeview_<username>.

It appears that these are temporary catalogs created while using AI Dashboards (formerly Lakeview). Regarding this, I would like to ask the following questions:

1. Creation Purpose and Process: What is the specific reason these catalogs are created, and what is the internal process behind their generation?

2. Management and Governance: Since these catalogs are likely generated whenever different users with various permissions view a dashboard, how does Databricks manage these at scale internally? (e.g., lifecycle management, visibility control, or cleanup policies).

I would appreciate any technical insights into how these internal catalogs function within the Unity Catalog framework.

[stbjelcevic · Accepted solution]

Hi @Seunghyun,

Databricks creates some system‑managed, internal Unity Catalog objects for certain features. They’re reserved for platform use and not intended for customer direct querying or permission management.

If you encounter the “__databricks_internal” namespace, treat it as a Databricks‑managed implementation detail. My best guess is it likely has to do with performance and permissions, but I can't share details on exactly how they are managed at scale.

[Seunghyun]

Thanks for answer.

I see. Since these catalogs are related to dashboards, do they specifically function as a mechanism for dashboard data caching? Additionally, are there any other potential side effects or impacts caused by the existence of these catalogs?

[Sumit_7]

@Seunghyun,

- No — __databricks_internal_catalog_lakeview_<user> catalogs are not a dashboard caching mechanism
- Yes — they indirectly participate in execution-time optimizations, but they do not store cached results
- No meaningful negative side effects for customers when understood correctly

Please verify @stbjelcevic 

[stbjelcevic]

@Sumit_7 ,

Yes, this is pretty much all correct - they are not caches of prior results, and they don’t alter data or dashboard behavior. They may appear to metastore admins in system tables and count toward standard UC quotas, which I guess could be considered an adverse side effect if vast numbers of assets are being created. Still, I doubt the average user would encounter any issue with their existence.

[Seunghyun]

Thanks for everybody!!

@stbjelcevic @Sumit_7