cancel
Showing results forย 
Search instead forย 
Did you mean:ย 
Administration & Architecture
Explore discussions on Databricks administration, deployment strategies, and architectural best practices. Connect with administrators and architects to optimize your Databricks environment for performance, scalability, and security.
cancel
Showing results forย 
Search instead forย 
Did you mean:ย 

System tables in Azure with private link

Matt4
New Contributor III

Hi,

Is there any restriction on what is able to be enabled with System tables when using Azure Databricks with a private link connection?  And is there a way around the restriction (that is secure)?

In a public network I am able to enable the following in Azure (North Europe)

{"schemas":[{"schema":"storage","state":"ENABLE_COMPLETED"},
{"schema":"operational_data","state":"UNAVAILABLE"},
{"schema":"access","state":"ENABLE_COMPLETED"},
{"schema":"billing","state":"ENABLE_COMPLETED"},
{"schema":"compute","state":"ENABLE_COMPLETED"},
{"schema":"marketplace","state":"ENABLE_COMPLETED"},
{"schema":"lineage","state":"ENABLE_COMPLETED"},
{"schema":"information_schema","state":"ENABLE_COMPLETED"}]}*

In a private network in UK South, I get the following options:

{"schemas":[{"schema":"operational_data","state":"UNAVAILABLE"},{"schema":"access","state":"ENABLE_COMPLETED"},
{"schema":"billing","state":"ENABLE_COMPLETED"},
{"schema":"compute","state":"ENABLE_INITIALIZED"},
{"schema":"marketplace","state":"ENABLE_COMPLETED"},{"schema":"information_schema","state":"ENABLE_COMPLETED"}]}*

The compute one specifically says the following when trying to enable it.  Anyone know why it is unavailable?

{"error_code":"INTERNAL_ERROR","message":"System schema \"compute\" is not currently available for enablement","details":[{"@type":"type.googleapis.com/google.rpc.RequestInfo","request_id":"624a5858-4969-4d1f-9704-c4213818d69e","serving_data":""}]}*

Thanks in advance

1 ACCEPTED SOLUTION

Accepted Solutions

Matt4
New Contributor III

Turns out it was an issue with Microsoft somewhere, they've now fixed it and all the system schemas have been enabled that I can enable

"schemas":[{"schema":"operational_data","state":"UNAVAILABLE"},{"schema":"access","state":"ENABLE_COMPLETED"},{"schema":"billing","state":"ENABLE_COMPLETED"},{"schema":"compute","state":"ENABLE_COMPLETED"},{"schema":"marketplace","state":"ENABLE_COMPLETED"},{"schema":"information_schema","state":"ENABLE_COMPLETED"}]}*

View solution in original post

2 REPLIES 2

Debayan
Databricks Employee
Databricks Employee

Hi, There are few restrictions, such as To enable system tables, you might need to grant network access to the system tables Blob storage endpoint. You can refer to https://learn.microsoft.com/en-us/azure/databricks/administration-guide/system-tables/ and https://learn.microsoft.com/en-us/azure/databricks/security/network/classic/private-link-standard

Matt4
New Contributor III

Turns out it was an issue with Microsoft somewhere, they've now fixed it and all the system schemas have been enabled that I can enable

"schemas":[{"schema":"operational_data","state":"UNAVAILABLE"},{"schema":"access","state":"ENABLE_COMPLETED"},{"schema":"billing","state":"ENABLE_COMPLETED"},{"schema":"compute","state":"ENABLE_COMPLETED"},{"schema":"marketplace","state":"ENABLE_COMPLETED"},{"schema":"information_schema","state":"ENABLE_COMPLETED"}]}*

Connect with Databricks Users in Your Area

Join a Regional User Group to connect with local Databricks users. Events will be happening in your city, and you wonโ€™t want to miss the chance to attend and share knowledge.

If there isnโ€™t a group near you, start one and help create a community that brings people together.

Request a New Group