cancel
Showing results for 
Search instead for 
Did you mean: 
Administration & Architecture
Explore discussions on Databricks administration, deployment strategies, and architectural best practices. Connect with administrators and architects to optimize your Databricks environment for performance, scalability, and security.
cancel
Showing results for 
Search instead for 
Did you mean: 

System tables in Azure with private link

Matt4
New Contributor III

Hi,

Is there any restriction on what is able to be enabled with System tables when using Azure Databricks with a private link connection?  And is there a way around the restriction (that is secure)?

In a public network I am able to enable the following in Azure (North Europe)

{"schemas":[{"schema":"storage","state":"ENABLE_COMPLETED"},
{"schema":"operational_data","state":"UNAVAILABLE"},
{"schema":"access","state":"ENABLE_COMPLETED"},
{"schema":"billing","state":"ENABLE_COMPLETED"},
{"schema":"compute","state":"ENABLE_COMPLETED"},
{"schema":"marketplace","state":"ENABLE_COMPLETED"},
{"schema":"lineage","state":"ENABLE_COMPLETED"},
{"schema":"information_schema","state":"ENABLE_COMPLETED"}]}*

In a private network in UK South, I get the following options:

{"schemas":[{"schema":"operational_data","state":"UNAVAILABLE"},{"schema":"access","state":"ENABLE_COMPLETED"},
{"schema":"billing","state":"ENABLE_COMPLETED"},
{"schema":"compute","state":"ENABLE_INITIALIZED"},
{"schema":"marketplace","state":"ENABLE_COMPLETED"},{"schema":"information_schema","state":"ENABLE_COMPLETED"}]}*

The compute one specifically says the following when trying to enable it.  Anyone know why it is unavailable?

{"error_code":"INTERNAL_ERROR","message":"System schema \"compute\" is not currently available for enablement","details":[{"@type":"type.googleapis.com/google.rpc.RequestInfo","request_id":"624a5858-4969-4d1f-9704-c4213818d69e","serving_data":""}]}*

Thanks in advance

1 ACCEPTED SOLUTION

Accepted Solutions

Matt4
New Contributor III

Turns out it was an issue with Microsoft somewhere, they've now fixed it and all the system schemas have been enabled that I can enable

"schemas":[{"schema":"operational_data","state":"UNAVAILABLE"},{"schema":"access","state":"ENABLE_COMPLETED"},{"schema":"billing","state":"ENABLE_COMPLETED"},{"schema":"compute","state":"ENABLE_COMPLETED"},{"schema":"marketplace","state":"ENABLE_COMPLETED"},{"schema":"information_schema","state":"ENABLE_COMPLETED"}]}*

View solution in original post

2 REPLIES 2

Debayan
Esteemed Contributor III
Esteemed Contributor III

Hi, There are few restrictions, such as To enable system tables, you might need to grant network access to the system tables Blob storage endpoint. You can refer to https://learn.microsoft.com/en-us/azure/databricks/administration-guide/system-tables/ and https://learn.microsoft.com/en-us/azure/databricks/security/network/classic/private-link-standard

Matt4
New Contributor III

Turns out it was an issue with Microsoft somewhere, they've now fixed it and all the system schemas have been enabled that I can enable

"schemas":[{"schema":"operational_data","state":"UNAVAILABLE"},{"schema":"access","state":"ENABLE_COMPLETED"},{"schema":"billing","state":"ENABLE_COMPLETED"},{"schema":"compute","state":"ENABLE_COMPLETED"},{"schema":"marketplace","state":"ENABLE_COMPLETED"},{"schema":"information_schema","state":"ENABLE_COMPLETED"}]}*

Join 100K+ Data Experts: Register Now & Grow with Us!

Excited to expand your horizons with us? Click here to Register and begin your journey to success!

Already a member? Login and join your local regional user group! If there isn’t one near you, fill out this form and we’ll create one for you to join!