- 7660 Views
- 6 replies
- 5 kudos
Resolved! Unity Catalog - Azure account console - how to access?
I'm trying to access account console in Azure but I only can see the list of workspaces and access them. I didn't find documentation about account console for Azure. Do you know how to access account console?
- 7660 Views
- 6 replies
- 5 kudos
- 5 kudos
Hello @Leszek​ . Please tell me is it works for you ?Did you find the root cause ?I still don't understand why I should grant to myself some extra permissions if I already global administrator, owner of subscription, owner of databricks workspace but...
- 5 kudos
- 2626 Views
- 2 replies
- 1 kudos
Unity Catalog metastore list is showing empty
Hi ,I am not able to list the meta-stores in databricks cli using the below command :databricks unity-catalog metastores list{}but when I tried databricks unity-catalog metastores get-summary I am able to get the meta-store info .Can anyone help me ...
- 2626 Views
- 2 replies
- 1 kudos
- 1 kudos
Hi @Kaniz Fatma​ , Unity catalog is enabled in my workspace and i have been assigned metastore admin and account admin also.databricks unity-catalog metastores list --debugHTTP debugging enabledsend: b'GET /api/2.1/unity-catalog/metastores HTTP/1.1\r...
- 1 kudos
- 4404 Views
- 3 replies
- 6 kudos
Resolved! Has Unity Catalog been available in Azure Gov Cloud?
We are using Databricks with Premium Tier in Azure Gov Cloud. We check the Data section but don't see any options to Create Metastore.
- 4404 Views
- 3 replies
- 6 kudos
- 6 kudos
Azure.gov does not have Unity Catalog (as of July 2024). I think previous responses missed the context of government cloud in OP's question. UC has been open sourced since this question was asked, and is a more comprehensive solution in commercial cl...
- 6 kudos
- 8823 Views
- 12 replies
- 3 kudos
Resolved! Unity catalog - How do you modify groups properly ?
Hello,What is the best practice to modify/delete/recreate groups properly ?In order to rename a group, the only mean was to delete/recreate. But after deletion in the account console, the permissions granted to the deleted groups in the tables were i...
- 8823 Views
- 12 replies
- 3 kudos
- 3 kudos
Hello,I have exactly the same issue - I am also using terraform.I deleted a group and the catalog permissions are in bad state. I am not able to revoke access to this group using the Databricks UI nor REST API. I also tried to recreate the group wit...
- 3 kudos
- 17005 Views
- 10 replies
- 6 kudos
Shared cluster configuration that permits `dbutils.fs` commands
My workspace has a couple different types of clusters, and I'm having issues using the `dbutils` filesystem utilities when connected to a shared cluster. I'm hoping you can help me fix the configuration of the shared cluster so that I can actually us...
- 17005 Views
- 10 replies
- 6 kudos
- 6 kudos
Can you not use a No Isolation Shared cluster with Table access controls enabled on workspace level?
- 6 kudos
- 3901 Views
- 3 replies
- 0 kudos
Primary Key not working as expected on Unity Catalog delta tables
Hi @Chetan Kardekar. I noticed that you had commented on Primary key on Delta tables. Do we have that feature already released in DataBricks Premium. I have a Unity Catalog and I created a table with Primary Key, though it doesnt act like Primary Key...
- 3901 Views
- 3 replies
- 0 kudos
- 0 kudos
Hi @Bala Akas​ Hope all is well! Just wanted to check in if you were able to resolve your issue and would you be happy to share the solution or mark an answer as best? Else please let us know if you need more help. We'd love to hear from you.Thanks!
- 0 kudos
- 19333 Views
- 8 replies
- 3 kudos
UnauthorizedAccessException: PERMISSION_DENIED: User does not have READ FILES on External Location
Our jobs have been running fine so far w/o any issues on a specific workspace. These jobs read data from files on Azure ADLS storage containers and dont use the hive metastore data at all.Now we attached the unity metastore to this workspace, created...
- 19333 Views
- 8 replies
- 3 kudos
- 3 kudos
@Wojciech_BUK I granted both in the GUI:) you can either search for display name there (and then it uses the Managed Identity Object ID), or you can search directly for the value of the "Managed Identity Application ID" and then it works correctly! ...
- 3 kudos
- 9148 Views
- 3 replies
- 0 kudos
Unable to use unity catalog in notebook
com.databricks.backend.common.rpc.SparkDriverExceptions$SQLExecutionException: org.apache.spark.sql.connector.catalog.CatalogNotFoundException: Catalog 'uc-dev' plugin class not found: spark.sql.catalog.uc-dev is not defined ....I get the above when ...
- 9148 Views
- 3 replies
- 0 kudos
- 0 kudos
I had the same error plugin class not found: spark.sql.catalog... is not defined immediately after attaching the workspace into Unity catalog.The error was resolved by restarting SQL Warehouse.It seems that if SQL Warehouse (or any cluster) is runnin...
- 0 kudos
- 8776 Views
- 4 replies
- 1 kudos
Accidentally removing the service principal that owns the view seems to put the Unity Catalog in an illegal state. Can you fix this?
I renamed our service principal in Terraform, which forces a replacement where the old service principal is removed and a new principal with the same permission is recreated. The Terraform succeeds to apply, but when I try to run dbt that creates tab...
- 8776 Views
- 4 replies
- 1 kudos
- 1 kudos
This is also true for removing groups before unassigning them (removing and unassigning in Terraform)│ Error: cannot update grants: Could not find principal with name <My Group Name>
- 1 kudos
- 20114 Views
- 9 replies
- 11 kudos
input_file_name() not supported in Unity Catalog
Hey, so our notebooks reading a bunch of json files from storage typically use a input_file_name() when moving from raw to bronze, but after upgrading to Unity Catalog we get an error message:AnalysisException: [UC_COMMAND_NOT_SUPPORTED] input_file_n...
- 20114 Views
- 9 replies
- 11 kudos
- 11 kudos
.withColumn("RECORD_FILE_NAME", col("_metadata.file_name"))Will work for spark.read to get the file name, or: .withColumn("RECORD_FILE_NAME", col("_metadata.file_path"))To get the whole file path
- 11 kudos
- 8367 Views
- 6 replies
- 2 kudos
Resolved! How to grant access to views without granting access to underlying tables
I have a medallion architecture: Bronze layer: Raw data in tablesSilver layer: Refined data in views created from the bronze layerGold layer: Data products as views created from the silver layerCurrently I have a data scientist that needs access to d...
- 8367 Views
- 6 replies
- 2 kudos
- 2 kudos
Single-user clusters use a different security mode which is the reason for this difference. On single-user/assigned clusters, you'll need the Fine Grained Access Control service (which is a Serverless service) - that is the solution to this problem (...
- 2 kudos
- 11982 Views
- 4 replies
- 1 kudos
Connection to Azure SQL Server: ODBC Driver 18 for SQL Server
Task: Setup connection to Azure SQL Server.A couple things have changed...*We've started using Unity Catalog, so need Unity Catalog -enabled clusters*Legacy init scripts have been deprecated, and this is how we had our pyodbc setup, etc. defined.Code...
- 11982 Views
- 4 replies
- 1 kudos
- 1 kudos
Hi @js54123875 did you manage to find a solution for this? I'm facing a similar problem.ThanksDiego
- 1 kudos
- 7078 Views
- 3 replies
- 6 kudos
Bug? Unity Catalog incompatible with Sparklyr in RStudio (on Driver) and as well if used on one cluster from multiple notebooks?
If I start a RStudio Server with in cluster init script as described here in a Unity Catalog Cluster the sparklyr connection fails with an error about a missing Credential Scope.=LI tried it both in 11.3LTS and 12.0 Beta. I tried it only in a Persona...
- 7078 Views
- 3 replies
- 6 kudos
- 6 kudos
Have run into this issue as well. Let me know if there was any resolution
- 6 kudos
- 12295 Views
- 3 replies
- 3 kudos
Resolved! Error creating external location in Unity Catalog
HelloWhen I try to create an external location I get this error:Failed to access cloud storage: [AbfsRestOperationException] HTTP Error -1CustomTokenProvider getAccessToken threw com.databricks.api.base.DatabricksServiceException : INTERNAL_ERROR: Un...
- 12295 Views
- 3 replies
- 3 kudos
- 3 kudos
I think I sould have something missconfigured, the way I solved the problem was to re-create the workspace and start from scratch, it was a small one for testing proposes.
- 3 kudos
- 10279 Views
- 2 replies
- 2 kudos
AnalysisException: [UC_COMMAND_NOT_SUPPORTED] Spark higher-order functions are not supported in Unity Catalog.;
Hello,forecast_date = '2017-12-01' spark.conf.set('spark.sql.shuffle.partitions', 500 ) # generate forecast for this data forecasts = ( history .where(history.date < forecast_date) # limit training data to prior to our forecast date .groupBy...
- 10279 Views
- 2 replies
- 2 kudos
- 2 kudos
@Mr_K ApplyInPandas is a higher order function in Python. As of now, we do not support higher order functions in Unity Catalog. We do support direct calls made to python UDFs. Here is an example of how to reference UDFs in UC - https://docs.databrick...
- 2 kudos