cancel
Showing results for 
Search instead for 
Did you mean: 
Data Engineering
Join discussions on data engineering best practices, architectures, and optimization strategies within the Databricks Community. Exchange insights and solutions with fellow data engineers.
cancel
Showing results for 
Search instead for 
Did you mean: 

Forum Posts

Anonymous
by Not applicable
  • 5986 Views
  • 7 replies
  • 24 kudos

Resolved! Couldn't create new catalog?

I used DBR version 11.0

  • 5986 Views
  • 7 replies
  • 24 kudos
Latest Reply
seanzy
New Contributor II
  • 24 kudos

In the 2.9 Comprehensive lab of getting started with data engineering on data bricks, I try to run:%run ../Includes/Classroom-Setup-11and get the following errorThe execution of this command did not finish successfully    Resetting the learning envir...

  • 24 kudos
6 More Replies
harraz
by New Contributor III
  • 6353 Views
  • 3 replies
  • 0 kudos

Unable to use unity catalog in notebook

com.databricks.backend.common.rpc.SparkDriverExceptions$SQLExecutionException: org.apache.spark.sql.connector.catalog.CatalogNotFoundException: Catalog 'uc-dev' plugin class not found: spark.sql.catalog.uc-dev is not defined ....I get the above when ...

  • 6353 Views
  • 3 replies
  • 0 kudos
Latest Reply
Tomas
New Contributor II
  • 0 kudos

I had the same error plugin class not found: spark.sql.catalog... is not defined immediately after attaching the workspace into Unity catalog.The error was resolved by restarting SQL Warehouse.It seems that if SQL Warehouse (or any cluster) is runnin...

  • 0 kudos
2 More Replies
js54123875
by New Contributor III
  • 2503 Views
  • 3 replies
  • 3 kudos

Setup for Unity Catalog, autoloader, three-level namespace, SCD2

I am trying to setup delta live tables pipelines to ingest data to bronze and silver tables. Bronze and Silver are separate schema. This will be triggered by a daily job. It appears to run fine when set as continuous, but fails when triggered.Table...

  • 2503 Views
  • 3 replies
  • 3 kudos
Latest Reply
Anonymous
Not applicable
  • 3 kudos

Hi @Jennette Shepard​ Thank you for posting your question in our community! We are happy to assist you.To help us provide you with the most accurate information, could you please take a moment to review the responses and select the one that best answ...

  • 3 kudos
2 More Replies
Ryu1
by New Contributor
  • 857 Views
  • 0 replies
  • 0 kudos

Other than the "account admin" permission, is there a small permission or role to collect only catalog information?

I am going to use an open source called "datahub" to collect and share metadata information of databricks. (https://datahubproject.io/)Recently, however, there has been a big challenge. That is, to collect the unity catalog information of databricks,...

  • 857 Views
  • 0 replies
  • 0 kudos
Anonymous
by Not applicable
  • 4697 Views
  • 6 replies
  • 2 kudos

Resolved! Delta Sharing - Unity Catalog difference

Delta Sharing and Unity catalog both have elements of data sharing. Can you please explain when one would use Delta sharing vs Unity Catalog?

  • 4697 Views
  • 6 replies
  • 2 kudos
Latest Reply
DBXC
Contributor
  • 2 kudos

Based on the Databricks reply from the post below: "Unity Catalog does not currently support separating data by workspace or Azure subscription. As you noted, data from all catalogs within a region can be accessed by any workspace within that region,...

  • 2 kudos
5 More Replies
Tannu858
by New Contributor
  • 1088 Views
  • 1 replies
  • 0 kudos
  • 1088 Views
  • 1 replies
  • 0 kudos
Latest Reply
Anonymous
Not applicable
  • 0 kudos

Hi @Bhupesh Aggarwal​ Hope everything is going great.Just wanted to check in if you were able to resolve your issue. If yes, would you be happy to mark an answer as best so that other members can find the solution more quickly? If not, please tell us...

  • 0 kudos
Leszek
by Contributor
  • 6015 Views
  • 5 replies
  • 5 kudos

Resolved! Unity Catalog - Azure account console - how to access?

I'm trying to access account console in Azure but I only can see the list of workspaces and access them. I didn't find documentation about account console for Azure. Do you know how to access account console?

  • 6015 Views
  • 5 replies
  • 5 kudos
Latest Reply
vimalii
New Contributor II
  • 5 kudos

Hello @Leszek​ . Please tell me is it works for you ?Did you find the root cause ?I still don't understand why I should grant to myself some extra permissions if I already global administrator, owner of subscription, owner of databricks workspace but...

  • 5 kudos
4 More Replies
asethia
by New Contributor
  • 4147 Views
  • 1 replies
  • 0 kudos

delta lake in Apache Spark

Hi,As per documentation https://docs.delta.io/latest/quick-start.html , we can configure DeltaCatalog using spark.sql.catalog.spark_catalog.The Iceberg supports two Catalog implementations (https://iceberg.apache.org/docs/latest/spark-configuration/#...

  • 4147 Views
  • 1 replies
  • 0 kudos
Latest Reply
Anonymous
Not applicable
  • 0 kudos

@Arun Sethia​ :Yes, Delta Lake also supports custom catalogs. Delta Lake uses the Spark Catalog API, which allows for pluggable catalog implementations. You can implement your own custom catalog to use with Delta Lake.To use a custom catalog, you can...

  • 0 kudos
maaaxx
by New Contributor III
  • 3620 Views
  • 5 replies
  • 0 kudos

Resolved! Can Unity catalog grant the access to a file inside azure datalake storage?

Hi databricks community,I have searched quite a while through the internet but did not find an answer. If I have configured the azure datalake connection in Unity data catalog, is it possible to grant the access to users for a specific file or a fold...

  • 3620 Views
  • 5 replies
  • 0 kudos
Latest Reply
Hubert-Dudek
Esteemed Contributor III
  • 0 kudos

As @werners said service principal needs to have access to the file level.In the unity catalog, you can use "READ FILES"/"WRITE FILES" permission to give someone the possibility of reading files from the storage level (but through databricks).

  • 0 kudos
4 More Replies
sajith_appukutt
by Honored Contributor II
  • 1176 Views
  • 1 replies
  • 0 kudos
  • 1176 Views
  • 1 replies
  • 0 kudos
Latest Reply
sajith_appukutt
Honored Contributor II
  • 0 kudos

You could leverage SHOW GRANT which displays the privilegesSHOW GRANT [<user>] ON [CATALOG | DATABASE <database-name> | TABLE <table-name> | VIEW <view-name> | FUNCTION <function-name> | ANONYMOUS FUNCTION | ANY FILE]You could use this code snippet ...

  • 0 kudos
Labels