cancel
Showing results for 
Search instead for 
Did you mean: 
Data Governance
Join discussions on data governance practices, compliance, and security within the Databricks Community. Exchange strategies and insights to ensure data integrity and regulatory compliance.
cancel
Showing results for 
Search instead for 
Did you mean: 
Data + AI Summit 2024 - Data Governance

Forum Posts

owlleg6
by New Contributor III
  • 14408 Views
  • 6 replies
  • 1 kudos

Models signature inaccessible in Unity Catalog

I am using Unity Catalog to store my models. My Unity Catalog meta-store is hosted in an Azure Storage Account, which has public network access disabled. Access is restricted to certain IP ranges and private endpoint connections only.Recently, my dev...

Data Governance
azure
models
networking
Unity Catalog
  • 14408 Views
  • 6 replies
  • 1 kudos
Latest Reply
lbdatauser
New Contributor II
  • 1 kudos

I have the same problem. It must be a permissions issue. In my case, it happens when the models are under a schema that was created without indicating a specific "MANAGED LOCATION". In those cases, I also have problems trying to load a model. When th...

  • 1 kudos
5 More Replies
Ashley1
by Contributor
  • 13080 Views
  • 8 replies
  • 9 kudos

Backup Unity Catalog and managed tables

Hi All, Can anyone point me to either documentation or personally tried and tested method of backing up (and restoring) Unity Catalog and its associated managed tables? We're running on Azure and using ADLS Gen2.Regards,Ashley

  • 13080 Views
  • 8 replies
  • 9 kudos
Latest Reply
prasad_vaze
New Contributor III
  • 9 kudos

Our UC managed  tables are stored on  prod ADLS storage which is different from UC root storage account.  So what's the best way to backup and restore UC managed tables into different region?   One option is deep clone tables, copy ADLS folders to an...

  • 9 kudos
7 More Replies
GMB
by New Contributor II
  • 2570 Views
  • 2 replies
  • 0 kudos

User ID, Username and Notebook not populated in History Tab

Hi,If I update a Tag on a table within a Noetbook as follows:sql = f"""ALTER TABLE {silver_catalog}.public.tablename SET TAGS ('Is_published' = 'Y');"""spark.sql(sql)Then the User ID and Username, nor Notebook is populated when I look in the History ...

  • 2570 Views
  • 2 replies
  • 0 kudos
Latest Reply
Kaniz_Fatma
Community Manager
  • 0 kudos

Hey there! Thanks a bunch for being part of our awesome community!  We love having you around and appreciate all your questions. Take a moment to check out the responses – you'll find some great info. Your input is valuable, so pick the best solution...

  • 0 kudos
1 More Replies
Dp15
by Contributor
  • 4088 Views
  • 2 replies
  • 1 kudos

Resolved! Schema enforcement from External Location

Hi, This might be a silly question to ask but is it possible to enforce schema on a table created from external location, I am trying to create a table from a s3 location and my command looks like thisquery = """CREATE TABLE IF NOT EXISTS {} ...

  • 4088 Views
  • 2 replies
  • 1 kudos
Latest Reply
Kaniz_Fatma
Community Manager
  • 1 kudos

I want to express my gratitude for your effort in selecting the most suitable solution. It's great to hear that your query has been successfully resolved. Thank you for your contribution. 

  • 1 kudos
1 More Replies
Yevhen
by New Contributor II
  • 3340 Views
  • 1 replies
  • 1 kudos

Metastore ErrorClass=QUOTA_EXCEEDED limit: 100000

I faced with error "[RequestId=b567c3bb-05fc-4ac8-b4db-2d826b2c00be ErrorClass=QUOTA_EXCEEDED] Cannot create 1 Table(s) in Metastore 95157916-5539-4dc4-a7d6-e9a854ba0591 (estimated count: 100088, limit: 100000). " but I didn't  see  here this limit  ...

  • 3340 Views
  • 1 replies
  • 1 kudos
Latest Reply
arpit
Valued Contributor
  • 1 kudos

@Yevhen Here is the doc for quota limit: https://docs.databricks.com/en/data-governance/unity-catalog/index.html#resource-quotas

  • 1 kudos
Yulei
by New Contributor III
  • 6817 Views
  • 3 replies
  • 2 kudos

Resolved! Migrate existing Metastore to a new Metastore in same region

Hi, Databricks CommunityI am currently plan to do migrate of existing metastore (not in a desired account and name) to a new one (a different desired account) within the same region. I understand it is not as straightforward it might be and complicat...

  • 6817 Views
  • 3 replies
  • 2 kudos
Latest Reply
Yulei
New Contributor III
  • 2 kudos

hi, @Kaniz_Fatma . Thank for the recommendations and links, they are helpful and I am going through them one by one.

  • 2 kudos
2 More Replies
lsun6234
by New Contributor II
  • 7639 Views
  • 6 replies
  • 0 kudos

help me in converting the below code equivalent to databricks sql PS. with recursive is not working

 SELECT position_no,      position_function,     work_function,      job_profile_id,     pos_cat as position_category,     pos_cat_desc,     job_posting_title as pos_title,     employee_status as emp_status,     emp_status_desc,      clevel,     subs...

  • 7639 Views
  • 6 replies
  • 0 kudos
Latest Reply
DBSolution
New Contributor II
  • 0 kudos

Do we have the solution of this now?

  • 0 kudos
5 More Replies
sqlshep
by New Contributor III
  • 16242 Views
  • 3 replies
  • 0 kudos

Azure Databricks Unity Catalogue Limits

In Azure, what are the limits of Unity Catalogue?  One UC per region per subscription, or is it one UC per region per tenant? 

  • 16242 Views
  • 3 replies
  • 0 kudos
Latest Reply
SiddeshSamarth
New Contributor II
  • 0 kudos

Does it mean we will have DEV, TEST, PROD catalog in a single Unity catalog?

  • 0 kudos
2 More Replies
iplantevin
by New Contributor III
  • 1782 Views
  • 0 replies
  • 0 kudos

Introducing PACE, the new open-source data security engine

Hi Databricks community! Exciting news: we open-sourced PACE (Policy As Code Engine) and launched on Product Hunt, and we'd love your input!PACE offers an abstraction on Unity Catalog data access policies, translating YAML/JSON policy definitions int...

Data Governance
Data Policies
Dynamic Views
  • 1782 Views
  • 0 replies
  • 0 kudos
Dp15
by Contributor
  • 14271 Views
  • 8 replies
  • 3 kudos

Resolved! Schema Changes to External table

Hi,I have an external table which reads data from a S3 bucket. The s3 bucket is expected to get new files frequently. With some changes to the underlying schema. I used Refresh Table table command to load new files from the s3 location and it worked ...

  • 14271 Views
  • 8 replies
  • 3 kudos
Latest Reply
shan_chandra
Esteemed Contributor
  • 3 kudos

@Dp15 - I am glad it worked. Happy to help!!!

  • 3 kudos
7 More Replies
ravi_kumar
by New Contributor
  • 3272 Views
  • 1 replies
  • 0 kudos

Best way to run a job/trigger on alter table SET ROW FILTER or SET MASK

I want to run SQL statements whenever we add ROW FILTER or MASK using ALTER TABLE command. https://docs.databricks.com/en/sql/language-manual/sql-ref-syntax-ddl-alter-table.htmlhttps://docs.databricks.com/en/sql/language-manual/sql-ref-syntax-ddl-alt...

  • 3272 Views
  • 1 replies
  • 0 kudos
Latest Reply
Walter_C
Honored Contributor
  • 0 kudos

The best way to run a jobon ALTER TABLE SET ROW FILTER or SET MASK.1. **Create the Function**  You need to start by creating a function that will be used to filter the rows or mask the columns. The function should return a boolean value that determin...

  • 0 kudos
BalaRamesh
by New Contributor II
  • 3244 Views
  • 1 replies
  • 0 kudos

couldn't able to find the Data explorer in my current databricks workspace

I am trying to create catelog and metastor but not able to create it

  • 3244 Views
  • 1 replies
  • 0 kudos
Latest Reply
Debayan
Esteemed Contributor III
  • 0 kudos

Hi, The Catalog Explorer UI is not live in all Databricks workspaces. For information about the previous UI, see Explore and create tables in DBFS. Refer: https://docs.databricks.com/en/catalog-explorer/index.html .  Also, could you please share the ...

  • 0 kudos
karthik_p
by Esteemed Contributor
  • 3925 Views
  • 1 replies
  • 0 kudos

Azure private link enable and system tables

we are planning to enabled azure private link config. as per article looks , we will have 2 private link services that need to be configured and 2 private endpoints. also we would like enable system tables. based on system tables limitations it says ...

karthik_p_0-1704834297422.png
  • 3925 Views
  • 1 replies
  • 0 kudos
Latest Reply
Debayan
Esteemed Contributor III
  • 0 kudos

Hi, In this scenario, are you mentioning about private preview of system tables? Also, you can refer to https://learn.microsoft.com/en-us/azure/key-vault/general/integrate-databricks-blob-storage , let us know if this helps. 

  • 0 kudos
MiroFuoli
by New Contributor II
  • 7065 Views
  • 3 replies
  • 1 kudos

Unity Catalog - Limited Options for Connection Objects

I’m currently trying to create a Foreign Catalog based on a Connection object of type SQLSERVER. This would allow me to directly access our on-premises MS SQL database from within Azure Databricks using Unity Catalog.As I’m part of a large organizati...

Data Governance
Connection
Foreign Catalog
JDBC
SQL Server
Unity Catalog
  • 7065 Views
  • 3 replies
  • 1 kudos
Latest Reply
Kaniz_Fatma
Community Manager
  • 1 kudos

Hi @MiroFuoli, Firstly, ensuring you’re using the correct versions of dbt-core and dbt-databricks is important. There have been instances where specific versions of these packages have caused issues. You might want to try using an older version, such...

  • 1 kudos
2 More Replies
karthik_p
by Esteemed Contributor
  • 6497 Views
  • 4 replies
  • 0 kudos

How to configure storage credentials for aws s3 buckets which use access keys

We have a requirement to configure storage credentials in UC for s3 buckets which have been previously mounted by using access keys  mounts has been depreciated and we need to configure storage credentials when we use UC. But while configuring storag...

  • 6497 Views
  • 4 replies
  • 0 kudos
Latest Reply
Kaniz_Fatma
Community Manager
  • 0 kudos

Thank you for posting your question in our community! We are happy to assist you.To help us provide you with the most accurate information, could you please take a moment to review the responses and select the one that best answers your question?This...

  • 0 kudos
3 More Replies

Connect with Databricks Users in Your Area

Join a Regional User Group to connect with local Databricks users. Events will be happening in your city, and you won’t want to miss the chance to attend and share knowledge.

If there isn’t a group near you, start one and help create a community that brings people together.

Request a New Group
Labels