cancel
Showing results for 
Search instead for 
Did you mean: 
Data Governance
cancel
Showing results for 
Search instead for 
Did you mean: 

Forum Posts

chandan_a_v
by Valued Contributor
  • 13381 Views
  • 10 replies
  • 6 kudos

Resolved! GC (Metadata GC Threshold) issue

Hi All,I am facing the GC metadata issue while performing distributed computing on Spark. 2022-01-13T22:02:28.467+0000: [GC (Metadata GC Threshold) [PSYoungGen: 458969K->18934K(594944K)] 458969K->18958K(1954816K), 0.0144028 secs] [Times: user=0.05 sy...

  • 13381 Views
  • 10 replies
  • 6 kudos
Latest Reply
Anonymous
Not applicable
  • 6 kudos

Hi @Chandan Angadi​ Hope you are doing well now.Thanks for getting back to us and sending in your solution. Would you like to mark an answer as best?Thanks!

  • 6 kudos
9 More Replies
Hubert-Dudek
by Esteemed Contributor III
  • 3116 Views
  • 2 replies
  • 24 kudos

Databricks Runtime 10.5 (Beta)  �� Auto Loader: new SQL function CLOUD_FILES_STATE You can use the new CLOUD_FILES_STATE function to query the ...

Databricks Runtime 10.5 (Beta) Auto Loader: new SQL function CLOUD_FILES_STATEYou can use the new CLOUD_FILES_STATE function to query the internal state of an Auto Loader stream. Delta Lake: new maxRecordsPerFile option for maximum records written t...

  • 3116 Views
  • 2 replies
  • 24 kudos
Latest Reply
SørenRavn
New Contributor II
  • 24 kudos

anyone that got CLOUD_FILES_STATE to work on 10.5 Beta/Azure. Syntax:%sqlSELECT * FROM cloud_files_state('abfss://test@test.dfs.core.windows.net/test/checkpoint/'); i get this error: com.databricks.backend.common.rpc.DatabricksExceptions$SQLExecu...

  • 24 kudos
1 More Replies
Mr__E
by Contributor II
  • 3993 Views
  • 3 replies
  • 1 kudos

Resolved! Table ACLs, secrets, and compute clusters

I am adding a credential to access MongoDB through the Python MongoDB Spark connector. It is stored as a secret and the `users` group has `MANAGE` ACL privileges. I turned on table ACLs and created a `High Concurrency` compute cluster. I then run a s...

  • 3993 Views
  • 3 replies
  • 1 kudos
Latest Reply
Atanu
Esteemed Contributor
  • 1 kudos

@Erik Louie​  looks the issue is resolved. Please let me know if you need further help on this. Thanks!

  • 1 kudos
2 More Replies
idmserialkey
by New Contributor
  • 19423 Views
  • 0 replies
  • 0 kudos

assadpc.com

IDM Serial Key with Internet Download Manager (IDM) is a tool to increase download speeds, resume, and schedule downloads. https://assadpc.com/idm-serial-number/

  • 19423 Views
  • 0 replies
  • 0 kudos
RantoB
by Valued Contributor
  • 11805 Views
  • 8 replies
  • 6 kudos

Resolved! Enable Unity Catalog and pre-requisite for Delta Sharing

Hi, I would like to use Delta Sharing but I have not understood how to configure the Delta Sharing Server. Is that fully managed by Databricks ? If so I tried to share a table :%sql create share first_shareand I got the following error message with t...

  • 11805 Views
  • 8 replies
  • 6 kudos
Latest Reply
jose_gonzalez
Moderator
  • 6 kudos

Hi @Bertrand BURCKER​ ,This feature is not available yet. If you would like to test it or explore it, then you will need to sign up to get more details once the feature is available. In addition, you could reach out to your account team (in case you...

  • 6 kudos
7 More Replies
jeffreym9
by New Contributor III
  • 5093 Views
  • 3 replies
  • 4 kudos

Resolved! Compatibility of Hive metastore schema version and Databricks 9.1 LTS

I found a page that tells the compatibility between Databricks and Hive metastore versions: https://docs.microsoft.com/en-us/azure/databricks/data/metastores/external-hive-metastoreBut the information is only up to Databricks 7.1. Anywhere I can find...

  • 5093 Views
  • 3 replies
  • 4 kudos
Latest Reply
Atanu
Esteemed Contributor
  • 4 kudos

it still should work with 7.x +. version. Are you seeing any issue on your cluster ? @Jeffrey Mak​ 

  • 4 kudos
2 More Replies
prakash_raj
by New Contributor II
  • 4573 Views
  • 2 replies
  • 3 kudos

Resolved! How to Create Cluster that should have access to AWS Glue Catalog in different account

How to Create Cluster that should have access to AWS Glue Catalog in different account

  • 4573 Views
  • 2 replies
  • 3 kudos
Latest Reply
Prabakar
Esteemed Contributor III
  • 3 kudos

Hi @Prakash Rajendran​ , If the Glue Data Catalog is in a different AWS account from where Databricks is deployed, a cross-account access policy must allow access to the catalog from the AWS account where Databricks is deployed.Please refer to the be...

  • 3 kudos
1 More Replies
User16826994223
by Honored Contributor III
  • 3459 Views
  • 2 replies
  • 1 kudos

What industry security controls and practices do Databricks follow?

What industry security controls and practices do Databricks follow?

  • 3459 Views
  • 2 replies
  • 1 kudos
Latest Reply
davidveuve
New Contributor II
  • 1 kudos

Databricks recently also updated the Security Addendum that is incorporated into our typical contracts. It's now written in plain language, and provides detailed content around security controls and practices. https://databricks.com/security-addendum...

  • 1 kudos
1 More Replies
Anonymous
by Not applicable
  • 4525 Views
  • 2 replies
  • 0 kudos
  • 4525 Views
  • 2 replies
  • 0 kudos
Latest Reply
sajith_appukutt
Honored Contributor II
  • 0 kudos

You could use delta sharing without unity catalog. It is an open protocol for secure real-time exchange of large datasets. There repo at https://github.com/delta-io/delta-sharing also has a reference implementation of the delta sharing server which c...

  • 0 kudos
1 More Replies
Anonymous
by Not applicable
  • 3122 Views
  • 1 replies
  • 1 kudos
  • 3122 Views
  • 1 replies
  • 1 kudos
Latest Reply
Anonymous
Not applicable
  • 1 kudos

Yes, UC will have a suite of REST APIs allowing external tools to query it.UC Data Dictionary tables will be available via SQL as well. It is possible to query the UC Data Dictionary via JDBC/ODBC connection, but this requires a running endpoint or c...

  • 1 kudos
Anonymous
by Not applicable
  • 3283 Views
  • 1 replies
  • 1 kudos
  • 3283 Views
  • 1 replies
  • 1 kudos
Latest Reply
Anonymous
Not applicable
  • 1 kudos

Yes, this is the vision. The first phase of Unity Catalog is focused on the operational aspects of accessing data i.e. data access controls (Table ACL model, data sharing etc) and auditing. Lineage will be introduced in future phases.

  • 1 kudos
Labels