Here's your Data + AI Summit 2024 - Data Governance recap as you navigate the explosion of AI, data and tools in efforts to build a flexible and scalable governance framework that spans your entire data and AI estate. Keynote: Evolving Data Governan...
I am adding a credential to access MongoDB through the Python MongoDB Spark connector. It is stored as a secret and the `users` group has `MANAGE` ACL privileges. I turned on table ACLs and created a `High Concurrency` compute cluster. I then run a s...
@Erik Louie​ looks the issue is resolved. Please let me know if you need further help on this. Thanks!
IDM Serial Key with Internet Download Manager (IDM) is a tool to increase download speeds, resume, and schedule downloads. https://assadpc.com/idm-serial-number/
Hi, I would like to use Delta Sharing but I have not understood how to configure the Delta Sharing Server. Is that fully managed by Databricks ? If so I tried to share a table :%sql create share first_shareand I got the following error message with t...
Hi @Bertrand BURCKER​ ,This feature is not available yet. If you would like to test it or explore it, then you will need to sign up to get more details once the feature is available. In addition, you could reach out to your account team (in case you...
I found a page that tells the compatibility between Databricks and Hive metastore versions: https://docs.microsoft.com/en-us/azure/databricks/data/metastores/external-hive-metastoreBut the information is only up to Databricks 7.1. Anywhere I can find...
it still should work with 7.x +. version. Are you seeing any issue on your cluster ? @Jeffrey Mak​
How to Create Cluster that should have access to AWS Glue Catalog in different account
Hi @Prakash Rajendran​ , If the Glue Data Catalog is in a different AWS account from where Databricks is deployed, a cross-account access policy must allow access to the catalog from the AWS account where Databricks is deployed.Please refer to the be...
What industry security controls and practices do Databricks follow?
Databricks recently also updated the Security Addendum that is incorporated into our typical contracts. It's now written in plain language, and provides detailed content around security controls and practices. https://databricks.com/security-addendum...
Yes I believe that is in pipeline
You could use delta sharing without unity catalog. It is an open protocol for secure real-time exchange of large datasets. There repo at https://github.com/delta-io/delta-sharing also has a reference implementation of the delta sharing server which c...
Yes, you'll have an option to use the existing metastore, an external metastore, and Unity at the same time.
Unity Catalog is planned to support multi-cloud architectures from one catalog instance.
Yes, UC will have a suite of REST APIs allowing external tools to query it.UC Data Dictionary tables will be available via SQL as well. It is possible to query the UC Data Dictionary via JDBC/ODBC connection, but this requires a running endpoint or c...
Yes, this is the vision. The first phase of Unity Catalog is focused on the operational aspects of accessing data i.e. data access controls (Table ACL model, data sharing etc) and auditing. Lineage will be introduced in future phases.
Initially UC will support Delta and Parquet formats only. We plan to support more formats with time
The delta sharing protocol is a mechanism to share live data in your Delta Lake without copying it to another system.Currently it supports delta as the source format. For more details on the advantages of using delta as a table format for data lakes ...
