Unity Catalog Lakeguard: Industry-first and only data governance for multi-user Apache™ Spark cluste

Sujitha — Thu, 25 Apr 2024 14:58:23 GMT

Run Scala, Python and SQL workloads on shared, cost-efficient multi-user compute.

We are thrilled to announce Unity Catalog Lakeguard, which allows you to run Apache Spark™ workloads in SQL, Python, and Scala with full data governance on the Databricks Data Intelligence Platform’s cost-efficient, multi-user compute. To enforce governance, traditionally, you had to use single-user clusters, which adds cost and operational overhead. With Lakeguard, user code runs in full isolation from any other users’ code and the Spark engine on shared compute, thus enforcing data governance at runtime. This allows you to securely share clusters across your teams, reducing compute cost and minimizing operational toil.

Click here to read more.

Re: Unity Catalog Lakeguard: Industry-first and only data governance for multi-user Apache™ Spark cl

ani-khadkikar — Fri, 26 Apr 2024 06:14:54 GMT

Hello Sujitha - Will this be extended to R? And by when? R continues to be used quite heavily in the industry. Would be good to know when the limitations of R and shared cluster usage will be taken away.

topic Re: Unity Catalog Lakeguard: Industry-first and only data governance for multi-user Apache™ Spark cl in Announcements

Unity Catalog Lakeguard: Industry-first and only data governance for multi-user Apache™ Spark cluste

Re: Unity Catalog Lakeguard: Industry-first and only data governance for multi-user Apache™ Spark cl