topic Re: how to share a unity PROD catalog to STAGING workspace in Data Engineering

how to share a unity PROD catalog to STAGING workspace

jeremy98 — Wed, 27 Nov 2024 09:19:54 GMT

Hello Community,

I’m looking for a secure way to share a production Unity Catalog with the staging workspace. My goal is to sync data from a schema in the production catalog to the staging workspace, enabling it to read the data and write it into some of my staging tables.

Currently, I’ve enabled access to the production catalog in the staging workspace. However, I haven’t implemented specific rules, which means the staging workspace can manipulate the production Unity Catalog—something I definitely want to avoid!

I’ve come across Delta Sharing as a potential solution, but I was wondering if there are other secure alternatives.

Thank you in advance for your help!

Re: how to share a unity PROD catalog to STAGING workspace

yumnus — Wed, 27 Nov 2024 10:19:12 GMT

Hi!

A potential solution to your issue could be configuring read-only access to the schema in your production catalog. This approach allows you to securely share the production catalog with your staging workspace while ensuring that users in the staging workspace can only read data and cannot manipulate it. You will have to assign the production catalog to your staging workspace.

To implement this:

Assign appropriate permissions (e.g. SELECT only) to the users or groups in the staging workspace for the specific schemas or tables you wish to share.
Ensure no write privileges are granted to the staging workspace for the production catalog to maintain data integrity.

If you are considering Delta Sharing, it’s a robust alternative, especially for use cases that involve sharing data with external systems or strict decoupling of environments. However, for internal use between your production and staging workspaces, managing permissions within Unity Catalog might suffice.

I hope this helps! Let me know if you have further questions.

Re: how to share a unity PROD catalog to STAGING workspace

jeremy98 — Wed, 27 Nov 2024 10:38:39 GMT

Thank you,
for this amazing answer! I was reflecting on what you said and wanted to clarify:
Are you suggesting assigning read-only access to the staging workspace for the shared production catalog (shared by assigning the production catalog to the staging workspace) only?

Re: how to share a unity PROD catalog to STAGING workspace

yumnus — Wed, 27 Nov 2024 10:56:28 GMT

Yes.

You can assign your production catalog to the staging workspace and then define the permissions so that the needed schemas and tables can only be read/selected and not manipulated.

Schema level permissions:

Re: how to share a unity PROD catalog to STAGING workspace

jeremy98 — Wed, 27 Nov 2024 11:01:33 GMT

Yes,
the privileges need to be assigned directly in the staging workspace for the shared prod_catalog. Currently, I seem to have full access.

Correct?