topic Re: Multiple Databricks cluster in same workspace in Data Engineering

Multiple Databricks cluster in same workspace

RajibRajib_Mand — Sat, 17 Dec 2022 01:36:30 GMT

Hi All,

I have created three cluster(dev,qa,prod)in the same workspace to isolate data for different environment.How do we differentiate environment while running job using dev it should update data for dev environment?

Regards,

Rajib

Re: Multiple Databricks cluster in same workspace

sher — Sat, 17 Dec 2022 08:02:09 GMT

you can create 2 workspaces dev n prod that also best practice

Re: Multiple Databricks cluster in same workspace

RajibRajib_Mand — Sat, 17 Dec 2022 14:31:48 GMT

@sherbin w -The customer doesn't want to create separate workspace.We need to create multiple cluster in same workspace.

Re: Multiple Databricks cluster in same workspace

Aviral-Bhardwaj — Sun, 18 Dec 2022 06:06:42 GMT

hey @Rajib Rajib Mandal , this is very easy, i have done this multiple times, you can segregate data using your IAM role that is attached to the cluster, it is known as an Instance profile, you can only give the dev data access to dev role and the so on for other 2 test and prod cluster.and it will work like as you want

Thanks

Aviral