topic Databricks Cross cloud in Data Engineering

Databricks Cross cloud

Searce — Fri, 09 Dec 2022 02:49:23 GMT

We have service with AWS Databricks. We are doing the same replica on GCP Databricks. Here we required all the services and functionalities should be run in AWS and AWS Databricks. The only thing data should be stored on the GCP Storage. Simply functions and services depend on the AWS side those related data should depend on GCP storage. Is this possible to do?

Re: Databricks Cross cloud

Ajay-Pandey — Fri, 09 Dec 2022 05:27:18 GMT

I think it's not mandatory you can access the data from multiple cross-cloud in databricks.

Just you have to configure the same.

Re: Databricks Cross cloud

Searce — Fri, 09 Dec 2022 05:44:07 GMT

Thanks for the update, Yes I understand, we can access the data from cross-cloud.

Here we need a solution for only storing the data in GCP rest of the things in AWS. For example, pipelines and clusters need to be placed in AWS, data should be stored in GCP

Re: Databricks Cross cloud

Aviral-Bhardwaj — Sat, 10 Dec 2022 15:15:17 GMT

no, right now i don't think they are supporting this type of architecture