Access the data from cross-cloud.

Phani1 — Mon, 30 Dec 2024 07:14:05 GMT

Hi All ,

We have a use case where we need to connect AWS Databricks to a GCP storage bucket to access the data. In Databricks We're trying to use external locations and storage credentials, but it seems like AWS Databricks only supports AWS storage bucket credentials and Cloudflare.

Is there any way to set up a connection and retrieve data from the GCP storage location using AWS Databricks?

Regards,

Phani

Re: Access the data from cross-cloud.

szymon_dybczak — Mon, 30 Dec 2024 12:26:36 GMT

Hi @Phani1 ,

You can use delta sharing. In that way you can create share that will allow you to access data stored in GCS and it's govern by UC permissions model.

What is Delta Sharing? | Databricks on AWS

You can also use legacy approach, but it doesn't support UC:

Connect to Google Cloud Storage | Databricks on AWS

topic Re: Access the data from cross-cloud. in Data Engineering

Access the data from cross-cloud.

Re: Access the data from cross-cloud.