Querying Mysql db from Azure databricks where public access is disabled

[ashraf1395]

Hi there,

We are trying to setup a infra that ingest data from MySQL hosted on awa EC2 instance with pyspark and azure databricks and dump to the adls storage.

Since databases has public accessibility disabled and how can I interact with MySQL from azure databricks?

Major challenge is public accessiblity disabled. Querying MySQL normally is doable

[-werners- · Accepted solution]

you will need some kind of tunnel that opens the db server to external access.
Perhaps a vpn is an option?
If not: won't be possible.
An alternative way would be to have some local LAN tool extract the data and then move it to S3/... and afterwards let Databricks pick up those files.
If I'm not mistaken, Databricks is integrating ETL functionality in their platform (I think it is in private preview), so it might be possible in the future (although you would still need some kind of opening to the public facing internet using a gateway/vpn whatever).