Options
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
07-17-2023 07:56 AM
Hey,
There is two possible way.
First, is to use this spark library to connect to the SFTP: https://github.com/springml/spark-sftp
You connect to the SFTP and load the data into your storage.
The second one is to use Azure Data Factory (if you are on Azure, i don't know the equivalent in other Cloud Provider) to ingest the data into your blob mounted in your databricks. When your ADF pipeline is finished you trigger your Databricks pipeline.
Cheers 🙂