topic Re: SFTP Connect in Data Engineering

SFTP Connect

ajain80 — Tue, 04 Oct 2022 21:05:14 GMT

How I can connect sftp server from databricks. So I can write files into tables directly?

Re: SFTP Connect

Hubert-Dudek — Thu, 13 Oct 2022 15:19:34 GMT

The classic solution is to copy data from FTP to ADLS storage using Azure Data Factory, and after the copy is done in the ADF pipeline, trigger the databricks notebook.

Re: SFTP Connect

Cronk — Thu, 11 May 2023 06:26:29 GMT

Thank you, I will try it. I hope it works. If you're looking for the best online writing services for your college assignments, be sure to visit https://www.topwritersreview.com/reviews/essayusa/ It's an excellent resource that can help you make an informed decision and achieve your academic goals with confidence.

Re: SFTP Connect

kalo1 — Mon, 15 May 2023 08:13:21 GMT

Thank you for the link, you made my day.

Re: SFTP Connect

Debayan — Fri, 07 Oct 2022 07:18:36 GMT

Hi @ARINJAY JAIN , Could you please check if this works: https://community.databricks.com/s/question/0D53f00001lbF65CAE/notebook-for-sftp-server-connectivity-without-password

Re: SFTP Connect

Michael_Appiah — Thu, 24 Aug 2023 13:11:56 GMT

Is there a way to copy data from FTP into ADLS using PySpark in a Databricks Notebook (i.e. without ADF)?
The following library (https://github.com/springml/spark-sftp) that some have recommended does not seem to include a PySpark API.