cancel
Turn on suggestions
Showing results for 
Search instead for 
Did you mean: 
Help
Data Engineering
Join discussions on data engineering best practices, architectures, and optimization strategies within the Databricks Community. Exchange insights and solutions with fellow data engineers.
cancel
Turn on suggestions
Showing results for 
Search instead for 
Did you mean: 

Forum Posts

Start a conversation
clant
by • New Contributor II
  • 1402 Views
  • 1 replies
  • 4 kudos

Structured Streaming from SFTP

Hello,Is it possible to use a SFTP location to load from for structured streaming.At the moment we are going from SFTP->S3->databricks via structured streaming. I would like to cut out the S3 part.CheersChris

Data Engineering
Reply
  • 1402 Views
  • 1 replies
  • 4 kudos
View Replies
Latest Reply
Anonymous
Not applicable
  • 4 kudos

Hi @Chris Lant​ Great to meet you, and thanks for your question! Let's see if your peers in the community have an answer to your question first. Or else bricksters will get back to you soon. Thanks.

  • 4 kudos
hetadesai
by • New Contributor II
  • 6095 Views
  • 1 replies
  • 3 kudos

How to download zip file from SFTP location and put that file into Azure Data Lake and unzip there ?

I have zip file on SFTP location. I want to copy that file from SFTP location and put it into Azure Data lake and want to unzip there using spark notebook. Please help me to solve this.

Data Engineering
Reply
  • 6095 Views
  • 1 replies
  • 3 kudos
View Replies
Latest Reply
Hubert-Dudek
Esteemed Contributor III
  • 3 kudos

I would go with @Kaniz Fatma​ approach and download data in Data Factory and after is downloaded on success trigger databricks spark notebook. With spark you can read also compressed data so maybe you will not need to do even separate unzip.

  • 3 kudos
Labels
Top