cancel
Showing results for 
Search instead for 
Did you mean: 
Data Engineering
cancel
Showing results for 
Search instead for 
Did you mean: 

Forum Posts

sunkam
by New Contributor
  • 1954 Views
  • 4 replies
  • 0 kudos

Unable to read from azure blob using SAS token\\

I have tried many times all the answers from the internet and stackover flowI have already created the config section before this steps, it passed but this below step is not executing.

image
  • 1954 Views
  • 4 replies
  • 0 kudos
Latest Reply
aockenden
New Contributor II
  • 0 kudos

We were getting this problem when using directory-scoped SAS tokens. While I know there are a number of potential issues that can cause this problem, one potential explanation is that it turns out there is an undocumented spark setting needed on the ...

  • 0 kudos
3 More Replies
vivek_sinha
by Contributor
  • 4648 Views
  • 4 replies
  • 4 kudos

Resolved! Getting Authentication Error while accessing Azure Blob table (wasb) URL using PySpark

I am trying to access the Azure Blob table using Pyspark but getting an Authentication Error. Here I am passing SAS token (HTTP and HTTPS enabled) but it's working only with WASBS (HTTPS) URL, not with WASB (HTTP) URL.Even I tried with Account key as...

  • 4648 Views
  • 4 replies
  • 4 kudos
Latest Reply
vivek_sinha
Contributor
  • 4 kudos

Hi @Arvind Ravish​  The issue got fixed after passing HTTP and HTTPS enabled token to spark executors.Thanks again for your help

  • 4 kudos
3 More Replies
halfwind22
by New Contributor III
  • 6521 Views
  • 11 replies
  • 12 kudos

Resolved! Unable to write csv files to Azure BLOB using pandas to_csv ()

I am using a Py function to read some data from a GET endpoint and write them as a CSV file to a Azure BLOB location.My GET endpoint takes 2 query parameters,param1 and param2. So initially, I have a dataframe paramDf that has two columns param1 and ...

  • 6521 Views
  • 11 replies
  • 12 kudos
Latest Reply
halfwind22
New Contributor III
  • 12 kudos

@Hubert Dudek​ I cant issue a spark command to executor node, throws up an error ,because foreach distributes the processing.

  • 12 kudos
10 More Replies
Labels