cancel
Showing results for 
Search instead for 
Did you mean: 
Data Engineering
Join discussions on data engineering best practices, architectures, and optimization strategies within the Databricks Community. Exchange insights and solutions with fellow data engineers.
cancel
Showing results for 
Search instead for 
Did you mean: 

Forum Posts

jwilliam
by Contributor
  • 2318 Views
  • 3 replies
  • 2 kudos

Resolved! How to mount Azure Blob Storage with OAuth2?

We already know that we can mount Azure Data Lake Gen2 with OAuth2 using this:configs = {"fs.azure.account.auth.type": "OAuth", "fs.azure.account.oauth.provider.type": "org.apache.hadoop.fs.azurebfs.oauth2.ClientCredsTokenProvider", ...

  • 2318 Views
  • 3 replies
  • 2 kudos
Latest Reply
mathijs-fish
New Contributor III
  • 2 kudos

Is there any update on this feature request? OAuth still seems not to be working with Azure Blob Storage... Configuration works fine for ADLS gen 2, but for Azure Blob Storage still only SAS and Account key seems to be working.

  • 2 kudos
2 More Replies
Woody
by New Contributor II
  • 718 Views
  • 2 replies
  • 3 kudos

This site has alot of issues....

I need to login and I can't to try Databricks..so you have a OAUTH issue...I cant try Databricks at all because the country icon doesn't work and it sends a URI issue from your front end to the back end..."Request_URI=&Geo_country_code=&Geo_country_i...

  • 718 Views
  • 2 replies
  • 3 kudos
Latest Reply
Anonymous
Not applicable
  • 3 kudos

Hi @Jessica Woods​ Thank you for posting your question in our community! We are happy to assist you.To help us provide you with the most accurate information, could you please take a moment to review the responses and select the one that best answers...

  • 3 kudos
1 More Replies
Josh_Stafford
by New Contributor II
  • 1316 Views
  • 2 replies
  • 1 kudos

Using dbutils.fs.ls on URI with square brackets results in error

Square brackets in ADLS are accepted, so why can't I list the files in the folder? I have tried escaping the square brackets manually, but then the escaped values are re-escaped from %5B to %255B and %5D to %255D. I get:URISyntaxException: Illegal ...

  • 1316 Views
  • 2 replies
  • 1 kudos
Latest Reply
Anonymous
Not applicable
  • 1 kudos

@Joshua Stafford​ :The URISyntaxException error you are encountering is likely due to the fact that square brackets are reserved characters in URIs (Uniform Resource Identifiers) and need to be properly encoded when used in a URL. In this case, it ap...

  • 1 kudos
1 More Replies
chanansh
by Contributor
  • 4843 Views
  • 3 replies
  • 0 kudos

Relative path in absolute URI when reading a folder with files containing ":" colons in filename

I am trying to read a folder with partition files where each partition is date/hour/timestamp.csv where timestamp is the exact timestamp in ISO format, e.g. 09-2022-12-05T20:35:15.2786966Z It seems like spark having issues with reading files with col...

  • 4843 Views
  • 3 replies
  • 0 kudos
Latest Reply
Kaniz
Community Manager
  • 0 kudos

Hi @Hanan Shteingart​  (Customer)​, We haven’t heard from you since the last response from @Debayan Mukherjee​  (Customer)​ , and I was checking back to see if his suggestions helped you.Or else, If you have any solution, please share it with the co...

  • 0 kudos
2 More Replies
espenol
by New Contributor III
  • 3773 Views
  • 4 replies
  • 3 kudos

Resolved! Can't read large multiline json,

Hey! So I'm struggling to read a multiline json. Some details:It's gzipped from the API I get it fromjust a single file in the folder currrentlystored in ADLS Gen2 storage. 95 MB zipped, approximately 1.2 GB unzippedI can read it just fine using the...

Can be read as text, multiline json Corrupted string relative path in absolute uri
  • 3773 Views
  • 4 replies
  • 3 kudos
Latest Reply
espenol
New Contributor III
  • 3 kudos

Thanks a lot for the help! Removing colon fixed it. Now I need to fix the Data Factory instance that writes to my storage container. Hope it's easy, Data Factory is such a hassle.

  • 3 kudos
3 More Replies
antoniok
by New Contributor II
  • 2057 Views
  • 1 replies
  • 3 kudos

dbutils.fs.ls is giving "null uri host This can be caused by unencoded / in the password string"

I'm trying to list number of files in s3 bucket. I've initially used "aws s3 ls <s3://>" to list the files and it worked. However, when trying to do the same using dbutils.fs.ls, I'm getting java.lang.NullPointerException: null uri host. This can be ...

  • 2057 Views
  • 1 replies
  • 3 kudos
Latest Reply
marcus1
New Contributor III
  • 3 kudos

You might be encountering an issue with bucket naming. Which I'm also getting with a bucket named something.[0-9]https://issues.apache.org/jira/browse/HADOOP-17241

  • 3 kudos
Malcoln_Dandaro
by New Contributor
  • 1316 Views
  • 0 replies
  • 0 kudos

Is there any way to navigate/access cloud files using the direct abfss URI (no mount) with default python functions/libs like open() or os.listdir()?

Hello, Today on our workspace we access everything via mount points, we plan to change it to "abfss://" because of security, governance and performance reasons. The problem is sometimes we interact with files using "python only" code, and apparently ...

  • 1316 Views
  • 0 replies
  • 0 kudos
Labels