Bug in db.fs.utils

740209
New Contributor II

‎09-27-2022 12:42 PM

When using db.fs.utils on a s3 bucket titled "${sometext}.${sometext}.${somenumber}${sometext}-${sometext}-${sometext}" we receive an error. PLEASE understand this is an issue with how it encodes the .${somenumber} because we verified with boto3 that we can query the s3 bucket. There is already a community issue that I believe is due to this, but does not provide this amount of detail. https://community.databricks.com/s/question/0D58Y00009BMsKGSA1/dbutilsfsls-is-giving-null-uri-host-t...

Command (works with other buckets that do not follow this format, but doesn't work on any bucket that follows format above):

dbutils.fs.ls("s3a://${bucket_name_here_follow_above_format}")

Traceback:

ExecutionError: An error occurred while calling z:com.databricks.backend.daemon.dbutils.FSUtils.ls. : java.lang.NullPointerException: null uri host. This can be caused by unencoded / in the password string at java.util.Objects.requireNonNull(Objects.java:228) at shaded.databricks.org.apache.hadoop.fs.s3native.S3xLoginHelper.buildFSURI(S3xLoginHelper.java:69) at shaded.databricks.org.apache.hadoop.fs.s3a.S3AFileSystem.setUri(S3AFileSystem.java:792) at shaded.databricks.org.apache.hadoop.fs.s3a.S3AFileSystem.initialize(S3AFileSystem.java:406) at org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:3469)

0 Kudos

Debayan
Databricks Employee
Databricks Employee

‎09-29-2022 11:30 PM

Hi, Could you please type in the whole command and enter? (sometimes copying will put the format incorrectly).

Reference: https://docs.databricks.com/data/data-sources/aws/amazon-s3.html

740209
New Contributor II

‎10-03-2022 09:43 AM

@Debayan Mukherjee​ 

All the information is there please read accurately. I am not going to give you the actual bucket name I am using on a public forum. As i said above here is the command:

dbutils.fs.ls("s3a://${bucket_name_here_follow_above_format}")

where format is "${sometext}.${sometext}.${somenumber}${sometext}-${sometext}-${sometext}"

so an example command is: dbutils.fs.ls("s3a://testtext.testtext.4303testtext-testtext-testtext").

We had to use an S3 access point name because the actual name of bucket did not work and has the above Traceback.

0 Kudos

Debayan
Databricks Employee
Databricks Employee
In response to 740209

‎10-05-2022 11:51 PM

Hi @arturo.iacobelli arturo.iacobelli​ , Thanks for the information. We appreciate your patience in this regard! I would request you to file a support case in Databricks portal with all the details so that we can look into the issue thoroughly and triage the same.

0 Kudos

740209
New Contributor II
In response to Debayan

‎10-18-2022 07:00 AM

We can't submit a support case apparently because this requires a support contract. Our databricks contact told us to submit a community request.

0 Kudos