โ04-24-2019 07:08 AM
Getting the error when try to load the uploaded file in py notebook.
# File location and type file_location = "//FileStore/tables/data/d1.csv"
file_type = "csv"
# CSV options
infer_schema = "true"
first_row_is_header = "false"
delimiter = ","
# The applied options are for CSV files. For other file types, these will be ignored.
df = spark.read.format(file_type)
.option("inferSchema", infer_schema)
.option("header", first_row_is_header)
.option("sep", delimiter)
.load(file_location)
display(df)
Error at highlighted line:
Py4JJavaError Traceback (most recent call last) <command-2996484959822084> in <module>() 9 10# The applied options are for CSV files. For other file types, these will be ignored.---> 11df = spark.read.format(file_type).option("inferSchema", infer_schema).option("header", first_row_is_header).option("sep", delimiter).load(file_location) 12 13 display(df)/databricks/spark/python/pyspark/sql/readwriter.py in load(self, path, format, schema, **options) 164 self.options(**options) 165if isinstance(path, basestring):--> 166return self.df(self.jreader.load(path)) 167elif path isnotNone: 168if type(path)!= list:/databricks/spark/python/lib/py4j-0.10.7-src.zip/py4j/java_gateway.py in call(self, *args) 1255 answer = self.gateway_client.send_command(command) 1256 return_value = get_return_value( -> 1257 answer, self.gateway_client, self.target_id, self.name) 1258
โ04-30-2019 03:12 AM
Hi @Praveen B
The file location need to have the filesystem scheme. so it should be of the format dbfs:/FileStore/tables/data/d1.csv
Let us know if it works.
Thanks
โ07-27-2019 09:28 PM
Yes, this worked - cheers
Rodney
โ10-02-2020 01:14 AM
what about setting the file location which points at blob storage associated with a storage account?
I am trying for https://storageaccountname.blob.core.windows.net/testfile.csv but I come back with:java.io.IOException: No FileSystem for scheme: httpsโ11-23-2020 12:45 AM
@naughtoneladโ Did you get the solution of the question you mentioned in the comment? If yes, please share as I am facing the same issue.
โ11-22-2020 06:29 AM
@naughtoneladโ if your issue is solved,please let me know as I am facing the same problem
Join a Regional User Group to connect with local Databricks users. Events will be happening in your city, and you wonโt want to miss the chance to attend and share knowledge.
If there isnโt a group near you, start one and help create a community that brings people together.
Request a New Group