- 9310 Views
- 2 replies
- 1 kudos
I have uploaded a csv file which have well formatted data and I was trying to use display(questions) where questions=spark.read.option("header","true").csv("/FileStore/tables/Questions.csv")This is throwing an error as follows:SparkException: Job abo...
- 9310 Views
- 2 replies
- 1 kudos
- 2685 Views
- 1 replies
- 1 kudos
I'm using:
frame = spark.read.csv(path=bucket+folder,
inferSchema = True,
header = True,
multiLine=True
)
to read in a series of CSV ...
- 2685 Views
- 1 replies
- 1 kudos
Latest Reply
Files saved in Windows operation system contain carriage return and line feed in every line.Please add following option it can help: .option("ignoreTrailingWhiteSpace", true)
- 6208 Views
- 1 replies
- 0 kudos
I'm reading binary file http://snap.stanford.edu/data/amazon/productGraph/image_features/image_features.b using pyspark.from io importStringIO import array
img_embedding_file = sc.binaryRecords("s3://bucket/image_features.b",4106)def mapper(featur...
- 6208 Views
- 1 replies
- 0 kudos
Latest Reply
Hi @tourist_on_road, please go through the below spark docs,https://spark.apache.org/docs/2.3.0/api/python/pyspark.html#pyspark.SparkContext.binaryFiles