data-guy
New Contributor III

Hello!

I got the same error a few days ago and I resolved it with this post that I found:

Accessing ADLS Gen 2 with RDD | Data Engineering (data-engineering.wiki)

Basically, the key is to setup the properties to "hadoop" using spark.sparkContext.hadoopConfiguration.set(...)

I hope you solve your problem!

View solution in original post