Relative path in absolute URI when reading a folder with files containing ":" colons in filename

[chanansh]

I am trying to read a folder with partition files where each partition is date/hour/timestamp.csv where timestamp is the exact timestamp in ISO format, e.g. 09-2022-12-05T20:35:15.2786966Z

It seems like spark having issues with reading files with colons, which is quiet ridiculous.

The issue was raised in 2019 https://issues.apache.org/jira/browse/SPARK-28841 but it seems like the issue remains.

What is the solution, rather than now renaming zillions of file names, which in s3 requires copy...

I am using Spark 3.2.1

[Debayan]

The issue was reopened again https://issues.apache.org/jira/browse/HDFS-14762

[chanansh]

I have renamed the files replacing : with - as the bug still exists