Hi @Sparktaculer, When reading a file from GCS using Scala, if the file path contains square brackets, you can try encoding the square brackets using URL encoding. For example, replace "[" with "%5B" and "]" with "%5D". Then use the encoded file path in your code.
Here's an example:
import org.apache.hadoop.fs.{FileSystem, Path}
import java.net.URI
val path = "gs://my-bucket/path/with/%5Bsquare%5D/brackets.csv"
val fs = FileSystem.get(new URI(path), sc.hadoopConfiguration)
val file = fs.open(new Path(path))
val lines = scala.io.Source.fromInputStream(file).getLines()
In this example, the file path contains the square brackets encoded as "%5B" and "%5D". The FileSystem.get
method is used to get a handle to the file system, and the fs.open
method is used to open the file. Finally, the scala.io.Source.fromInputStream
method is used to read the contents of the file.Sources:
- https://docs.databricks.com/data/data-sources/read-gcs.html
- https://en.wikipedia.org/wiki/Percent-encoding