I used to use dbfs with mounted directories and now I want to switch to Volumes for storing my jars and application.conf for pipelines.
I see the file my application.conf in Data Explorer > Catalog > Volumes, I also see the file with dbutils.fs.ls("/Volumes/.../path/to/file"),
but I'm not able to read it with Scala...
> dbutils.fs.ls("/Volumes/.../application.conf")
Cell output:
res52: Seq[com.databricks.backend.daemon.dbutils.FileInfo] = ArrayBuffer(FileInfo(/Volumes/.../application.conf, application.conf, 1602, 1693576640000))
My approaches:
1. pureconfig + custom case class
import pureconfig.{ConfigSource, ConfigReader, ConfigConvert}
import pureconfig.generic.auto._
val config = ConfigSource.file("/Volumes/.../application.conf")
config.loadOrThrow[Config]
Error: ConfigReaderException: Cannot convert configuration to a Config. Failures are: - Unable to read file /Volumes/.../application.conf (No such file or directory).
2. Scala io Source
import scala.io.Source
Source.fromFile("/Volumes/.../application.conf")
Error: FileNotFoundException: /Volumes/.../application.conf (No such file or directory)
3. java io
import java.io.{FileReader, File}
val fr = new FileReader(new File("/Volumes/.../application.conf"))
Error: FileNotFoundException: /Volumes/.../application.conf (No such file or directory)
4. java nio
import java.nio.charset.StandardCharsets
import java.nio.file.{Files, Paths}
val path = Paths.get("/Volumes/.../application.conf")
new String(Files.readAllBytes(path), StandardCharsets.UTF_8)
Error: FileNotFoundException: /Volumes/.../application.conf (No such file or directory)