Hello,
We are trying to run a job with a spark-sumit task on cluster mode, this spark submit task requires a configuration file application.conf that we provide using --files flag option in the spark-submit parameters and put an alias on it using \#application.conf (\ is just used to avoid highlight in this text box), we specify the configuration file using "spark.driver.extraJavaOptions=-Dconfig.file=application.conf and we are getting a java.io.FileNotFoundException each time, either this file is placed on DBFS or local file system. log4j logs show that file is fetched and copied with timestamp but then fails when main class tries to parse it. Any idea how to fix this issue?
Thanks in advance
