[SQL_CONF_NOT_FOUND] The SQL config "/Volumes/xxx...." canot be found. Please verify that the confi

[DataGeek_JT]

I am getting the below error when trying to stream data from Azure Storage path to a Delta Live Table ([PATH] is the path to my files which I have redacted here):

[SQL_CONF_NOT_FOUND] The SQL config "/Volumes/[PATH]" cannot be found. Please verify that the config exists. SQLSTATE: 42K0I

I can read data from the volume in Unity Catalogue using spark.read.csv("/Volumes/[PATH]") but it seems to think there is an issue doing it this way. 

The notebook code is:

import dlt

from pyspark.sql import functions as F

data_source_path = spark.conf.get("/Volumes/[PATH]")

@dlt.table

def ticks():

    return (spark.readStream

        .format("cscloudFilesv")

        .option("cloudFiles.format", "csv")

        .option("header", True)

        .load(data_source_path )

        .select("*", F.col("_metadata.file_path").alias("source_file_name"),  F.split(F.col("_metadata.file_path"), "/")[8].alias("Symbol"))

    )

Any idea what is wrong please? 

[NandiniN]

I believe you are not setting 

spark.conf.set("/Volumes/[PATH]", "your_actual_path_here")

hence when you try to get the conf, it fails. 

data_source_path = spark.conf.get("/Volumes/[PATH]")

"/Volumes/[PATH]" becomes the conf name, you would not want to name it with / and [] rather like read_path.