Currently I am learning how to use databricks-connect to develop Scala code using IDE (VS Code) locally. The set-up of the databricks-connect as described here https://docs.microsoft.com/en-us/azure/databricks/dev-tools/databricks-connect was succuesful. I can compile PySpark scripts with no problem. However, I struggle to run the scala example as provided in the Scala demo object. I receive the java.lang.ClassNotFoundException:
My build.sbt follows the suggested pattern:
Do you have any suggestion on what I am missing ? The error occurs when building the SparkSession.
val spark = SparkSession.builder.master("local").getOrCreate()
A 'hello world' prints succesfully.
Note: I am using Databricks on Azure