<?xml version="1.0" encoding="UTF-8"?>
<rss xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#" xmlns:taxo="http://purl.org/rss/1.0/modules/taxonomy/" version="2.0">
  <channel>
    <title>topic How to resolve &amp;quot;java.lang.ClassNotFoundException: com.databricks.spark.util.RegexBasedAWSSecretKeyRedactor&amp;quot; when running Scala Spark project using databricks-connect ? in Data Engineering</title>
    <link>https://community.databricks.com/t5/data-engineering/how-to-resolve-quot-java-lang-classnotfoundexception-com/m-p/24787#M17251</link>
    <description>&lt;P&gt;Currently I am learning how to use databricks-connect to develop Scala code using IDE (VS Code) locally. The set-up of the databricks-connect as described here &lt;A href="https://docs.microsoft.com/en-us/azure/databricks/dev-tools/databricks-connect" alt="https://docs.microsoft.com/en-us/azure/databricks/dev-tools/databricks-connect" target="_blank"&gt;https://docs.microsoft.com/en-us/azure/databricks/dev-tools/databricks-connect&lt;/A&gt; was succuesful. I can compile PySpark scripts with no problem. However, I struggle to run the scala example as provided in the Scala &lt;A href="https://docs.databricks.com/dev-tools/databricks-connect.html#language-scala" alt="https://docs.databricks.com/dev-tools/databricks-connect.html#language-scala" target="_blank"&gt;demo object&lt;/A&gt;. I receive the java.lang.ClassNotFoundException:&lt;/P&gt;&lt;P&gt;&lt;/P&gt;&lt;P&gt;&lt;span class="lia-inline-image-display-wrapper" image-alt="image"&gt;&lt;img src="https://community.databricks.com/t5/image/serverpage/image-id/1997i644A913AD80ED262/image-size/large?v=v2&amp;amp;px=999" role="button" title="image" alt="image" /&gt;&lt;/span&gt;My build.sbt follows the suggested pattern:&lt;/P&gt;&lt;P&gt;&lt;/P&gt;&lt;P&gt;&lt;span class="lia-inline-image-display-wrapper" image-alt="build.sbt"&gt;&lt;img src="https://community.databricks.com/t5/image/serverpage/image-id/2008i19E21EEE3645BA8B/image-size/large?v=v2&amp;amp;px=999" role="button" title="build.sbt" alt="build.sbt" /&gt;&lt;/span&gt;Do you have any suggestion on what I am missing ? The error occurs when building the SparkSession. &lt;/P&gt;&lt;PRE&gt;&lt;CODE&gt;val spark = SparkSession.builder.master("local").getOrCreate()&lt;/CODE&gt;&lt;/PRE&gt;&lt;P&gt; A 'hello world' prints succesfully. &lt;/P&gt;&lt;P&gt;&lt;/P&gt;&lt;P&gt;Note: I am using Databricks on Azure&lt;/P&gt;</description>
    <pubDate>Thu, 24 Mar 2022 09:34:46 GMT</pubDate>
    <dc:creator>Databach</dc:creator>
    <dc:date>2022-03-24T09:34:46Z</dc:date>
    <item>
      <title>How to resolve "java.lang.ClassNotFoundException: com.databricks.spark.util.RegexBasedAWSSecretKeyRedactor" when running Scala Spark project using databricks-connect ?</title>
      <link>https://community.databricks.com/t5/data-engineering/how-to-resolve-quot-java-lang-classnotfoundexception-com/m-p/24787#M17251</link>
      <description>&lt;P&gt;Currently I am learning how to use databricks-connect to develop Scala code using IDE (VS Code) locally. The set-up of the databricks-connect as described here &lt;A href="https://docs.microsoft.com/en-us/azure/databricks/dev-tools/databricks-connect" alt="https://docs.microsoft.com/en-us/azure/databricks/dev-tools/databricks-connect" target="_blank"&gt;https://docs.microsoft.com/en-us/azure/databricks/dev-tools/databricks-connect&lt;/A&gt; was succuesful. I can compile PySpark scripts with no problem. However, I struggle to run the scala example as provided in the Scala &lt;A href="https://docs.databricks.com/dev-tools/databricks-connect.html#language-scala" alt="https://docs.databricks.com/dev-tools/databricks-connect.html#language-scala" target="_blank"&gt;demo object&lt;/A&gt;. I receive the java.lang.ClassNotFoundException:&lt;/P&gt;&lt;P&gt;&lt;/P&gt;&lt;P&gt;&lt;span class="lia-inline-image-display-wrapper" image-alt="image"&gt;&lt;img src="https://community.databricks.com/t5/image/serverpage/image-id/1997i644A913AD80ED262/image-size/large?v=v2&amp;amp;px=999" role="button" title="image" alt="image" /&gt;&lt;/span&gt;My build.sbt follows the suggested pattern:&lt;/P&gt;&lt;P&gt;&lt;/P&gt;&lt;P&gt;&lt;span class="lia-inline-image-display-wrapper" image-alt="build.sbt"&gt;&lt;img src="https://community.databricks.com/t5/image/serverpage/image-id/2008i19E21EEE3645BA8B/image-size/large?v=v2&amp;amp;px=999" role="button" title="build.sbt" alt="build.sbt" /&gt;&lt;/span&gt;Do you have any suggestion on what I am missing ? The error occurs when building the SparkSession. &lt;/P&gt;&lt;PRE&gt;&lt;CODE&gt;val spark = SparkSession.builder.master("local").getOrCreate()&lt;/CODE&gt;&lt;/PRE&gt;&lt;P&gt; A 'hello world' prints succesfully. &lt;/P&gt;&lt;P&gt;&lt;/P&gt;&lt;P&gt;Note: I am using Databricks on Azure&lt;/P&gt;</description>
      <pubDate>Thu, 24 Mar 2022 09:34:46 GMT</pubDate>
      <guid>https://community.databricks.com/t5/data-engineering/how-to-resolve-quot-java-lang-classnotfoundexception-com/m-p/24787#M17251</guid>
      <dc:creator>Databach</dc:creator>
      <dc:date>2022-03-24T09:34:46Z</dc:date>
    </item>
  </channel>
</rss>

