cancel
Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
Showing results for 
Search instead for 
Did you mean: 
Help
Data Engineering
Join discussions on data engineering best practices, architectures, and optimization strategies within the Databricks Community. Exchange insights and solutions with fellow data engineers.
cancel
Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
Showing results for 
Search instead for 
Did you mean: 

Forum Posts

Start a conversation
ahuarte
by • New Contributor III
  • 12953 Views
  • 17 replies
  • 3 kudos

Resolved! Getting Spark & Scala version in Cluster node initialization script

Hi there, I am developing a Cluster node initialization script (https://docs.gcp.databricks.com/clusters/init-scripts.html#environment-variables) in order to install some custom libraries.Reading the docs of Databricks we can get some environment var...

Data Engineering
Reply
  • 12953 Views
  • 17 replies
  • 3 kudos
View Replies
Latest Reply
Lingesh
New Contributor III
  • 3 kudos

We can infer the cluster DBR version using the env $DATABRICKS_RUNTIME_VERSION. (For the exact spark/scala version mapping, you can refer to the specific DBR release notes)Sample usage inside a init script, DBR_10_4_VERSION="10.4" if [[ "$DATABRICKS_...

  • 3 kudos
16 More Replies
sarvesh242
by • Contributor
  • 9239 Views
  • 3 replies
  • 2 kudos

Resolved! java.lang.NoSuchMethodError in databricks

I have created a package in scala. Now, I am calling a method from that package and using it in my notebook. During run time, it throws me an error java.lang.NoSuchMethodError. The method exists in the package but still, I am getting this error. Plea...

Data Engineering
Reply
  • 9239 Views
  • 3 replies
  • 2 kudos
View Replies
Latest Reply
sarvesh242
Contributor
  • 2 kudos

Hi! @Kaniz Fatma​ . I am using scala version 2.11 with spark 2.4.3. According to Apache spark official website https://spark.apache.org/docs/2.4.3/#:~:text=For%20the%20Scala%20API%2C%20Spark,x.) Spark 2.4.3 uses Scala 2.12. (https://spark.apache.org/...

  • 2 kudos
2 More Replies
sarvesh242
by • Contributor
  • 1694 Views
  • 1 replies
  • 2 kudos

Resolved! java.lang.NoSuchMethodError in databricks

I have created a package. Now I am calling a method from this package in my notebook but it is throwing me java.lang.NoSuchMethodError in databricks. The method exists in the package. Can you please guide me regarding the same.Thanks!

Data Engineering
Reply
  • 1694 Views
  • 1 replies
  • 2 kudos
View Replies
Latest Reply
sarvesh242
Contributor
  • 2 kudos

Hi! I am sharing the error stack with you. I can't share the code with you due to confidentiality of the code. Can you please guide me ?java.lang.NoSuchMethodError: com.iig.utils.common.IIGCommonConstants$.flowProperties()Ljava/lang/String; at com.ii...

  • 2 kudos
Labels