topic Re: how to create a scala jar using db notebook and save it in a file path inside databricks in Data Engineering

how to create a scala jar using db notebook and save it in a file path inside databricks

swatish0395 — Thu, 11 May 2023 09:59:39 GMT

I have scala function as below, i am unable to understand how to write a scala jar with the same,

please find below code i have used Enforcing Column-Level Encryption - Databrick

%scala

import com.macasaet.fernet.{Key, StringValidator, Token}

import org.apache.hadoop.hive.ql.exec.UDF;

import java.time.{Duration, Instant};

class Validator extends StringValidator {

override def getTimeToLive() : java.time.temporal.TemporalAmount = {

Duration.ofSeconds(Instant.MAX.getEpochSecond());

}

class udfDecrypt extends UDF {

def evaluate(inputVal: String, sparkKey : String): String = {

if( inputVal != null && inputVal!="" ) {

val keys: Key = new Key(sparkKey)

val token = Token.fromString(inputVal)

val validator = new Validator() {}

val payload = token.validateAndDecrypt(keys, validator)

payload

} else return inputVal

}

please tell ema solution to create a jar with the above scala function inside DB notebook

#Scala #DBnotebook #Jars #[Jar Scala]

Re: how to create a scala jar using db notebook and save it in a file path inside databricks

-werners- — Fri, 12 May 2023 08:32:07 GMT

for a jar you need to build it (using SBT for example). Notebooks are not able to do that.

I create jars in IntelliJ with SBT.

What you can do is not using a jar at all but use package cells.

However I do not think those are used a lot because a while ago I asked a question on the forum who uses package cells and nobody confirmed they did.

Re: how to create a scala jar using db notebook and save it in a file path inside databricks

Anonymous — Mon, 22 May 2023 06:43:43 GMT

Hi @swati sharma

Thank you for posting your question in our community! We are happy to assist you.

To help us provide you with the most accurate information, could you please take a moment to review the responses and select the one that best answers your question?

This will also help other community members who may have similar questions in the future. Thank you for your participation and let us know if you need any further assistance!

Re: how to create a scala jar using db notebook and save it in a file path inside databricks

swatish0395 — Tue, 23 May 2023 10:50:49 GMT

I had to finally create the jar using teh intellij and sbt iconfiguration on the same env. and then installed the jar in the cluster it worked