Databricks Community

Witold · 07-23-2024

The Databricks runtime is shipped with two Java Runtimes: JRE 8 and JRE 17. While the first one is used by default, you can use the environment variable JNAME to specify the other JRE: JNAME: zulu17-ca-amd64.FWIW, AFAIK JNAME is available since DBR 1...

Witold

@AlexeyEgorovThis post is bit outdated, as, starting from Databricks Runtime 16, JDK 17 is the new default.

Witold · 09-30-2024

Now I got your point. No, you can't create Append Blob files directly in Volumes, as this is a native Azure functionality. A volume is basically just an abstraction over a native storage.You will still need to use libraries like azure-storage-blob wi...

Witold · 09-30-2024

Since Databricks uses Spark, you should be able to use e.g. Columnar EncryptionBesides, you can look into this and aes-specific functions.

Witold · 09-30-2024

It makes sense that you learn how the security model of Unity Catalog works, including long-term storage credentials, external locations, etc: https://learn.microsoft.com/en-us/azure/databricks/data-governance/unity-catalog/#other-securable-objects

Witold · 09-19-2024

Multiple things.First very obvious thing: "5000000" - It's not surprising that you run OOM when loading such a huge amount of records.The DBR has a built-in MySQL driver, use it insteadUse fetchSize to control the number of records. Start e.g. with 1...

Databricks Community

User Stats

User Activity

Databricks runtime and Java Runtime

Re: Databricks runtime and Java Runtime

Re: Writing append blob files to unity catalog volum

Re: Parquet Encryption/Decryption in Databricks

Re: Writing append blob files to unity catalog volum

Re: Java heap issue, GC allocation failure while writing data from mysql to adls