โ04-11-2023 11:35 PM
I am new to azure databricks and I want to install a library on a cluster and to do that I need to install bazel build tool first.
I checked the site bazel but I am still not sure how to do it in databricks?
I appriciate if any can help me and write me some steps
โ04-14-2023 12:25 AM
Databricks migrated over from the standard Scala Build Tool (SBT) to using Bazel to build, test and deploy our Scala code. Follow this doc https://www.databricks.com/blog/2019/02/27/speedy-scala-builds-with-bazel-at-databricks.html
โ04-14-2023 12:41 AM
Thanks, I read this article but I am still not sure how to make use of it. The original aim is to install library which explicitly mentioned the requirement to install first bazel. The article just explain how to use bazel without pointing how to install it or build it. Any more assistance , appriciate it
โ04-14-2023 12:51 AM
Hi Found this github repo talking about rules https://github.com/acqio/rules_databricks
โ04-14-2023 01:39 AM
Thanks, but isn't this repo also assume that bazel is installed?
I am saying this because of the second step in the link
"Add the flag --build_python_zip following to your bazelrc to create a python executable zip: "
so it assume i have bazelrc ! ?
โ04-14-2023 02:13 AM
I think we are looking at it wrong i think bezel need to be installed in a separate linux vm and then execute after configuring databricks cli on VM as it ask in authentication section. Post we are able to interact with databricks cluster via cli we can install libraries using this https://github.com/acqio/rules_databricks/blob/master/docs/dbk_libraries.md
Join a Regional User Group to connect with local Databricks users. Events will be happening in your city, and you wonโt want to miss the chance to attend and share knowledge.
If there isnโt a group near you, start one and help create a community that brings people together.
Request a New Group