cancel
Showing results for 
Search instead for 
Did you mean: 
Data Engineering
Join discussions on data engineering best practices, architectures, and optimization strategies within the Databricks Community. Exchange insights and solutions with fellow data engineers.
cancel
Showing results for 
Search instead for 
Did you mean: 

Installing Bazel on databricks cluster

JGil
New Contributor III

I am new to azure databricks and I want to install a library on a cluster and to do that I need to install bazel build tool first.

I checked the site bazel but I am still not sure how to do it in databricks?

I appriciate if any can help me and write me some steps

5 REPLIES 5

Avinash_94
New Contributor III

Databricks migrated over from the standard Scala Build Tool (SBT) to using Bazel to build, test and deploy our Scala code. Follow this doc https://www.databricks.com/blog/2019/02/27/speedy-scala-builds-with-bazel-at-databricks.html

JGil
New Contributor III

Thanks, I read this article but I am still not sure how to make use of it. The original aim is to install library which explicitly mentioned the requirement to install first bazel. The article just explain how to use bazel without pointing how to install it or build it. Any more assistance , appriciate it

Avinash_94
New Contributor III

Hi Found this github repo talking about rules https://github.com/acqio/rules_databricks

JGil
New Contributor III

Thanks, but isn't this repo also assume that bazel is installed?

I am saying this because of the second step in the link

"Add the flag  --build_python_zip following to your bazelrc to create a python executable zip: "

so it assume i have bazelrc ! ?

Avinash_94
New Contributor III

I think we are looking at it wrong i think bezel need to be installed in a separate linux vm and then execute after configuring databricks cli on VM as it ask in authentication section. Post we are able to interact with databricks cluster via cli we can install libraries using this https://github.com/acqio/rules_databricks/blob/master/docs/dbk_libraries.md

Connect with Databricks Users in Your Area

Join a Regional User Group to connect with local Databricks users. Events will be happening in your city, and you won’t want to miss the chance to attend and share knowledge.

If there isn’t a group near you, start one and help create a community that brings people together.

Request a New Group