topic Re: examples on python sdk for install libraries in Data Engineering

examples on python sdk for install libraries

saiprasadambati — Tue, 09 May 2023 23:00:52 GMT

Hi Everyone,

I'm planning to use databricks python cli "install_libraries"

can some one pls post examples on function install_libraries

https://github.com/databricks/databricks-cli/blob/main/databricks_cli/libraries/api.py

Re: examples on python sdk for install libraries

pvignesh92 — Wed, 10 May 2023 07:09:09 GMT

@saiprasad ambati prabhakar Are you having any issues while installing that?

Re: examples on python sdk for install libraries

artsheiko — Wed, 10 May 2023 12:14:01 GMT

databricks libraries install --cluster-id 1234-567890-lest123 --jar dbfs:/test-dir/test.jar

Doc : https://docs.databricks.com/dev-tools/cli/libraries-cli.html

Re: examples on python sdk for install libraries

saiprasadambati — Wed, 10 May 2023 13:50:38 GMT

i dont have any issues with installation, im looking for examples to use the python function install_libraries , i want to install libraries via python sdk

Re: examples on python sdk for install libraries

saiprasadambati — Wed, 10 May 2023 13:51:41 GMT

hi @Artem Sheiko , im looking for python sdk example to install libraries in the cluster, pls let me know if you have any

here is the github link im following

https://github.com/databricks/databricks-cli/blob/main/databricks_cli/libraries/api.py

Re: examples on python sdk for install libraries

saiprasadambati — Wed, 10 May 2023 14:24:05 GMT

okay, i figured it out , here is the example

from databricks_cli.sdk.api_client import ApiClient

from databricks_cli.libraries.api import LibrariesApi

awsapi_client = ApiClient(

host="********************",

token="*******************",

)

awslibrary_api = LibrariesApi(awsapi_client)

awslibrary_api.install_libraries('0510-140152-aaaaaaaa', {'pypi': {'package': 'mlflow==2.2.2'}} )

Re: examples on python sdk for install libraries

Loop-Insist — Fri, 25 Aug 2023 14:15:54 GMT

Here you go using Python SDK

from databricks.sdk import WorkspaceClient

from databricks.sdk.service import compute

w = WorkspaceClient(host="yourhost", token="yourtoken")

# Create an array of Library objects to be installed

libraries_to_install = [

compute.Library(pypi=compute.PythonPyPiLibrary(package="packagename==version")) # Specify the PyPi package

]

w.libraries.install(cluster_id="yourclusterid", libraries=libraries_to_install)

Re: examples on python sdk for install libraries

brina — Tue, 14 Nov 2023 19:14:42 GMT

Hello! I was able to install a library using python sdk - Task configuration. Please see the code below! Hope it helps! 🙂

from databricks.sdk.service.jobs import Task from databricks.sdk.service.compute import ( Library, PythonPyPiLibrary, ) Task(...<other config>, libraries=[Library(pypi=PythonPyPiLibrary(package="package_name==version"))], )