โ02-13-2024 11:45 PM
We aim to streamline the UCX installation process by utilizing Databricks CLI and automating the manual input of required details at each question level .
Could you please guide us how can we achieve to automate the parameter's while installation? what are the various options to achieve this? do we any pre provided API or should it be done through programming?
Is there a reason for disabling the manual/clone based installation and is only available option being through Databricks CLI?
โ02-14-2024 12:04 AM
Hi @Phani1, Automating the UCX installation process can significantly enhance efficiency and reduce human error.
Letโs explore the options for automating parameter input during installation:
Databricks CLI:
Configuration Files:
Environment Variables:
Pre-Provided APIs:
Custom Scripts:
Regarding the reason for disabling manual/clone-based installation:
Remember to document your automated installation process thoroughly, including the parameters used and any specific considerations for your environment.
Happy automating! ๐
โ02-14-2024 12:12 AM
How do we pass parameters via config.yml file of UCX ?
โ02-14-2024 12:16 AM
Hi @Phani1,
To pass parameters via the config.yml
file of UCX in Databricks, you can follow these steps:
Create a Databricks Job:
databricks jobs create --json-file ./deploy/databricks/config/job.config.json
job.config.json
), define the job properties such as name, cluster settings, libraries, timeout, and schedule.Add Parameters to the Notebook Task:
notebook_task
property to include base_parameters
. For instance:
{
"notebook_task": {
"notebook_path": "/notebooks/python_notebook",
"base_parameters": {
"argument1": "value 1",
"argument2": "value 2"
}
}
}
dbutils.widgets.get("argument1")
and similar methods.Execute the Job:
โ02-15-2024 10:41 PM
Hi Team,
We don't see option at UCX command level passing parameters as json /config file, could you please help me in this case how we can automate the installation.
Join a Regional User Group to connect with local Databricks users. Events will be happening in your city, and you wonโt want to miss the chance to attend and share knowledge.
If there isnโt a group near you, start one and help create a community that brings people together.
Request a New Group