Can't enable CLI 2.1 on CI

[alejandrofm]

Hi! this is my CI configuration, I added the databricks jobs configure --version=2.1 command but it stills showing this error, any idea of what can I be doing wrong?

Error:

Resetting Databricks Job with job_id 1036...

WARN: Your CLI is configured to use Jobs API 2.0. In order to use the latest Jobs features please upgrade to 2.1: 'databricks jobs configure --version=2.1'. Future versions of this CLI will default to the new Jobs API. Learn more at https://docs.databricks.com/dev-tools/cli/jobs-cli.html

YML

image: python:3.6-alpine
 
stages:
 
 - build
 
 - deploy
 
​
 
before_script:
 
   - apk add jq
 
   - echo "Installing Databricks CLI..." 
 
   - pip3 install databricks-cli
 
   - pip3 show databricks-cli
 
   - echo "Configuring Databricks profile..."
 
   - echo "[DEFAULT]" > ~/.databrickscfg
 
   - echo "host = ${DATABRICKS_HOST}" >> ~/.databrickscfg
 
   - echo "token = ${DATABRICKS_TOKEN}" >> ~/.databrickscfg
 
   - databricks jobs configure --version=2.1
 
   - source ci/utils.sh
 
​
 
build:
 
   stage: build
 
   only:
 
       - master
 
   script:
 
       - echo "Compiling Python package in EGG distribution..."
 
       - python3 setup.py bdist_egg
 
       - export PACKAGE_VERSION=$(python setup.py --version)
 
       - echo "Pushing compiled library to DBFS..."
 
       - dbfs cp --overwrite dist/${CI_PROJECT_NAME}-${PACKAGE_VERSION}-py3.6.egg dbfs:/FileStore/build/${CI_PROJECT_NAME}.egg
 
​
 
deploy:
 
   stage: deploy
 
   only:
 
       - master
 
   script:
 
       - databricks jobs configure --version=2.1
 
       - echo "Pushing compiled library to DBFS..."
 
       - dbfs cp --overwrite dbfs:/FileStore/build/${CI_PROJECT_NAME}.egg dbfs:/FileStore/eggs/${CI_PROJECT_NAME}.egg
 
       - dbfs rm dbfs:/FileStore/build/${CI_PROJECT_NAME}.egg
 
       - upload_process_scripts ${DATABRICKS_MAIN_SCRIPT_PATH} ${CI_PROJECT_NAME}
 
       - create_jobs ${DATABRICKS_JOBS_PATH}
 
   when: manual
 
​

Thanks!

[Anonymous]

Hello, @Alejandro Martinez​ - My name is Piper, and I'm a moderator for Databricks. I apologize that it has taken so long to respond to you. 😞

We will give it a bit longer to see what the community has to say before we come back around if we need to.

Thanks for your patience.

[Kaniz · Accepted solution]

Hi @Alejandro Martinez​ , To set up and use the Databricks jobs CLI (and the job runs CLI) to call the Jobs REST API 2.1, Update the CLI to version 0.16.0 or above.

Run pip install databricks-cli --upgrade using the appropriate version of pip for your Python installation.

To list the version of the CLI that is currently installed, run 

databricks --version (or data bricks -v).