databricks api to create a serverless job

anazen13
New Contributor III

I am trying to follow your documentation on how to create serverless job via API https://docs.databricks.com/api/workspace/jobs/create#environments-spec-environment_version So i see that sending the json request resulted for me to see serverless cluster for my test job in Databricks UI. However even though i specify the environment or dependencies in my json, they seem to be completely ignored. By default i get serverless environement 3. Can you give me a hint of what is going wrong in this case?

Here is my json 

{
              "environments": [
                {
                  "environment_key": "default_python",
                  "spec": {
                    "environment_version": "1",
                    "dependencies": [
                      "/Volumes/<mycatalog>/..."
                    ]
                  }
                }
              ],
              "name": "${{ parameters.TEST_RUN_TITLE }}",
              "tasks": [
                {
                  "task_key": "pytest_task",
                  "notebook_task": {
                    "notebook_path": "${{ parameters.NOTEBOOK_PATH }}",
                    "base_parameters": {
                      "build_id": "$(Build.BuildId)"
                    }
                  }
                }
              ]
            }