Your YAML is valid, but the reason the libraries are not being installed is because base_environment_path is not supported for serverless compute. Serverless jobs use a fully managed environment and you can’t override it with a custom base environment.

If you need specific libraries, you have two options:

Use a non-serverless cluster and point base_environment_path to your YAML.

For serverless, install the required packages either by defining them under libraries in the job spec or by using %pip install inside the notebook.

In short, base_environment_path has no effect on serverless jobs.