ThomazNeto
Databricks Partner

Hi Ankit,

Before building anything custom, have a look at three things that already exist in the platform. Together they cover most of what you described.

  1. Workspace base environments. A workspace admin defines a YAML with the serverless environment version and a pinned list of Python dependencies (requirements file, wheels in a UC volume, an internal index-url). Databricks pre-builds and caches it, users pick it from the Environment panel, and admins can star one as the workspace default. You're limited to 10 per workspace, so think "approved profiles" (data-eng, ml-sklearn, ml-mlflow) rather than one per person. Base environments also work on classic compute through "manage dependencies using environments". This is probably the closest thing to what you're prototyping, without the app and the Lakebase sync.
    https://docs.databricks.com/aws/en/admin/workspace-settings/base-environment
    https://docs.databricks.com/aws/en/compute/serverless/dependencies

  2. Compute policies with libraries. For classic compute, a policy can carry up to 500 libraries (PyPI, wheels in volumes, requirements.txt) that get installed automatically, and the docs are explicit about the side effect you want: "Users can't install or uninstall compute-scoped libraries on compute that use this policy." Databricks also recommends policies over init scripts for library installs. Note this blocks compute-scoped libraries; notebook-scoped %pip still works unless you cut off the index (see next point).
    https://docs.databricks.com/aws/en/admin/clusters/policies

  3. Control the source, not the list. The docs say private mirrors like Nexus or Artifactory are supported via --index-url in %pip and in base environment YAML, and admins can set a private repo as the default pip source for serverless. If InfoSec curates what's in the mirror, the approval happens once, at the repository, and every install path inherits it. That's cleaner than approving versions inside Databricks.
    https://docs.databricks.com/aws/en/libraries/

One more, since you tagged Unity Catalog: on standard access mode, JARs, Maven coordinates and init scripts need to be in the UC artifact allowlist (MANAGE ALLOWLIST privilege). It doesn't cover PyPI, so for Python the mirror is the control point.
https://docs.databricks.com/aws/en/data-governance/unity-catalog/manage-privileges/allowlist

Your app might still make sense as the request/approval UI, but I'd have it write base environments and policy libraries rather than manage installs itself.

Hope this helps.

Thomaz A. Rossito Neto
Principal Data Architect & AI Strategy — CI&T
thomazn@ciandt.com
linkedin.com/in/thomaz-antonio-rossito-neto