Databricks App deployment failing due to mysqlclient build error (pkg-config not found)

Data Engineering

Join discussions on data engineering best practices, architectures, and optimization strategies within the Databricks Community. Exchange insights and solutions with fellow data engineers.

Hi Team,

I am trying to deploy a Python project as a Databricks App, but the deployment fails during dependency installation with the following error:

ERROR: Failed to build 'mysqlclient' when getting requirements to build wheel pkg-config: not found Exception: Can not find valid pkg-config name.

From the logs, it looks like pip is attempting to build mysqlclient from source, but the Databricks App runtime does not have access to system-level packages such as pkg-config or MySQL client libraries.

Relevant log snippet:

Exception: Can not find valid pkg-config name. Specify MYSQLCLIENT_CFLAGS and MYSQLCLIENT_LDFLAGS env vars manually

The dependency appears to be pulled indirectly via:

ibis-framework[mysql]

Are there any Databricks-approved workarounds for native dependencies in Apps, or is using pure-Python drivers the only option?

Any guidance or best practices from the community would be really helpful.

Thanks in advance!