Greetings @bragoarefur , Thanks for laying out the context so clearlyโyour read of the current state is spot on. Below is what the Databricks docs actually say, and a recommendation with an implementation path that avoids polling while staying cloud-agnostic.
What the docs say
- Workspace Model Registry webhooks are in Public Preview and let you trigger on events such as MODEL_VERSION_CREATED, but they are explicitly not available when using Models in Unity Catalog (UC).
-
The UC migration guideโs โmanual approvalโ alternative effectively suggests using job notifications to call out to external CI/CD systems for approvals, rather than event-driven registry webhooks; it does not provide an equivalent eventing facility for UC models.
-
Models in Unity Catalog do not have an activity log in the UI; tracking activity requires using audit logs.
Viable alternatives in UC
You identified the two main paths; hereโs how they stack up against the platformโs capabilities:
-
Option #1: Poll UC for new model versions. This works via MLflow client calls (for example, listing model versions), but thereโs no native activity log to subscribe to, so you end up implementing your own state tracking and timers. Itโs simple, but inefficient and introduces latency and drift.
-
Option #2: Consume Databricks audit logs (System Tables).
- UC emits audit events for model lifecycle actions, including createModelVersion, finalizeModelVersion, getModelVersion, updateModelVersion, deleteModelVersion, and setRegisteredModelAlias, among othersโexactly the signals you need to detect new versions and promotions.
-
These are available via the system.access.audit system table (Public Preview) governed by UC, with a stable schema and retention, and they can be consumed with streaming or scheduled queries. This is Databricks-hosted and consistent across clouds, removing the need to set up cloud-specific storage to receive logs.
-
Note that system tables are updated throughout the day (not true real-time). If you need sub-minute latencies, you may still want a narrow, model-specific overlay (e.g., short-interval polling) to bridge the gap for critical models.
Recommendation
Given your requirements and trade-offs, I recommend Option #2โuse audit log System Tables as the backbone, with a small overlay for truly latency-sensitive models:
-
Use system.access.audit to detect UC model events (e.g., createModelVersion, finalizeModelVersion, setRegisteredModelAlias). Itโs scalable, robust, and cloud-agnostic from your perspective because Databricks hosts the storage and shares it via Delta Sharing across clouds.
-
For the small subset of models where sub-minute reaction time matters, add a targeted polling overlay (MLflow client list/search) with short intervals. Keep this scoped to only those models to avoid system-wide inefficiency. The general model activity is not exposed via a UC activity log, so polling would be the only way to get that near-real-time signal.
-
If you still require a manual approval step, use job notifications to fan out to your external CI/CD systems and gate deployment there; this is the UC guideโs recommended approach for manual approvals post-train.
Reference implementation sketch (System Tables)
You can stream from the audit system table and trigger downstream jobs or notifications when specific UC model events arrive. Example in PySpark:
from pyspark.sql.functions import col
# Stream audit logs
audit_stream = (
spark.readStream.table("system.access.audit")
.where(col("service_name") == "unityCatalog")
.where(col("action_name").isin(
"createModelVersion",
"finalizeModelVersion",
"setRegisteredModelAlias",
"updateModelVersion",
"deleteModelVersion"
))
)
# Write matching events to a Bronze table or directly to a trigger function
query = (
audit_stream.writeStream
.option("checkpointLocation", "dbfs:/checkpoints/uc_model_events")
.toTable("prod.ml_ops.uc_model_events_bronze")
)
And a simple SQL view over system.access.audit for scheduled detection:
CREATE OR REPLACE VIEW prod.ml_ops.uc_model_events AS
SELECT
event_time,
action_name,
request_params,
user_identity.email AS actor
FROM system.access.audit
WHERE service_name = 'unityCatalog'
AND action_name IN (
'createModelVersion',
'finalizeModelVersion',
'setRegisteredModelAlias',
'updateModelVersion',
'deleteModelVersion'
);
Operational notes:
- Ensure your workspace is enabled for Unity Catalog and that metastore/account admins grant USE/SELECT on the system schemas so your pipelines can read system tables.
-
System table data is Databricks-hosted in the same region as your metastore; you donโt need to configure S3/ABFS/GCS delivery to consume these via SQL/streaming, which addresses cloud-specific storage concerns.
-
Expect โupdated throughout the dayโ semantics rather than true real-time; tune trigger intervals and watermarking accordingly, and consider the hybrid overlay for the smallest set of latency-sensitive models.
Whatโs missing today
- There is currently no webhook equivalent for UC modelsโthe docs explicitly state UC models do not support registry webhooks.
- For manual approval, the recommended path is post-train job notifications to external CI/CD rather than registry webhooks.
Bottom line
Use the audit log system.access.audit table for event-driven workflows on UC models, and reserve polling for the few cases where ultra-low latency is critical. This keeps the architecture elegant, scalable, and cloud-agnostic, while acknowledging the current absence of UC webhooks.
Hope this helps, Louis.
