- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
09-23-2025 08:18 AM
Hi @Malthe
The recommended method is to manage and trigger the sync via table-level APIs or management interfaces, not the pipeline-level job triggers:
-
For Unity Catalog synced tables (e.g., syncing to Postgres), triggering a sync or refresh is performed by interacting with the table’s own API endpoint rather than the pipeline overview, such as using the
refreshAPI or a manual command at the table level. -
If using the Unity Catalog or related table management system, you can issue an API call or use the Databricks CLI to manually trigger a sync (refresh), schedule triggers, or monitor the status of the ingest process at the table level.
-
This design separates table sync pipelines from user-managed job pipelines to ensure that ingestion is strictly controlled via the target table’s management UI/API, maintaining data integrity and reliability.
If your use case demands automated or programmatic triggering (for instance, after upstream data changes), it’s best practice to:
-
Use the synced table’s refresh API endpoint in your automation/orchestration tools.
-
Monitor and manage ingestion through the table sync management interface, not the pipeline jobs dashboard.
For additional details please refer to the below doc
https://docs.databricks.com/aws/en/dev-tools/ci-cd/best-practices