Saritha_S
Databricks Employee
Databricks Employee

Hi @Malthe 

The recommended method is to manage and trigger the sync via table-level APIs or management interfaces, not the pipeline-level job triggers:

  • For Unity Catalog synced tables (e.g., syncing to Postgres), triggering a sync or refresh is performed by interacting with the table’s own API endpoint rather than the pipeline overview, such as using the refresh API or a manual command at the table level.

  • If using the Unity Catalog or related table management system, you can issue an API call or use the Databricks CLI to manually trigger a sync (refresh), schedule triggers, or monitor the status of the ingest process at the table level.

  • This design separates table sync pipelines from user-managed job pipelines to ensure that ingestion is strictly controlled via the target table’s management UI/API, maintaining data integrity and reliability.

If your use case demands automated or programmatic triggering (for instance, after upstream data changes), it’s best practice to:

  • Use the synced table’s refresh API endpoint in your automation/orchestration tools.

  • Monitor and manage ingestion through the table sync management interface, not the pipeline jobs dashboard.

For additional details please refer to the below doc 

https://docs.databricks.com/aws/en/dev-tools/ci-cd/best-practices

View solution in original post