DATABRICKS LAKEFLOW SQL SERVER INGESTION PIPELINE ERROR

[ShivangiB1]

Hey Team,

I am getting below error while creating pipeline :

com.databricks.pipelines.execution.extensions.managedingestion.errors.ManagedIngestionNonRetryableException: [INGESTION_GATEWAY_DDL_OBJECTS_MISSING] DDL objects missing on table 'coedb.dbo.songs_new'. Execute the DDL objects script and full refresh the table on the Ingestion Pipeline. Error message: ' Reason: - Catalog is not properly configured to capture DDL changes for the provided set of tables.

I did enabled cdc and ct both, can u please help on same.

[ShivangiB1]

Hey Team, can anyone help on this

[mark_ott]

The error you are seeing means Databricks cannot capture DDL (table definition) changes, even though CDC (Change Data Capture) and CT (Change Tracking) are enabled. You must run the specific DDL support objects script for Databricks ingestion and then do a full refresh of the table.

Required Actions

• Download and run the utility script (ddl_support_objects.sql) for your SQL Server. This script creates the database objects needed for Databricks ingestion to track DDL (schema) changes alongside CDC/CT changes.​

• Make sure you run the script with the correct mode (use BOTH if you want to support both CDC and CT; use CDC if only CDC is needed). Example:

  text

  -- Run this as a SQL command in your database EXEC dbo.lakeflowSetupChangeDataCapture @Tables = 'coedb.dbo.songs_new', @user = 'your_ingestion_user';

• You may need to grant the required permissions on the tables for the user that will run the pipeline.​

• After running the script, do a “full refresh” of the table in the Databricks ingestion pipeline. This will allow the pipeline to recognize the new DDL support objects.

Troubleshooting Tips

• Ensure you’re using the exact table name (coedb.dbo.songs_new) and database user relevant to your setup.

• If you still see errors, check that the table has a primary key. CDC prefers tables with primary keys, though tables without them can be supported with the script.​

• Double-check that CT and/or CDC is enabled and running using SQL Server's system views and jobs.

• If you only enabled CDC/CT but did not run the DDL support script, Databricks ingestion cannot capture schema changes and will show this error.