cancel
Showing results forย 
Search instead forย 
Did you mean:ย 
Data Engineering
Join discussions on data engineering best practices, architectures, and optimization strategies within the Databricks Community. Exchange insights and solutions with fellow data engineers.
cancel
Showing results forย 
Search instead forย 
Did you mean:ย 

lakeflow connect postgres sql in uae north

tasa
Visitor

Hi can anyone tell me how to enable tc LakeFlow Connect for Direct Cdc Managed Ingestion Pipeline. it shows that this mode is not available for this workspace for the selected connector.

2 REPLIES 2

balajij8
Contributor III

Hi, 

The Integrated CDC architecture pipeline is currently not available for Postgres SQL and its feature availability issue. It's available for SQL Server and MySQL connectors in beta, Oracle in Private Preview

PostgreSQL currently uses the gateway-based architecture that uses an ingestion gateway & a separate ingestion pipeline. More details here

Ashwin_DSA
Databricks Employee
Databricks Employee

Hi @tasa,

It looks like your workspace is not currently enabled for Direct CDC/integrated CDC for that connector. In Databricks, this commonly surfaces as the DIRECT_CDC_NOT_ENABLED validation error, which means the feature is not enabled for the org or workspace yet.

If you are using SQL Server, the public documentation refers to this as an integrated CDC pipeline. That feature is currently in Beta, and the docs state that workspace admins control access from the Previews page.

Here are some options you can try...

  • Ask a workspace admin to open the workspace Previews page and check whether the relevant preview is available and enabled for that workspace.
  • If the preview does not appear, contact your Databricks account team. The preview release docs note that if you do not see a Beta feature in your workspace, you should contact your Databricks account team.
  • If you are creating the pipeline through the API, CLI, notebook, or Bundles, make sure the pipeline spec includes channel: "PREVIEW", because the integrated CDC docs require that for pipeline creation.
  • If you need to validate the error itself, see the Databricks error reference for MANAGED_INGESTION_PIPELINE_SPEC_VALIDATION_ERROR, which includes DIRECT_CDC_NOT_ENABLED.

If this answer resolves your question, could you mark it as โ€œAccept as Solutionโ€? That helps other users quickly find the correct fix.

Regards,
Ashwin | Delivery Solution Architect @ Databricks
Helping you build and scale the Data Intelligence Platform.
***Opinions are my own***