<?xml version="1.0" encoding="UTF-8"?>
<rss xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#" xmlns:taxo="http://purl.org/rss/1.0/modules/taxonomy/" version="2.0">
  <channel>
    <title>topic Re: Lakebase Postgre updating Delta Table. in Data Engineering</title>
    <link>https://community.databricks.com/t5/data-engineering/lakebase-postgre-updating-delta-table/m-p/167243#M55655</link>
    <description>&lt;P&gt;&lt;a href="https://community.databricks.com/t5/user/viewprofilepage/user-id/177778"&gt;@rkhbo3003&lt;/a&gt;&amp;nbsp;&lt;/P&gt;&lt;P&gt;You can use Lakebase Change Data Feed (CDF) for it. It is the pattern for this Postgres to Delta ingestion path. It continuously streams every INSERT, UPDATE and DELETE from the Lakebase Postgres write-ahead log (WAL) directly into Unity Catalog Delta tables with short latency. You don't need additional external CDC tooling or maintain custom streaming clusters. You can enable &lt;STRONG&gt;REPLICA IDENTITY FULL&lt;/STRONG&gt; on the Postgres &lt;STRONG&gt;tables&lt;/STRONG&gt; and start the feed directly from the Lakebase UI. POS transactions land automatically in Delta as lb_pos_transactions_history appended with standard CDC metadata columns (_pg_change_type, _pg_commit_lsn and _pg_commit_timestamp).&lt;/P&gt;&lt;P&gt;It completes a fully managed bi-directional sync loop with zero custom ETL overhead combined with the existing Delta-to-Postgres synced tables&lt;/P&gt;&lt;P&gt;&lt;STRONG&gt;OLTP Writes (Postgres) -&lt;/STRONG&gt; POS terminals write to writable Postgres tables for millisecond latency transaction processing.&amp;nbsp;OLTP to Delta (Lakebase CDF) WAL changes stream asynchronously to Delta history tables for analytics and downstream consumption.&lt;BR /&gt;&lt;STRONG&gt;Delta to Postgres (Synced Tables) -&lt;/STRONG&gt; Curated reference data such as product catalogs, pricing and customer dimensions replicates back to Postgres for fast, read-only lookups&lt;/P&gt;&lt;P&gt;Lakebase CDF only generates append only change logs (lb_&amp;lt;table_name&amp;gt;_history). It does not maintain an inplace and current state replica table out of the box. To maintain a live current-state table such as pos_transactions_current, you can use the CDC history table and &lt;STRONG&gt;process&lt;/STRONG&gt; the &lt;STRONG&gt;change feed downstream&lt;/STRONG&gt;. You can run a periodic &lt;STRONG&gt;MERGE&lt;/STRONG&gt; statement on a &lt;STRONG&gt;scheduled job&lt;/STRONG&gt; every few minutes, or create an &lt;STRONG&gt;auto updating&lt;/STRONG&gt; Lakeflow &lt;STRONG&gt;Streaming Table / Materialized View&lt;/STRONG&gt; that &lt;STRONG&gt;applies the CDC events&lt;/STRONG&gt; continuously. It provides an immutable audit trail in the _history table while delivering an up-to-date snapshot for operational analytics. You can check the bi directional sync &lt;A href="https://community.databricks.com/t5/lakebase-articles/databricks-lake-base-enterprise-healthcare-data-intelligence-via/td-p/156407" target="_self"&gt;here&lt;/A&gt;&lt;/P&gt;&lt;P&gt;More details &lt;A href="https://docs.databricks.com/aws/en/oltp/projects/lakebase-cdf#compute-current-inventory-with-a-materialized-view" target="_self"&gt;here&lt;/A&gt;&lt;/P&gt;</description>
    <pubDate>Wed, 02 Sep 2026 04:43:14 GMT</pubDate>
    <dc:creator>balajij8</dc:creator>
    <dc:date>2026-09-02T04:43:14Z</dc:date>
    <item>
      <title>Lakebase Postgre updating Delta Table.</title>
      <link>https://community.databricks.com/t5/data-engineering/lakebase-postgre-updating-delta-table/m-p/167234#M55653</link>
      <description>&lt;P&gt;I am using Postgre for OLTP processing for POS application.Lag is reduced a lot, however when there is updation on Postgre table, I need to sync back to delta table. There is one way from delta table sync table (read only). Any design pattern sas to keep delta table and postgre table in sync.&lt;/P&gt;</description>
      <pubDate>Wed, 02 Sep 2026 01:44:54 GMT</pubDate>
      <guid>https://community.databricks.com/t5/data-engineering/lakebase-postgre-updating-delta-table/m-p/167234#M55653</guid>
      <dc:creator>rkhbo3003</dc:creator>
      <dc:date>2026-09-02T01:44:54Z</dc:date>
    </item>
    <item>
      <title>Re: Lakebase Postgre updating Delta Table.</title>
      <link>https://community.databricks.com/t5/data-engineering/lakebase-postgre-updating-delta-table/m-p/167243#M55655</link>
      <description>&lt;P&gt;&lt;a href="https://community.databricks.com/t5/user/viewprofilepage/user-id/177778"&gt;@rkhbo3003&lt;/a&gt;&amp;nbsp;&lt;/P&gt;&lt;P&gt;You can use Lakebase Change Data Feed (CDF) for it. It is the pattern for this Postgres to Delta ingestion path. It continuously streams every INSERT, UPDATE and DELETE from the Lakebase Postgres write-ahead log (WAL) directly into Unity Catalog Delta tables with short latency. You don't need additional external CDC tooling or maintain custom streaming clusters. You can enable &lt;STRONG&gt;REPLICA IDENTITY FULL&lt;/STRONG&gt; on the Postgres &lt;STRONG&gt;tables&lt;/STRONG&gt; and start the feed directly from the Lakebase UI. POS transactions land automatically in Delta as lb_pos_transactions_history appended with standard CDC metadata columns (_pg_change_type, _pg_commit_lsn and _pg_commit_timestamp).&lt;/P&gt;&lt;P&gt;It completes a fully managed bi-directional sync loop with zero custom ETL overhead combined with the existing Delta-to-Postgres synced tables&lt;/P&gt;&lt;P&gt;&lt;STRONG&gt;OLTP Writes (Postgres) -&lt;/STRONG&gt; POS terminals write to writable Postgres tables for millisecond latency transaction processing.&amp;nbsp;OLTP to Delta (Lakebase CDF) WAL changes stream asynchronously to Delta history tables for analytics and downstream consumption.&lt;BR /&gt;&lt;STRONG&gt;Delta to Postgres (Synced Tables) -&lt;/STRONG&gt; Curated reference data such as product catalogs, pricing and customer dimensions replicates back to Postgres for fast, read-only lookups&lt;/P&gt;&lt;P&gt;Lakebase CDF only generates append only change logs (lb_&amp;lt;table_name&amp;gt;_history). It does not maintain an inplace and current state replica table out of the box. To maintain a live current-state table such as pos_transactions_current, you can use the CDC history table and &lt;STRONG&gt;process&lt;/STRONG&gt; the &lt;STRONG&gt;change feed downstream&lt;/STRONG&gt;. You can run a periodic &lt;STRONG&gt;MERGE&lt;/STRONG&gt; statement on a &lt;STRONG&gt;scheduled job&lt;/STRONG&gt; every few minutes, or create an &lt;STRONG&gt;auto updating&lt;/STRONG&gt; Lakeflow &lt;STRONG&gt;Streaming Table / Materialized View&lt;/STRONG&gt; that &lt;STRONG&gt;applies the CDC events&lt;/STRONG&gt; continuously. It provides an immutable audit trail in the _history table while delivering an up-to-date snapshot for operational analytics. You can check the bi directional sync &lt;A href="https://community.databricks.com/t5/lakebase-articles/databricks-lake-base-enterprise-healthcare-data-intelligence-via/td-p/156407" target="_self"&gt;here&lt;/A&gt;&lt;/P&gt;&lt;P&gt;More details &lt;A href="https://docs.databricks.com/aws/en/oltp/projects/lakebase-cdf#compute-current-inventory-with-a-materialized-view" target="_self"&gt;here&lt;/A&gt;&lt;/P&gt;</description>
      <pubDate>Wed, 02 Sep 2026 04:43:14 GMT</pubDate>
      <guid>https://community.databricks.com/t5/data-engineering/lakebase-postgre-updating-delta-table/m-p/167243#M55655</guid>
      <dc:creator>balajij8</dc:creator>
      <dc:date>2026-09-02T04:43:14Z</dc:date>
    </item>
  </channel>
</rss>

