topic Building a Spark Declarative Pipeline OSS with Apache Iceberg and AWS Glue Catalog in Data Engineering https://community.databricks.com/t5/data-engineering/building-a-spark-declarative-pipeline-oss-with-apache-iceberg/m-p/156843#M54480 <P>Hey everyone,</P><P>I recently worked on building a modern financial data lakehouse using Spark Declarative Pipeline OSS (SDP OSS), Apache Iceberg, and AWS Glue Catalog.</P><P>The blog covers:</P><P>- Building declarative data pipelines with Spark</P><P>- Using Apache Iceberg as the table format</P><P>- Managing metadata with AWS Glue Catalog</P><P>- Streaming + batch style processing patterns</P><P>- End-to-end lakehouse architecture ideas</P><P>Blog link : <A href="https://medium.com/@pranavsadagopan/building-a-spark-declarative-pipeline-a-modern-financial-data-lakehouse-with-sdp-apache-iceberg-36ae6c6523ae" target="_blank">https://medium.com/@pranavsadagopan/building-a-spark-declarative-pipeline-a-modern-financial-data-lakehouse-with-sdp-apache-iceberg-36ae6c6523ae</A></P><P>Looking forward to feedback and discussions <span class="lia-unicode-emoji" title=":slightly_smiling_face:">🙂</span></P><P>&nbsp;</P> Wed, 13 May 2026 15:25:02 GMT Pranav_1699 2026-05-13T15:25:02Z Building a Spark Declarative Pipeline OSS with Apache Iceberg and AWS Glue Catalog https://community.databricks.com/t5/data-engineering/building-a-spark-declarative-pipeline-oss-with-apache-iceberg/m-p/156843#M54480 <P>Hey everyone,</P><P>I recently worked on building a modern financial data lakehouse using Spark Declarative Pipeline OSS (SDP OSS), Apache Iceberg, and AWS Glue Catalog.</P><P>The blog covers:</P><P>- Building declarative data pipelines with Spark</P><P>- Using Apache Iceberg as the table format</P><P>- Managing metadata with AWS Glue Catalog</P><P>- Streaming + batch style processing patterns</P><P>- End-to-end lakehouse architecture ideas</P><P>Blog link : <A href="https://medium.com/@pranavsadagopan/building-a-spark-declarative-pipeline-a-modern-financial-data-lakehouse-with-sdp-apache-iceberg-36ae6c6523ae" target="_blank">https://medium.com/@pranavsadagopan/building-a-spark-declarative-pipeline-a-modern-financial-data-lakehouse-with-sdp-apache-iceberg-36ae6c6523ae</A></P><P>Looking forward to feedback and discussions <span class="lia-unicode-emoji" title=":slightly_smiling_face:">🙂</span></P><P>&nbsp;</P> Wed, 13 May 2026 15:25:02 GMT https://community.databricks.com/t5/data-engineering/building-a-spark-declarative-pipeline-oss-with-apache-iceberg/m-p/156843#M54480 Pranav_1699 2026-05-13T15:25:02Z