Join discussions on data engineering best practices, architectures, and optimization strategies within the Databricks Community. Exchange insights and solutions with fellow data engineers.
I'm looking to develop generic code that can read multiple data formats, such as Parquet, Delta, Iceberg and save it as delta. Can you provide some insights or guidance on how to achieve this?
The Lakehouse Engine is a configuration driven Spark framework, written in Python, serving as a scalable and distributed engine for several lakehouse algorithms, data flows and utilities for Data Products. - lakehouse-engine/lakehouse_engine at master ยท adidas/lakehouse-engine