topic Re: Data Migration from SAP S/4HANA to Databricks in Data Engineering

Data Migration from SAP S/4HANA to Databricks

Pratikmsbsvm — Sun, 08 Feb 2026 15:53:53 GMT

May someone please help me designing the Migration of SAP S/4 HANA to Databricks. How to design this. what all we need to consider as LLD.

1. How Data needs to be extracted and by which tool ? near–real-time replication is required

2. Each layer for Data flow

3. SAP Extraction

4. and if possible and any idea on how to DesignHub-and-Satellite (Hub & Spoke) architecture, central hub for harmonized SAP data models, satellites for domain-specific analytics.

Thanks a lot

Re: Data Migration from SAP S/4HANA to Databricks

WiliamRosa — Tue, 10 Feb 2026 03:16:32 GMT

SAP HANA SMART INTEGRATION

Re: Data Migration from SAP S/4HANA to Databricks

SteveOstrowski — Fri, 26 Jun 2026 05:23:17 GMT

Hi @Pratikmsbsvm,

Here is an updated view of the options for moving SAP S/4HANA data into Databricks, including the SAP and Databricks partnership path that is now the recommended low-friction approach. I will cover the integration options first, then the data flow layers and the Hub-and-Satellite design.

1. SAP BUSINESS DATA CLOUD CONNECT TO DATABRICKS (EVALUATE FIRST IF YOU ARE ON SAP BDC)

The most significant recent development is the SAP and Databricks partnership. If you want SAP data landing in your own Databricks workspace, the relevant product is SAP Business Data Cloud Connect to Databricks, generally available since October 2025. It shares curated SAP data products into your Databricks workspace through Delta Sharing as live, zero-copy, bidirectional data, so you can work with S/4HANA data without building a traditional extraction pipeline and without physically replicating it, and you can push results back into SAP. There is also a sibling product, SAP Databricks, which is an SAP-managed edition of Databricks embedded directly inside BDC (generally available since April 2025), for teams that want to work entirely within the SAP environment. Recent enhancements automatically sync SAP semantic metadata such as descriptions, keys, and governance tags into Unity Catalog (one direction, SAP to Databricks), so the business context travels with the data.

This path is built on Delta Sharing, an open protocol that has been generally available and in production use at scale for years, so the underlying data-movement foundation is well proven. The integration products themselves are generally available but relatively new, so if your organization is already on, or actively adopting, SAP Business Data Cloud, this is the most direct, lowest-ETL option and I would evaluate it first. For net-new or non-BDC landscapes, the SAP-certified partner extraction tools in the next sections remain the proven route.

References:
https://www.databricks.com/blog/announcing-general-availability-sap-business-data-cloud-connect-databricks
https://www.databricks.com/blog/announcing-general-availability-sap-databricks-sap-business-data-cloud
https://docs.databricks.com/aws/en/delta-sharing/sap-bdc/

2. LAKEFLOW CONNECT (MANAGED INGESTION)

Lakeflow Connect provides fully managed ingestion connectors. As of today there is no dedicated SAP or S/4HANA managed extraction connector, so for classic per-table SAP extraction you would use one of the partner tools below, or the Business Data Cloud path above. The connector catalog continues to expand, with recent additions including Zendesk and Confluence, plus Public Preview query-based connectors for Oracle, Teradata, SQL Server, MySQL, MariaDB, PostgreSQL, and Lakehouse Federation, so it is worth checking the release notes periodically.

https://docs.databricks.com/aws/en/ingestion/lakeflow-connect/

3. SAP-CERTIFIED PARTNER EXTRACTION TOOLS (NEAR REAL-TIME / CDC)

When you need classic extraction or change data capture directly from S/4HANA (ODP, BAPI, RFC, IDoc, CDS Views), these validated partners are production-proven and land data in Delta on Databricks:
- Fivetran: Databricks-validated, available via Partner Connect, SAP connectors via ODP
- Informatica Cloud Data Integration: SAP-certified, BAPI / RFC / IDoc / ODP
- Qlik (formerly Attunity): near real-time SAP change data capture
- Precisely Connect: near real-time replication from S/4HANA
- Theobald Xtract Universal and SNP Glue: lightweight, SAP-focused extraction

4. DIRECT AND DO-IT-YOURSELF EXTRACTION OPTIONS

- OData on CDS Views for a supported, semantically rich extraction interface
- JDBC direct to SAP HANA for batch pulls where appropriate
- PyRFC for custom RFC / BAPI extraction
- Landing SAP exports in cloud storage and ingesting with Auto Loader

These give you flexibility, but you own more of the engineering and operations compared with options 1 to 3.

DATA FLOW LAYERS (MEDALLION)

Regardless of the extraction method, land the data into a medallion architecture:
- Bronze: raw SAP data as ingested, or the live BDC shared tables, with minimal transformation and full history
- Silver: cleansed, conformed, deduplicated, with business keys resolved
- Gold: curated, aggregated, business-ready models for analytics and serving

Lakeflow Spark Declarative Pipelines (SDP) are a good fit for building and orchestrating these layers, with built-in data quality expectations.

HUB-AND-SATELLITE WITH UNITY CATALOG

Use Unity Catalog as the governance backbone for the Hub-and-Satellite design: a central hub catalog or schema for shared, conformed entities, with satellite schemas for domain or source-specific context. Unity Catalog gives you one permission model, end-to-end lineage, and discovery across all of it. If you use the Business Data Cloud path, the SAP governance tags and semantic metadata flow into that same model.

For near-real-time, combine CDC from option 1 or 3 with streaming or triggered SDP pipelines so the Silver and Gold layers stay current.

Happy to go deeper on any one of these.

* This reply used an agent system I built to research and draft this response based on the wide set of documentation I have available and previous memory. I personally review the draft for any obvious issues and for monitoring system reliability and update it when I detect any drift, but there is still a small chance that something is inaccurate, especially if you are experimenting with brand new features.

If this answer resolves your question, could you mark it as "Accept as Solution"? That helps other users quickly find the correct fix.

Re: Data Migration from SAP S/4HANA to Databricks

dima — Thu, 29 Oct 2026 17:45:54 GMT

@Pratikmsbsvm I will try to be brief but accurate.

1. Getting data out of SAP

You need a properly supported connector/extraction method. You can start researching Fivetran, SNP Glue and alike. What you need to understand is that this can be expensive, and you need to check the SAP licensing implications of your chosen extraction method with the customer and the vendor. Also PoC the solutions to verify that they support your required CDC, delete handling, latency, etc.

2. Existing SAP data flows

There is no readily available tool to convert all SAP data flows, whatever their implementation is: BW transformations, ABAP, AMDP, Calculation Views, Graphical Views, etc. You may be able to accelerate your efforts with BW MCP and ABAP MCP servers, but keep in mind which ones are official and which are community-built.

In any case, significant manual effort will be required, so proper scoping and prioritisation of data flows are a must. “Move it all in one go” is in most cases not on the menu unless you are planning a multi-million-dollar, multi-year project.

3. SAP extraction and data modelling

SAP has out-of-the-box extractors, and you can build your own custom extractors. ABAP CDS is one option worth looking into, especially if you need to expose business logic or support delta extraction.

But don't assume CDS is the answer for everything. Depending on your requirements, standard extractors, ODP, existing APIs or third-party CDC connectors may be more appropriate.

Also, “give me all the tables” will not work here. There are hundreds of thousands of tables and other objects in an SAP system, and you need to understand what you are extracting and why. In many cases, you can extract source-aligned data and do most of the transformation and harmonisation in Databricks.

4. Layers and Hub-and-Spoke architecture

I'd start with a standard Bronze/Silver/Gold architecture:

Bronze: Raw or source-aligned SAP data, landed in Delta tables, with ingestion metadata and a way to recover or replay data.
Silver: A central hub with harmonised SAP business entities, shared master data, consistent keys and reusable business definitions.
Gold: Domain-specific satellites, e.g. Finance, Sales and Supply Chain, building their own models, KPIs and analytical data products on top of the shared hub.

The idea is to harmonise what is genuinely shared across domains, without trying to squeeze everything into one giant central model. Databricks Unity Catalog can help with governance, access control and lineage.

For the LLD, I'd also explicitly define the required latency per flow, initial load strategy, CDC and delete handling, data volumes, reconciliation with SAP, monitoring, security and recovery procedures.

And I'd definitely start with a PoC covering one representative business process, from SAP extraction all the way to the final analytical model, before committing to the full architecture.