Design Oracle Fusion SCM to Azure Databricks

[Datalight]

Hello Techie,

I am planning to migrate All module of Oracle fusion scm data to Azure Databricks.

Do we have only option of BICC (Business Intelligence Cloud Connector), OR any other option avaialble.

Can anyone please help me with reference architecture.

Thanks a lot.

[mark_ott · Accepted solution]

BICC (Business Intelligence Cloud Connector) is not the only option for migrating Oracle Fusion SCM data to Azure Databricks, though it is the most common and recommended tool for high-volume, scheduled extracts according to Oracle’s official guidance.​

Alternative Extraction Options

• Oracle BI Publisher (BIP): Suited for extracting smaller, customized datasets where control over output format or embedded logic is required. Not optimal for large-scale or incremental extractions, and generally considered a reporting tool rather than a bulk data extraction tool.​

• Oracle Fusion REST APIs: For low-volume or real-time data extraction needs, Fusion exposes rich REST and SOAP interfaces. These APIs are practical for incremental or event-driven pipelines but require custom development and may have API call limits.​

• Data Integration/ETL Tools: Third-party tools and services (like Azure Data Factory, BryteFlow, or RiteSync) can orchestrate extraction, often leveraging BICC, BIP, or direct API access—sometimes with additional features like change data capture (CDC), scheduling, and monitoring.​

• Direct JDBC/ODBC Connections: Primarily for on-premise Oracle environments, but not typically an option for Oracle Fusion Cloud data; access is usually limited to the SaaS application layer.​

Reference Architecture Overview

A typical reference architecture for Oracle Fusion SCM to Azure Databricks includes:

1. Extraction Layer:

   • Use BICC (scheduled bulk extract), BI Publisher (for special cases), or Fusion APIs to extract raw data or CSVs.

2. Data Ingestion/Storage:

   • Store extracted files or API payloads in Azure Data Lake Storage Gen2 (ADLS Gen2).​

   • Optionally use Azure Data Factory for orchestrating the extract and load operations for both bulk and incremental data.​

3. Processing and Analytics:

   • Use Azure Databricks to transform, cleanse, and enrich the ingested data.

   • Apply the medallion architecture (Bronze, Silver, Gold layers using Delta tables).​

4. Consumption:

   • Curated data sets feed downstream tools such as Power BI, reporting solutions, or machine learning workloads.

Example Reference Pipeline

• Extract: BICC/BI Publisher scheduled extracts to ADLS Gen2 (via SFTP, REST, or native ADLS sinks).

• Orchestrate: Azure Data Factory manages and monitors extracts, error handling, and incremental loads.

• Transform: Databricks notebooks/scripts convert raw data to Delta Lake formats and maintain SCD/CDC logic.

• Consume: Final data available for analytics and reporting.

This architecture ensures scalability, modularity, and compliance with both Oracle and Azure best practices.​

Summary Table

Extraction Method	Bulk Loading	Real-Time	Complexity	Usage Scenario
BICC	Yes	No	Medium	All modules, large volumes​
BI Publisher (BIP)	Limited	No	High	Custom formats, reports​
REST/SOAP APIs	No	Yes	High	Real-time, low-volume​
ETL Tools/ADF	Yes	Yes	Medium	Orchestration​
3rd Party (RiteSync, Orbit)	Yes	Yes	Medium	Enhanced scheduling, CDC​

In summary, while BICC remains the main enterprise-grade method, additional strategies are possible for specific needs or custom orchestration—especially if lighter extracts, more control over formatting, or real-time data is needed.​

[Datalight]

@mark_ott : Thanks a ton. sorry for late reply, as Client was not sure on the approach. your solution helps a lot. Thanks Again.

[DineshBI1234]

You may want to check out the BI Connector. It helps companies with pipelining Oracle Fusion data (from raw database tables, BIP Data Models, OTBI reports) to Azure Databricks in hours, without having to write code.

[masonreed11]

BICC is the standard choice for bulk Oracle Fusion SCM extraction, but REST APIs and OIC are also good options depending on your use case. Your architecture and data freshness requirements will determine the best approach.