Walter_C
Databricks Employee
Databricks Employee

Costs:

  1. Egress Costs:

    • Delta Sharing: Within the same region, Delta Sharing incurs no egress costs. However, sharing data across different regions or clouds may result in egress fees charged by the cloud provider. Databricks supports sharing from Cloudflare R2, which incurs no egress fees. Tools and recommendations are available to monitor and avoid egress fees (source: "Monitor and manage Delta Sharing egress costs (for providers)").
    • CosmosDB: Egress costs apply when data is read from CosmosDB and transferred out of the Azure region. The exact costs depend on the amount of data transferred and the regions involved.
  2. Computation Costs:

    • Delta Sharing: The data provider incurs storage costs for Delta or Parquet files, but there are no compute costs on the provider's side for sharing data. The recipient incurs data processing costs when they access and process the shared data (source: "Delta Sharing Cost Breakdown").
    • CosmosDB: Costs are associated with the provisioned throughput (RU/s) for read and write operations, as well as storage costs.

Performance Issues:

  • Delta Sharing: There can be performance issues when accessing the Delta Sharing table concurrently through API calls. For example, if the number of API requests exceeds the predefined rate limit, it can result in "Too Many Requests" errors (HTTP 429). 
  • CosmosDB: CosmosDB is designed to handle high throughput and low latency for concurrent operations. However, performance can be impacted if the provisioned throughput is not sufficient for the workload