topic Re: TPC-DI Benchmark in Databricks (ETL) in Data Engineering

TPC-DI Benchmark in Databricks (ETL)

Stephanraj — Tue, 06 Dec 2022 11:06:17 GMT

Similar to TPC-DS and TPC-H benchmark, is it possible to execute TPC-DI benchmark in Databricks. What is the feasibility to execute TPC-DI benchmark, please let me know if there is any existing ETL Tool available in Databricks for TPC-DI.

For reference: https://www.tpc.org/tpc_documents_current_versions/pdf/tpc-di_v1.1.0.pdf

Re: TPC-DI Benchmark in Databricks (ETL)

franco_patano — Wed, 22 Mar 2023 02:21:45 GMT

Actually, we are working on it. We talked about it last summit.

https://www.youtube.com/watch?v=zMbeP6Zc_Xk

Re: TPC-DI Benchmark in Databricks (ETL)

Shannon_Barrow — Thu, 13 Apr 2023 00:49:23 GMT

Hello, we recently made public a repo that makes it easy to run the TPC-DI benchmark end-to-end - in a variety of ways. The repo can be found here. Feel free to let me know if you have any issues running it!