topic Re: TPC -DS test On databricks in Data Engineering

TPC -DS test On databricks

User16826994223 — Tue, 22 Jun 2021 09:17:45 GMT

If I want to run TPC-DS test on databricks what are the steps involved, do we have already daya available on databricks file system or I have to download or create from somewhere.

Re: TPC -DS test On databricks

sajith_appukutt — Tue, 22 Jun 2021 22:58:15 GMT

For TPC -DS, checkout https://github.com/databricks/tpcds-kit . Databricks includes a variety of datasets mounted to the Databricks File System (DBFS) and this includes TPC-H

Re: TPC -DS test On databricks

aladda — Wed, 23 Jun 2021 03:54:52 GMT

See the spark-sql-perf repo for details on how to run benchmark tests using TPC-DS - https://github.com/databricks/spark-sql-perf

Re: TPC -DS test On databricks

hillel1 — Mon, 20 Nov 2023 19:43:34 GMT

I am new here 🙂 can you please explain one would run spark-sql-perf from a notebook?
meaning how to reset a new spark application with the jars set.