Options
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Friday
Hi Thiamlee,
You can use below if you want to learn underlying dependencies directly from existing time series
- SDV (Synthetic Data Vault) - It is usually the quickest starting point. SDV includes dedicated time series synthesizers like PAR (Probabilistic Auto Regressive) that handle multivariate tabular dependencies well and install easily in Databricks.
- ydata-synthetic - Its built around Time GAN architectures, this library is specifically tailored for sequential data. If preserving strict autocorrelation, lag properties, and feature correlations across time steps is critical for your ML models, this is typically stronger than standard tabular generators.
- TimeGAN - You can run raw TimeGAN implementations in PyTorch/TensorFlow if you want low-level control over the generator/discriminator networks, though ydata-synthetic wraps much of this functionality cleanly.
- Gretel Synthetics - Its a good option if you are open to commercial/managed workflows with open-source roots. It handles temporal sequences well out of the box and includes built-in privacy controls like Differential Privacy.
You can consider a native Spark approach using dbldatagen (Databricks package) or Faker with PySpark Pandas UDFs. Deep generative models like TimeGAN can be compute intensive to fit and difficult to scale across distributed worker nodes during training. Defining parametric distributions (sine waves for seasonality, linear trends, and probabilistic spike injections) in native Spark allows you to generate millions of rows in seconds usually on serverless or standard compute without single-node memory bottlenecks.
