How do you train forecasting models for events that never happened?

[ThiamLee]

Most of my time-series work hits the same wall. The models only learn from scenarios that already happened.

Rare events barely show up in the training data. Demand spikes, stockouts, and extreme regimes are exactly the cases I care about. They are also the ones history is thin on.

Lately I have been testing synthetic time-series data as a way around this. The idea is to generate new scenarios from patterns already in your data, then train or stress-test against them.

One tool I came across is Remix Labs. It synthesizes new time-series datasets from data you already have. The pipeline is no-code, and it runs models like N-BEATS, NHITS, LSTM, and GRU under the hood. No new data collection needed.

Curious how others handle this. Do you augment with synthetic data, or stick to real historical records only? What has worked for you?

[ivanvyd]

@ThiamLee thanks for sharing, this is a really interesting problem and useful topic to discuss.

I'd be open to trying augmentation here. I'd compare the same model with and without synthetic data on identical rolling backtests, keeping the test data real and fitting the generator only on each training split. I'd also check real spike periods separately, so the overall score doesn't hide weaker results there.

For events you've never observed, realistic, domain-reviewed scenarios could help with stress testing.Those results are useful, but they still depend on the assumptions behind the scenarios.

One wrinkle with stockouts: sales can understate demand when inventory runsout, so I'd account for that before augmentation.

Have you had a chance to test whether the improvements carry over to held-out real events?

[ivanvyd]

One other thing I'd probably validate is whether the synthetic spikes preserve the relationships with things like promotions, price, holidays, and inventory. A spike can look realistic on its own but still teach the model the wrong context if those relationships get distorted.