Page 1 of 1

Walk-Forward Hygiene — Stop Fitting Yesterday

Posted: Sat Sep 05, 2026 9:43 am
by NYC88
If your "edge" only exists inside the window you optimized, you built a diary, not a system.

I've fitted enough yesterday to know what it smells like. Pretty curve. Ugly next month. The fix isn't a smarter optimizer — it's hygiene you write down before you peek.

### Minimum standard I'll take seriously on this board

1. **Train window** and **test window** declared up front.
2. **Purge/embargo** around events if labels overlap (especially on high-frequency bars).
3. **One change at a time** when you iterate parameters.
4. **Costs** stated — spread + fees + slippage in cents/share or bps.
5. **No peeking** at test metrics until a preset number of iterations is done. Write the number down. Yes, really.

### Simple walk-forward sketch

Code: Select all

For fold k:
  fit on [T0, T1)
  freeze params
  trade/sim on [T1, T2)
  log: trades, avg edge after costs, max DD, #trades
Roll forward
Aggregate: median fold edge, worst fold DD
### Instant disqualifiers

Optimized stop and target on the same bars you report as "OOS." Universe selected because those tickers "worked." Ignoring open auction mess while testing 1-minute signals — I've seen that one more times than coffee.

### Posting template

Code: Select all

Market: US equities / list
Bar size:
Train → Test folds:
Params frozen:
Cost model:
Median OOS expectancy (after costs):
Worst fold DD:
Known failure regimes:
Sticky because every month someone will post a curve without folds. Ask for this block or move on.

How many folds do you run before you trust a parameter set?
What cost model (¢/share or bps) are you assuming at the open vs midday?
Anyone catch themselves peeking at OOS mid-iteration — how do you lock yourself out?
What's your known failure regime that the average curve quietly hides?

Re: Walk-Forward Hygiene — Stop Fitting Yesterday

Posted: Sat Sep 05, 2026 10:24 pm
by Fairman
If the edge only exists inside the window you optimized, you built a diary, not a system. I've fitted enough yesterday to know the smell: pretty curve, ugly next month.

Minimum I'll take seriously for stock and ETF work on this board:

- Train and test windows declared up front.
- Purge or embargo around overlapping event labels, especially on high-frequency bars.
- One parameter change at a time.
- Costs stated: spread, fees, and slippage in cents per share or bps.
- No peeking at test metrics until a preset number of iterations is done. Write that number down before you start.

Walk-forward isn't a smarter optimizer. It's hygiene you write down before you peek.

A worked example of how I lay it out for an intraday ETF scalp model. Say I have 1-minute bars covering several months. I split into rolling folds: train on six weeks, test on the next two weeks, then roll forward two weeks and repeat. Labels look 15 minutes ahead, so I purge the last 15 minutes of each training window and embargo the first 15 minutes of each test window, so no label overlaps the boundary. The parameter I'm testing is one thing, say an opening-range length of 5 versus 15 minutes. Costs go in as 1 cent per share per side of slip plus the actual commission schedule. I set the iteration count at five before I look at any test fold. After five, I look once, and whatever it says is the answer for this version.

What I'm looking for is not the best fold. It's whether the test folds look roughly like the train folds after costs. If train looks strong and test is flat or negative after the cent of slip, the edge was probably the window, not the rule.

Failure modes:

- Re-running the split until the test period "behaves."
- Adding a second parameter because the first didn't work, without resetting the iteration count.
- Leaving the 09:30 ET open in train and cutting it from test, or the other way around. Keep the session definition identical.
- Quoting results in gross ticks and never converting to cents after costs.

When to skip the model entirely: if the edge disappears once one cent of slip is added, it's not a scalping edge on liquid US names, however good the chart looks.

If someone posts an equity curve without costs and without a held-out window, I treat it as a screenshot, not evidence.

Diagram: fold sketch + the five hygiene rules.