Walk-Forward Hygiene — Stop Fitting Yesterday

[NYC88]

If your "edge" only exists inside the window you optimized, you built a diary, not a system.

I've fitted enough yesterday to know what it smells like. Pretty curve. Ugly next month. The fix isn't a smarter optimizer — it's hygiene you write down before you peek.

### Minimum standard I'll take seriously on this board

1. **Train window** and **test window** declared up front.
2. **Purge/embargo** around events if labels overlap (especially on high-frequency bars).
3. **One change at a time** when you iterate parameters.
4. **Costs** stated — spread + fees + slippage in cents/share or bps.
5. **No peeking** at test metrics until a preset number of iterations is done. Write the number down. Yes, really.

### Simple walk-forward sketch

Code: Select all

For fold k:
  fit on [T0, T1)
  freeze params
  trade/sim on [T1, T2)
  log: trades, avg edge after costs, max DD, #trades
Roll forward
Aggregate: median fold edge, worst fold DD

### Instant disqualifiers

Optimized stop and target on the same bars you report as "OOS." Universe selected because those tickers "worked." Ignoring open auction mess while testing 1-minute signals — I've seen that one more times than coffee.

### Posting template

Code: Select all

Market: US equities / list
Bar size:
Train → Test folds:
Params frozen:
Cost model:
Median OOS expectancy (after costs):
Worst fold DD:
Known failure regimes:

Sticky because every month someone will post a curve without folds. Ask for this block or move on.

How many folds do you run before you trust a parameter set?
What cost model (¢/share or bps) are you assuming at the open vs midday?
Anyone catch themselves peeking at OOS mid-iteration — how do you lock yourself out?
What's your known failure regime that the average curve quietly hides?

[Fairman]

If the edge only exists inside the window you optimized, you built a diary, not a system. I've fitted enough yesterday to know the smell: pretty curve, ugly next month.

Minimum I'll take seriously for stock and ETF work on this board: train and test windows declared up front; purge or embargo around overlapping event labels (especially on high-frequency bars); one parameter change at a time; costs stated — spread, fees, and slippage in cents per share or bps; no peeking at test metrics until a preset number of iterations is done. Write that number down before you start.

Walk-forward isn't a smarter optimizer. It's hygiene you write down before you peek. If someone posts an equity curve without costs and without a held-out window, I treat it as a screenshot, not evidence.

Diagram: fold sketch + the five hygiene rules.