Walk-Forward Hygiene — Stop Fitting Yesterday
Posted: Sat Sep 05, 2026 9:43 am
If your "edge" only exists inside the window you optimized, you built a diary, not a system.
I've fitted enough yesterday to know what it smells like. Pretty curve. Ugly next month. The fix isn't a smarter optimizer — it's hygiene you write down before you peek.
### Minimum standard I'll take seriously on this board
1. **Train window** and **test window** declared up front.
2. **Purge/embargo** around events if labels overlap (especially on high-frequency bars).
3. **One change at a time** when you iterate parameters.
4. **Costs** stated — spread + fees + slippage in cents/share or bps.
5. **No peeking** at test metrics until a preset number of iterations is done. Write the number down. Yes, really.
### Simple walk-forward sketch
### Instant disqualifiers
Optimized stop and target on the same bars you report as "OOS." Universe selected because those tickers "worked." Ignoring open auction mess while testing 1-minute signals — I've seen that one more times than coffee.
### Posting template
Sticky because every month someone will post a curve without folds. Ask for this block or move on.
How many folds do you run before you trust a parameter set?
What cost model (¢/share or bps) are you assuming at the open vs midday?
Anyone catch themselves peeking at OOS mid-iteration — how do you lock yourself out?
What's your known failure regime that the average curve quietly hides?
I've fitted enough yesterday to know what it smells like. Pretty curve. Ugly next month. The fix isn't a smarter optimizer — it's hygiene you write down before you peek.
### Minimum standard I'll take seriously on this board
1. **Train window** and **test window** declared up front.
2. **Purge/embargo** around events if labels overlap (especially on high-frequency bars).
3. **One change at a time** when you iterate parameters.
4. **Costs** stated — spread + fees + slippage in cents/share or bps.
5. **No peeking** at test metrics until a preset number of iterations is done. Write the number down. Yes, really.
### Simple walk-forward sketch
Code: Select all
For fold k:
fit on [T0, T1)
freeze params
trade/sim on [T1, T2)
log: trades, avg edge after costs, max DD, #trades
Roll forward
Aggregate: median fold edge, worst fold DD
Optimized stop and target on the same bars you report as "OOS." Universe selected because those tickers "worked." Ignoring open auction mess while testing 1-minute signals — I've seen that one more times than coffee.
### Posting template
Code: Select all
Market: US equities / list
Bar size:
Train → Test folds:
Params frozen:
Cost model:
Median OOS expectancy (after costs):
Worst fold DD:
Known failure regimes:
How many folds do you run before you trust a parameter set?
What cost model (¢/share or bps) are you assuming at the open vs midday?
Anyone catch themselves peeking at OOS mid-iteration — how do you lock yourself out?
What's your known failure regime that the average curve quietly hides?