Hand-labelling my setups before any model gets to see them
Posted: Wed Sep 30, 2026 9:11 pm
Before I let any model, simple or fancy, near my trade data, I label a sample of setups by hand. It's slow. It's also the only way I've found to know whether my labels mean anything. A model trained on sloppy labels just reproduces the sloppiness very confidently.
## Rails
- Each setup gets a label from a fixed list of four: ORB break, VWAP reclaim, pullback to level, and fade at extension. Anything that doesn't fit is labelled OTHER. I don't create a new class on the fly.
- I label from a screenshot taken at the moment of entry, with everything to the right of entry hidden. The outcome must not influence the label.
- I label the same sample twice, a week apart, without looking at my first pass. If the two passes disagree on a lot of trades, the definitions need tightening before any modelling.
- I write down the disagreement count and which pairs of labels get confused. I report my own numbers and don't borrow anyone else's benchmark.
## Worked example (illustrative)
One trade, 09:47 ET, a $29 name. Price broke the opening range high at 29.40, pulled back to 29.34, which was also VWAP, and I bought 29.36 with a stop at 29.27.
First pass I labelled it "pullback to level." A week later, blind, I labelled it "VWAP reclaim." Both are defensible, and that's exactly the problem. I tightened the definitions: VWAP reclaim now requires price to have closed at least one 1-minute bar below VWAP within the prior five bars. This trade never did, so the correct label is "pullback to level."
The cents didn't change. My exit was 29.52, +16 cents. What changed is that the trade now sits in the right bucket when I later check which setup type actually works for me.
## Failure modes
- Labelling after seeing the result, so winners drift into the "good setup" classes.
- Letting OTHER become a dumping ground. If it grows large, the playbook is missing a setup or I'm taking trades that aren't in it.
- Skipping the second blind pass because the first one "felt right."
## When to skip
- I don't train or test anything until the sample has been labelled twice and the definitions are stable.
- If I've changed my playbook this month, older labels get re-checked before any comparison.
- Small samples stay descriptive. I don't pretend they're predictive.
Diagram: a screenshot box with the right side blacked out, feeding into four label bins plus OTHER, with a "second blind pass" loop arrow back to the start.
## Rails
- Each setup gets a label from a fixed list of four: ORB break, VWAP reclaim, pullback to level, and fade at extension. Anything that doesn't fit is labelled OTHER. I don't create a new class on the fly.
- I label from a screenshot taken at the moment of entry, with everything to the right of entry hidden. The outcome must not influence the label.
- I label the same sample twice, a week apart, without looking at my first pass. If the two passes disagree on a lot of trades, the definitions need tightening before any modelling.
- I write down the disagreement count and which pairs of labels get confused. I report my own numbers and don't borrow anyone else's benchmark.
## Worked example (illustrative)
One trade, 09:47 ET, a $29 name. Price broke the opening range high at 29.40, pulled back to 29.34, which was also VWAP, and I bought 29.36 with a stop at 29.27.
First pass I labelled it "pullback to level." A week later, blind, I labelled it "VWAP reclaim." Both are defensible, and that's exactly the problem. I tightened the definitions: VWAP reclaim now requires price to have closed at least one 1-minute bar below VWAP within the prior five bars. This trade never did, so the correct label is "pullback to level."
The cents didn't change. My exit was 29.52, +16 cents. What changed is that the trade now sits in the right bucket when I later check which setup type actually works for me.
## Failure modes
- Labelling after seeing the result, so winners drift into the "good setup" classes.
- Letting OTHER become a dumping ground. If it grows large, the playbook is missing a setup or I'm taking trades that aren't in it.
- Skipping the second blind pass because the first one "felt right."
## When to skip
- I don't train or test anything until the sample has been labelled twice and the definitions are stable.
- If I've changed my playbook this month, older labels get re-checked before any comparison.
- Small samples stay descriptive. I don't pretend they're predictive.
Diagram: a screenshot box with the right side blacked out, feeding into four label bins plus OTHER, with a "second blind pass" loop arrow back to the start.