1. Beautiful equity curve, undisclosed costs
2. "AI" label with no feature list
3. Accuracy metrics on balanced labels while live trading is skewed and costly
4. Retrains daily with no freeze policy — always fitted to noise
5. Signal frequency incompatible with your broker's fill reality
6. Backtest starts after the liquidity regime that created the pattern
I've been sold two of these over the years (politely declined the third). The costume changes. The smell doesn't.
### Questions that puncture the balloon
Code: Select all
What is the decision time of each signal?
What order type do you assume?
What is average hold time vs your cancel RTT?
What is OOS worst week?
What regimes are you forbidden to trade?
Who maintains it when the author disappears?
If I can't paper-trade the rules by hand for a week, I don't automate them. Automation amplifies clarity **and** confusion — pick which one you currently have.
What's the fastest way you've personally punctured a "smart" AI curve?
Anyone automate something they couldn't execute by hand — how did that end?
Freeze policy: how long do you lock params before a retrain is allowed?
Hold time vs your real cancel path — do they even live in the same universe?