Out-of-Sample Testing: Why Last Month's Best Traders Often Stop Winning
Any list of “best traders” is picked with hindsight. Some of those traders are skilled; many were lucky in a short window. A copier who starts after they top a list gets their future, not their past.
What out-of-sample means here
On every copyability board, each trader is ranked using one period. The out-of-sample ROI column then shows the simulated follower ROI, at the same delay, in the following window (7 or 14 days) — data the ranking never saw. If rankings carry real information, high-ranked traders should keep doing better than low-ranked ones out of sample.
How to read it
- A high rank with a negative out-of-sample ROI is a warning sign: the result may not persist.
- Short windows are noisy. Prefer the longest window available, and treat the shortest window as a hint, not proof.
- Aggregate results of this check across all boards: TBD (published with the monthly report).
Why we don’t just rank by PnL
Ranking by PnL selects for size and luck, and says nothing about whether a copier could follow in time. Our boards rank by returns kept after delay, and the out-of-sample column tests whether that ranking holds up.
Past results don’t predict future results. Related: methodology.