algorithmic trading / backtesting / overfitting / quantitative research / ai trading
Why Trading Strategies Fail: You Shipped Version One
A great equity curve in a backtest can hide an overfit trading strategy. Austin Starks explains why shipping the first promising version skips the research that makes a strategy worth testing further.
The video walks through generating strategy variants, testing them together, inspecting their behavior, checking changing market conditions, and validating on unseen data. It connects that iterative research process to the AI workflow he built into NexusTrade.
Transcript
0:00Your trade strategy didn't fail
0:01because the strategy was bad.
0:03It failed because you shipped
0:04the first version
0:05that looked half-decent
0:06in a backtest.
0:07Here's exactly what happened.
0:08You backtested the strategy once,
0:10and the equity curve
0:11looked amazing.
0:12You decided to immediately go live,
0:14risking your entire account balance.
0:17Two weeks later,
0:17the account is bleeding dry,
0:19and you've lost thousands of dollars
0:21before you start blaming
0:22the market makers.
0:23This is because the strategy wasn't nearly as good
0:25as you thought it was.
0:26That backtest that you saw
0:28was over-fit.
0:29It had no ability to generalize
0:31to unseen data.
0:33Professional quants,
0:34the folks that are making
0:35billions of dollars
0:36at Citadel and Jane Street,
0:39don't just create a trading strategy
0:40once and ship the first version
0:42that they like.
0:43They run a loop on repeat,
0:46and this is what it looks like.
0:48Step 1:
0:49They generate multiple different
0:51strategy variants.
0:52They don't just have one strategy.
0:54They might generate 10.
0:56They might change
0:57the look-back period.
0:58They might change the definitions
0:59of certain indicators.
1:01So they might say
1:02momentum means stocks
1:03with a rate of change
1:04greater than 1%.
1:05Or it might mean
1:06momentum means stocks
1:07with an RSI greater than 60.
1:09They basically create different variants
1:11of their trading strategy.
1:13Step 2 is they test all variants simultaneously,
1:16and they don't look for the one
1:17that has the highest returns.
1:19They look for the strategies
1:20that are consistently doing very well.
1:24Step 3 is
1:25they might find some promising candidates
1:26and do a little bit
1:28of a deeper dive and a deeper analysis on them.
1:31For example, they might inspect the events
1:32that happens throughout the lifetime
1:34of a backtest,
1:35and change little things
1:37to make the performance
1:38of that strategy more favorable.
1:41Step 4 is
1:42they see if the strategy's
1:43still doing well.
1:45Every single trading strategy
1:47you'll ever create
1:49has a shelf life.
1:50No strategy is going to be good
1:51all the time,
1:52in every regime,
1:54forever and here on out.
1:56It's just not possible.
1:58If you don't believe me,
1:59look up the efficient market hypothesis.
2:01Finally, we validate the strategy
2:03on data that it's never seen.
2:05What this looks like is
2:05you'll build a strategy
2:07on one period of time,
2:08and then you'll validate it
2:09on a smaller next period of time,
2:11and seeing if it actually holds up.
2:14If it only works on the segment
2:16the strategy was trained on,
2:18that means the strategy was over-fit,
2:22which basically means
2:23it memorized the data.
2:24You didn't find an actual edge.
2:27This exact loop is exactly
2:30what I built my AI
2:32trading platform on.
2:33It's not just an AI
2:35that spits out backtests.
2:36It's an AI that autonomously
2:39performs this research loop
2:41over and over again,
2:43until it finds a strategy
2:44that conforms to what you want,
2:47not just some generic
2:48boilerplate LLM garbage.
2:51The link to it
2:51is in my bio.
2:53Try it out,
2:53kill your strategy,
2:55and test it out
2:55before you risk a single dollar.
Join the conversation
Loading conversation…