Pssst, You! The original story is on Medium! Clap and share it with your friends.
|
Three thousand feet below Manhattan, your iPhone flashlight is scattered by the dust permeating the air around you. Natural light can’t possibly make it this far below the surface, and yet, an illuminating and ominous green flicker exists just at the end of the tunnel. You advance cautiously, every step echoing through the silence, until the light takes shape — reflecting off an old man’s snow-white beard. He hovers above a relic of immaculate craftsmanship, its form almost otherworldly. The man looks up, his eyes shimmering with the same ancient light, as if he and the relic are bound by some arcane pact. The man smirks, delighted that someone believed in the legend enough to come down here — he turns to you and says one thing: “What is your first wish, master?”
“I want to conquer Wall Street”, you say as the dust between you starts to shine bright, emitting its own eerie unnatural light and forming a vortex around the man. “Become a stock trading master. I want to maximize the percent I gain for each trade I make. I want to minimize the drawdown I experience in my portfolios. I want to become a stock-trading legend.”
The green dust starts to dance between you two, illuminating the cavern. Your phone heats up, and a small icon appears on the screen. The ancient man’s smirk turns into a smile. He looks at you; his eyes shine as bright as the dust around you. “Done,” his voice echoes before he dematerializes into a cloud of green mist.
Your phone’s temperature returns to normal, and a new app materializes on your home screen. Labeled “NexusTrade,” the app icon is a vivid display of nucleotides spiraling through the letter ’N’, with a striking candlestick graph set against the backdrop. “The genie came through,” you muse to yourself, confident in your newfound advantage. Eagerly, you tap on the app to explore its features. But instead of a dashboard or trading options, you’re greeted with a mysterious countdown: 8,758 hours, 57 minutes, and 13 seconds.
|
The Aftermath — wish and you shall receive
|
In the ensuing weeks, NexusTrade performs like a symphony of precision and profit. Your email is a constant parade of notifications, each one heralding another brilliantly executed trade, buying at rock-bottom prices and selling at staggering highs. Feeling invincible, you say goodbye to your monotonous data entry job and set out on a luxury shopping extravaganza. Gucci, Dior, Prada — you splurge as if the labels are going out of fashion. You’ve beaten the system; you’ve won capitalism.
One year goes by and the countdown that has been teasing you for the past year is gone. Your fingertips tremble with anticipation as you navigate through NexusTrade’s analytics. The average Profit and Loss (PnL) per trade? A jaw-dropping 320%. And what about the portfolio’s drawdown? A practically negligible 2%. It appears the genie has delivered to the letter on your extravagant wish.
|
Eager to quantify your windfall, you switch to the portfolio graph, already imagining the exponential curve you’ll find. Yet, what greets you seems like a flat line. “Must be a glitch,” you mutter, shutting down the app before reopening it. To your disbelief, the graph is unchanged. Your portfolio’s net value has barely budged, showing a decline of 0.06%. “How can this be?” you wonder, flabbergasted. All the dazzling statistics, the flawless trades, and yet, your portfolio remains nearly stagnant. As you sit there in disbelief, your eyes scan the room filled with your luxury acquisitions — Gucci bags, Dior sunglasses, Prada shoes — all now feeling like monuments to your foolishness. You realize you’ve quit your stable job, traded it for a life of luxury you assumed was funded, and now the cruel irony sets in. What could have possibly gone wrong?
|
The Pitfall of Over-Optimization: A Postmortem
|
The problem isn’t the genie or even the NexusTrade app; it’s the nature of your wish. When utilizing non-linear optimization methods like genetic algorithms, it’s imperative that the optimization function is exactly what you want it to be. You asked to “maximize the percent gain for each trade” and “minimize drawdown of your portfolio,” both of which sound like the dreams of any trader. But did you really know what you were asking for?
|
Maximizing Percent Gain Per Trade
|
When you focus exclusively on maximizing the percentage gain for each individual trade, you inadvertently introduce a distortion into your trading strategy. Imagine this: the algorithm detects an extremely volatile asset that has wild price swings. You buy a single share for $1, and the stock skyrockets to $4 before you sell. That’s a 300% gain, right? While technically true, this creates an illusion of success, because you’re mistaking high percentages for high returns.
Here’s the rub: such a strategy often ignores important factors like trade volume, liquidity, and overall portfolio impact. What if you had $100,000 to invest? A 300% gain on a single share earning you $3 doesn’t make much of a dent. The algorithm, adhering strictly to your wish, disregards the importance of capital allocation. It fulfills the letter of your wish while failing the spirit.
Additionally, there’s the issue of risk. In pursuing these high-percentage but low-impact gains, the algorithm might expose you to highly volatile assets that could just as easily plummet, wiping out other gains. However, because your wish was for high percentage gains “per trade,” the algorithm might not take a holistic view of risk.
|
Minimizing Drawdown of Your Portfolio
|
When it comes to the second part of your wish — minimizing drawdown — the algorithm might go to great lengths to avoid any significant downturns in your portfolio. Ironically, the surest way to prevent losing money in the stock market is not to invest in it at all. If you don’t invest, you can’t lose. The algorithm takes this maxim to heart, interpreting your wish as an instruction to safeguard your portfolio at all costs — even if it means missing out on lucrative opportunities.
By choosing to not invest, you keep your drawdown close to zero. This also result in opportunity costs. Markets go up and down, but historically, they have trended upwards. By not participating fully in the market, you miss out on potential gains, and your money’s value may even be eroded by inflation over time.
In the end, by taking your words literally, the algorithm creates a portfolio that’s effectively ‘too safe to fail’ but also ‘too cautious to succeed.’
The devil is in the details. Specify your wish too narrowly, and you might just get what you asked for — at the expense of what you actually wanted.
|
Other Risks of Over-Optimization
|
Besides the cautionary tales exemplified by our genie story, there are further dangers when it comes to over-optimizing trading algorithms.
|
Overfitting: The Illusion of Mastery
|
Overfitting occurs when an algorithm adapts too closely to its training data, optimizing for every little idiosyncrasy and aberration it encounters. In doing so, it becomes a mirror of the noise in the data rather than a model of the underlying pattern. It’s much like cramming for an exam by memorizing the answers to previous tests; you’ll ace those specific questions but falter when presented with new, unanticipated queries. In the world of trading, an overfit algorithm may show stellar performance during backtesting but can falter dramatically when exposed to live, ever-changing market conditions.
|
Overconfidence: The Hubris of Success
|
When an algorithm aces its backtests and even clears the validation set, it’s tempting to believe you’ve unlocked the Midas touch of trading. However, the algorithm isn’t a crystal ball. Markets are shaped by an unfathomable array of elements — geopolitical shifts, economic signals, and even the collective psychology of traders. Succumbing to overconfidence can lead you to sideline risk management, overlook red flags, and invest more than you can actually afford to lose, setting the stage for potential financial ruin.
|
Given these pitfalls, how can you better align your trading algorithm with your actual objectives?
|
Choosing appropriate optimization objectives (fitness functions)
|
Firstly, be mindful when selecting your optimization objectives or fitness functions. Don’t just choose parameters that maximize or minimize one aspect of trading. Consider the holistic impact on your portfolio. For example, instead of just maximizing percentage gains per trade, a better objective could be maximizing the Sharpe Ratio, which considers both return and risk. This ensures you are not just aiming for high returns but also managing the associated risks adequately.
|
Deploying multiple portfolios
|
One of the best ways to mitigate risk is diversification — not just within a portfolio but across multiple portfolios. You could deploy several trading algorithms, each optimized for different conditions or market sectors. By running multiple, uncorrelated portfolios, you can spread the risk. When one portfolio is underperforming, another might be excelling, creating a more balanced and resilient overall trading strategy.
|
Out-of-Sample Testing (Validation Set)
|
An essential aspect of mitigating the risks associated with over-optimization is the use of out-of-sample testing, often referred to as testing on a validation set. Instead of solely relying on historical data (in-sample data) to gauge a trading algorithm’s effectiveness, out-of-sample testing evaluates the strategy’s performance using new, unseen data. This provides a more robust measure of an algorithm’s likely future performance, offering a guard against overfitting and providing an additional layer of risk management. NexusTrade integrates this critical step automatically, ensuring that your trading strategy isn’t just finely-tuned to past market conditions but is also resilient and adaptable to future fluctuations and trends.
|
Monitoring and reevaluation of your strategy
|
Constant vigilance is key when employing algorithmic trading strategies. No algorithm, no matter how well-crafted, can be set up and left to run indefinitely without oversight. Market conditions change, rules are updated, and unexpected events can occur. Therefore, it’s essential to have a structured monitoring and reevaluation plan in place.
Regular monitoring allows you to track performance metrics and see how the algorithm is interacting with the market in real-time. Is the system making unexpected trades? Is it failing to execute when it should? Such anomalies could be signs that the algorithm is acting based on overfit models or outdated rules.
Reevaluation, on the other hand, is a more in-depth process and involves revisiting the core assumptions and parameters of your algorithm. Is the data set still relevant? Do the fitness functions still align with your trading objectives? This step may require backtesting the algorithm on new data, modifying fitness functions, or even re-optimizing the entire portfolio.
By integrating continuous monitoring and periodic reevaluation into your risk management plan, you fortify your algorithm against the changing tides of market conditions and inherent risks of over-optimization.
|
Over-optimization is a seductive pitfall, offering the illusion of perfect control and spectacular results. However, as the story of the subterranean genie illustrates, it’s crucial to consider the broader implications of your specific wishes or trading objectives.
The NexusTrade app’s story isn’t just a crafted fable; it’s a real-life illustration of what can happen when you don’t fully understand what you’re asking for.
|
When it comes to trading algorithms, as in life, specificity is a double-edged sword. It can lead to either unparalleled success or unexpected failure, depending on how well your chosen parameters align with actual, complex reality. Be careful what you wish for; you just might get it.
|
|
