ai agents / backtesting / software engineering
Building the AI Agent Is the Easy Part
Building my AI trading agent took about two weeks. Almost all of my time goes into the backtesting infrastructure underneath it. Data quality means the market cap behind every screen is right on every date, through splits and data errors. Performance engineering means a backtest on minute-by-minute options data finishes in minutes instead of an hour, so I can try the next idea.
Transcript
0:00The hardest part about building your own AI agent is not actually building the agent.
0:05Like building the agent is easy.
0:07You can build your own agent in like two weeks and have it production ready.
0:12No, the hard part about building your AI agent is building the infrastructure around it.
0:18Let me give you a concrete example.
0:19My AI agent can take a plain English sentence and then transform it into a testable trading strategy.
0:25You can say something like, I want to buy NVIDIA if it beats it's earnings.
0:29I want to buy Apple if it's price is below it's 30 day average price.
0:33A plain English sentence transformed into a trading strategy, and with that strategy you can do several things.
0:38You can paper trade it so you're basically risking no money to see how that strategy performs.
0:42You can connect a brokerage and live trade it so you can actually do a fully automated trading strategy.
0:48And most importantly, you can backtest it so you can see exactly how that set of rules performed in any period in the past.
0:55The backtesting infrastructure is pretty much the thing I build.
0:59I don't really build out the AI agent.
1:01I don't really build out the web app.
1:04Yes, I do build it, but all that can mostly be automated with AI.
1:07The part that I spend all my time on is the backtesting infrastructure.
1:11Specifically, it's two key things.
1:13So number one is the data quality.
1:15You can say something like, I want to buy options on the top 10 stocks on market cap every two weeks.
1:20Then you must have accurate data on what are the top 10 stocks on market cap.
1:26But more than that, if you don't have it, if it's wrong, let's say there's a stock split and you didn't update the market cap of the stock after the split.
1:33Or let's say there's a data issue and it ran a $500 million market cap as $500 billion market cap.
1:39If you did any mistake in the data, that will significantly affect your backtesting results.
1:45So you have to build infrastructure around making sure you have high data quality.
1:52The second thing that I build that takes a long time is performance engineering.
1:56I want my backtest to take minutes.
1:58I want to launch a backtest on my actual portfolio and it takes one hour per backtest.
2:04For one, it's expensive.
2:05It costs me a lot of money to run that backtest.
2:07I don't want to wait an hour.
2:08But two, I'm bored.
2:09Like I'm bored.
2:10No one wants to wait an hour per backtest.
2:12Why do you want to run a second backtest after that?
2:15Why don't you try a slightly different set of rules after that?
2:18You know, you can't wait an hour per backtest and these backtests work on minute by minute historical options data pulls full options chains each minute.
2:28That's so much data.
2:30So I spend all my time performance engineering making sure that I only fetch the data that's needed and that the data is processed efficiently.
2:39I'm not doing repeated allocations.
2:41I'm not fetching the same data twice.
2:42I'm actually caching the data.
2:44I'm co-locating my machine next to where the data is stored.
2:48Like all this stuff is a lot of work.
2:50So if you're thinking about building an AI agent, yes, you need to understand how building AI agents work, but that's not what you're going to be spending your time on.
2:58You're going to be spending your time on the infrastructure around your AI agent.
Join the conversation
Loading conversation…