Pssst, You! The original story is on Medium! Please read the article there and engage with it (clap, comment, and share!) It spreads my reach significantly. Thank you for your cooperation 😃
|
I first heard of “Reinforcement Learning” when I was a junior at Cornell University in a course called “Foundations of Artificial Intelligence”. This course was an eye-opener, introducing me to various subfields within AI such as neural networks and genetic optimization. But what really caught my attention was Reinforcement Learning.
|
For those not steeped in technical jargon, Reinforcement Learning (RL) is a subfield of Machine Learning (ML). Unlike most ML which relies on supervised learning — training algorithms with labeled examples — Reinforcement Learning takes a different approach. It uses a system of rewards and punishments to ‘teach’ the computer, much like how you might train a dog.
|
For example, imagine training a new puppy you got from the shelter. When the puppy sits on command, you give him a treat and lots of “good boys”. However, when the puppy pees on the carpet, you scold him, so he learns to not do it again. RL applies this same principle to computers, enabling them to learn from their actions.
|
Reinforcement Learning Image Stolen From This Article
|
I was obsessed with the idea of applying RL to the stock market. But I didn’t know how. Despite completing a course in the Fundamentals of Reinforcement Learning, the gap between academic theory and practical application seemed vast. I lacked the necessary technical skills to independently apply RL in such a complex domain as stock trading. So, I gave up.
|
However, an unexpected opportunity emerged two years later, during my Master’s in Software Engineering at Carnegie Mellon. I encountered a course titled “Intro to Deep Learning,” renowned for its rigor and depth. It demanded proficiency in multivariable calculus and linear algebra — advanced and complicated high-level mathematics that I hadn't taken previously. Yet, the allure of the course was irresistible. Studying at the best computer science school in the world, I was aware that such an opportunity to expand my knowledge might not come again. Embracing the challenge, I enrolled in the course.
|
And as anticipated, the course proved to be exceptionally challenging. The professor was amazing and the content was very interesting. But the complexity was overwhelming at times. I had to self-study on linear algebra and calculus just to understand the fundamentals.
|
Image Generated by ChatGPT – man studying mathematics
|
This endeavor, though arduous, set the stage for a significant breakthrough. The course culminated in a final project, offering students the freedom to apply AI in a domain of their choice. Seizing this chance, I formed a team dedicated to realizing the vision I had harbored for years: developing a reinforcement learning-based stock trading algorithm.
|
At the heart of Reinforcement Learning (RL) lies a simple yet powerful concept: it involves a series of states and actions. In each state, when an action is performed, the system transitions to a new state and the agent receives a reward. The overarching objective is to maximize the total sum of rewards by selecting the best action in every state.
|
The RL Algorithm that I Implemented
|
This project was intellectually demanding. Training the model was a nightly task, stretching into the early hours. The reliance on cloud GPUs for this intensive computation came with a substantial cost, amounting to hundreds of dollars. And yet, despite my best efforts, I failed to develop an algorithm that could even outperform Buy and Hold.
|
You can find the code for the project below. There’s also a detailed paper that explains in depth the experiments I performed.
|
The amount of effort I expended upon my goal was substantial. I learned reinforcement learning, starting from the fundamentals. I took a course in deep learning, and learned advanced mathematics. And I spent weeks on this project, learning how to implement reinforcement learning algorithms and how to apply it to stock trading. Yet, despite all of my best effort, I simply failed.
|
The “Cartpole” game — a renowned reinforcement learning challenge where an agent learns to balance a pole on a moving cart
|
Nonetheless, despite my failure, this phase of my journey was not devoid of value. Through these challenges, I became somewhat of an “AI Expert”, understanding deeply complicated fields of AI such as deep reinforcement learning. I became a better software engineer and researcher, as I had to learn how to implement these complex algorithms from scratch. I now understand neural networks better than 99.995% of the population. And most importantly, I developed a passion for Artificial Intelligence, even before it became “cool” because of ChatGPT.
|
The culmination of this journey led to the inception of NexusTrade — an AI-powered automated investing platform. The lessons learned and skills acquired became instrumental in exploring practical AI applications, such as strategy optimization and leveraging Large Language Models (LLMs) for strategy generation. This venture has been a remarkable success, amassing over 1300 users.
|
User Growth for the NexusTrade App
|
Additionally, my journey into Reinforcement Learning also led to the creation of NexusGenAI, my LLM Configuration app that powers NexusTrade’s AI-Powered Chat. I would’ve never been interested in Large Language Models if I hadn’t developed this passion for AI.
|
My life would be a lot different — no exaggeration. While I set out to accomplish an impossible goal, and failed, I couldn’t be happier that I tried. I hope this article serves as inspiration — set impossible goals for yourself. If you aim for the stars and miss, you at least landed on the moon.
|
|
