How AI Agents Actually Remember (Memory + RAG Explained)

0:04Welcome back to the course on how to build an AI agent from scratch In the last video we introduced the concept of memory An agent should be able to remember basic things so it doesn't have to learn everything

0:19from scratch every single time you create an agent And in that video I mostly talked about simple memory architecture For example storing a raw dump of the entire conversation

0:34in a file and then fetching that file when you create a new agent I talked about maybe some smarter improvements Maybe storing separate different files and in each file you'll just store a small amount

0:49of relevant information that the agent can fetch when it needs to However if you have a 100 page document or a Harry Potter book or you're trying to

1:02do the law under the bunch of case law he has to ingest then accuracy is critical You need a way to store and fetch all of this information when you need to and

1:15a basic markdown file just isn't going to cut it In this video I'm going to be talking about advanced memory architecture and I'm going to focus on two different approaches

1:29Number one is using a vector database and number two is using a regular database with semantic information Both of these are different but important strategies

1:43for building long term memory in your AI agent Let's first start with the vector database and specifically I'm going to be focusing on

1:56a technique called retrieval augmented generation Let's talk about that Retrieval augmented generation is how you search through a large number of documents using

2:11vectors First you need a vector database like this A vector database is simply a database that stores numerical embeddings or vectors and these vectors

2:24are the meaning of the inputs So instead of searching text like you would in a regular database you are searching for the similarity in

2:37vector space This sounds really vague but let me give you a concrete example with this image I generated using the AI So first

2:51the parts of retrieval augmented generation are first you want to take all of your inputs and transform them into vectors and store them in a vector database So we have this cat

3:06put it into an embedding model it will form this cat vector We have this other cat put it through an embedding database it will store this cat vector This is actually a kitten this is a cat but they're similar but not exactly the same as you

3:19can see by the numbers Next we have a dinosaur and this dinosaur is pretty far away from this cat but it's sort of similar ish but it's pretty far away because they're not

3:33the same thing And then finally we have the moon This is way off It's not even kind of close to any of these other things that we've stored So these things are stored in the vector

3:47database and then at runtime we'll have a query Let's say our query is cute animals We'll literally search through vector space to find

3:58the things that are close in space and then return them back to the user and then we'll put it back into our system prompt as information We just

4:11inject it into the prompt as retrieved information or not even in the prompt We can also put it in the conversation The point is this information will be given to the language model Now you don't

4:25need to know how all this works under the hood You don't need to know about cosine similarity and the different types of similarity algorithms because most of the time this is done for you

4:39There are managed vector databases that do a lot of this You can just create an account You also oftentimes don't even need a specialized vector database

4:52You can use something like Postgres with pgVector or something like MongoDB And if you have a short amount of vectors you can just store in blob storage You don't really need a vector database but you do need some way of retrieving

5:07these vectors quickly if you're implementing retrieval augmented generation So this is one way of storing a large amount of documents It relies on

5:19transforming the inputs into vectors and then doing a similarity search Another more advanced technique is just to simply store information in your database with tags that you can query later

5:35Here's a concrete example within my app NexusTrade This is my agent summary data model After an agent run we store the iteration start iteration end

5:49and also certain things like some semantic insights and some procedural lessons Also for each portfolio we store things like what the instrument type is If it has options what's the option structure

6:05How good did this strategy actually perform Which tickers were involved And we store that for every single portfolio that is generated That way at runtime if you say I want to create another option trading strategies we

6:19can query for similar strategies that you've already created that already did well And now we have a great starting point We don't have to start learning everything from scratch because we've already learned a lot of these

6:33lessons So this structure is actually injected at runtime if you use my app NexusTrade And this makes it so the agents improve

6:45over time automatically You're already injecting insights that have been proven to work into your agent at runtime And so this improves the agents

7:01Now here's a curveball question How do we know our AI agents are actually improved What evidence do we have to suggest that our agents are getting any better at all

7:16Are we just going to read through the agent and make a judgment call Yeah this looks nice This looks good Yeah yeah Not for a production AI agent That's not going to make you pass the AI native software engineer interview at Meta You need a

7:30systematic way of evaluating your AI agents This is so important and something that most people don't think about at all In the next video I'm

7:44going to be talking about agent evals And how do you know your agent is doing what you want it to do