AI Agent Observability: Why Did One Easy Task Cost $1,000?

0:00You're sitting in front of a senior software engineer at Meta and he's looking at you asking you

0:06Why did this on call agent fail Why did it spend a thousand dollars on such an easy

0:14task when it has worked all the time before and because you watch this video You know exactly

0:23what to say and you're gonna get that big tech job making 200 000 300 000 400 000

0:31per year In the previous parts of the series we talked about the agentic loop how we have

0:38a thought and an action the action executes and gets some sort of tool call response Large responses

0:45are kept out of the conversation context and we append it all to a conversation log And when

0:53that log gets too long we want to compact it but in such a way that we can

0:59search it in the future We can see what we've done before so that we don't have to

1:05lose important information and we don't have to rediscover everything from scratch every single time The last part

1:13that we need to discuss is observability and you kind of get some of this for free depending

1:19on your architecture but you still want to mention this explicitly You want to make sure you're saving

1:25every single request in response to some sort of data store like a database You want to see

1:31how long these are taking with little latency metrics and you're going to want to have some sort

1:38of dashboard so you can see what are the tools that are always taking so long What are

1:43the tool calls that are have extremely high responses and why is it so high I thought we

1:50saved it in an external place but this little response is 100 000 tokens Oh look it's a

1:56bug that you only found because you monitored And there are other things that you might want to

2:01talk about during your interview like how does the agent actually start Is there a UI where I

2:05start it or does it start by a webhook or an API call or something that's more autonomous

2:12You might want to talk about what model are we actually using for this AI agent Are we

2:16going to give Anthropic 10 million a year Or are we going to use an open source model

2:21on our own hostile hardware Maybe we're going to use open router or prompt optimization or a prompt

2:27evaluation so you see which models are actually good enough for the task at hand And I have

2:34an entire course on how you pass this AI systems design round All of it is available with

2:42the link in my bio