ai / ai engineering / llm / mapreduce / prompt engineering / ocr / document processing / agentic engineering
MapReduce for LLMs: How Cheap Models Beat Claude Opus 5 on 100 PDFs
Here is the problem I use to teach this. You have 100 PDF documents and you need to know whether the words Hello World appear anywhere in them. You could upload all of them to one language model and ask. That might work, and it might not, because the documents are messy, some are photographs of pages, and the model gets it wrong. You could OCR everything first with a tool like Tesseract and send the text instead, which is a little better, and now you are relying on the accuracy of the OCR. If the OCR is wrong, your answer is wrong, and I need 100% accuracy.
MapReduce is the technique I reach for instead. I split the task up, I map it across several cheap models looking at different views of the same input, and then I have one reducer model combine their answers. The same page goes out as OCR text, as images, and as the raw PDF, and each view gets its own cheap model. The reducer reads all of their answers plus the few documents they flagged, and its system prompt says outright that some of the inputs are wrong. I also fan out by batch: instead of sending all 100 documents in one request, I send 10 batches of 10.
The reason this is worth doing is cost. Output prices per million tokens on OpenRouter the day I recorded this: DeepSeek V4 Flash at $0.16, GPT-5.6 Luna at $1.20, Muse Spark 1.3 Contributor at $0.20, against $25.00 for Claude Opus 5 and $50.00 for Claude Fable 5 and GPT-6 Astra. Three mappers and a reducer on the cheap end still costs a fraction of one call to the expensive one, and the mapping stage runs in parallel, so it is not slower either. It works anywhere a single prompt struggles: needle in a haystack searches, code review, any task where one model on one view of the input keeps getting it almost right.
Transcript
0:00I'm a senior AI engineer, and today I'm going to teach you about an agentic engineering technique that you've never heard of.
0:08Now I want you to agree that if you've never heard of this, you must leave a comment below telling
0:14others you've never heard of it, and you have to follow me. I post content like this literally all the
0:21time. So a little bit of background about me is I have a master's degree in software engineering from Carnegie Mellon,
0:29which according to U.S. News and World Report has two awards, two big wins.
0:36Number one, they're the best school in the entire world for software engineering. But also, lesser heard of, number two,
0:45they're the best school in the entire world for artificial intelligence. And yes,
0:51I did take artificial intelligence classes there,
0:55graduate-level AI classes while I was there. I've also built production-ready AI agents throughout my career, including as
1:04a software engineer at Oscar Health, and building my own agentic algorithmic trading platform. So if you want to learn
1:12from a literal AI expert who is publishing about using AI,
1:17using language models for algorithmic trading since 2023,
1:23stop and save this video. The technique I'm going to teach you is called MapReduce.
1:31Now let me illustrate this with an example.
1:33Let's say you have 100 PDF documents and your only task is to figure out if the words Hello World
1:41are in those PDFs. How would you do it? Naive solution number one is you just upload all those PDFs
1:49and send it to the language model. You might send it as images, or with modern models like GPT-5.6 Luna
1:55or Muse Spark you can actually upload the raw PDFs, and it's asked: is Hello World in any of these documents?
2:02And it might work, but it might not. If you do it this way,
2:09what you'll find is that the language models are wrong. They don't give you the right answer, especially if the
2:17raw PDFs are messy,
2:19maybe they're like pictures of photographs, or they're just blurry. They can give you the wrong answer.
2:25So here's what you want to do instead. Naive solution number two is you could just OCR the document. You
2:33could use an open source tool like Tesseract,
2:36extract the text from the document and just use that. Or even use like an open source OCR language model
2:42like Mistral's language model, and see what it says.
2:45And this is a little bit better,
2:48but you're still relying on the accuracy of the OCR. The OCR stands for Optical Character Recognition. I can make
2:55another video about that later. But if you OCR it and the OCR is wrong,
3:00then you're still going to get the wrong answer. You need 100%
3:05accuracy. What are you going to do? This is where we can do MapReduce.
3:11What we'll do is we'll do both independently.
3:16We'll have one OCR it. We'll get the OCR text and send the text to a language model.
3:22We'll also send the raw images and/or the raw PDFs. Or shoot, you might have a third one: that
3:28one does the images and one does the PDF. You just basically fan out. You do different things.
3:33And then, this is a quick addendum to the video I recorded post hoc, because I almost forgot the
3:39most important thing. With the PDFs and images, you know, you're originally sending all 100 of them in one request
3:49to the language model. Instead, you might want to fan that out too. You might want to send
3:5510 batches of 10 documents. Then you'll have one language model take the output of all of the inputs to
4:06the previous step. Your system prompt will explain that sometimes the inputs are wrong, they can do things incorrectly, you'll
4:14have good few-shot examples,
4:15and the LLM you send it to,
4:18called the reducer, will then give you the final answer. In this architecture,
4:24MapReduce significantly improves your accuracy on nearly any LLM task you can think of. And using modern day cheap LLMs
4:34like DeepSeek V4 Flash,
4:36GPT-5.6 Luna,
4:38Muse Spark 1.3,
4:40if you use these and you use MapReduce,
4:43this is almost always less expensive and more accurate and faster than using an expensive model like Claude Opus 5,
4:53Claude Fable 5, and yes, GPT-6 Astra. It works for needle in the haystack problems. It works for things like
5:02code review. And again, because of how cheap these powerful models are getting, and because the mapping stage can be
5:09parallelized,
5:10it's really not all that bad. Like, yeah, if you have something extremely performance critical,
5:15you're not going to want to use it. But you're probably not using a language model in the first place,
5:20right? So next time you get a hard problem in engineering and you can't solve it with a simple prompt,
5:25no matter all the prompt engineering that you're doing,
5:28try splitting it up,
5:30mapping it, and then having a reducer LLM. I promise you,
5:35you're going to be shocked at the results. If you learned something,
5:38click that follow button and leave a comment below.
Join the conversation
Loading conversation…