I’ve been writing about genetic algorithms (GAs) for a few months now. These algorithms are elegant and powerful, drawing inspiration from the natural process of evolution. When I discuss GAs, many of my readers are wondering what a practical use-case for it is outside of the realm of finance. Well, I just found one.
|
I will discuss the researcher’s approach, some of the benefits I see, and some of the downsides. I’ll also discuss what the paper is missing (in my opinion) and how it could be improved.
|
This paper presents a novel approach to prompt engineering. Traditional prompt engineering is usually extremely manual, and requires substantial human investment with little concrete guidance.
|
So, the authors had the idea of using evolutionary algorithms for prompt engineering, with some caveats. With traditional evolutionary algorithms, the operators tend to independently operate on tokens. This is detrimental for prompt optimization because we would lose coherence in the sentence structure. Just think about it logically – if we replace a random word in a sentence, that has the potential effect of creating a nonsensical phrase.
|
To overcome this limitation, the authors had the brilliant idea of using a Large Language Model to simulate evolutionary algorithms instead of implementing it in the traditional way. The LLM performs the operations needed to generate new candidate solutions.
|
They call this architecture EvoPrompt. In their evaluations, EvoPrompt significantly outperforms manual prompt engineering and other automated approaches. It’s extremely easy to implement, only requiring a Large Language Model. And, it works with relatively small population sizes.
|
|