TL;DR
NexusTrade has published 23 dated model evaluations from 2024-06-25 to 2026-08-24, covering 83 exact model versions. Each study keeps its own task, questions and grading, so compare models within one study and read every result with its publication date.
- Published studies
- 23
- Exact model versions
- 83
- Earliest publication
- 2024-06-25
- Latest publication
- 2026-08-24
Read a result with its date
Each row links to a study page with the full results table, the test procedure and the original article. Results describe the model versions available on the publication date. Newer versions of the same model family were not tested in older studies.
Compare models only within one study. Different studies used different questions, graders and harnesses, so their scores are not on a shared scale.
2026 evaluations
Open every published row
Scroll sideways to compare all columns. Select a heading to sort.
| 2026-08-24 | Voice-cloning systems ranked blind on one speaker's voice | Voice Cloning | 16 | Fine-tuned Dots.TTS soar ranked first, ahead of fine-tuned MOSS-TTS v1.5 and OmniVoice + LoRA checkpoint 800. Among zero-shot arms, Dots.TTS soar ranked highest (5) and MOSS-TTS v1.5 zero-shot ranked 13. |
| 2026-08-08 | Artifact grading and repeatability study | Artifact Review | 3 | Luna and Hy3 share the reported recall. Luna repeated its grade on 19/23 reruns, Hy3 on 18/23 and MiniMax M3 on 16/23. |
| 2026-08-08 | Five models completing one sandbox research task | Sandbox Code Generation | 5 | Luna reached B+ three times and had the lowest reported completed-run wall time and cost. The oscillating grades and single-task sample remain material limitations. |
| 2026-08-08 | Models answering 22 stock questions with SQL | Stock Screening | 6 | Gemini 3.6 Flash has the highest published average answer score (0.841) and success rate (86.4%) in this six-row SQL comparison. GPT 5.6 Luna scores 0.550 on the same table. |
| 2026-08-08 | Next-action execution model bakeoff | Agent Execution | 12 | The published deployment decision selected GPT 5.6 Luna because it improved score, schema validity, billed cost and latency versus the old DeepSeek default. GPT 5.6 Luna Pro had a higher raw score and a substantially higher cost. |
| 2026-08-08 | Planning models on 29 frozen conversations | Agent Planning | 14 | GPT 5.6 Luna was the published cost-oriented selection among similar-quality planning candidates. Muse Spark 1.1 had the highest raw mean; those are different claims. |
| 2026-04-06 | Eleven models building an options strategy | Options Strategy Development | 11 | Gemini 3 Flash Preview has the highest published score, 66/100. The evaluator still calls that mixed and asks for more iteration; no result proves the account-doubling objective. |
No matching rows. Clear the filter to see all records.
7 studies published in 2026. Each result applies to its own task, date and model versions.
Voice-cloning systems ranked blind on one speaker's voice ↗Artifact grading and repeatability study ↗Five models completing one sandbox research task ↗Models answering 22 stock questions with SQL ↗Next-action execution model bakeoff ↗Planning models on 29 frozen conversations ↗Eleven models building an options strategy ↗
2025 evaluations
Open every published row
Scroll sideways to compare all columns. Select a heading to sort.
| 2025-10-03 | MongoDB time-series collections vs TimescaleDB for backtest data | Time-Series Database | 2 | MongoDB time-series collections used 7.73 GB against 136.93 GB and answered backtest queries in 274 ms against 549 ms. TimescaleDB answered portfolio queries faster, 716 ms against 938 ms. |
| 2025-08-11 | Models turning plain-English trading rules into strategy configs | Strategy Configuration Generation | 14 | Claude Opus 4.1 has the highest published median (1.000) and average score (0.945) with a 100.0% success rate and the lowest average time (85,175 ms). Claude Sonnet 4 has the lowest average score among models with a 100.0% success rate (0.684). |
| 2025-08-10 | GPT-5 family and peers generating financial SQL queries | Financial SQL Generation | 8 | Gemini 2.5 Pro has the highest published median (0.967), average score (0.788) and success rate (88.76%). GPT-5 scores a 0.950 median and 0.699 average with the slowest average time (113,743 ms), while GPT-5 Mini scores 0.933 and 0.717 at $0.25/M input. |
| 2025-08-06 | GPT-oss and Gemini generating financial SQL queries | Financial SQL Generation | 4 | Gemini 2.5 Pro Preview and GPT-OSS-120B share the highest median score (0.900), but Gemini 2.5 Pro Preview has the higher average score (0.706 vs 0.614) and success rate (83.15% vs 70.79%). GPT-OSS-120B has the fastest average time (38,538 ms). |
| 2025-07-11 | Models generating financial SQL queries, July 2025 | Financial SQL Generation | 11 | Gemini 2.5 Pro has the highest published average score (0.757) and ties o4 Mini on median score (0.967). Grok-4 scores a 0.767 median and 0.666 average with the slowest average time (74,812 ms). |
| 2025-04-15 | Models generating SQL for financial questions, April 2025 | Financial SQL Generation | 11 | GPT-4.1 has the highest published average score (0.884) and success rate (93.3%) in this 11-model table, with Gemini 2.5 Pro at 0.880 and 92.5%. GPT-4.1 Nano has the lowest average score (0.382). |
| 2025-04-10 | Seven models generating financial SQL queries | Financial SQL Generation | 7 | Grok-3-Beta has the highest published average score (0.747) and perfect-score share (67.5%). Gemini 2.0 Flash has the highest success rate (92.5%) and fastest average execution time (1,296 ms), and Llama 4 Maverick has the lowest average score (0.565). |
| 2025-02-24 | Claude 3.7 Sonnet and o3-mini building a vague leveraged-ETF strategy | Strategy Configuration Generation | 2 | Claude 3.7 Sonnet's strategy returned 36.02% with a 1.03 Sharpe ratio and 17.55% max drawdown. o3-mini's strategy returned 31.87% with a 0.54 Sharpe ratio and 41.74% max drawdown. |
| 2025-02-07 | Claude 3.5 Sonnet and o3-mini building a Bitcoin trading strategy | Strategy Configuration Generation | 2 | The article reports that o3-mini's buy rule matched the request (Pass), while Claude 3.5 Sonnet's buy rule used 1 plus the rate of change instead of 1 standard deviation (Fail). |
| 2025-02-05 | Gemini 2.0 Flash, DeepSeek R1 and o3-mini writing financial SQL | Financial SQL Generation | 3 | Gemini 2.0 Flash scored 1/1 on both questions at $0.10 input and $0.40 output per million tokens. o3-mini scored 0.7/1 and 1/1, and DeepSeek R1 scored 0.7/1 and 0/1. |
| 2025-02-01 | DeepSeek R1 and o3-mini writing a dividend stock screen in SQL | Financial SQL Generation | 2 | In this single February 2025 request, o3-mini scored 1 on accuracy in 2 minutes 24 seconds, and DeepSeek R1 scored 0 after 41 minutes. Both runs were estimated at 6c. |
| 2025-01-21 | DeepSeek R1 and OpenAI o1 answering SPY drawdown questions with SQL | Financial SQL Generation | 2 | On question 1, R1 reported 90 qualifying 7-day drops and o1 reported 150, and the article judged both correct. Both reported an average 365-day return of 20.29% after the drops. R1's list price was $0.55/M input and $2.19/M output against o1's $15.00/M and $60.00/M. |
No matching rows. Clear the filter to see all records.
12 studies published in 2025. Each result applies to its own task, date and model versions.
MongoDB time-series collections vs TimescaleDB for backtest data ↗Models turning plain-English trading rules into strategy configs ↗GPT-5 family and peers generating financial SQL queries ↗GPT-oss and Gemini generating financial SQL queries ↗Models generating financial SQL queries, July 2025 ↗Models generating SQL for financial questions, April 2025 ↗Seven models generating financial SQL queries ↗Claude 3.7 Sonnet and o3-mini building a vague leveraged-ETF strategy ↗Claude 3.5 Sonnet and o3-mini building a Bitcoin trading strategy ↗Gemini 2.0 Flash, DeepSeek R1 and o3-mini writing financial SQL ↗DeepSeek R1 and o3-mini writing a dividend stock screen in SQL ↗DeepSeek R1 and OpenAI o1 answering SPY drawdown questions with SQL ↗
2024 evaluations
Open every published row
Scroll sideways to compare all columns. Select a heading to sort.
| 2024-07-19 | GPT-4o mini and GPT-3.5 Turbo generating a portfolio JSON object | Strategy Configuration Generation | 2 | GPT-4o mini passed, generating a valid portfolio that passed a backtest configuration, while GPT-3.5 Turbo failed after running out of context. |
| 2024-07-19 | GPT-4o mini and GPT-3.5 Turbo generating a valid SQL query | Financial SQL Generation | 2 | GPT-4o mini passed, delivering accurate results on the first try, while GPT-3.5 Turbo failed to generate a valid query. |
| 2024-06-25 | Claude 3.5 Sonnet and GPT-4o generating trading portfolio configs | Strategy Configuration Generation | 2 | On the custom-indicator prompt (Test 2), Claude 3.5 Sonnet's configuration was graded 100% accurate after about 180 seconds, while GPT-4o's was misconfigured after 44 seconds. GPT-4o's final response was faster on Test 1 as well (about 53.5 seconds against 83.5 seconds). |
| 2024-06-25 | Claude 3.5 Sonnet and GPT-4o turning a screen into a MongoDB query | NoSQL Stock Screening | 2 | GPT-4o returned 5 companies in 26 seconds. Claude 3.5 Sonnet returned no response because its query never terminated. |
No matching rows. Clear the filter to see all records.
4 studies published in 2024. Each result applies to its own task, date and model versions.
Continue exploring
Discussion
Sign in or create a free account to join the discussion.
No comments yet.