TL;DR
Gemini 2.5 Flash (google/gemini-2.5-flash) appears in 3 published studies. 2025-08-10, GPT-5 family and peers generating financial SQL queries: 0.900 for median score. 2025-08-06, GPT-oss and Gemini generating financial SQL queries: 0.867 for median score. 2025-08-11, Models turning plain-English trading rules into strategy configs: 0.825 for median score. Each result belongs to its own task, date and grading method.
- Exact tested ID
- google/gemini-2.5-flash
- Published studies
- 3
- Provider
Inspect this exact model
GPT-5 family and peers generating financial SQL queries
Gemini 2.5 Flash: source evidence dated 2025-08-11.
- Median score
- 0.900
- Avg score
- 0.657
- Success rate
- 78.65%
- Avg time
- 40,236 ms
- Input cost
- $0.30/M
- Output cost
- $2.50/M
The displayed values belong to the identified model versions and this published task, with their original metric units.
How did this version perform on the job I need?
The first measured row below belongs to this model, not its provider's other models. Open that study to see the other arms and tested input.
2025-08-10: GPT-5 family and peers generating financial SQL queriesSelect a model for the specific tested task. A profile with one study does not establish performance on other jobs.
Test a strategy with Aurora
Select a model for the specific tested task. A profile with one study does not establish performance on other jobs.
Test a strategy with AuroraKeep reporting dates and units beside the answer, and inspect its original citations.
GPT-5 family and peers generating financial SQL queries
See the complete gpt-5 family and peers generating financial sql queries
Gemini 2.5 Flash recorded 0.900 for median score in the 2025-08-10 publication. EvaluateGPT financial question set (count not restated in this article), 8 models.
| Published metric | Value |
|---|---|
| Median score | 0.900 |
| Avg score | 0.657 |
| Success rate | 78.65% |
| Avg time | 40,236 ms |
| Input cost | $0.30/M |
| Output cost | $2.50/M |
No matching rows. Clear the filter to see all records.
GPT-5 family and peers generating financial SQL queries: read these measurements
View task, sample and measurement units
EvaluateGPT financial question set (count not restated in this article), 8 models
Generate syntactically and semantically valid SQL queries for real-world financial questions.
Scoring
LLM-judge scores from 0 to 1 as described in the linked GPT-oss article (Claude Sonnet 4, Gemini 2.5 Pro and GPT-4.1 judges in that article). This article does not restate the judges.
Cost units
List prices per million tokens, from $0.05/M input and $0.40/M output (GPT-5 Nano) to $1.25/M input and $10.00/M output (Gemini 2.5 Pro, GPT-5, GPT-5 Chat). Total spend was reported as around $200.
Time units
Average time in milliseconds, from 40,236 ms (Gemini 2.5 Flash) to 113,743 ms (GPT-5).
GPT-oss and Gemini generating financial SQL queries
Open every published row
Gemini 2.5 Flash recorded 0.867 for median score in the 2025-08-06 publication. 90 financial questions per the article text, 4 models.
| Published metric | Value |
|---|---|
| Median score | 0.867 |
| Avg score | 0.615 |
| Success rate | 75.28% |
| Avg time | 39,237 ms |
| Input cost | $0.30/M |
| Output cost | $2.50/M |
| Context window | 1,048,576 |
No matching rows. Clear the filter to see all records.
GPT-oss and Gemini generating financial SQL queries: read these measurements
View task, sample and measurement units
90 financial questions per the article text, 4 models
Generate BigQuery SQL queries that answer complex financial questions.
Scoring
Three LLM judges score each query and its results from 0 to 1. The scores are averaged.
Cost units
List prices per million tokens, from $0.05/M input and $0.20/M output (GPT-OSS-20B) to $1.25/M input and $10.00/M output (Gemini 2.5 Pro Preview). No measured run spend is published.
Time units
Average time in milliseconds, from 38,538 ms (GPT-OSS-120B) to 56,393 ms (Gemini 2.5 Pro Preview).
Models turning plain-English trading rules into strategy configs
Open every published row
Gemini 2.5 Flash recorded 0.825 for median score in the 2025-08-11 publication. 14 models on one sample of strategy requests, this article does not state the count (the companion article of 2025-08-12 describes the same table as 12 financial questions).
| Published metric | Value |
|---|---|
| Median score | 0.825 |
| Avg score | 0.746 |
| Success rate | 100.0% |
| Avg time (ms) | 114,618 |
| Input cost | $0.30/M |
| Output cost | $2.50/M |
No matching rows. Clear the filter to see all records.
Models turning plain-English trading rules into strategy configs: read these measurements
View task, sample and measurement units
14 models on one sample of strategy requests, this article does not state the count (the companion article of 2025-08-12 describes the same table as 12 financial questions)
Convert a natural-language trading strategy request into a NexusTrade portfolio configuration of actions and conditions.
Scoring
LLM judges score each configuration from 0 to 1 using published guidelines (1.0 perfect match, 0.8 to 0.9 minor issues, 0.5 to 0.7 significant discrepancies, 0.2 to 0.4 major errors, 0.0 to 0.1 complete mismatch). Success rate is reported separately.
Cost units
The table lists each model's list price per million input and output tokens. It does not publish total run cost.
Time units
Average time per generation in milliseconds, as published in the table.
No comments yet.