TL;DR
Grok 3 Mini Beta (x-ai/grok-3-mini-beta) appears in 3 published studies. 2025-04-10, Seven models generating financial SQL queries: 0.645 for average score. 2025-08-11, Models turning plain-English trading rules into strategy configs: 0.900 for median score. 2025-04-15, Models generating SQL for financial questions, April 2025: 0.645 for average score. Each result belongs to its own task, date and grading method.
- Exact tested ID
- x-ai/grok-3-mini-beta
- Published studies
- 3
- Provider
- x-ai
Inspect this exact model
Seven models generating financial SQL queries
Grok 3 Mini Beta: source evidence dated 2025-08-11.
- Average score
- 0.645
- Median score
- 1.000
- Std deviation
- 0.459
- Success rate
- 72.5%
- Avg execution time (ms)
- 1,593
- Perfect score (1.0) share
- 60.0%
The displayed values belong to the identified model versions and this published task, with their original metric units.
How did this version perform on the job I need?
The first measured row below belongs to this model, not its provider's other models. Open that study to see the other arms and tested input.
2025-04-10: Seven models generating financial SQL queriesSelect a model for the specific tested task. A profile with one study does not establish performance on other jobs.
Test a strategy with Aurora
Select a model for the specific tested task. A profile with one study does not establish performance on other jobs.
Test a strategy with AuroraKeep reporting dates and units beside the answer, and inspect its original citations.
Seven models generating financial SQL queries
See the complete seven models generating financial sql queries
Grok 3 Mini Beta recorded 0.645 for average score in the 2025-04-10 publication. 7 models on a shared SQL question set, the article does not state the number of questions.
| Published metric | Value |
|---|---|
| Average score | 0.645 |
| Median score | 1.000 |
| Std deviation | 0.459 |
| Success rate | 72.5% |
| Avg execution time (ms) | 1,593 |
| Perfect score (1.0) share | 60.0% |
No matching rows. Clear the filter to see all records.
Seven models generating financial SQL queries: read these measurements
View task, sample and measurement units
7 models on a shared SQL question set, the article does not state the number of questions
Generate SQL queries that answer natural-language financial questions over NexusTrade's stock data.
Scoring
Each query receives a 0 to 1 score. The table reports the distribution in buckets 0.0-0.2, 0.3-0.5, 0.6-0.7, 0.8-0.9 and 1.0. The grader is not described in this article.
Cost units
The article publishes no per-model cost.
Time units
Average execution time in milliseconds per model, from 1,296 ms (Gemini 2.0 Flash) to 26,892 ms (DeepSeek V3). The article does not define whether this includes model generation time.
Models turning plain-English trading rules into strategy configs
Open every published row
Grok 3 Mini Beta recorded 0.900 for median score in the 2025-08-11 publication. 14 models on one sample of strategy requests, this article does not state the count (the companion article of 2025-08-12 describes the same table as 12 financial questions).
| Published metric | Value |
|---|---|
| Median score | 0.900 |
| Avg score | 0.836 |
| Success rate | 100.0% |
| Avg time (ms) | 109,802 |
| Input cost | $0.30/M |
| Output cost | $0.50/M |
No matching rows. Clear the filter to see all records.
Models turning plain-English trading rules into strategy configs: read these measurements
View task, sample and measurement units
14 models on one sample of strategy requests, this article does not state the count (the companion article of 2025-08-12 describes the same table as 12 financial questions)
Convert a natural-language trading strategy request into a NexusTrade portfolio configuration of actions and conditions.
Scoring
LLM judges score each configuration from 0 to 1 using published guidelines (1.0 perfect match, 0.8 to 0.9 minor issues, 0.5 to 0.7 significant discrepancies, 0.2 to 0.4 major errors, 0.0 to 0.1 complete mismatch). Success rate is reported separately.
Cost units
The table lists each model's list price per million input and output tokens. It does not publish total run cost.
Time units
Average time per generation in milliseconds, as published in the table.
Models generating SQL for financial questions, April 2025
Open every published row
Grok 3 Mini Beta recorded 0.645 for average score in the 2025-04-15 publication. 60 financial questions per the article text, 11 models.
| Published metric | Value |
|---|---|
| Average score | 0.645 |
| Median score | 1.000 |
| Std dev | 0.459 |
| Success rate | 72.5% |
| Avg execution | 1,593 ms |
| Input cost | $0.30/M |
| Output cost | $0.50/M |
No matching rows. Clear the filter to see all records.
Models generating SQL for financial questions, April 2025: read these measurements
View task, sample and measurement units
60 financial questions per the article text, 11 models
Generate SQL queries that correctly answer financial analysis questions.
Scoring
The article reports scores and success rates but refers readers to an earlier article for the methodology. The grader is not described in this article.
Cost units
List prices per million tokens, from $0.10/M input for GPT-4.1 Nano and Gemini 2.0 Flash to $3.00/M input and $15.00/M output for Claude 3.7 Sonnet and Grok-3-Beta. No measured run spend is published.
Time units
Average execution time in milliseconds, from 1,296 ms (Gemini 2.0 Flash) to 38,614 ms (GPT-4.1 Nano).
No comments yet.