TL;DR
Published 2025-02-01. In this single February 2025 request, o3-mini scored 1 on accuracy in 2 minutes 24 seconds, and DeepSeek R1 scored 0 after 41 minutes. Both runs were estimated at 6c.
- Published
- 2025-02-01
- Published model rows
- 2
- Evidence
- Historical task evaluation
Follow this evaluation
Published results
DeepSeek R1 and o3-mini writing a dividend stock screen in SQL: source evidence dated 2025-02-01.
- Exact model version
- OpenAI o3-mini
- Accuracy
- 1
- Estimated cost
- 6c
- Latency
- 2 minutes 24 seconds
- Input price
- $1.10/M
- Output price
- $4.4/M
The displayed values belong to the identified model versions and this published task, with their original metric units.
What did this task require the model to return?
Read the tested version, task and rubric together. Check failures alongside successful answers; compare costs only within the reported unit.
2025-02-01: DeepSeek R1 and o3-mini writing a dividend stock screen in SQLKeep the study date and exact model ID beside the result. Do not average different tasks into one winner.
Test a strategy with Aurora
Keep the study date and exact model ID beside the result. Do not average different tasks into one winner.
Test a strategy with AuroraKeep reporting dates and units beside the answer, and inspect its original citations.
Published results
See the complete published results
Scroll sideways to compare all columns. Select a heading to sort.
| OpenAI o3-mini | 1 | 6c | 2 minutes 24 seconds | $1.10/M | $4.4/M |
| DeepSeek R1 | 0 | 6c | 41 minutes | $0.75/M | $2.4/M |
No matching rows. Clear the filter to see all records.
1 natural-language screening request, 2 models, R1 asked twice, o3-mini once. Published 2025-02-01. Values retain their original units and grading scale.
Task and tested input
Convert a plain-English dividend stock screen into an executable SQL query, run it and summarize the results.
Published prompt excerpt
Show me large-cap stocks with: - Dividend yield >3% - 5 year dividend growth > 5% - Debt/Equity <0.5
How the runs worked
1 natural-language screening request, 2 models, R1 asked twice, o3-mini once
NexusTrade Aurora text-to-SQL pipeline with LLM grading and automatic retry, over the NexusTrade BigQuery fundamentals tables
Step 1
Send the same screening request to Aurora with DeepSeek R1 selected. The article reports asking it two separate times.
Step 2
Let the application generate SQL, execute it, have an LLM grade the response and retry automatically below the threshold.
Step 3
Repeat the request once with o3-mini, then grade accuracy by inspecting the query and read latency from database and log timestamps.
How to read the measurements
Grading and valid output
The author assigned 1 when manual inspection found the query conformed to the request and 0 otherwise. For R1 the article lists using total liabilities instead of debt, full-year instead of latest-quarter data and an average dividend yield as the defects.
Cost
Estimated relative cost: with c as one R1 request, R1 used 5 retries plus 1 request (6c), and o3-mini used 1 retry, 1 evaluation and 1 summarization request at 2c each (6c). List prices shown were $0.75/M input and $2.4/M output for R1 on OpenRouter and $1.10/M input and $4.4/M output for o3-mini.
Time
R1 gave up after 41 minutes according to the database logs. O3-mini responded 2 minutes 24 seconds after the 6:26 PM request, including one retry.
Why this model was selected
The article concluded o3-mini was better than R1 in every way for this example and said it was integrated for all NexusTrade users, as R1 had been.
See the published results
This image was published with the 2025-02-01 article. Read the source for the full test procedure and every result it reports.
Sample and scope
February 2025 single-question comparison. The article itself frames the result as "this one example", and DeepSeek R1 and o3-mini have since been replaced.
Accuracy is a 0 or 1 grade from the author's manual query inspection. The o3-mini answer screenshot includes a row with a negative debt-to-equity (-20.4350) and a 433540.00 dividend growth figure, which the article does not discuss.
Cost is an estimate in relative units built from request counts and list prices at the time, not a measured bill. R1 was served through OpenRouter (provider Nebius in the log) during a period the app flagged as high demand.
No comments yet.