TL;DR
DeepSeek R1 (deepseek/deepseek-r1) appears in 3 published studies. 2025-02-05, Gemini 2.0 Flash, DeepSeek R1 and o3-mini writing financial SQL: 0.7/1 for q1: rddt vs spy return correlation. 2025-01-21, DeepSeek R1 and OpenAI o1 answering SPY drawdown questions with SQL: 90 for q1 drop count. 2025-02-01, DeepSeek R1 and o3-mini writing a dividend stock screen in SQL: 0 for accuracy. Each result belongs to its own task, date and grading method.
- Exact tested ID
- deepseek/deepseek-r1
- Published studies
- 3
- Provider
- deepseek
Inspect this exact model
Gemini 2.0 Flash, DeepSeek R1 and o3-mini writing financial SQL
DeepSeek R1: source evidence dated 2025-02-05.
- Q1: RDDT vs SPY return correlation
- 0.7/1
- Q2: biotech revenue growth
- 0/1
- Input price (per 1M tokens)
- $0.75
- Output price (per 1M tokens)
- $2.4
The displayed values belong to the identified model versions and this published task, with their original metric units.
How did this version perform on the job I need?
The first measured row below belongs to this model, not its provider's other models. Open that study to see the other arms and tested input.
2025-02-05: Gemini 2.0 Flash, DeepSeek R1 and o3-mini writing financial SQLSelect a model for the specific tested task. A profile with one study does not establish performance on other jobs.
Test a strategy with Aurora
Select a model for the specific tested task. A profile with one study does not establish performance on other jobs.
Test a strategy with AuroraKeep reporting dates and units beside the answer, and inspect its original citations.
Gemini 2.0 Flash, DeepSeek R1 and o3-mini writing financial SQL
See the complete gemini 2.0 flash, deepseek r1 and o3-mini writing financial sql
DeepSeek R1 recorded 0.7/1 for q1: rddt vs spy return correlation in the 2025-02-05 publication. 2 natural-language financial questions, 3 models, one attempt each.
| Published metric | Value |
|---|---|
| Q1: RDDT vs SPY return correlation | 0.7/1 |
| Q2: biotech revenue growth | 0/1 |
| Input price (per 1M tokens) | $0.75 |
| Output price (per 1M tokens) | $2.4 |
No matching rows. Clear the filter to see all records.
Gemini 2.0 Flash, DeepSeek R1 and o3-mini writing financial SQL: read these measurements
View task, sample and measurement units
2 natural-language financial questions, 3 models, one attempt each
Generate a BigQuery SQL query that answers a natural-language financial question over the NexusTrade stock dataset.
Scoring
Author-assigned per-question scores. On question 1, R1 misspelled adjustedClosingPrice and o3-mini used the ticker REDDIT, so both received 0.7 after a manual fix. On question 2, R1's query failed to execute (0/1), and o3-mini-high graded both the Gemini and o3-mini queries 9/10, which the author recorded as full credit.
Cost units
OpenRouter list prices per million tokens as shown in the article: Gemini Flash 2.0 $0.1 input and $0.4 output. DeepSeek R1 $0.75 input and $2.4 output. O3-mini $1.1 input and $4.4 output. No per-run cost is published.
Time units
The article publishes no per-model timing. It describes Gemini as responding in a couple of seconds, R1 as taking more than 30 seconds on question 1, and o3-mini as taking a few seconds.
DeepSeek R1 and OpenAI o1 answering SPY drawdown questions with SQL
Open every published row
DeepSeek R1 recorded 90 for q1 drop count in the 2025-01-21 publication. 3 chained SPY research questions in one conversation per model, 2 models, single run each.
| Published metric | Value |
|---|---|
| Q1 drop count | 90 |
| Q1 verdict | Correct |
| Q2 avg 180d drawdown | -16.69% |
| Q2 avg 365d drawdown | -19.33% |
| Q3 avg 180d return | +8.58% |
| Q3 avg 365d return | +20.29% |
| Input price | $0.55/M |
| Output price | $2.19/M |
No matching rows. Clear the filter to see all records.
DeepSeek R1 and OpenAI o1 answering SPY drawdown questions with SQL: read these measurements
View task, sample and measurement units
3 chained SPY research questions in one conversation per model, 2 models, single run each
Answer multi-step historical market questions about SPY by generating SQL, executing it and summarizing the results.
Scoring
Author inspection only. Question 1 was judged 100% correct for both models, with o1 described as a little better for including a total-occurrences field and an untruncated table. Question 2 favored R1 for including a ratio. Question 3 was described as almost exactly alike.
Cost units
List prices quoted in the article: R1 $0.55/M input and $2.19/M output tokens. O1 $15.00/M input and $60.00/M output tokens. No per-question spend is published.
Time units
The article publishes no per-model timing.
DeepSeek R1 and o3-mini writing a dividend stock screen in SQL
Open every published row
DeepSeek R1 recorded 0 for accuracy in the 2025-02-01 publication. 1 natural-language screening request, 2 models, R1 asked twice, o3-mini once.
| Published metric | Value |
|---|---|
| Accuracy | 0 |
| Estimated cost | 6c |
| Latency | 41 minutes |
| Input price | $0.75/M |
| Output price | $2.4/M |
No matching rows. Clear the filter to see all records.
DeepSeek R1 and o3-mini writing a dividend stock screen in SQL: read these measurements
View task, sample and measurement units
1 natural-language screening request, 2 models, R1 asked twice, o3-mini once
Convert a plain-English dividend stock screen into an executable SQL query, run it and summarize the results.
Scoring
The author assigned 1 when manual inspection found the query conformed to the request and 0 otherwise. For R1 the article lists using total liabilities instead of debt, full-year instead of latest-quarter data and an average dividend yield as the defects.
Cost units
Estimated relative cost: with c as one R1 request, R1 used 5 retries plus 1 request (6c), and o3-mini used 1 retry, 1 evaluation and 1 summarization request at 2c each (6c). List prices shown were $0.75/M input and $2.4/M output for R1 on OpenRouter and $1.10/M input and $4.4/M output for o3-mini.
Time units
R1 gave up after 41 minutes according to the database logs. O3-mini responded 2 minutes 24 seconds after the 6:26 PM request, including one retry.
No comments yet.