TL;DR
o1 (openai/o1) appears in 1 published study. 2025-01-21, DeepSeek R1 and OpenAI o1 answering SPY drawdown questions with SQL: 150 for q1 drop count. Each result belongs to its own task, date and grading method.
- Exact tested ID
- openai/o1
- Published studies
- 1
- Provider
- openai
Inspect this exact model
DeepSeek R1 and OpenAI o1 answering SPY drawdown questions with SQL
o1: source evidence dated 2025-01-21.
- Q1 drop count
- 150
- Q1 verdict
- Correct
- Q2 avg 180d drawdown
- -16.74%
- Q2 avg 365d drawdown
- -19.14%
- Q3 avg 180d return
- 8.58%
- Q3 avg 365d return
- 20.29%
The displayed values belong to the identified model versions and this published task, with their original metric units.
How did this version perform on the job I need?
The first measured row below belongs to this model, not its provider's other models. Open that study to see the other arms and tested input.
2025-01-21: DeepSeek R1 and OpenAI o1 answering SPY drawdown questions with SQLSelect a model for the specific tested task. A profile with one study does not establish performance on other jobs.
Test a strategy with Aurora
Select a model for the specific tested task. A profile with one study does not establish performance on other jobs.
Test a strategy with AuroraKeep reporting dates and units beside the answer, and inspect its original citations.
DeepSeek R1 and OpenAI o1 answering SPY drawdown questions with SQL
See the complete deepseek r1 and openai o1 answering spy drawdown questions with sql
o1 recorded 150 for q1 drop count in the 2025-01-21 publication. 3 chained SPY research questions in one conversation per model, 2 models, single run each.
| Published metric | Value |
|---|---|
| Q1 drop count | 150 |
| Q1 verdict | Correct |
| Q2 avg 180d drawdown | -16.74% |
| Q2 avg 365d drawdown | -19.14% |
| Q3 avg 180d return | 8.58% |
| Q3 avg 365d return | 20.29% |
| Input price | $15.00/M |
| Output price | $60.00/M |
No matching rows. Clear the filter to see all records.
DeepSeek R1 and OpenAI o1 answering SPY drawdown questions with SQL: read these measurements
View task, sample and measurement units
3 chained SPY research questions in one conversation per model, 2 models, single run each
Answer multi-step historical market questions about SPY by generating SQL, executing it and summarizing the results.
Scoring
Author inspection only. Question 1 was judged 100% correct for both models, with o1 described as a little better for including a total-occurrences field and an untruncated table. Question 2 favored R1 for including a ratio. Question 3 was described as almost exactly alike.
Cost units
List prices quoted in the article: R1 $0.55/M input and $2.19/M output tokens. O1 $15.00/M input and $60.00/M output tokens. No per-question spend is published.
Time units
The article publishes no per-model timing.
No comments yet.