TL;DR
GPT-4o (openai/gpt-4o) appears in 2 published studies. 2024-06-25, Claude 3.5 Sonnet and GPT-4o turning a screen into a MongoDB query: Returned 5 companies for response. 2024-06-25, Claude 3.5 Sonnet and GPT-4o generating trading portfolio configs: Correct for test 1 result. Each result belongs to its own task, date and grading method.
- Exact tested ID
- openai/gpt-4o
- Published studies
- 2
- Provider
- openai
Inspect this exact model
Claude 3.5 Sonnet and GPT-4o turning a screen into a MongoDB query
GPT-4o: source evidence dated 2024-06-25.
- Response
- Returned 5 companies
- Time to response
- 26s
The displayed values belong to the identified model versions and this published task, with their original metric units.
How did this version perform on the job I need?
The first measured row below belongs to this model, not its provider's other models. Open that study to see the other arms and tested input.
2024-06-25: Claude 3.5 Sonnet and GPT-4o turning a screen into a MongoDB querySelect a model for the specific tested task. A profile with one study does not establish performance on other jobs.
Test a strategy with Aurora
Select a model for the specific tested task. A profile with one study does not establish performance on other jobs.
Test a strategy with AuroraKeep reporting dates and units beside the answer, and inspect its original citations.
Claude 3.5 Sonnet and GPT-4o turning a screen into a MongoDB query
See the complete claude 3.5 sonnet and gpt-4o turning a screen into a mongodb query
GPT-4o recorded Returned 5 companies for response in the 2024-06-25 publication. 1 screening request, 2 models, single run each.
| Published metric | Value |
|---|---|
| Response | Returned 5 companies |
| Time to response | 26s |
No matching rows. Clear the filter to see all records.
Claude 3.5 Sonnet and GPT-4o turning a screen into a MongoDB query: read these measurements
View task, sample and measurement units
1 screening request, 2 models, single run each
Translate a plain-English stock screen into a MongoDB query, execute it and summarize the matching companies.
Scoring
Response or no response only. The article did not verify GPT-4o's returned figures.
Cost units
The article publishes no per-model cost for this test.
Time units
GPT-4o responded in 26 seconds. Claude 3.5 Sonnet never responded.
Claude 3.5 Sonnet and GPT-4o generating trading portfolio configs
Open every published row
GPT-4o recorded Correct for test 1 result in the 2024-06-25 publication. 2 portfolio-creation prompts, 2 models, single run each.
| Published metric | Value |
|---|---|
| Test 1 result | Correct |
| Test 1 final response | ~53.5s |
| Test 2 result | Misconfigured |
| Test 2 final response | 44s |
| Input price | $5.00/M |
| Output price | $15.00/M |
No matching rows. Clear the filter to see all records.
Claude 3.5 Sonnet and GPT-4o generating trading portfolio configs: read these measurements
View task, sample and measurement units
2 portfolio-creation prompts, 2 models, single run each
Generate a complete, syntactically valid portfolio configuration (nested strategies, conditions and indicators) from a plain-English trading request, plus a user-facing summary.
Scoring
Author inspection of the generated JSON. In Test 2, GPT-4o's buy strategy included conditions that were not specified and its stop loss was described as odd, while Claude's configuration was described as exactly what was articulated.
Cost units
List prices shown in the article: GPT-4o $5.00/M input and $15.00/M output tokens. Claude 3.5 Sonnet $3/MTok input and $15/MTok output. No per-run spend is published.
Time units
Approximate time to the final response: GPT-4o about 53.5 seconds (Test 1) and 44 seconds (Test 2). Claude 3.5 Sonnet about 83.5 seconds (Test 1) and around 180 seconds (Test 2). Intermediate responses took about 3 to 3.5 seconds for GPT-4o and 3.6 to 4 seconds for Claude.
No comments yet.