TL;DR
Claude 3.5 Sonnet (anthropic/claude-3.5-sonnet) appears in 3 published studies. 2024-06-25, Claude 3.5 Sonnet and GPT-4o turning a screen into a MongoDB query: No response for response. 2024-06-25, Claude 3.5 Sonnet and GPT-4o generating trading portfolio configs: ~83.5s for test 1 final response. 2025-02-07, Claude 3.5 Sonnet and o3-mini building a Bitcoin trading strategy: Fail for buy rule matched request. Each result belongs to its own task, date and grading method.
- Exact tested ID
- anthropic/claude-3.5-sonnet
- Published studies
- 3
- Provider
- anthropic
Inspect this exact model
Claude 3.5 Sonnet and GPT-4o turning a screen into a MongoDB query
Claude 3.5 Sonnet: source evidence dated 2025-02-07.
- Response
- No response
- Time to response
- —
The displayed values belong to the identified model versions and this published task, with their original metric units.
How did this version perform on the job I need?
The first measured row below belongs to this model, not its provider's other models. Open that study to see the other arms and tested input.
2024-06-25: Claude 3.5 Sonnet and GPT-4o turning a screen into a MongoDB querySelect a model for the specific tested task. A profile with one study does not establish performance on other jobs.
Test a strategy with Aurora
Select a model for the specific tested task. A profile with one study does not establish performance on other jobs.
Test a strategy with AuroraKeep reporting dates and units beside the answer, and inspect its original citations.
Claude 3.5 Sonnet and GPT-4o turning a screen into a MongoDB query
See the complete claude 3.5 sonnet and gpt-4o turning a screen into a mongodb query
Claude 3.5 Sonnet recorded No response for response in the 2024-06-25 publication. 1 screening request, 2 models, single run each.
| Published metric | Value |
|---|---|
| Response | No response |
| Time to response | — |
No matching rows. Clear the filter to see all records.
Claude 3.5 Sonnet and GPT-4o turning a screen into a MongoDB query: read these measurements
View task, sample and measurement units
1 screening request, 2 models, single run each
Translate a plain-English stock screen into a MongoDB query, execute it and summarize the matching companies.
Scoring
Response or no response only. The article did not verify GPT-4o's returned figures.
Cost units
The article publishes no per-model cost for this test.
Time units
GPT-4o responded in 26 seconds. Claude 3.5 Sonnet never responded.
Claude 3.5 Sonnet and GPT-4o generating trading portfolio configs
Open every published row
Claude 3.5 Sonnet recorded ~83.5s for test 1 final response in the 2024-06-25 publication. 2 portfolio-creation prompts, 2 models, single run each.
| Published metric | Value |
|---|---|
| Test 1 result | — |
| Test 1 final response | ~83.5s |
| Test 2 result | 100% accurate |
| Test 2 final response | ~180s |
| Input price | $3/MTok |
| Output price | $15/MTok |
No matching rows. Clear the filter to see all records.
Claude 3.5 Sonnet and GPT-4o generating trading portfolio configs: read these measurements
View task, sample and measurement units
2 portfolio-creation prompts, 2 models, single run each
Generate a complete, syntactically valid portfolio configuration (nested strategies, conditions and indicators) from a plain-English trading request, plus a user-facing summary.
Scoring
Author inspection of the generated JSON. In Test 2, GPT-4o's buy strategy included conditions that were not specified and its stop loss was described as odd, while Claude's configuration was described as exactly what was articulated.
Cost units
List prices shown in the article: GPT-4o $5.00/M input and $15.00/M output tokens. Claude 3.5 Sonnet $3/MTok input and $15/MTok output. No per-run spend is published.
Time units
Approximate time to the final response: GPT-4o about 53.5 seconds (Test 1) and 44 seconds (Test 2). Claude 3.5 Sonnet about 83.5 seconds (Test 1) and around 180 seconds (Test 2). Intermediate responses took about 3 to 3.5 seconds for GPT-4o and 3.6 to 4 seconds for Claude.
Claude 3.5 Sonnet and o3-mini building a Bitcoin trading strategy
Open every published row
Claude 3.5 Sonnet recorded Fail for buy rule matched request in the 2025-02-07 publication. 1 strategy request, 2 models, one attempt each.
| Published metric | Value |
|---|---|
| Buy rule matched request | Fail |
| Generated buy condition (as displayed) | (1 Day BTC Rate of Change > ROC(BTC Price) + 1) or (Positions Percent Change of (BTC) < -15) |
No matching rows. Clear the filter to see all records.
Claude 3.5 Sonnet and o3-mini building a Bitcoin trading strategy: read these measurements
View task, sample and measurement units
1 strategy request, 2 models, one attempt each
Turn a plain-English trading request into a syntactically valid, nested NexusTrade strategy JSON object with the requested conditions.
Scoring
Author inspection of the generated buy rule. Claude 3.5 Sonnet produced a syntactically valid strategy on its first try, but its buy rule used 1 plus the 3-day rate of change instead of 1 standard deviation. The article states o3-mini created the strategies exactly.
Cost units
The article publishes no per-run cost. It quotes list prices of $1.10 input and $4.40 output per million tokens for o3-mini and $15 output per million tokens for Claude 3.5 Sonnet. Its text gives Claude 3.5 Sonnet's input price as $3.75, while its pricing screenshot shows $3 input and $3.75 for prompt caching writes.
Time units
The article publishes no per-model timing.
No comments yet.