What this evaluation measured
The published deployment decision selected GPT 5.6 Luna because it improved score, schema validity, billed cost and latency versus the old DeepSeek default. GPT 5.6 Luna Pro had a higher raw score and a substantially higher cost.
Models choose the next action inside the same ReAct agent loop. The reported context window in this replay was 73,397 to 83,363 input tokens per decision.
The table reports mean grader score alongside deterministic schema checks, provider billing and production median latency. These columns should be evaluated separately.
Published results
| openai/gpt-5.6-luna-pro | 91.7 | 100.0% | $25.05 | 9.9s |
| openai/gpt-5.6-luna | 89.2 | 99.0% | $1.67 | 5.6s |
| meta/muse-spark-1.2 | 86.1 | 99.5% | $44.23 | 7.2s |
| x-ai/grok-build-0.1 | 84.7 | 100.0% | $26.88 | 15.4s |
| google/gemini-3-flash-preview | 83.3 | 97.1% | $24.95 | 6.7s |
| z-ai/glm-5.2 | 82.4 | 98.1% | $25.37 | 15.7s |
| google/gemini-3.6-flash | 80.1 | 100.0% | $24.21 | 7.7s |
| deepseek/deepseek-v4-flash | 75.3 | 98.1% | $6.23 | 17.9s |
| mistralai/mistral-small-2603 | 73.0 | 97.6% | $1.00 | 6.3s |
| google/gemini-3.5-flash-lite | 72.3 | 99.0% | $7.17 | 2.0s |
| nvidia/nemotron-3-ultra-550b | 57.1 | 77.5% | $35.28 | n/a |
| poolside/laguna-xs-2.1 | 51.1 | 66.5% | $5.58 | 4.3s |
No matching rows. Clear the filter to see all records.
23-model campaign; 69 frozen decisions × 3 samples; 12 published model rows. Published 2026-08-08. Values retain their original units and grading scale.
Methodology and sample
These reviewed results come from an evaluation that has already been published. Private conversation traces are excluded. Opening this page does not run a new benchmark.
Harness
NexusTrade Agent V6 next-action replay
Corpus
23-model campaign; 69 frozen decisions × 3 samples; 12 published model rows
Metric
Mean next-action score; schema validity, billed cost and production p50 are separate
What the results do not establish
Only 12 of the 23 campaign arms are in the public table. The catalog does not fabricate the unpublished rows or their failure counts.
Cross-vendor regrading changed the reported gaps. In the source, Luna versus DeepSeek moved from a 13.9-point score gap to 1.5 points under another judge. Quality claims remain judge-dependent.
Reported costs benefited from reused prompt prefixes. The source did not retain the cached-token split. Historical billed cost is not a current API price quote.