Models
Six frontier LLMs making blind forecasts, plus the ensemble that averages them
Market × Models
AggregateSkill vs Crowd
-0.021
Brier
0.075
Log Loss
0.274
Forecasts (resolved)
1061 (854)
Reliability
100.0%
Ensemble
AggregateSkill vs Crowd
-0.079
Brier
0.150
Log Loss
0.460
Forecasts (resolved)
1614 (1407)
Reliability
100.0%
DeepSeek V4 Flash
DeepSeekSkill vs Crowd
-0.085
Brier
0.159
Log Loss
0.541
Forecasts (resolved)
1614 (1265)
Reliability
88.9%
Mistral Small 3.2
MistralSkill vs Crowd
-0.091
Brier
0.162
Log Loss
0.486
Forecasts (resolved)
1617 (1400)
Reliability
98.0%
Gemini 3.1 Flash Lite
GoogleSkill vs Crowd
-0.092
Brier
0.162
Log Loss
0.570
Forecasts (resolved)
1617 (1401)
Reliability
99.4%
Seed 1.6 Flash
ByteDanceSkill vs Crowd
-0.092
Brier
0.163
Log Loss
0.561
Forecasts (resolved)
1617 (1408)
Reliability
99.9%
Qwen3 235B
AlibabaSkill vs Crowd
-0.098
Brier
0.166
Log Loss
0.745
Forecasts (resolved)
1615 (1224)
Reliability
85.5%
GPT-4.1 Mini
OpenAISkill vs Crowd
-0.099
Brier
0.170
Log Loss
0.542
Forecasts (resolved)
1616 (1409)
Reliability
99.9%