Models
Six frontier LLMs making blind forecasts, plus the ensemble that averages them
Market × Models
AggregateSkill vs Crowd
-0.047
Brier
0.107
Log Loss
0.347
Forecasts (resolved)
2165 (1614)
Reliability
78.1%
Ensemble
AggregateSkill vs Crowd
-0.086
Brier
0.156
Log Loss
0.476
Forecasts (resolved)
2718 (2167)
Reliability
82.6%
Mistral Small 3.2
MistralSkill vs Crowd
-0.093
Brier
0.162
Log Loss
0.493
Forecasts (resolved)
2721 (2055)
Reliability
77.7%
Gemini 3.1 Flash Lite
GoogleSkill vs Crowd
-0.095
Brier
0.164
Log Loss
0.562
Forecasts (resolved)
2721 (2158)
Reliability
82.1%
DeepSeek V4 Flash
DeepSeekSkill vs Crowd
-0.094
Brier
0.164
Log Loss
0.552
Forecasts (resolved)
2718 (1863)
Reliability
70.6%
Seed 1.6 Flash
ByteDanceSkill vs Crowd
-0.098
Brier
0.168
Log Loss
0.553
Forecasts (resolved)
2721 (2164)
Reliability
82.3%
Qwen3 235B
AlibabaSkill vs Crowd
-0.103
Brier
0.169
Log Loss
0.682
Forecasts (resolved)
2719 (1824)
Reliability
69.3%
GPT-4.1 Mini
OpenAISkill vs Crowd
-0.104
Brier
0.174
Log Loss
0.552
Forecasts (resolved)
2720 (2168)
Reliability
82.5%