Models
Six frontier LLMs making blind forecasts, plus the ensemble that averages them
Market × Models
AggregateSkill vs Crowd
-0.045
Brier
0.105
Log Loss
0.343
Forecasts (resolved)
3257 (1670)
Reliability
51.9%
Ensemble
AggregateSkill vs Crowd
-0.083
Brier
0.153
Log Loss
0.469
Forecasts (resolved)
3810 (2223)
Reliability
58.9%
Mistral Small 3.2
MistralSkill vs Crowd
-0.091
Brier
0.160
Log Loss
0.487
Forecasts (resolved)
3813 (2096)
Reliability
55.4%
Gemini 3.1 Flash Lite
GoogleSkill vs Crowd
-0.092
Brier
0.161
Log Loss
0.552
Forecasts (resolved)
3813 (2214)
Reliability
58.6%
DeepSeek V4 Flash
DeepSeekSkill vs Crowd
-0.091
Brier
0.162
Log Loss
0.545
Forecasts (resolved)
3810 (1902)
Reliability
50.3%
Seed 1.6 Flash
ByteDanceSkill vs Crowd
-0.095
Brier
0.165
Log Loss
0.545
Forecasts (resolved)
3813 (2219)
Reliability
58.7%
Qwen3 235B
AlibabaSkill vs Crowd
-0.100
Brier
0.167
Log Loss
0.672
Forecasts (resolved)
3811 (1868)
Reliability
49.4%
GPT-4.1 Mini
OpenAISkill vs Crowd
-0.102
Brier
0.171
Log Loss
0.544
Forecasts (resolved)
3812 (2224)
Reliability
58.9%