Scoring Methodology
Our scoring system uses three complementary metrics to evaluate prediction performance, borrowed from meteorology and decision science.
Accuracy Score
The percentage of your predictions that were correct. Simple and intuitive.
Accuracy = (Correct Predictions / Total Resolved Predictions) × 100%
How it works:
- ✓Correct: Full point (1.0)
- ~Partially Correct: Half point (0.5)
- ✗Incorrect: No points (0.0)
- ?Unfalsifiable: Excluded from calculation
Example:
If you made 10 predictions: 6 correct, 2 partially correct, 2 incorrect:
Score = (6 × 1.0 + 2 × 0.5 + 2 × 0.0) / 10 = 70%
Domain-Specific Scores
All three metrics are calculated separately for each AI domain, allowing you to identify your areas of expertise:
Shadow Score
For the annual Shadow Report, domains receive one of three shadow scores based on collective prediction accuracy and sensing gap incidents:
Deep Shadow
Major setbacks, failed promises, or significant safety incidents. AI winter continues.
Partial Shadow
Mixed results with some progress but notable gaps between hype and reality.
No Shadow
Genuine breakthroughs validated. Spring is here for this domain!
Remember: Good calibration beats overconfidence
The goal isn't to be right all the time, but to accurately express your uncertainty and learn from the outcomes.