TITANPHOEBE

How it works

PHOEBE Forecasting Platform

A structured probabilistic forecasting system for geopolitical and international affairs intelligence. Analyst estimates are aggregated, weighted by track record and confidence, and updated continuously as evidence emerges.

About PHOEBE

Probabilistic Intelligence for International Affairs

PHOEBE is a forecasting platform developed by TITAN, the analytical division of the Atlas Institute for International Affairs. It applies structured probabilistic methods to questions of geopolitical significance — producing aggregate probability estimates that make analytical uncertainty explicit and trackable over time.

Unlike narrative intelligence assessments, PHOEBE produces a number: a probability. That number is the product of multiple independent estimates — from signed analysts and anonymous contributors alike — weighted by track record where one exists, domain expertise, and stated confidence. As evidence accumulates, contributors update their estimates and the aggregate evolves.

Forecasts are probability assessments produced by TITAN analysts and anonymous contributors alike. They reflect structured judgement under uncertainty, not certainty, and are updated as new evidence emerges.

What is PHOEBE

Structured forecasting for geopolitical intelligence

PHOEBE — Probabilistic Heuristic Observatory for Evidence-Based Estimation — is a forecasting platform operated by TITAN, the analytical division of the Atlas Institute for International Affairs. Signed analyst status is by invitation, but PHOEBE also accepts anonymous submissions from anyone, so contributing an estimate is not restricted to a fixed panel.

PHOEBE is not a prediction market. There is no trading and no financial incentive. It is a structured expert aggregation system: analysts — signed and anonymous alike — submit calibrated probability estimates, those estimates are weighted by track record, confidence, and identity, and the result is a single aggregate probability that updates as the situation evolves.

Why this methodology works

Making uncertainty explicit

Traditional intelligence assessments use qualitative language — "likely", "probably", "may" — that conveys different levels of confidence to different readers. A probability forces precision: 70% means something specific that can be tested against outcomes over time, in a way "likely" never can.

Structured aggregation works because it corrects for the two failure modes that undermine both individual expert judgement and simple crowd polling. A single expert, however skilled, is prone to overconfidence and to anchoring on their own prior view. A plain unweighted average of many opinions dilutes genuine expertise with noise. PHOEBE addresses both: analysts are weighted by a demonstrated track record of calibration, not by seniority or self-reported confidence alone, so someone who has been reliably right in the past carries more influence than someone who hasn't — and that weight is earned continuously, forecast by forecast, rather than assigned once and left static.

The result degrades gracefully instead of catastrophically. An outlier estimate or a single analyst having a bad day is automatically discounted rather than allowed to swing the aggregate. A forecast with too few submissions to be reliable is flagged as sparse rather than presented with false confidence. And because every forecast is scored against its actual outcome once it resolves, the system's own accuracy is auditable — anyone can check whether PHOEBE's aggregates have genuinely tracked reality over time, a standard a qualitative assessment can never be held to.

Reading a forecast

What the number is telling you, and what it isn't

A PHOEBE probability is a statement about a class of situations, not a prediction about this one. 70% means that across many occasions where the analysts said 70%, the thing should happen about seventy times. A single 70% forecast that doesn't materialise tells you almost nothing — the 30% happened, which it should, roughly three times in ten. Judge the platform on many forecasts, never on one.

5%
Very unlikely
20%
Unlikely
35%
Possible
50%
Even odds
65%
Probable
80%
Likely
95%
Near certain

Those words are a courtesy, not a definition. Studies of intelligence language repeatedly find readers take "likely" to mean anything from 55% to 90% — which is precisely why PHOEBE publishes the number instead.

Three things matter as much as the headline figure:

What you seeWhat it meansHow to read it
The number The weighted aggregate of current estimates Treat movement as more informative than level. A forecast that has held at 60% for three weeks is a different claim from one that reached 60% yesterday from 30%.
⚠ Sparse Fewer than three signed analysts The figure is real but thin. It reflects a handful of people, not an analytical consensus, and can move sharply on one new submission.
⚠ High disagreement Spread above 0.20 Not a warning that the forecast is bad — often the most interesting state. It means informed people are looking at the same evidence and reaching different conclusions, so the aggregate is hiding a real split.
UKR-05 · YES/NO
Will a formal ceasefire be signed before end of 2026?
49%
▲ 6 pts
6 analystsUpdate 4 · closes in 9 days
⚠ HIGH DISAGREEMENT
49%The weighted aggregate. Not a prediction — a statement about how often this should happen across similar situations.
▲ 6Movement since the last window closed. Often more informative than the level: a figure that reached 49% from 20% is a different claim from one that has sat there for a month.
6How many analysts are behind it. Below three signed analysts a forecast is flagged sparse.
⚠Spread above 0.20. Worth opening the forecast to see the distribution rather than trusting the headline.

Where a forecast shows high disagreement, the distribution on the forecast page is worth more than the aggregate. Six analysts evenly spread between 20% and 80% average to the same 50% as six analysts who all said 50% — and they mean completely different things.

Scope

What PHOEBE is not

It is not a prediction market. Nobody stakes anything, so nobody is financially punished for being wrong — the discipline comes from scored track records instead, which is weaker than money but harder to game with capital.

It is not a model. There is no simulation of events underneath, no dataset being extrapolated. Every number originates in a human judgement; the platform's contribution is how it weighs and combines them.

It is not a source. A PHOEBE probability summarises what a set of analysts currently believe. It is downstream of reporting and evidence, never a substitute for either, and should be cited as an aggregate of assessments rather than as a finding.

The system

How PHOEBE produces a forecast

PHOEBE works in cycles. Coordinators open a forecasting question with explicit resolution criteria and a deadline. Analysts submit probability estimates during a defined submission window. Those estimates are aggregated, weighted, and published as a single probability. The window closes, the aggregate is locked, and the cycle repeats for the next update period.

Forecast types

Four question formats

A basic idea of each. Full detail on how each type is scored and aggregated lives in a dedicated reference.

Yes / No

Will something happen or not. You submit one probability that it does.

"Will a formal ceasefire between Russia and Ukraine be signed before end of 2025?"

Multi-scenario

Several possible outcomes. You split 100% across all of them.

"Which scenario best describes PRC strategic posture toward Taiwan over the next 12 months?"

Election

Who wins, or what the result looks like. You assign probabilities per candidate or party.

"Who will win the first round of the 2027 French presidential election?"

Numeric

A number within a range. You submit one estimated value.

"What will UK GDP growth be in Q2 2026, as reported by ONS?"

Windows and updates

The submission cycle

Each forecast is divided into update windows — typically seven days. Within a window, you can submit or revise your estimate at any time. When the window closes, the aggregate is computed and locked as a permanent record. A new window opens for the next update period.

1
Window opens
Coordinator sets the question, criteria and deadline.
›
2
Analysts submit
Each estimate is weighted as it arrives. Revise freely.
›
3
Aggregate updates
Recomputed on every submission. Live, not batched.
›
4
Window closes
The figure is locked as a permanent snapshot.
›
5
Carry forward
Non-submitters' last estimates roll over, decayed.
↶
and the next window opens — typically every 7 days until the question resolves

If you submitted to the previous window and have not yet submitted to the current one, your last estimate is carried forward rather than dropped — otherwise every window would start from nothing. Carried-forward estimates arrive discounted by age: the decay rate is set per-forecast by the coordinator, typically 0.85 per day, so an estimate loses 15% of its weight for each day since it was made. Submitting to the current window replaces the carried-forward estimate at full weight.

7 days
Window length
0.85/day
Decay rate
2.0σ
Outlier threshold
70%
Signed pool weight
30%
Anonymous weight

Within a window, timing does not matter. An estimate submitted on Day 1 counts exactly as much as one submitted on Day 7 — decay is applied when a window closes, not continuously. What decays is your previous window's estimate if you have not yet submitted to the current one: after six days it carries roughly 0.856 ≈ 37% of its weight. And only the immediately preceding window is carried forward, so missing two windows in a row removes you from the aggregate entirely.

Evidence

Coordinator-posted intelligence updates

Coordinators can post evidence items at any time during a window. Each item has a type, a directional signal, and a credibility rating — visible to everyone, and the trigger for estimate updates.

Evidence type

Diplomatic
Talks, summits, formal positions, recognitions
Military
Deployments, strikes, procurement, posture
Political
Elections, cabinets, legislation, leadership
Economic
Sanctions, trade, currency, energy
Data release
Official statistics on a scheduled date
Official statement
On-record remarks by a named principal

Directional signal

▼
Supports no
Makes the outcome less likely
▾
De-escalatory
Tension easing
▴
Escalatory
Tension rising
▲
Supports yes
Makes the outcome more likely
← Away from the outcomeToward the outcome →
◆ Mixed Cuts both ways
▬ Neutral Context, not a signal

A worked example

Military ▴ Escalatory Primary source
Additional armoured brigade confirmed redeploying to the eastern oblast border.
Satellite imagery corroborated by two independent OSINT groups. Movement began three days ago.
Posted by coordinator · 2 days agoPrior nudge +3%
TypeOne of six categories. It changes nothing mechanically — it exists so analysts can filter and weigh evidence by kind.
SignalWhich way it points, on the scale above.
SourcePrimary, secondary or unverified — visible to every analyst so they can discount accordingly.
NudgeThe coordinator's judgement of how far the aggregate ought to move. Recorded, not applied.

Coordinators can record a prior nudge of up to 5% against an evidence item, indicating how far the aggregate ought to move before analysts have had time to respond. This is currently recorded but not applied — the published figure moves only when analysts actually revise their estimates. We would rather state that than let the page imply an automatic adjustment that is not running.

Resolution

How a forecast closes

A forecast resolves when the outcome is known and verified against the resolution criteria. The coordinator logs the outcome as one of the following:

StatusMeaningTrack record weight
CleanUnambiguous outcomeFull weight
ContestedReasonable dispute about whether criteria were metReduced weight
DisputedSignificant ambiguityQuarter weight (×0.25)

On resolution, Brier scores are computed automatically for every analyst's current estimate, and feed into their running track record, which sets their weight on future forecasts. Superseded estimates are not scored — you are judged on where you finished, not on every position you held along the way. Numeric forecasts are scored for accuracy but do not move the track record: percent error and Brier score are different quantities and averaging them together would make the multiplier meaningless.

Reference

The four forecast types in depth

What each type is, how it's aggregated and scored, and how it's categorised.

Yes / no

One event. One number.

No distribution to allocate, no candidate list to weigh — just a single, precise judgement call between two outcomes.

"Will a formal ceasefire between Russia and Ukraine be signed before end of 2025?"

0 — no100 — yes
The closer your number lands to the eventual outcome, the lower — better — your Brier score.
Categories

No formal sub-type — every Yes/No question is scored identically, using the same weighted aggregation and the same Brier calculation regardless of how it's framed. What varies is only the shape of the trigger that decides when the forecast resolves.

Fixed deadline
Resolves against a set date — the window is finite, so decay pressure builds toward a known close.

"...before end of 2025?"

Open trigger
Resolves whenever the condition is met, no set date — the forecast can stay open indefinitely.

"...once talks resume, if ever?"

Why binary framing specifically

Judgment research consistently finds people are best calibrated when asked for one probability on one clear proposition — accuracy degrades as the number of things being weighed at once goes up. Across a hundred calls where you said "70%," roughly seventy should come true — a testable claim "probably" never gives you.

Multi-scenario

Several outcomes. Split 100% across all of them.

Three to six scenarios, defined by the coordinator when the forecast opens. Every analyst allocates against the same fixed list — useful whenever a question has genuinely distinct branches rather than a clean yes-or-no split, and no single branch is obviously the whole story.

"Which scenario best describes PRC strategic posture toward Taiwan over the next 12 months?"

Demo: mutually exclusive — always sums to 100%
One shared distribution — each scenario aggregated across analysts, then scored with the multi-category Brier score.
Demo: independent outcomes — each stands alone
Nothing here has to sum to anything — each gets its own Brier score.
Categories

A real, stored setting — every multi-scenario forecast is set up as one of two subtypes, chosen when the coordinator builds it. This isn't cosmetic: it changes which aggregation model runs underneath, and whether your allocation is required to sum to 100% or not.

Mutually exclusive
Exactly one will happen — probabilities sum to 100%. Choose this when the scenarios genuinely can't overlap.

"Which of these four outcomes occurs?"

Independent outcomes
Any, all, or none could happen — each resolves on its own. Choose this when the scenarios could genuinely coexist.

"Which of these three, if any, occur?"

Why decomposition specifically

Splitting a question into its real branches forces you to weigh them against each other explicitly, rather than reasoning about one scenario in isolation and letting the rest sit as an unexamined "everything else." A forced allocation surfaces disagreements a single probability would hide.

Election

Candidates or parties. Real electoral rules.

Structurally a scenario list, like multi-scenario — but built around how elections actually resolve: winners, seat shares, and sometimes multiple rounds. Pick this over Yes/No the moment there's more than one candidate genuinely in play; pick Yes/No instead for a referendum or a simple "does the incumbent keep power" question.

First-past-the-post
Whichever color fills the most squares wins outright.
Categories

The electoral system is stored on the forecast and set once — it shapes what "the outcome" even means, from a single named winner to a full seat distribution across several parties.

FPTP
Most votes wins, single round. Simple, fast, but can hand victory to a candidate most voters didn't choose.

"Will Labour hold the seat of Bury North?"

Two-round
A runoff between the top two if no majority in round one, forcing a genuine head-to-head.

"Who wins the French presidency?"

AV
Ranked choice — last place eliminated and redistributed until a majority forms.

"Does the Greens candidate win Melbourne?"

Proportional
Seats allocated to parties by vote share, not one winner — the legislature mirrors the electorate.

"What's the Bundestag seat split?"

STV
Multi-seat proportional system using ranked preferences — like AV, but electing several winners per constituency instead of one.

"Does Sinn Féin take 2 of Dublin Bay South's 4 seats?"

Why the system matters

The same vote share can mean total victory under one system and a hung parliament under another. Forecasting an election means forecasting the system's arithmetic as much as the voters' choice.

Numeric

A number, not a probability. Your single best guess.

For questions with a continuous, measurable answer — a growth rate, a price level, a count — rather than a discrete set of outcomes. There's no hedging with a distribution here: you commit to one number, and you're judged on how close it lands.

"What will UK GDP growth be in Q2 2026, as reported by ONS?"

Count
A whole number of things — each icon is one unit.
Categories

A real, stored setting — every numeric forecast is one of three subtypes, chosen when it's set up. It controls how your estimate is rounded before scoring, so a whole-number question can't accidentally accept a fractional guess.

#
Count
A whole number of things — ships sunk, incidents, seats.

"How many ships does Iran lose in 2026?"

Duration
Time until something happens — months until a ceasefire.

"Months until a ceasefire holds?"

Continuous
A fractional value — GDP growth, inflation.

"What's UK inflation in Q3 2026?"

Why a point estimate specifically

A range feels safer, but it lets you hedge without ever being wrong. One committed number is the only way to genuinely test whether your model of the world is calibrated on a continuous quantity, the same way one probability tests it on a binary one.

Anyone can forecast

Submitting your first estimate

No account needed. Here's exactly what happens from opening a question to your estimate counting toward the aggregate.

Step by step
1
Submit an estimate

The form matches the question's type — one probability for Yes/No, a percentage split for multi-scenario, a candidate list for elections, a single number for anything continuous.

i
Read resolution criteria and evidence (optional)

Every question states exactly what has to happen for it to resolve Yes, and coordinators post evidence as the situation develops. Neither is required — but skipping them means guessing at a question someone already answered precisely.

2
Set your confidence

A five-point rating of how firmly you'd hold this view against new information — not how likely the event is. It genuinely changes your estimate's weight; see the arc below.

3
Your estimate joins the aggregate

There's no draft state and no confirmation step — the moment you submit, it's folded into the published probability. You can revise as often as you like before the window closes; each submission supersedes your previous one, so you always count exactly once no matter how many times you change your mind.

Rating your confidence

Alongside your estimate, you rate how confident you are on a five-point scale. This isn't decorative: it changes how much your estimate is weighted in the aggregate.

Sweeps on its own — click a level to lock in, click it again to release.

Ready to contribute?

Submissions are open to all — no account required. Create an account to track your history and see your calibration over time.

Submit an estimate → Create account

Deeper guidance

Five habits worth building

Read resolution criteria carefully

Ambiguous criteria are the most common source of contested resolutions — and of forecasting the wrong thing entirely.

Form a view before the aggregate

Seeing the crowd estimate first anchors your judgement — analysts who look early tend to cluster instead of adding signal.

Submit a precise number

60% and 65% aren't the same. Round numbers often reflect uncertainty — put that in your confidence rating instead.

Update on signal, not schedule

A 1–2 point move is noise; 10+ is signal. There is no penalty for leaving an estimate alone within a window — timing inside a window does not affect its weight. What costs you is missing a window entirely: your previous estimate carries over at a decayed weight, and after two consecutive misses it drops out of the aggregate altogether.

Track your calibration over time

Well-calibrated means right about 70% of the time when you say 70%. Insights shows your curve once enough forecasts resolve — above the diagonal is overconfident, below is underconfident.

What signed analysts get

A dashboard that tracks what you're actually good at

Every resolved forecast you've signed feeds three real, growing things.

45 achievements to earn
Well-calibrated
Century club
Dewey defeats Truman
Suspiciously lucky

Real ones, computed from your actual history — not decoration. Reach 10 and you're a Bronze Rank analyst; 40 and you're Platinum.

Where you're actually sharp

A live radar of your Brier score across every region and topic you've forecast in — so you find out whether you're actually sharper on Europe or on economics, instead of guessing.

A record that carries weight

Your Brier score doesn't just sit on a profile — it directly multiplies how much your future estimates count in the aggregate. Good calibration compounds; every signed analyst starts equal, but not every analyst stays that way.

Start building yours

Every resolved forecast from your first signed submission onward counts toward all of this.

Sign up Sign in

Technical specification

How estimates become a probability

PHOEBE averages its analysts — but not equally. Every submission carries a weight built from who made it, how confident they were, how accurate they have been before, and how far it sits from everyone else. The published figure is the weighted mean of those estimates. The sections that follow document exactly how each weight is derived, because that is where all the substance is.

The pipeline has three stages. Each submission is assigned a weight, the product of five factors: the analyst's base weight, their track record in that domain, their stated confidence, their pool, and a discount if the estimate is a statistical outlier. Estimates carried over from a closed window decay by age, so an analyst who stops participating fades out rather than counting forever. Finally the weighted estimates are aggregated — normalised within each pool, recombined at the pool ratio, and published, with the spread checked against the disagreement threshold.

Once a forecast resolves, the same machinery runs in reverse: every submission is scored against the verified outcome — by Brier score for probabilistic forecasts, by percent error for numeric ones — and those scores feed back into the track record multiplier that weighted the analyst in the first place. The model is designed to be fully transparent. Every parameter that enters the aggregate is documented below, and none of them are hidden from analysts.

What is running today. All five weight factors are computed and applied, along with temporal decay and the scoring loop that feeds accuracy back into an analyst's weight. Section 2 documents each factor individually, including where a deliberate limit applies — numeric forecasts are scored for accuracy but do not move the track record multiplier, for reasons set out there.

1. Aggregation model

The constants, and the arithmetic they produce

Model parameters

The constants the model runs on

Every constant the model uses, grouped by what it governs. The bar beneath each shows where the default sits inside its permitted range.

Who counts, and how much
Signed base weight
1.0
Every signed analyst starts here
Anonymous base
0.5
Half of a signed submission
Track record
0.5–2×
Earned on resolved forecasts. New analysts sit at 1.0
Confidence
0.7–1.3×
From the analyst's own 1–5 rating
Pools and ageing
Signed pool
70%
Share of the result when both pools are active
Anonymous pool
30%
Hard ceiling, however many arrive
Decay rate
λ 0.85
Per day, on carried-forward estimates only
Decay floor
0.01
Discounted heavily, never erased
Guardrails
Outlier threshold
2.0σ
Distance at which discounting begins
Max outlier discount
50%
The hard floor on any single estimate
Disagreement flag
σ > 0.20
Above this the forecast is flagged

Worked example

Three analysts, one forecast

Two signed analysts and one anonymous contributor submit on a Yes/No forecast. Their weights differ by confidence, track record and pool.

Raw weights
Analyst A signed, confidence 5, track record 1.4 estimate 0.70 w = 1.0 × 1.4 × 1.30 × 0.7 = 1.274 Analyst B signed, confidence 3, no track record estimate 0.55 w = 1.0 × 1.0 × 1.00 × 0.7 = 0.700 Analyst C anonymous, confidence 4, no history estimate 0.20 w = 0.5 × 1.0 × 1.15 × 0.3 = 0.173

Pools are normalised separately. The signed pool (A and B) is scaled to hold 70% of the result; the anonymous pool (C alone) holds 30%:

Effective weights, after pool capping
Signed pool total = 1.274 + 0.700 = 1.974 A: 1.274 / 1.974 × 0.7 = 0.4518 B: 0.700 / 1.974 × 0.7 = 0.2482 Anonymous pool total = 0.173 C: 0.173 / 0.173 × 0.3 = 0.3000 aggregate = 0.4518×0.70 + 0.2482×0.55 + 0.3000×0.20 = 0.3163 + 0.1365 + 0.0600 = 0.5128 → published as 51%

Note what the cap does. Analyst C carried a raw weight of 0.173 against a signed total of 1.974 — barely 8% of the arithmetic. Because they are the only anonymous submission, the pool normalisation lifts them to a full 30% of the published figure, pulling it down from 65% to 51%. A single anonymous contributor on a thinly-covered forecast has real influence; a hundred of them would still have exactly 30% between them.

2. Five-factor weight pipeline

How each submission weight is computed

Every submission enters the aggregate with a weight wᵢ computed as the product of five factors:

Composite weight
wᵢ = w_base × m_track × m_conf × m_pool × m_outlier

Implementation status. All five factors are computed and applied. Base weight and the confidence multiplier are set when a submission is recorded; the track record multiplier updates whenever a forecast the analyst worked on resolves; pool capping and the outlier discount are applied at aggregation time, since both depend on the distribution of submissions rather than any single one. Each factor below is marked accordingly. We publish this status rather than describe the design as though every part were self-evidently live, because an analyst deciding how much to trust an aggregate deserves to know what is actually behind it.

Base
weight
1.0
0.5 anon
×
Track
record
1.4
0.5 – 2.0
×
Confidence
1.30
0.7 – 1.3
×
Pool
0.7
0.7 / 0.3
×
Outlier
1.0
0.5 – 1.0
=
Final weight
1.274
this submission

A signed analyst with a strong record, submitting at maximum confidence. Every factor is multiplicative, so the spread compounds: the same analyst at the floor of every range would carry 0.245, under a fifth of the influence, and that is before any outlier discount.

Each factor is described below.

Factor 1 — Base weightLive

All signed-in analysts begin with w_base = 1.0. Anonymous submissions begin with w_base = 0.5. The base weight reflects the irreducible difference in accountability between identified and anonymous submissions.

Factor 2 — Track record multiplier (m_track)Live

Derived from the analyst's historical Brier score on resolved forecasts in the same domain. The multiplier is computed relative to the platform average:

m_track = clip(1 + (B_avg − Bᵢ) / B_avg × k_scale, 0.5, 2.0)

where Bᵢ is the analyst's domain Brier score, B_avg is the platform average, and k_scale = 1.5. New analysts with no track record receive m_track = 1.0.

Scoring runs automatically the moment a forecast is resolved, and the resulting multiplier feeds into the analyst's next submission. Yes/No, multi-scenario and election forecasts all contribute. Analysts with no resolved forecasts sit at 1.0 — no record is treated as no evidence, not as a poor record.

Multi-scenario and election forecasts are scored with the multi-category Brier score, halved so that it agrees exactly with the Yes/No formula, then normalised by the number of options. That normalisation matters: an uninformative forecast on a two-way question scores 0.25, but the same non-answer spread evenly across six options scores 0.417. Without correcting for this, analysts working on complex questions would be penalised for the complexity rather than for being wrong. After normalisation, zero information is worth a multiplier of exactly 1.0 whatever the number of options.

Numeric forecasts are scored but do not affect weight. Percent error is recorded against each numeric submission and shown on the analyst's record, but it is not folded into the track record multiplier. A 28% error and a Brier score of 0.28 are not the same quantity, and averaging them together would make the multiplier meaningless.

Factor 3 — Confidence multiplier (m_conf)Live

Derived from the analyst's stated confidence rating c ∈ {1, 2, 3, 4, 5}:

m_conf = 0.7 + 0.15 × (c − 1) → range [0.7, 1.3]

A confidence of 1 reduces weight to 70% of base; a confidence of 5 increases it to 130%. This rewards analysts who are willing to commit strongly to high-quality estimates.

Factor 4 — Pool weight (m_pool)Live

Signed-in and anonymous submissions are aggregated separately into two pools, then combined with a coordinator-set weighting. The default is 70% signed pool / 30% anonymous pool. m_pool reflects the submission's share within its own pool and the pool's weight in the final aggregate.

Signed pool  70%
Anonymous  30%
However many signed analysts submitHowever many anonymous

The cap is enforced at aggregation time, not per submission. Each pool's weights are normalised to sum to that pool's share, so the anonymous pool cannot exceed 30% of the published figure however many anonymous submissions arrive. One signed analyst at 0.90 against nine anonymous submissions at 0.10 aggregates to 0.66, not 0.18. Where a forecast has submissions in only one pool, that pool takes the whole aggregate — capping an empty signed pool's counterpart at 30% would mean publishing 30% of a number.

Factor 5 — Outlier discount (m_outlier)Live

Estimates that deviate from the current aggregate by more than the coordinator-set outlier threshold (default 2.0σ) receive a discount:

m_outlier = 1.0 − clip((|p̂ᵢ − μ| − θ·σ) / σ, 0, 0.5)
THRESHOLD 2σ 100% 50% DISTANCE FROM AGGREGATE

where μ is the current aggregate, σ is the standard deviation across submissions, and θ is the threshold. An estimate exactly at the threshold receives full weight; an estimate 3σ away receives 50% weight. Extreme outliers are not excluded — they are discounted.

Because the discount compares an estimate against an aggregate that does not exist until the estimate is recorded, it runs in three passes at aggregation time: compute the mean and σ, derive each submission's discount against them, then re-weight and recompute. The resulting discount and flag are written back onto the submission itself, so a coordinator can see which estimates were discounted and by how much rather than only that the published figure moved.

3. Temporal decay

How estimates carried across windows lose weight

When a window closes and a new one opens, analysts who submitted to the previous window but have not yet submitted to the new one still have their last estimate counted — otherwise every window would start from an empty aggregate. Those carried-forward estimates are discounted according to how long ago they were made:

Decay weight at time t days after submission
w_decay(t) = λᵗ where λ ∈ (0, 1) is the decay rate
Default λ = 0.85 per day: Day 0 (submission): w_decay = 1.000 Day 1: w_decay = 0.850 Day 3: w_decay = 0.614 Day 5: w_decay = 0.444 Day 7: w_decay = 0.321
32% 100% 0 DAYS SINCE SUBMISSION

Coordinators set this per forecast. A fast-moving military situation warrants aggressive decay; a question about 2027 GDP does not.

The decay weight replaces the composite weight wᵢ for a carried-forward estimate. Coordinators may set a different decay rate per-forecast and per-window, ranging from 0.50 (aggressive decay) to 0.99 (minimal decay). A floor of 0.01 prevents estimates from reaching zero weight.

An estimate submitted to the current window is not decayed, however old it is within that window — decay is applied at the window boundary, not continuously. Only the immediately preceding window is carried forward; an analyst who has missed two consecutive windows drops out of the aggregate entirely rather than being carried at a very small weight.

Practical implication: submitting to the current window replaces your carried-forward estimate and restores it to full weight. An analyst who stops participating sees their influence fall across one window and then disappear at the next, so the aggregate reflects the analysts who are currently engaged with the evidence rather than everyone who has ever looked at the question.

4. Disagreement detection

When analysts diverge significantly

The standard deviation σ of the weighted submission distribution is computed at each aggregate update. When σ exceeds the high-disagreement threshold (default 0.20), the forecast is flagged and a coordinator alert is generated:

Standard deviation of weighted submissions (Yes/No)
σ = √[ Σᵢ wᵢ (p̂ᵢ − μ)² / Σᵢ wᵢ ] High disagreement flag: σ > 0.20
Consensusσ = 0.04
AGGREGATE 50% 0% 100%
Everyone near 50%. The aggregate means what it says.
Diffuseσ = 0.19
AGGREGATE 50% 0% 100%
Nobody is sure. The aggregate is a shrug, honestly reported.
Splitσ = 0.30
AGGREGATE 50% 0% 100%
Two camps, no middle. The aggregate describes a view nobody holds.

All three publish 50%. Only the third is telling you something a single number cannot: that informed people are looking at the same evidence and reaching opposite conclusions. This is also the honest limit of σ — it separates the first case from the other two, but not the second from the third, and the third is the analytically interesting one. Where disagreement is flagged, the distribution on the forecast page is worth more than the headline figure.

The flag does not alter the aggregate — it signals that the probability figure should be interpreted with greater caution and that analyst divergence warrants examination. Common causes: different interpretations of the resolution criteria, asymmetric access to evidence, or genuine uncertainty about a bifurcated situation.

The published flag is binary at σ > 0.20. Coordinators additionally see a three-band breakdown in the admin console (low below 0.10, medium 0.10–0.20, high above 0.20); those bands are an internal review aid and do not appear on any public forecast.

Numeric forecasts are excluded from the disagreement flag. Their spread is measured in the units of the quantity being forecast — dollars, seats, percentage points — so it cannot be compared against a threshold defined on a probability scale. A σ of 5 is tight for a forecast of the S&P 500 and enormous for one about a central bank rate. Numeric forecasts therefore carry no high-disagreement badge, and the spread is presented directly on the forecast page instead.

On the estimator. The denominator is Σwᵢ − Σwᵢ²/Σwᵢ, not Σwᵢ. Dividing by Σwᵢ gives the population variance, which is biased downward as an estimate of the underlying dispersion — by roughly a third at three submissions. Since forecasts routinely run with few analysts, that would understate disagreement precisely where the aggregate is least reliable. A window with a single submission returns no σ at all rather than zero: one estimate carries no information about spread, and reporting zero would read as unanimity.

A sparse window flag is raised when fewer than three signed analysts have submitted. Anonymous submissions do not count toward clearing this flag, since the anonymous pool contributes at most 30% of the aggregate. Aggregates from sparse windows carry reduced epistemic weight and are not used in track-record calculations.

5. Brier scoring and calibration

Measuring forecast accuracy

PHOEBE uses the Brier score as its primary accuracy metric. The Brier score is a proper scoring rule — it is maximised in expectation only when the forecaster reports their true subjective probability. This means there is no strategic benefit to misreporting.

For a Yes/No forecast, the Brier score for a single estimate is:

Brier score — Yes/No forecast
BS = (f − o)²
where: f = stated probability (0 to 1) o = outcome (1 = Yes occurred, 0 = No occurred) Range: 0 (perfect) to 1 (worst possible for a single binary forecast) Example: Stated 70%, event occurred → BS = (0.70 − 1)² = 0.09 Example: Stated 70%, event did not occur → BS = (0.70 − 0)² = 0.49

For multi-scenario forecasts with K outcomes, the Brier score sums squared errors across all scenarios:

Brier score — multi-scenario forecast
BS_raw = Σₖ (fₖ − oₖ)² / 2 halved, so it matches the Yes/No scale BS = BS_raw × 0.25 / baseline normalised for the number of options
where oₖ = 1 for the outcome that occurred, 0 for all others, and baseline = (K−1) / 2K is what an uninformative K-way answer scores. Example (3 scenarios, A occurred, stated A=60%, B=30%, C=10%): BS_raw = [(0.60−1)² + 0.30² + 0.10²] / 2 = 0.26 / 2 = 0.13 baseline = (3−1) / 6 = 0.3333 BS = 0.13 × 0.25 / 0.3333 = 0.0975

Two adjustments matter here. Halving makes the multi-scenario score agree exactly with the Yes/No formula, so scores from different forecast types can be averaged together into one track record. Normalising corrects for the number of options: an uninformative answer scores 0.25 on a two-way question but 0.417 on a six-way one, so without this an analyst working on complex questions would be penalised for the complexity rather than for being wrong. After normalisation, no-information is worth exactly 0.25 whatever the number of options.

What a score feels like

Perfect
Unattainable in practice
Superforecaster
Metaculus ~0.11 · GJP ~0.15
Coin flip
Saying 50% to everything
Worse than none
Confidently wrong
Lower is betterHigher is worse

The useful range is narrower than the formula suggests. Everything from 0.25 upward is worse than answering 50% to every question, so the entire contest happens in the left-hand third of that bar. Moving from 0.20 to 0.15 is a far larger achievement than the numbers make it look.

Calibration is assessed by grouping forecasts by stated probability bucket (0–10%, 10–20%, … 90–100%) and comparing the stated probability to the observed resolution rate within each bucket. A perfectly calibrated analyst has a linear calibration curve through the origin at slope 1.

PERFECT OVERCONFIDENT UNDERCONFIDENT YOU SAID IT HAPPENED

Calibration is fixable in a way raw accuracy is not. An overconfident analyst who learns to shade their 90s down to 75s improves immediately, without learning anything new about the world.

Brier score (Yes/No)InterpretationBenchmark
0.00Perfect — stated probability matches outcome with certainty every timeTheoretical maximum
0.10–0.15Excellent calibration — top-tier superforecaster levelGJP superforecaster aggregate: ~0.15
0.15–0.20Good — better than most expert forecastersMetaculus community: ~0.11
0.20–0.25Average — comparable to informed non-specialistGJP regular forecaster: ~0.26
0.25 and aboveBelow average — systematic calibration issues likelyRandom guessing: 0.25 on binary forecasts

6. Numeric forecasts

Scoring a continuous estimate

verified outcome
−25%−10%exact+10%+25%

Numeric forecasts do not resolve to a binary outcome, so the Brier score does not apply. Instead PHOEBE scores a numeric estimate by its percent error against the verified outcome value — how far your submitted number was from what actually happened, as a proportion of that outcome:

Percent error — Numeric forecast
PE = |f − o| / |o|
where: f = your submitted value o = the verified outcome value (o ≠ 0) Example: Stated 3.2% GDP growth, actual outcome 3.0% → PE = |3.2 − 3.0| / |3.0| ≈ 6.7% Example: Stated 3.2% GDP growth, actual outcome 2.5% → PE = |3.2 − 2.5| / |2.5| = 28.0%

Lower is better, with 0% meaning your estimate matched the outcome exactly. There is no fixed upper bound — an estimate off by more than the outcome's own magnitude can exceed 100% error.

Percent errorInterpretation
Under 5%Excellent — the estimate is very close to the verified outcome
5–15%Good — a reasonable margin for most continuous quantities
15–30%Average — meaningful miss, worth reviewing your assumptions
Above 30%Poor — the estimate and outcome diverge substantially

One practical consequence of the formula: outcomes near zero produce very large or unstable percent errors even for a numerically small miss. Coordinators should account for this when setting resolution criteria on numeric forecasts where the outcome could plausibly land near zero — a fixed absolute-error criterion may be more appropriate than percent error in those cases.

7. Anonymous submissions

How unidentified estimates are handled

PHOEBE permits anonymous submissions to allow participation without account creation. Anonymous estimates are segregated into a separate pool that contributes a maximum of 30% of the final aggregate weight by default. This prevents anonymous submissions from dominating the aggregate while still incorporating potentially useful signal.

Anonymous submissions receive base weight 0.5 and are subject to the same outlier discounting as signed submissions. They are scored when a forecast resolves — the score attaches to the browser session that made the submission, which accumulates a running Brier average and earns a session weight multiplier:

Anonymous session multiplier
m_session = clip(1.2 − 0.8 × B̄_session, 0.3, 1.2) B̄ = 0.00 → 1.20 best an anonymous session can earn B̄ = 0.25 → 1.00 no better than answering 50% every time B̄ = 0.50 → 0.80 B̄ = 1.00 → 0.40

The ceiling is 1.2 against 2.0 for a signed analyst: an anonymous contributor with a good record earns real additional weight, but never as much as someone accountable for their record under a name. The session record lives in the browser and is lost if the person clears their storage — it is not an identity, and it cannot be transferred to an account later, since there is no way to prove after the fact which anonymous estimates were whose. Coordinators can set the anonymous pool weight to zero for forecasts where identity accountability is required.

Frequently asked questions

Common questions about PHOEBE

Questions about how PHOEBE works, how to interpret its forecasts, and how analyst access is managed.

The basics

What does a PHOEBE probability actually mean? ▼
A probability of 65% means that, based on current analyst assessments, the event is estimated to occur in approximately 65 out of 100 similar situations. It does not mean it will definitely happen. Probabilities are not predictions of single outcomes — they are calibrated statements of uncertainty. A 65% forecast that does not materialise is not necessarily a bad forecast; it means the 35% scenario occurred. Accuracy is assessed over many forecasts, not individual outcomes.
What is a Brier score and why does it matter? ▼
The Brier score is the squared difference between your stated probability and the outcome (0 or 1). Lower is better. It is a proper scoring rule, meaning the only way to maximise your expected score over time is to state your genuine probability — there is no strategic benefit to over- or under-stating confidence. A random forecaster achieves approximately 0.25 on binary questions. Well-calibrated analysts typically score 0.10–0.20. Your Brier score is used to compute your track record multiplier, which determines how heavily your estimates are weighted in the aggregate.
How are numeric forecasts scored? ▼
By percent error — how far your estimate was from the verified outcome, as a proportion of that outcome. Numeric forecasts don't have a binary outcome to score against, so the Brier score doesn't apply here; see the Methodology tab for the full formula and interpretation thresholds.
How are election forecasts displayed? ▼
It depends on the electoral system. Single-winner systems — FPTP, two-round, and AV — display each candidate with their aggregate probability of winning. Proportional and STV display parties with their projected seat share instead. The probability shown on the forecast card is the leading candidate or party's aggregate; full breakdowns and per-system detail are covered in the Forecast type tab.
What's the difference between mutually exclusive and independent multi-scenario forecasts? ▼
It's a real, stored setting chosen when the forecast is built. Mutually exclusive means exactly one scenario will happen, so your allocation must sum to 100%, and it's scored with the multi-category Brier score — squared error across every scenario, normalised so that a no-information answer scores the same whether the question has two options or six. Independent means any, all, or none could happen, and each scenario stands alone. Note that scoring currently treats every multi-scenario forecast as mutually exclusive; independent subtypes are aggregated correctly but not yet scored separately.

Participating

How do I request analyst access? ▼
Signed analyst status is by invitation from TITAN coordinators, since it carries a persistent track record. You don't need an invitation to participate, though — you can submit estimates anonymously on any open forecast without an account. If you are affiliated with Atlas Institute or TITAN and believe you should have signed analyst access, contact your programme coordinator.
Can I submit without creating an account? ▼
Yes. Anonymous submissions carry half the base weight of signed ones and the whole anonymous pool is capped at 30% of the published figure, however many anonymous estimates arrive. They are scored when a forecast resolves: your browser session accumulates a Brier record and earns a weight multiplier between 0.3 and 1.2, so an anonymous contributor who forecasts well counts for more over time. What you don't get is a persistent identity — the record lives in that browser session, disappears if you clear it, and carries no achievements, no domain specialisation and no dashboard. A signed account gets all of those plus a higher weight ceiling.
If I sign up after submitting anonymously, do those estimates count toward my track record? ▼
No. Anonymous scores attach to the browser session that made them, not to you, and there is no way to prove after the fact which anonymous estimates were yours — that's what makes them anonymous. Your signed track record starts from your first signed submission. If building one matters, sign in before you submit rather than after.
What are achievements and ranks? ▼
Signed analysts unlock achievements — real milestones computed from your actual resolved forecast history, not decoration — covering things like calibration streaks, domain specialisation, and timing. Earning enough of them promotes you through four ranks, from Bronze at 10 achievements to Platinum at 40. Anonymous submissions don't accumulate achievements, since they carry no persistent history to track.

Access and privacy

What does PHOEBE store about me? ▼
If you submit anonymously, a random session token is stored in your browser and against your submission. It is not linked to an email, a name, or an IP address, and it exists for one reason: so that revising an estimate replaces your earlier one instead of counting twice, and so your session can accumulate an accuracy record. Clearing your browser storage destroys it permanently and there is no way to recover it — including for us.

If you have a signed account, we hold your email address, display name, submissions with timestamps and confidence ratings, and your computed scores. Coordinators can see all of that. Other analysts never see individual submissions of any kind — only the aggregate is published.

There is no advertising, no third-party analytics on submission data, and nothing is sold.
Does it cost anything? ▼
No. Reading forecasts and submitting estimates are both free, with or without an account. PHOEBE is operated by TITAN as analytical infrastructure rather than as a product — there is nothing to buy and no paid tier.
Who decides which questions get asked? ▼
TITAN coordinators. They write the question, the resolution criteria, and the deadline, and they judge the outcome when it resolves. That is real editorial power and worth being aware of when reading any forecast: the framing of a question shapes the answer, and a question nobody thought to ask produces no forecast at all. Coordinators cannot alter submitted estimates or override a published aggregate.

Trust and mechanics

What happens if I think the resolution criteria are wrong? ▼
Raise it with your coordinator immediately — before submitting. Forecasts should only be attempted once you understand and accept the resolution criteria as written. Disputed resolutions (where analysts believe the criteria were applied incorrectly) carry reduced weight in track record calculations, but do not alter the published outcome. Coordinators have final authority on resolution.
How does the aggregate change when I update my estimate? ▼
Immediately. Each update triggers a real-time recomputation of the aggregate. Your new estimate supersedes the previous one — that one is marked superseded and kept as revision history, but stops counting toward the aggregate, so you are never double-counted no matter how often you revise. Your contribution stays at full weight for the rest of the window; decay only applies if the window closes and you carry the estimate into the next one without updating.
What does "high disagreement" mean on a forecast card? ▼
It means the weighted standard deviation of submitted estimates is above 0.20. As a rough feel, that is contributors spread across something like 40 percentage points rather than clustered. Numeric forecasts carry no disagreement flag — their spread is in the units of the thing being forecast, so it can't be compared against a threshold set on a probability scale. This does not mean the forecast is wrong or unreliable. It means there is genuine uncertainty or conflicting information among those who submitted. High disagreement forecasts should be interpreted with more caution than those where contributors are closely aligned.
Can coordinators see my individual submission? ▼
For signed submissions, yes — coordinators have access to the full submissions table, including individual analyst identities, timestamps, and confidence ratings. This is necessary for dispute resolution, quality monitoring, and track record computation. Anonymous submissions carry no identity to reveal in the first place. Either way, individual submissions are never shared with other contributors — only the aggregate is visible on the platform.
What is the difference between a "sparse" and "healthy" forecast? ▼
A sparse forecast has fewer than three signed analyst submissions in the current window. The aggregate is still computed and published, but carries reduced epistemic weight — it reflects the views of very few analysts and should not be treated as representative of broader analytical consensus. A healthy forecast has adequate participation, low disagreement, and recent updates from multiple analysts.

Need more information? For any questions not covered above, contact us at info@atlasinstitute.org.