How the skill metrics work

What this page is for. Every trader profile, board and Builder ranking is built from the same catalog of 84 metrics. This page explains the handful that carry the product — the ones in the verdict at the top of a profile — and the rules every metric follows: which way is better, how percentiles are formed, and when a number is too thin to show. One-line definitions of the rest are in the Glossary; the machine-readable catalog (id, meaning, direction, minimum sample, windows) is GET /v1/metrics/catalog.

Three questions that sound alike

A trader's page answers three different questions about the same settled positions. They are shown side by side because they disagree often, and the disagreement is the point.

Win rate and prediction accuracy are not adjusted for the price paid: selling 99¢ favourites gives a 99 % win rate and no edge, while a long-shot specialist can win one in five and still be very profitable. Edge vs the odds subtracts what the entry price already implied, so buying an 80¢ favourite that wins is not credited as a call. Above zero means the outcome landed their way more often than the price said it would.

The money version of that last figure is Hit rate vs implied (hit_rate_vs_implied): how much more often the position made money than its entry price implied, as a probability difference (+0.05 = beat the implied odds by 5 points).

Calibration

Brier score (brier) — the squared error between the price paid and whether the side bought won: 0 is perfect, always paying 50¢ scores 0.25, voided markets are left out, lower is better. It grades the forecast, not the money: a position sold at a profit before its side lost still counts as a miss. Raw Brier rewards buying favourites (near-certain 95¢ entries score well for reasons unrelated to skill), and it scores the entry prices as the forecasts — so a trader who wins by buying underpriced sides can post a worse Brier than one farming favourites. Read it beside the luck-adjusted edge.

Brier skill score — how much better than a flat base-rate guess the entry prices scored. Below 0 does not mean bad forecasting: a trader who keeps buying underpriced sides scores below 0 by construction.

Calibration chart and ECE — the profile's reliability diagram buckets entries into ten price bins and plots, per bin, how often the side bought at that price won. Expected calibration error (ece) is the position-weighted mean gap between each bin's win rate and its entry price, on the same bins, from 0 to 1, lower is better.

Luck-adjusted edge (calibration_excess_z) — how many standard deviations more often a trader's positions won than positions bought at the same prices won across the whole platform. Each position is judged against the real win rate of its own entry price, so cheap and expensive entries are on equal terms: a trader who buys 20¢ long shots that win half the time scores strongly, one who buys 80¢ favourites that win only 70 % of the time scores poorly. It grows with both the size of the edge and the number of positions, so a large value on few positions is noise.

Execution and copyability

Alpha vs mid (alpha_vs_mid) — the size-weighted gap between the mid price just before each fill and the price actually paid, signed by side, on the 0–1 price scale (0.01 = one cent a share). Positive means the trader consistently transacted better than the market — an execution or liquidity-provision edge that a copier crossing the spread after them cannot reproduce.

Maker share (maker_share) — the share of a wallet's volume filled on the maker side (resting orders someone else crossed), 0–100 %. It is descriptive, not a quality score, and its direction is neutral: a maker-built record can be a good one and still be one a follower cannot mirror. If you intend to copy, use it as a ceiling (for example ≤ 30 %), not something to maximise. Where Polyrank cannot see the fills behind an entry it says so rather than guessing.

The verdict at the top of a profile

The verdict is computed by one shared module (@polyrank/metrics, verdict.ts) so the trader page, Builder rows, boards and the x402 agent read the same answer. It scores three axes from percentiles, each averaging only the components that have real data:

The overall gauge weights Skill 0.4, Copyability 0.4 and Risk 0.2, re-normalised when an axis has no data, so a missing axis is never shown as a zero. Below 30 resolved markets the verdict says Insufficient track record and treats every number as provisional; a strong claim about edge needs 100. Edge over the price paid deliberately outweighs raw calibration: calibration alone rewards favourite-farming. How the header reads on the page: Trader profiles.

Percentiles, windows and sample floors

• Percentiles rank a wallet against the measured cohort for that metric and window, and are direction-flipped so that higher is better for every metric whatever its raw direction. Raw values are shown verbatim beside them.
• Windows: most metrics exist at lifetime, 90-day and 30-day windows; some are lifetime only. The free GET https://api-agent.polyrank.app/v1/agent/metrics/windows lists which metric exists at which window.
• Sample floors: every metric declares a minimum sample (for example 30 resolved positions for Brier and hit rate vs implied, 50 for win rate, 20 trades for alpha vs mid). Below it the cell is unmeasured (— in the app, null with breakdown_measured: false in the API), never a zero.
• Converters: on a wallet that mostly trades through negative-risk converts, the calibration and alpha rows measure convert arithmetic, not forecasting, and are marked Not skill-rankable — see P&L and resolutions.

Related

• Glossary — every metric in one line
• Trader profiles — where these numbers appear
• P&L and resolutions — how profit and settlement are counted
• Ranking Builder — weight any of the 84 metrics yourself