How we poll

Poll the Models is built on one principle: measure what AI actually does, not what it claims to do. Every number on this site comes from real model outputs, processed the same way, every week.

What we measure

We ask the major LLMs realistic buying-intent questions — the kind a real buyer types into a chat window. We then record which vendor each model most strongly endorses as its first pick, and which vendors it recommends at all.

The headline metric is Share of Answer (SoA): the fraction of the time a vendor is a model's single first choice. We report it with a 95% confidence interval so you can see how much to trust a ranking based on the sample size. A vendor ranked #1 with a tight interval is a signal; one with wide whiskers is a trend worth watching.

The questions

Each category has a bank of diverse buying questions — varying company size, budget, geography, technical level, and specific constraints. We run every question across every model in the panel, multiple times, to smooth out random variation. Base questions are the same each week so results are comparable over time.

The model panel

We poll the default flagship from each major provider — no fine-tuned or grounded variants, no system prompts. The user-default experience is exactly the product being measured. We pin exact model versions and record which version actually responded, so every ranking is traceable.

How rankings are built

A separate model reads each answer and identifies the first pick and full shortlist — no human judgment in the loop. Vendor names are normalised against a reference list so "Gusto" and "gusto.com" count as the same pick. We check the unresolved rate before publishing; if too many vendor mentions can't be matched, the snapshot doesn't go live.

Rankings are sorted by pooled Share of Answer, then by Consideration Rate as a tiebreaker. The concentration score (HHI) and consensus score show how dominated a category is and how much the models agree.

← Back to the index