How this works

A candidate formation starts from one of two sources. Direct membership uses YC’s own tags and industries (e.g. “Manufacturing and Robotics”), with generic labels like AI, B2B, SaaS, and broad geographies filtered out so they don’t drown out sharper categories. Semantic clustering catches everything YC hasn’t tagged consistently: every company description is embedded locally with all-MiniLM-L6-v2 (quantized to 8-bit for speed, run entirely on this pipeline’s own hardware — no hosted API calls, no per-company cost), then grouped by spherical k-means into up to 80 clusters: cosine similarity as the distance metric, with farthest-point-seeded starting centroids so early cluster picks don’t cluster together themselves. Model choice and clustering method both materially shape which categories emerge — a different embedding model or a Euclidean-distance clustering method would draw different boundaries around the same company descriptions.

For every candidate from either source, the pipeline compares two adjacent four-batch windows: the four batches ending at a given point, against the four batches before that. A formation only gets flagged as accelerating once it clears a specific bar — and a bar for cooling, in the other direction:

Worked through: say the prior four batches had 400 completed companies, 20 of them robotics infrastructure — a 5% share. If the most recent four batches have 500 companies and 40 are robotics infrastructure, that’s an 8% share — 1.6× the old share, a swing of +3 percentage points, and +20 companies, clearing all three bars comfortably. The percentage-point rule matters as its own, separate check from the multiple: a swing from 0.10% to 0.20% is technically 2× growth, but it can just mean one company became two. Requiring a real percentage-point move alongside the ratio — not the ratio alone — is what keeps that kind of noise out. The same logic runs in reverse for cooling: a category with only one company before can “fall” 100% by going to zero, which is why cooling also requires at least 10 companies in the prior window before it counts at all.

Anything short of those bars — even a real-looking bump — doesn’t get flagged. The thresholds exist specifically to keep noise off the site; a formation that shows up here moved by more than chance and small-sample variance would usually explain.

Where these exact numbers — 1.35×, 0.72×, the percentage-point and company-count floors — come from, stated plainly: they’re a heuristic floor, not an empirically optimized one. They were chosen to require a real ratio move and a real percentage-point move and a minimum company count together, for the small-sample-noise reasons worked through above, not tuned against a held-out precision target. That’s a real difference from Radar and Whisper below, whose 69%/49% and 54%/39% numbers come from an actual backtest against this site’s own batch history. No equivalent backtest has been run for this core threshold; if one changes these numbers later, that’ll be logged here, not silently adjusted.

Two formations sharing a handful of companies isn’t automatically meaningful — with dozens of tracked formations there are thousands of possible pairs, and some overlap is pure chance. Every pair that shares at least one company is run through a hypergeometric enrichment test against the full company universe, producing a lift score (how much more overlap than random chance would predict) and a p-value.

Worked through: if 300 of roughly 6,000 tracked companies are Agent Infrastructure and 200 are AI Security, pure chance predicts about (300⁄6,000) × 200 ≈ 10 companies belonging to both by coincidence. Finding 45 companies that actually do — a lift of 4.5× over that random expectation — is the kind of gap the test exists to catch; a lift near 1× is exactly what coincidence looks like. The p-value that comes with it isn’t “the odds we’re wrong” — it’s the probability of seeing an overlap this large if the two categories really were unrelated. A p-value of 0.0004 means data this extreme would turn up roughly 4 times in 10,000 under pure chance.

But testing thousands of pairs at once changes the math: even if none were genuinely related, testing 2,000 pairs at the usual 5% significance threshold would predict about 100 false alarms from chance alone. So every tested pair is corrected together with a Benjamini–Hochberg false-discovery-rate adjustment, which raises the bar collectively across every relationship being tested rather than trusting any single p-value in isolation. Only pairs that clear that corpus-wide bar are shown as a validated relationship on screen — not just because two bubbles happened to share a company.

Beyond describing what’s already happened, the site also surfaces forward-looking, falsifiable predictions: given a fast-growing formation and the real companies building in it, what capability or piece of infrastructure is likely needed next? Each prediction is grounded in specific real companies from that formation — not general industry knowledge — states a concrete falsifier (the exact observation that would prove it wrong), and a rough date to check back once more evidence exists. They’re clearly marked as predictions, not settled facts, and they’re meant to be checked against reality over time rather than left standing unexamined.

Generation is deliberately bounded, not run against every formation that has ever qualified as accelerating: only formations from the batch that just completed, or the current still-filling one, are eligible. And because different formations can independently produce essentially the same idea in different words, every run’s live predictions are grouped before anything is published: pairs above a calibrated embedding-similarity floor are proposed as a candidate duplicate set, then a separate model call decides whether they genuinely describe the same opportunity — merging them into one investigation credited to every supporting formation — or are only superficially similar, in which case they’re kept separate. The model can and does reject a proposed merge; it isn’t just trusting the similarity score. As of this writing, that step collapses 201 live predictions into 199 investigations.

Every investigation then goes through an automated editorial pass — the same model used everywhere else on this site that calls a hosted model at all, claude-sonnet-5, given the prediction’s rationale and falsifier and asked to judge four things directly: is the rationale specific and evidence-grounded rather than generic AI-startup boilerplate, is it concrete enough to actually investigate, is the falsifier genuinely checkable, and would it read as non-obvious to someone who already follows the space. That single call produces two separate outputs, not one derived from the other: a status — reviewed, needs-work, or rejected — and an independent 0–100 publication rank. There is no numeric rank a prediction needs to clear to be marked reviewed; status is the model’s own direct categorical judgment against the four criteria above, not a threshold applied to the rank. The prediction’s own generation-time confidence label is deliberately never used as that rank either, since it was never calibrated to mean that. As of this writing, of 199 live investigations, 21 are reviewed, 168 need work, and 10 are rejected. Nothing is deleted for failing this pass — a rejected investigation moves to the archive tier below, visible rather than erased.

What’s next reads as four tiers built on that pass, not one flat, ever-growing list: Priority investigations is the reviewed set, ranked by that score — read these first. By formation holds everything else still live and evidence-checked (needs-work, or not yet reviewed), filterable by formation. Resolved record is the confirmed/expired predictions described below. Research archive, collapsed by default rather than deleted, holds the hand-authored curated dossiers alongside investigations that didn’t clear the review bar.

A subset of these live predictions also carry a “supporting research” line: the prediction’s own title is embedded with the same local model used for semantic clustering, compared against recent arXiv papers fetched by keyword, and kept as a match only above a 0.5 cosine-similarity threshold — the level that separated real matches from noise in manual spot-checks. This is deliberately weak evidence, not confirmation: it says a related technical idea already exists in the published literature, nothing about whether the predicted company or product exists yet. It’s also incomplete by construction — only live, database-backed predictions get checked against arXiv this way; the curated hypothesis dossiers elsewhere on the site don’t share a stable identifier with this table yet, so they’re not included. And because arXiv skews toward technical-capability predictions over business-process ones, only a minority of live predictions turn up a match at all — the absence of a research line on a card means no match was found, not that none was looked for.

Separately, every live prediction is resolved on a full recompute each run: it’s anchored to the batch its source formation was detected at (not to when the prediction script happened to run), and it’s marked confirmed only if a different formation, detected at a strictly later batch, semantically matches the prediction text above a 0.65 similarity bar — deliberately stricter than the 0.5 bar used for the “supporting research” line above (that one only needs topical relatedness; this one is the site’s strongest self-claim, that a prediction came true). Chosen relative to that other threshold, not independently tuned against inspected score distributions the way the deduplication threshold above was — as of this writing, only one of 149 live predictions has cleared it. Anything not yet confirmed is open while its check-back date hasn’t passed, or expired once it has — expired predictions stay visible in a collapsed section rather than being deleted. No precision or hit-rate number is shown for this: unlike Radar and Whisper above, which can be backtested against a fixed historical sweep, there is no enumerable universe of predictions that could have fired and didn’t, so there is no honest denominator to build a base rate from. Confirmed count, expired count, and lead time are facts; a precision claim here would not be.

The Radar page runs a second, faster, deliberately noisier check on the same data: instead of waiting for the four-batch comparison above, it flags a YC tag or industry whose membership count rises for two consecutive batches in a row. That catches things sooner, but it’s a real tradeoff — most of what fires this way doesn’t go on to become a confirmed accelerating formation.

This was backtested against the site’s own full batch history rather than asserted: of every early signal since Winter 2018 old enough to have a known outcome, 69% went on to clear the accelerating bar above, versus 49% for an equally-active formation picked at random with no signal at all. That gap is the honest way to read it — not “69% accurate,” but a 1.41× lift over what an equally-active category does anyway, meaning Radar’s pick is right about 41% more often than a coin flip weighted to the baseline. Real, and worth showing — not magic.

A third, even faster tier — Whisper — uses the same looser-trigger idea: 3 companies instead of 6, a 1.3× rise instead of 1.5×, +2 companies instead of +3. It originally ran on the same YC tags Radar does, but that made its earliest, noisiest read also its broadest — a tag like “Machine Learning” ticking up isn’t news, that category has existed for over a decade. So Whisper now runs on the same description clusters as Radar’s granular section instead (below), trading tag/industry’s larger sample for actual specificity. Backtested the same way: 54% precision against a 39% base rate — a 1.39× lift, higher than granular Radar’s own 1.12×, off a real but thin sample (~60 total clusters). Whisper is meant to be glanced at, not trusted the way Radar’s numbers can be — that’s the deliberate tradeoff of a noisier, earlier tier, not an oversight.

Radar’s second section works differently again. “Health tech” or “education” are too broad to be interesting on their own, and YC doesn’t tag companies at a finer grain than that — so those rows come from a local embedding model clustering company descriptions into specific groups instead (Coding Agent, Voice Agent, and similar). There are only around 60 of these clusters, far too few to backtest with the confidence the tag/industry signal has, so no precision number is attached to them. They’re shown as a plain fact — how many companies in that cluster joined in the last two batches — not a validated prediction.

Everything above — Formation, Radar, Whisper, even the arXiv paper-matching on What’s next — starts from something YC already tagged or a prediction already written. That’s confirmation, not discovery: it can only ever find what YC has already touched. The Frontier tab on Signal works the other way. Paper abstracts from sixteen CS-adjacent arXiv categories are embedded with the same local model used everywhere else on the site and clustered on their own — no YC company, tag, or prediction ever seeds which papers get pulled in or how they get grouped. A cluster flagged there is evidence of something happening in the literature, full stop, whether or not any YC company has touched it yet.

This costs something real: there is no backtest. Radar and Whisper’s numbers exist because a YC tag firing early can be checked against whether the production accelerating-formation detector later confirms the same label — that’s the ground truth the whole precision/lift calculation depends on. An arXiv-only cluster has no such ground truth to check against; there is no YC-confirmed anything for it to be right or wrong about. So each cluster is tested against its own history instead: a cluster younger than six months is labeled new — too young to say anything more than “here’s a real, specific cluster.” Once it has real history, its most recent month is compared to its own historical monthly average; a real surprise (at least 3× that average) makes it surging, and staying elevated across a later run makes it sustained. Real activity that falls short of that bar (1.5×–3×) is labeled watching instead of discarded — the lowest-confidence claim on the site, shown so the page reflects the real underlying signal rather than only the handful of clusters that cross a hard line. There is still no precision, no lift, and no confirmed/expired outcome attached, ever — that’s not an oversight or a smaller version of Radar’s rigor, it’s a structurally different, weaker kind of claim, and it’s labeled that way everywhere it appears.

The Frontier tab’s second source works on the exact same principle as its arXiv one, applied to a different corpus: Hugging Face model cards across sixteen AI-adjacent task categories — pipeline tags like text-generation, robotics, and text-to-video, not YC labels — are embedded with the same local model used everywhere else on the site and clustered on their own. No YC company, tag, or prediction seeds which models get pulled in or how they get grouped, and neither does any curated “trending” list from Hugging Face or anyone else — the embedded text is built from each model’s own declared pipeline tag, tags, and name. A cluster flagged here is evidence of models actually being built and released, full stop, whether or not any YC company is behind them.

The same cost applies, for the same reason: there is no backtest, since there is no YC-confirmed formation a Hugging Face-only cluster could ever be checked against. Growth is read the same way as arXiv’s — against each cluster’s own history, not a flat comparison — using the identical new / surging / sustained / watching tiers and thresholds: too young for a baseline, a real surprise (3× or more) against its own historical monthly average, still elevated on a later check, or real above-baseline activity (1.5×–3×) that doesn’t clear the surging bar. There is still no precision, no lift, and no confirmed/expired outcome attached, ever — the same structurally weaker kind of claim as its arXiv counterpart, labeled that way everywhere it appears.

Everything above tests each source against itself. A separate step goes further and asks the site’s actual thesis directly: does independent research activity show up before the YC formation it’s plausibly related to, not after? For every formation, its label is embedded and compared against every arXiv cluster’s and Hugging Face trend’s label (same local model, same approach as the arXiv paper-matching above), and a match is kept only if it clears a stricter similarity bar and the cluster’s own onset — the month by which a real quarter of its eventual member papers or models already existed, not the month the detection script happened to last run — comes before the formation’s own anchor batch (the start of the four-batch window that first showed it accelerating, not the later batch where that acceleration was strong enough to detect). Getting the direction of that comparison right is most of the difficulty here: naively comparing against either source’s wall-clock detection date, or against a formation’s detection batch instead of its anchor, both systematically make research look later than it really was.

As of this writing, that comparison currently finds too few qualifying pairs to state a median lead time honestly — below a floor of 3 matched pairs per source, the same “too small a population to claim precision” reasoning applied everywhere else on this site. Hugging Face’s ingestion is still a narrow one-month pilot slice (see the repository README), so its onset dates can’t yet predate much of anything — that should resolve as ingestion continues. arXiv’s corpus already spans multiple years of real paper history, so its own null result is a genuine finding, not a data-volume artifact: no cluster both specific enough to a formation and early enough currently exists. This is not the same claim as “structurally impossible” — the mechanism runs on every monthly refresh, and any qualifying pair (per formation, per source) appears on the Frontier tab as soon as it clears the bar, with the aggregate median shown once enough real pairs accumulate.

A separate, simpler check runs on the site’s Simple view, under “Is activity elsewhere moving in the same direction?” For each of the most active accelerating formations, it asks whether independent activity elsewhere — GitHub repositories, Hugging Face models, public Tech.eu funding rounds — shows comparable growth over the same recent-vs-prior window YC’s own formation detection uses. This isn’t a model judgment call; it’s the same recent-count ÷ prior-count multiple used throughout the site, computed per outside source, with a >1× floor before it counts as growth at all. A formation with no connected outside evidence yet says so plainly rather than being silently left off.

One source is honestly weaker than the others: Hugging Face’s search endpoint matches a category name as a loose substring against model and dataset repo names, not a topic or description search, and it doesn’t support quoted-phrase matching the way GitHub’s search does — tested directly against the live API before writing this: search=“Infrastructure” (quoted) returns zero results, so that fix isn’t available here. A generic category name can pick up repos that mention the word incidentally, not because they belong to that category. Multiples from Hugging Face specifically should be read with that in mind.

The question is deliberately “is activity elsewhere moving in the same direction,” not “did YC see it first.” It used to ask the latter, using a lead-time claim built from a client-side proxy — a plain keyword-overlap match between a formation’s title and nearby arXiv cluster labels. That proxy was both unvalidated and one-sided: it kept only cases favorable to YC and silently dropped any case where research or outside activity was actually earlier or simultaneous. That framing has been retired. When a lead-time claim appears now, it comes exclusively from the calibrated cross-source match described just above — which, by construction, only ever measures research leading a formation, never the reverse — so a shown lead time always means the outside research predates the YC formation’s own detection. That match currently has no accepted pairs anywhere on the site, so no formation claims a lead time today. That’s the honest state of the underlying data, not a missing feature.

No multiple-comparison correction on the core formation-detection threshold. With roughly 76 categories tested across roughly 15 batch windows, some flagged signals are false positives at any reasonable significance level. This threshold is already disclosed above as a heuristic, not a backtested or corrected one; this is the concrete consequence of that.

No confidence intervals on the headline ratios. “9× as common” from a base of 1 company is a very different claim than the same ratio off a base of 20. The raw counts are already shown alongside every ratio (87/652 → 50/706, for example), which lets a careful reader gauge this by hand, but a real interval isn’t computed or shown.

The null model behind “accelerating” is a heuristic, not a formal significance test, as already stated above. This is different from the relationship test between formations, which does have a fully specified null (hypergeometric, Benjamini–Hochberg corrected) and real p-values; that part of the site isn’t affected by this limitation.

Category taxonomy inherits Y Combinator’s own tag inconsistencies. Some categories overlap, split, or duplicate across time purely because YC’s own tagging changed, not because the underlying trend did. Not yet corrected.

The outside-evidence check against GitHub and Hugging Face doesn’t baseline against the platform’s own overall growth. A repo or model count ratio needs to be compared against how fast the whole platform grew over the same window to mean anything on its own; right now it isn’t.

Radar and Whisper are backtested against this site’s own batch history (69% vs. a 49% baseline; 54% vs. a 39% baseline), a real, held-out check of whether an early flag predicts later confirmation. What isn’t measured is different: whether a formation that’s already confirmed keeps growing afterward. The site never claims that it does (confidence there is explicitly not a persistence claim), but the claim isn’t tested either way.