Methodology and accuracy

How forecasts are made, and how accurate they are

1stSeen predicts when a recurring internship, co-op, or new-grad program is likely to open its next round of applications, from the openings it has observed for that program before. This page explains what it observes, how a forecast is computed, what the confidence score means, and where the evidence on accuracy stands today.

Last updated September 17, 2026

Why the evidence model matters

1. Dates keep their precision
ExactBoundedObserved by
A board's own publication date, a window between two archive captures, and a date something was merely seen by are stored apart and never promoted into one another. More
2. Numbers come from statistics
A fixed statistical model weighs past openings by their precision and produces the window and the confidence score. More
3. Replays refuse hindsight
A backtest may use only facts 1stSeen had recorded before its cutoff, so it never scores what it could not have known at the time. More

Where accuracy stands

Not yet measured. The latest backtest, finished Oct 3, 2026, went back to 890 past openings and tried to forecast each one from only what 1stSeen had recorded 60 days before it opened. It could score none of them.

That is the backtest refusing hindsight, not failing. A past opening may be forecast only from facts 1stSeen had recorded before it, and 1stSeen has recorded everything it holds since it began collecting, including archive captures from years earlier. So no past opening yet has a forecast made in time. The first scorable cases will be programs that open after collection began and were forecast before they did, and their number grows with each recruiting season. Until then 1stSeen publishes no accuracy figure, because it has none measured without hindsight.

Why each past opening could not be scored, in the backtest's own words
Reason	Openings
no temporal evidence was available by the forecast cutoff	689
held-out opening is observed-by only and has no scoreable actual interval	201

Today 421 of 845 early-career roles have a forecast. 351 of those rest on one recruiting cycle of the program's own or none, and take the rest of their timing from comparable programs, as described below. Every forecast says how many of its own cycles it rests on.

Roles by the program's own recruiting cycles behind today's forecast
No cycle of its own, from comparable programs only	0
One cycle of its own	351
Two cycles	45
Three or more cycles	25
No forecast	424

What 1stSeen observes

1stSeen reads public job postings from companies' applicant tracking systems (Greenhouse, Lever, Ashby, and SmartRecruiters), their public career pages, and archived copies of those pages in the Internet Archive's Wayback Machine. Every observation is stored with the address it came from, when it was observed, and a fingerprint of its content, and is never rewritten afterwards. Data sources lists what is read and how.

From those observations it reconstructs when each program opened in earlier years. Every reconstructed date keeps the precision its source supports:

Exact: The job board published the date itself.
Bounded: A complete archive capture shows the program absent and a later one shows it present, so it opened between the two.
Observed by: The program was visible by that date and may have opened earlier. An archive capture timestamp is never treated as an exact opening date.

The three are stored apart and never promoted into one another, and every date on the site carries its class.

A repost is not a new year. Postings that name the same cohort, such as Summer 2027, count as one recruiting cycle, and without a stated cohort, openings less than 300 days apart count as one.

How a forecast is made

A fixed statistical model, hierarchical-circular-shrinkage-v2, computes every date, interval, and score.

It places each past opening on an annual circle, so late December and early January count as close together, and weighs each by its precision, the quality of its source, how surely it belongs to this program, and its age.
It produces an expected date and an 80% prediction interval: the range it expects the next opening to fall in. The 80% is the interval's design target. How often real openings land inside it is what the backtest measures.
A program with fewer than three recruiting cycles of its own is forecast only when comparable programs (at the same company, or in the same field, with the same level and recruiting season) give the model a timing to borrow. Their influence shrinks as the program's own history grows, and such forecasts are wider: roughly 42, 32, or 22 days either side of the expected date at the least, for no, one, or two cycles of its own.
When there is too little of the program's own history and nothing comparable to borrow from, the model does not forecast it. The role is listed with the evidence that exists, and no date.
Recruiting signals, such as a careers page adding an internship section, can raise or lower the confidence score. They never move a date or an interval.
No language model produces a date, an interval, or a score. A language model may help sort an ambiguous job title or read a question to the agent; the numbers come only from the statistical model.
Every forecast is stored with the observations it used, the model version, and a fingerprint of its inputs. A forecast is never edited: when the evidence changes, a new version is added beside the old one, and the program page lists both.

What a window rests on

Every forecast says which evidence its window mainly rests on, read from the weights the model recorded for it:

Own history: Most of the window's weight comes from this program's own past openings.
Borrowed timing: Most of the window's weight comes from comparable programs (this company's other programs, or similar programs elsewhere at the same level and season), because this program's own history is short.

The percentage beside it is that basis's share of the window's weight. The number of recruiting cycles does not say this on its own: a program with one recorded cycle can take most of its weight from that cycle or almost none, depending on how consistent the comparable programs are. Borrowed timing is a real signal, since programs at one company or level and season tend to open together, but it describes those programs, not this one.

What the confidence score means

Every forecast carries a confidence score from 0 to 100. It scores how much consistent evidence backs the window: how much of the program's own history there is, how consistent its timing has been, the quality and precision of its sources, how recent they are, how narrow the interval is, and any current signals. The program page shows each factor.

It is not the chance that the window is right, and it has not been calibrated against outcomes, which is why it is written as 48 / 100 and never as a percentage. A program with little history of its own is capped whatever else supports it: the cap rises from 48 through 56, 69, and 82 to 92 as its own weighted history grows.

How a backtest keeps hindsight out

A backtest goes back to a cutoff before a past opening and forecasts it again from only what 1stSeen knew then. Forecast Replay runs one such case on request.

Evidence is admitted only when both its date and the moment 1stSeen recorded it fall before the cutoff.
The opening being predicted is held out by its identity, and comparable programs contribute only evidence that meets the same cutoff.
An Observed by opening has no defensible actual date, so it is never used to score a forecast; it is skipped, with that reason.

A single replay is one case. Accuracy is the backtest's measurement over every scorable case, stated at the top of this page.

What the agent can and cannot do

The recruiting agent answers a question by choosing among fixed tools: looking up stored evidence and forecasts, checking current postings, and asking the statistical model for a forecast. It cannot write a date or a score itself. While it works it shows the tools it ran and what they returned, in counts, and never its prompts or private reasoning.

What 1stSeen covers

Early-career technical programs: internships, co-ops, new-grad and graduate programs, rotational programs, and technical apprenticeships, in software, machine learning, data, quantitative roles, hardware and robotics, every engineering discipline, and product and design. Other roles at the same companies are kept as evidence but never listed or forecast.

What the model does not know

It assumes a program recurs about once a year. It cannot foresee a program being cancelled, split into several cohorts, or moved.
Public sources are incomplete. A company can open applications somewhere 1stSeen does not read, and archives capture pages irregularly, which is why an archived date usually says only that a program was visible by then.
Nothing here comes from the companies themselves. A forecast is a prediction from public evidence, and the company's own announcement always wins.