An AI model speed test only becomes useful when the method is clear enough for someone else to understand—or challenge—the numbers. AI Speed Index measures first-token latency, sustained tokens per second, total completion time, and consistency across repeated runs under ordinary U.S. home-office conditions. This article describes exactly how those measurements are collected, what is recorded, and what is left out.
I run every test from a home office in Austin, Texas, on a MacBook Pro connected to a 1 Gbps fiber line. The setup is deliberately ordinary. Most of the people who will use these models also work from home offices, co-working spaces, or small-team environments rather than inside a data-center lab. The goal is to produce numbers that reflect that reality.
What We Measure and Why
Four core metrics form the foundation of every speed test:
First-token latency: the time from request submission until the first usable token appears.
Sustained tokens per second: the average rate at which tokens are generated after the first token, measured across the full response.
Total completion time: the full duration from request start until the final token is received.
Consistency: average, median, and range across a fixed number of identical runs.
These four numbers answer different practical questions. First-token latency tells a product manager how quickly a user sees something start to appear. Sustained throughput shows whether the model stays fast once generation begins. Total completion time is the number that actually appears on a stopwatch when a real task finishes. Consistency reveals whether the model can be trusted for automated or high-volume work.
We deliberately avoid treating a single peak tokens-per-second figure as the headline. A model that spikes high on one short prompt and then slows on longer or repeated requests is not reliable for daily work.
Test Conditions That Stay Fixed
Every speed test records the same baseline conditions so results can be compared over time:
Exact model name and version string
Provider and API endpoint used
Date and local time (U.S. Central)
Geographic region of the test machine
Network type and approximate bandwidth
Number of test runs
Prompt text or task description
Input token count and output token count for each run
Any observed failures, retries, or timeouts
When a provider offers multiple endpoints or regions, the endpoint used is stated. If the test is run during a period of known high load or after a model update, that context is noted. The point is not to claim laboratory perfection. The point is to make the conditions visible so a reader can decide how much the numbers apply to their own environment.
Concurrent-load tests follow the same documentation rules. When five or ten simultaneous requests are sent, the method, the tools used to generate the load, and the observed behavior (including any throttling or queuing) are described in the same article.
How a Typical Latency Run Is Conducted
A standard single-request latency series uses the following sequence:
The prompt is prepared and tokenized so input length is known in advance.
The request is sent and the wall-clock timer starts.
First-token arrival is recorded.
Generation continues until the model signals completion or a predefined maximum length is reached.
Total elapsed time and final token count are recorded.
Tokens per second is calculated from the post-first-token portion of the response.
The process is repeated for the planned number of runs (commonly ten) with the same prompt and the same settings.
Between runs a short cool-down interval is observed so that any temporary rate-limit or warm-up effects are less likely to distort the next measurement. If a run fails or times out, it is logged as a failure rather than discarded. The failure rate itself becomes part of the published result.
For longer tasks the same principles apply, only the prompt length and expected output size change. The method stays consistent so that a short-prompt latency number and a long-document completion time can still be read against each other.
What We Do Not Claim
A transparent method also requires clear boundaries. AI Speed Index does not claim:
That the numbers represent every possible network or region.
That a single set of runs proves universal superiority.
That laboratory-grade isolation is achieved in a home office.
That provider-stated theoretical maxima are being independently verified at the silicon level.
The measurements are observational under stated conditions. They are useful for comparing models on the same tasks, under the same network, with the same documentation standard. They are not a substitute for a reader’s own production monitoring.
Pricing is calculated separately from pure speed. When cost figures appear in a speed article they are derived from the observed token counts and the publicly listed rates at the time of testing. Any assumptions (for example, whether caching applied or whether a discounted tier was used) are stated.
Why Consistency Matters More Than Peak Speed
In product and operations work, a model that finishes in 4.2 seconds nine times and 11 seconds once is often worse than a model that finishes in 5.5 seconds every time. Variance creates uncertainty in automation, in user-facing interfaces, and in cost forecasting.
That is why every speed article reports the range and, when useful, a simple distribution of the runs. A tight cluster of results supports higher confidence for repeated workflows. A wide spread is itself a finding, even if the average looks attractive.
The same logic applies to concurrent requests. A model that maintains acceptable latency under five simultaneous calls is more useful for a small team’s internal tools than a model that looks fast only when it is the only request on the wire.
How Results Are Presented
Every substantial speed test ends with the same structure:
Use it when…
Skip it when…
What we measured
What we still do not know
The first two lines are practical recommendations based on the observed numbers. The third line restates the key measured values. The fourth line lists open questions—different regions, longer context, different load patterns, or future model versions—that have not yet been tested.
This format keeps the difference between data and judgment visible. A reader who weights cost higher than first-token latency, or reliability higher than peak throughput, can still use the raw measurements.
The method described here is the baseline for every Speed Tests article that follows. When the method itself changes—new tools, new concurrent-load approach, additional metrics—the change will be documented in the same transparent style. Fast is a feature. Reliable is the product. The only way to keep both statements honest is to show the stopwatch, the spreadsheet, and the exact conditions under which the numbers were collected.
Letters
No letters yet — be the first to write.