Meta says its next-generation Andromeda advertising system can score eligible ads at more than one billion ads per second, a 100-fold throughput increase over its 2024 predecessor. The company detailed the production system in an October 28, 2026 engineering post, describing a redesign spanning the prediction model, serving software and GPU execution.
The number measures candidate evaluation, not ads displayed to users. Andromeda handles retrieval, the early selection stage that decides which ads deserve further consideration. Meta’s claim is that it can now bring model quality approaching its more expensive ranking stage to this much larger workload.
That distinction matters, especially because a separate, higher figure of 10 billion ads per second describes model inference alone.
Retrieval Chooses the Candidates; Ranking Examines Them
Meta’s ad recommendation pipeline splits the work into stages because evaluating every eligible ad with its most computationally expensive models would be too costly.
| Stage | What it does | Main constraint |
|---|---|---|
| Retrieval | Narrows a large eligible pool to a small set of promising candidates | Evaluate many candidates within milliseconds |
| Ranking | Scores the surviving candidates in greater detail | Spend more computation on a much smaller set |
Andromeda performs the first job. Its scores help determine which ads advance; they are not the final decision about which ad someone sees. Meta describes ranking as a later, more computationally expensive stage in its technical explanation.
That creates an important limit on downstream improvements: a better ranking model cannot select an ad that retrieval has already discarded. Improving the shortlist can therefore improve the final result even if the ranking system stays unchanged. Retrieval quality deserves attention alongside the headline throughput figure for this reason.
Meta says Andromeda’s new model approaches ranking-level performance while operating within retrieval’s latency constraints. That is a company characterization of model quality, not a claim that retrieval and ranking have become interchangeable.
Generative Ad Growth Changed the Workload
Meta identifies its own generative AI creative tools as a major reason for the redesign. Since the 2024 Andromeda deployment, those tools have helped advertisers create hundreds of millions of new ads, expanding the candidate pool that retrieval must handle.
The earlier system evaluated tens of millions of ads per second per GPU card, according to Meta. The company says its newer requirements call for evaluating multiple billions per second per card.
More candidates create two connected demands. The system needs enough throughput to examine the expanded pool, and it needs enough predictive quality to distinguish useful candidates from weak ones. Adding capacity without improving selection would address only part of the problem.
Meta also says the bottleneck changed at this scale. Moving intermediate tensors, the arrays holding model data, between GPU operations generated terabytes per second of memory traffic. Transferring ad information between CPU hosts and remote accelerators could saturate network connections.
The practical consequence is straightforward: faster arithmetic helps little if the GPU spends too much time waiting for its inputs. Meta’s account describes a data-movement problem as well as a model-computation problem.
What the 100x Throughput Claim Measures
The published figures cover different scopes and hardware descriptions. They should not be treated as interchangeable benchmarks.
| Published figure | Measurement scope | Hardware description |
|---|---|---|
| Tens of millions of ads per second | 2024 Andromeda evaluation throughput | Per GPU card |
| More than 10 billion ads per second | Next-generation model inference alone | One GH200 GPU |
| More than one billion ads per second | Next-generation production retrieval pipeline | A single GPU server, according to coverage |
Meta supplies the 2024 per-card figure and the headline production claim in its engineering post. Tensor Journal reports the separate inference-only result on a single NVIDIA GH200 GPU. PPC Land describes the production figure as running on a single GPU server; Tensor Journal describes it as a single-GPU pipeline.
The one-billion figure is the relevant production headline. The 10-billion figure concerns a narrower component: executing the model. A complete retriever must also find candidates, move data and select the highest-scoring results. A component’s peak processing rate does not tell readers how quickly that entire sequence runs.
The distinction also prevents two common misreadings:
- One billion candidate evaluations per second does not mean one billion ad impressions served per second.
- It does not mean Meta has one billion distinct eligible ads for every individual request.
I’d read the headline as a candidate-processing capacity claim. Its importance is how much selection work fits inside the serving budget, not how many advertisements people will see.
All these performance figures originate with Meta; the supplied reporting provides no independent audit. The rounded baselines and differing hardware descriptions also leave insufficient detail to reconstruct an exact, like-for-like 100x calculation. That multiplier should remain attributed to Meta.
The Model Uses Specialized Experts to Reduce Computation
Meta says the new retrieval architecture approaches ranking-level quality with approximately 300 times fewer inference-time FLOPs than the 2024 Andromeda retrieval model. FLOPs are floating-point operations, a measure of arithmetic work. They are not a direct measure of elapsed time, electricity consumption or total operating cost.
The model makes three coordinated changes:
- Specialized user models. An ensemble of user towers replaces a single tower. These experts represent different aspects of user interests using signals including engagement sequences, cohort information, LLM-derived semantic features and engagement across surfaces.
- Context-aware ad representations. The candidate-ad encoder incorporates real-time serving context, giving later interactions more information than an isolated ad representation would provide.
- Expert routing. An attention-based gating network routes candidates toward the most relevant user experts.
Meta describes this as a retrieval mixture-of-experts architecture in its engineering post. The LLM connection concerns architectural ideas and semantic features. This system’s job is to evaluate ads, not generate their copy or images.
The 300x FLOPs reduction and 100x throughput increase measure different things. The first concerns model arithmetic; the second concerns the serving system’s processing rate. It would be misleading to divide them and declare that the difference measures serving overhead. Memory access, workload shape, hardware utilization and candidate selection also affect throughput, and the published figures do not isolate each contribution.
The useful conclusion is narrower: Meta changed the computational work required per evaluation while also changing how the system executes that work.
Keeping the Retrieval Pipeline on the GPU
Meta’s serving redesign addresses the cost of moving data between stages.
According to Tensor Journal’s account, index matching, scoring and top-K selection now sit on the same GPU. Top-K selection keeps a specified number of the highest-scoring candidates. The outlet reports Meta’s claim that this co-location reduces network traffic by roughly 100 times.
The same report describes a fused GPU kernel that keeps intermediate results in registers and thread-block shared memory instead of writing them out to high-bandwidth memory between operations. Registers and shared memory provide fast, local storage for work in progress. Avoiding repeated writes and reads can reduce traffic through the GPU’s broader memory system.
Frequently Asked Questions
3 questions
1Does Andromeda Replace Ad Ranking?
No. Andromeda retrieves promising candidates. A later, more computationally expensive ranking stage evaluates the survivors in greater detail.
