𝕏 For You feed — request path and labeling path

𝕏 For You feed — request path and labeling path An architecture diagram generated by Archify. Post event stream · Kafka · Labeling path · continuous, off the request path Post event stream Kafka Understanding models · content · account models · Labeling path · continuous, off the request path Understanding models content · account models Labeling rules · Scarecrow · Botmaker · AES · Labeling path · continuous, off the request path Labeling rules Scarecrow · Botmaker · AES Safety labels · offline write · online read · Labeling path · continuous, off the request path Safety labels offline write · online read Home Mixer · candidate pipeline · Rust · Request path · per request, ~300 ms server budget · per request Home Mixer candidate pipeline · Rust per request Thunder · in-network · in memory · Request path · per request, ~300 ms server budget Thunder in-network · in memory Phoenix retrieval · OON · two-tower ANN · Request path · per request, ~300 ms server budget Phoenix retrieval OON · two-tower ANN SimClusters · OON · engagement clusters · Request path · per request, ~300 ms server budget SimClusters OON · engagement clusters Feature hydration · post · author · labels · Request path · per request, ~300 ms server budget Feature hydration post · author · labels Phoenix ranking · Transformer · P(action) · Request path · per request, ~300 ms server budget · largest cost Phoenix ranking Transformer · P(action) largest cost VM Ranker · DPP diversity re-rank · Request path · per request, ~300 ms server budget VM Ranker DPP diversity re-rank Visibility filtering · allow · interstitial · drop · Request path · per request, ~300 ms server budget Visibility filtering allow · interstitial · drop in-network OON candidates candidates ~10² scored ranked read labels write new posts posts & media scores Request path · per request, ~300 ms server budget Labeling path · continuous, off the request path Legend Backend Database Security Message bus

Request path

  • • Home Mixer orchestrates the pipeline per request: retrieve, hydrate, filter, score, select, blend, respond
  • • Three retrieval channels in parallel; one Transformer scores them all
  • • OON = out-of-network: posts from accounts the viewer does not follow

Ranking ≠ visibility

  • • Ranking decides the order
  • • Visibility filtering decides whether a post can be shown at all, from labels produced offline

No separate pre-ranking layer

  • • Retrieval plus rule filters hand a few hundred posts straight to the heavy ranker
  • • Candidate isolation keeps scores cacheable