Pasted

Input and Query Hydration:

Starts with a user's "For You" request.
Fetches user-specific data: engagement history (likes, replies, etc.), following list, preferences, and features like muted keywords or blocked accounts.



Candidate Sourcing:

In-Network (Thunder Component): Pulls recent posts from followed accounts using real-time Kafka streams and in-memory stores. This ensures low-latency access to familiar content, with automatic trimming of expired posts.
Out-of-Network (Phoenix Retrieval): Uses a two-tower embedding model to search a global corpus of posts. One tower encodes the user's profile and history; the other encodes posts. Similarity is computed via dot products to retrieve top candidates. This step gathers a large pool (e.g., thousands) of potential posts.



Candidate Hydration and Pre-Scoring Filtering:

Enriches candidates with metadata: text, media, author details, video length, etc.
Applies filters to remove ineligible content, such as duplicates, old posts, self-authored items, blocked/muted authors, or previously seen content. This reduces the pool while respecting user preferences.



Scoring and Ranking (Phoenix Scorer):

The core of the system: A Grok-based transformer predicts probabilities for 15 user actions per candidate post.




Predicted Actions:
Category
Examples
Impact on Score




Positive
Like, Reply, Repost, Quote, Click, Profile Click, Video View, Photo Expand, Share, Dwell, Follow Author
Increases score (weighted positively)


Negative
Not Interested, Block Author, Mute Author, Report
Decreases score (weighted negatively)




Final score = Sum of (weight × probability) for each action. Weights prioritize meaningful engagement like replies over likes (which are "nearly worthless" in boosting visibility).
Additional adjustments: Author diversity scorer penalizes repeated authors to promote variety; out-of-network scorer tweaks scores for broader discovery.
Key innovation: Candidates are isolated during attention (no inter-candidate influence), allowing cacheable, consistent scoring.



Selection and Post-Selection Filtering:

Sorts by final score and selects top results.
Final filters handle edge cases: spam detection, deleted content, violence, or thread deduplication.



Output:

Delivers the ranked feed via a gRPC endpoint.