Now Twitter's Recommendation Algorithm is open-source - An era of transparency

Today I completely stayed away from all my routine to check the Twitter's recommendation algorithm which is open sourced couple of days back.

I was going through their 472,945 lines of code (scrolling over), but here are some mind-blowing facts which I would like to share with you all. Following are those

1. This algo serves 150 billion Tweets to people’s devices.. every day. Now you can have all answers to designing a recommendation engine available for free. It's a terrific time to learn the algorithm line by line which has evolved from ~2 decades of world-class engineering based on communication from politics to generic posts.

2. The algorithm uses a neural network with a whopping 48 million parameters that is continuously trained on tweet interactions to optimize engagement.

3. Twitter's GraphJet analyzes hundreds of millions of tweets to extract the best 1500 tweets for each user request and runs approximately 5 billion times per day with an average completion time of under 1.5 seconds.

4. The "For You" timeline consists of 50% In-Network Tweets and 50% Out-of-Network Tweets on average, with around 15% of Home Timeline Tweets served through Twitter's graph traversal heuristics.

5. Twitter's SimClusters identifies over 145k communities, updated every three weeks, ranging in size from a few thousand users to hundreds of millions.

6. The entire recommendation pipeline consists of three main stages and uses various heuristics and filters to ensure a balanced and diverse feed, including visibility filtering, author diversity, and content balance.

There could be many reasons why Elon Musk open sourced this massive code base but having access to this code is a great resource to learn and build.

Check GitHub code repo link https://github.com/twitter/the-algorithm if you are interested.