Twitter API Calls Doubled Since April: Now Serving 70,000 Every Second

Curtis C. Chen, September 16th, 2010

TwitterHave you ever wondered how much traffic Twitter handles in a given day, or what software sits behind the curtain of the popular service?  A recent presentation reveals some of the answers. Twitter’s incredible growth becomes obvious when you compare the recent numbers to those announced at Chirp.

On September 9th, Twitter’s university recruiting team stopped by UC Berkeley to talk about the company and what it does. The slides from platform engineer Raffi Krikorian’s talk, Twitter by the Numbers, are now online, and they disclose some fascinating technical details about the social media giant’s operations.

Twitter serves over 70 million tweets per day, totaling over 12GB of tweet text alone.  Many of those messages are delivered to client apps and web sites through the Twitter API to the tune of six billion API calls per day (double what was announced at Chirp in April), or about 70,000 API calls per second.  All told, the service generates 8TB of data every day, which is eight times more than the New York Stock Exchange.

Pop quiz, engineers: Your web service needs to deliver real-time message traffic to an asymmetric digraph of over 150 million users.  What database do you use?  WHAT DATABASE DO YOU USE?

    • Shoot the hostage
    • Oracle
    • MySQL
    • Write your own database

      If you’re Twitter, the correct answer is #4:  Create your own database software, call it FlockDB, and release it on github.  (By the way, if you picked option 1, maybe software engineering isn’t the right career choice for you.)

      FlockDB is just one of the home-grown, high-performance software systems Twitter uses to support its tremendous growth.  Others include:

      • hosebird, a “near real-time” streaming API back-end (instead of REST, which is only “pseudo real-time”); and
      • snowflake (also on github), a network service to generate unique IDs at high scale (MySQL couldn’t keep up, and was a single point of failure).

      With a stated goal of supporting “half the world and all its devices,” Twitter faces many engineering challenges.  This peek under the hood (full slides are embedded below) shows that they’re aware of the potential problems, and are working hard to steer clear of the fail whale.

      Via Delyn Simons

      Both comments and pings are currently closed.

      8 Responses to “Twitter API Calls Doubled Since April: Now Serving 70,000 Every Second”

      September 16th, 2010
      at 7:47 pm
      Comment by: Twitter API Calls Doubled Since April: Now Serving 70000 Every Second | Erence Teknoloji - Teknoloji Portalı

      [...] is the original post: Twitter API Calls Doubled Since April: Now Serving 70000 Every Second BENZER YAZILARRemember That DM on Twitter? So Do Your Apps: Tech News «Itsy Is a Simple, [...]

      September 17th, 2010
      at 12:17 pm
      Comment by: The Latest from TechCrunch | Y Not Share

      [...] Yesterday, Twitter engineer Raffi Krikorian presented a number of new stats about the microblogging service as part of a recruiting talk at UC Berkeley. One interesting stat involves the number of API calls that the site is seeing: Krikorian says that Twitter is seeing 6 billion API calls per day, or 70,000 per second. That’s more than double the amount (3 billion API calls) that Twitter was seeing in April, according to the Programmable Web. [...]

      September 17th, 2010
      at 1:01 pm
      Comment by: Twitter Seeing 6 Billion API Calls Per Day, 70K Per Second » Tech Reviews

      [...] Twitter Seeing 6 Billion API Calls Per Day, 70K Per Second Yesterday, Twitter engineer Raffi Krikorian presented a number of new stats about the microblogging service as part of a recruiting talk at UC Berkeley. One interesting stat involves the number of API calls that the site is seeing: Krikorian says that Twitter is seeing 6 billion API calls per day, or 70,000 per second. That’s more than double the amount (3 billion API calls) that Twitter was seeing in April, according to the Programmable Web. [...]

      September 17th, 2010
      at 3:01 pm
      Comment by: Twitter API Calls Doubled Since April: Now Serving 70,000 Every Second (Curtis C. Chen/ProgrammableWeb) | Home Stereo Equalizer

      [...] C. Chen / ProgrammableWeb:Twitter API Calls Doubled Since April: Now Serving 70,000 Every Second  —  Have you ever wondered how much traffic Twitter handles in a given day, or what [...]

      September 17th, 2010
      at 3:05 pm
      Comment by: DVRHDMI » Twitter Seeing 6 Billion API Calls Per Day, 70K Per Second

      [...] Yesterday, Twitter engineer Raffi Krikorian presented a number of new stats about the microblogging service as part of a recruiting talk at UC Berkeley. One interesting stat involves the number of API calls that the site is seeing: Krikorian says that Twitter is seeing 6 billion API calls per day, or 70,000 per second. That’s more than double the amount (3 billion API calls) that Twitter was seeing in April, according to the Programmable Web. [...]

      September 17th, 2010
      at 5:40 pm
      Comment by: Twitter Seeing 6 Billion API Calls Per Day, 70K Per Second | TechGoo

      [...] Last week, Twitter engineer Raffi Krikorian presented a number of new stats about the microblogging service as part of a recruiting talk at UC Berkeley. One interesting stat involves the number of API calls that the site is seeing: Krikorian says that Twitter is seeing 6 billion API calls per day, or 70,000 per second. That’s more than double the amount (3 billion API calls) that Twitter was seeing in April, according to the Programmable Web. [...]

      September 17th, 2010
      at 9:47 pm
      Comment by: Twitter Seeing 6 Billion API Calls Per Day, 70K Per Second | freedork.org - austin, texas, news feeds, current events, opinion

      [...] Last week, Twitter engineer Raffi Krikorian presented a number of new stats about the microblogging service as part of a recruiting talk at UC Berkeley. One interesting stat involves the number of API calls that the site is seeing: Krikorian says that Twitter is seeing 6 billion API calls per day, or 70,000 per second. That’s more than double the amount (3 billion API calls) that Twitter was seeing in April, according to the Programmable Web. [...]

      October 7th, 2010
      at 7:26 pm
      Comment by: Square Trash Receptacle (35-Gallon) Beige (RCP 3958) | Compactor Trash Bags

      [...] C. Chen / ProgrammableWeb:Twitter API Calls Doubled S&#1110&#1495&#1089&#1077 April: Now Serving 70,000 E&#957&#1077r&#1091 Se…  —  H&#1072&#957&#1077 &#1091&#959&#965 &#1077&#957&#1077r wondered &#1211&#959w [...]

      Follow the PW team on Twitter

      ProgrammableWeb
      APIs, mashups and code. Because the world's your programmable oyster.

      John Musser
      Founder, ProgrammableWeb

      Adam DuVander
      Executive Editor, ProgrammableWeb. Author, Map Scripting 101. Lover, APIs.