This guest post comes from Simon de la Rouviere, a masters student in Socio-Informatics at the MIH Media Lab (at Stellenbosch University) and founder of Tweekly.fm and TwimeMachine. When he is not on the web, he is making music.
During undergrad, I worked on a few side projects, with most of them using some sort of social-networking API. It was great learning some new skills and at the same time it was rewarding to tap into the vast networks of established sites such as Facebook and Twitter. Cue to postgrad and I had to complete a research component as part of my course requirements. With a nudge from my supervisor I asked the question: “Is it viable to have a generic social networking API? Will it be useful?”
Each social network chooses their own standard of how the data must be structured, how the API functions, how developers must interface with it, etc. As a newbie developer, getting grips with each API has its hurdles. Often times, new developers (myself included) start by using an external library instead of resorting to using something like
curl. These external libraries for each programming language in turn require more effort to learn and understand. The possibility of it working properly is not guaranteed as well.
For a senior developer who has worked with several APIs and doesn’t necessarily need external libraries, learning a new social-networking API still takes some time. Flickr’s API is very different from Facebook’s graph API and YouTube’s GData API is difficult to discern if you just want some simple data. As an example, see all the data it returns for a video.
There is an issue of dealing with different social-networking APIs. They are different and require effort to learn each one. It would be easier for a developer to learn only one API instead of several. The question was whether the resulting API would end up being too generic and therefore not useful.
So given these problems, I set out to create an initial prototype of a generic social-networking API. The scope was as follows:
- Find features that are common to most social-networks (i.e. API methods that most social-networks have).
The common methods/endpoints used ended up being “user”, “user_feed” and “search”.
- Design new generic formats for this common data/resources/features.
Twitter stores their user data in a different format than Facebook, although the concept of a ‘user’ is still the same. One might hold more data than the other. I settled for a format which holds the most basic information. Part of the research was to test if this was enough.
- Limited to GET requests.
The scope becomes much larger when you have to deal with POST requests (storing authentication tokens for each user and so forth).
- Determine a set of social-networks to use in the research.
Based on social-network size and API usage, Facebook, Twitter, Last.fm, Flickr and YouTube were used in the prototype.
So it basically worked as follows:
The URI format was in the form of “/api/resource/service/parameters”. So fetching user information for a Twitter user would look as follows: “/api/user/twitter/simondlr”. This would return basic user information from Twitter in JSON. Sending a GET request to “/api/user_feed/lastfm/troegie” would fetch all the recent tracks I listened to.
After this was done, I drafted up some questions to send to developers, testing two important constructs. Is the concept useful? Would developers want something like this? Secondly, how good is the implementation?
The results came back as follows. Developers found the concept useful. There is a desire to simplify the social-networking API experience, especially when dealing with various APIs. The implementation I created needs work. Developers found that more data and more functions would be useful.
What does it all mean?
As social-networking APIs explode and their data usage continues to flow across the web, a standard way of interfacing with them would be beneficial. Social networks are homogenous enough to provide basic API services that function the same across them all. Whether this solution is through a separate interface, or through standardization of the format (such as what activity streams are proposing), will be interesting.
This implementation has its flaws, including dealing with SLA’s, uptime, dealing with intermediate tokens and rate limiting. However it shows that there is a desire to battle with this problem that will increase as more APIs start permeating the web.
You can find the test site at shotbeak.com/genapi (I took some time to make it user-friendly so that most developers can understand how it works).
Covering it all in a blog post is difficult, so if anyone has questions, I’ll answer them in the comments.