1,000 Web APIs

John Musser, November 3rd, 2008   Comments(36)

Last week ProgrammableWeb crossed one of its biggest milestones thus far when we added the 1,000th web service API to our API directory. This is a long way from the 32 APIs we started with back in the summer of 2005. Back then even the phrase “web mashup” was only a few months old.

And the change is not just in quantity of APIs, but in quality, variety, rate of growth, and most of all, impact. Over the past 3 years, APIs have shown they can be truly disruptive in a wide range of online markets. We’ve seen this with the introduction of ground breaking APIs like:

• eBay’s API, the first open API of this class starting back in 2001 and now over 60% of eBay listings come from their APIs to the tune of over 6 billion API calls per month.
• The original Amazon API from 2002, the eCommerce API (now called the Associates Web Service), and today with innovative API-driven cloud services like Amazon S3 and Amazon EC2
• Salesforce.com, the industry’s leading enterprise API and one that should make most SaaS vendors understand why they need to offer an API
• The Google Maps API which for many people is synonymous with the word mashup
• Flickr API, the inspiration for hundreds of photo mashups
• YouTube API, also used in hundreds of mashups and now nearly every online video provider has followed suit by offering APIs
• Facebook platform, whose launch 18 months ago shook-up the world of social networks and now comprable “platforms” are available on most socnets
• Twitter, whose API does 10x the traffic of their web site

One of the trends we’ve seen in open APIs is that within a given market segment, once one API becomes a huge success (like Google Maps or Facebook), or a sufficient number of leaders offer APIs (like in eCommerce), that a competitive or defensive reaction occurs and soon everyone else in that sector feels like they need to offer an API. It can be a domino effect. Within a year after the Facebook F8 launch of their platform nearly everyone from Google’s Orkut to MySpace offered open APIs. Likewise this has happened in mapping, video, photo, music, messaging, voice (the telcos are opening-up, often begrudgingly), and even in the news business (with new APIs for NPR, The New York Times, and lots of other news and media outlets coming soon).

How do these 1000 APIs break down by type? The following chart, derived from our database, shows the the top 15 sectors or markets with the greatest number of competing API providers. As you can see there are already 71 mapping-related APIs alone:

As each of those largest API markets have grown, we’ve added new Topic Areas on ProgrammableWeb to track the news, APIs, mashups and sub-trends:

• Government
• Mapping
• Mobile / Telephony
• Music
• Photo
• Shopping
• Social
• Video

From a technology perspective, where are we on the API front? For one thing, the success of open Web APIs has had a big impact on the rise of RESTful design patterns. To provide some light on this from an open API perspective, we’ve created a new mashup, using our own ProgrammableWeb API and the Google Chart API, to show the distribution of protocol usage by API across our directory. As you can see, 63% of the APIs in our directory are REST-based (and including Atom, also RESTful, there’s more).

And what was API number 1000 in our directory? It was the New York Times Community API, a notable new API provider, one that besides giving you all the news that’s fit to mix, symbolizes just how much the world is opening-up.

Share This

500 Photo Mashups

John Musser, October 21st, 2008   Comments(0)

While map mashups in many ways defined this genre of application, the second most popular type of web mashup here on ProgrammableWeb are photo mashups. How popular? Just this past week the number of photo-related mashups passed the 500 mark, and there are now 505 listed.

And even though there are now 48 photo APIs, it’s still Flickr and their ever-popular API that rule this segment. There are now 382 Flickr mashups listed here. The next most popular photo APIs include ImageLoop, Panoramio and Picasa (the latter two being Google products).

Lately we’ve seen a trend in mashups that combine music APIs and photo APIs. Here is a sample of three from this month:

• Flokoon: Flokoon is a visual search engine that lets you explore websites in a Flash-based visualization. Uses APIs including Last.fm for music data and YouTube for video. More details at our Flokoon mashup profile.



• TuneChimp: TuneChimp lets you discover the best music from any artist: Listen to top tracks, learn more about them through videos, photos, bios, news, and lyrics. This mashup uses APIs including Yahoo Boss, Last.fm and Flickr. More at our TuneChimp mashup profile.



• SoundPushr: Pick and collect items while surfing through last.fm artists, YouTube videos, Flickr shots and Amazon albums. More details at our SoundPushr mashup profile.

Track this topic at our photo API and mashup dashboard.

Share This

5 Best Practices for Mashups

Andres Ferrate, September 11th, 2008   Comments(3)

A wide array of content and functionality has been incorporated into the ever-growing number of mashups out on the web today. From enterprise mashups to proof-of-concept hacks, developers and would be developers are leveraging the power of mashups to provide information in new and compelling ways.

Mashups are still a relatively new phenomena, and as this new type of online application evolves it will become increasingly more important to ensure that your mashup adheres to a variety of best practices. Summarized below are five key best practices that you should strive to use in the development of your mashup.

Security

Perhaps the most important yet often overlooked part of a mashup is security. Mashups are inherently Web 2.0, which means that your mashup will likely rely on AJAX. AJAX is a great set of web development techniques, but it also poses a avriety of risks for end-users, including cross-site scripting (XSS), Cross-Site Request Forgeries (CSRF) and JSON Hijacking, and more. The Open AJAX Alliance has a great white paper on AJAX and Mashup Security that outlines specific security risks and best practices to address these risks. IBM and AJAXWorld have additional information on AJAX security as well.

In addition to securing mashups against AJAX vulnerabilities, end-user information also presents another security risk. A good number of social networking and social media sites provide APIs that allow third parties to access information for a specific user or users.

Consequently, there is a risk that a mashup may expose user credentials (including passwords, emails, etc.). The best way to avoid this scenario is to use open web standards such as OAuth or OpenID for user authentication/authorization. Regardless of whether either of these standards is used (or the site that provides the API supports them), a mashup should be designed to minimize the exposure of user information as much as possible. The Open Web Application Security Project (OWASP) has a wiki that includes a Guide to Authorization, and Niall Kennedy provides a good primer on the subject as well.

Compatibility

Mashups are applications, but they are also web sites. Depending on the target audience, a mashup may have reach across various types of users, who utilize different browsers and operating systems. An enterprise mashup developed in-house for a specific browser and operating system used throughout an organization will be less likely to require application logic to address differences in browsers (and differences within the same browser on different operating systems). However, a mashup developed for public consumption is likely to be used by a diverse base of end users, each with browsers that may require server-side and/or client-side modifications to your mashup code.

Whether it be CSS, JavaScript, HTML, or server-side scripting, chances are that modifications may need to be made to accommodate how a browser renders and interacts with a mashup. Apple’s Developer Center summary on web page development best practices provides some good guidance that is relevant for mashup developers.

Performance

As with compatibility, a mashup’s client-side and server-side performance can greatly affect an end user’s experience. On the server-side, site performance can be compromised under high loads related to complex database queries, high user loads, or memory-intensive server-side scripting. Invest time in optimizing queries and code, and balancing user load.

Additionally, a mashup may load more slowly on certain browsers, depending on the content type and whether there is any JavaScript code being processed. A great example of this is seeing a browser freeze when trying to load a large number of map markers using the Google Maps API. Gear mashups towards limiting memory-intensive processes in the browser, and break up data/information into smaller sets using strategies such as paging.

Attribution

As evidenced by our API Directory, there is an immense amount of content that is accessible via APIs. Each API typically carries terms of use that specify who can use content, how the content can be used, and how the content should be attributed. There are several attribution licenses out there, including Creative Commons and the GNU Free Documentation License.

Developers should be aware of the terms of attribution for any content, and adhere to the attribution method specified by those terms. Note also that some API methods include query parameters that can be used to specify what type of content should be retrieved (e.g., Creative Commons Share Alike vs. privately copyrighted).

Legality

In addition to attribution, developers should be aware of the terms of use for APIs used for mashups. Some APIs, such as the Google Maps API, have terms of use that explicitly state that the API can only be used for “services that are generally accessible to consumers without charge.” Other APIs, such as Yelp’s API, have terms of use that prohibit use of “the Yelp API or any Yelp Content in a manner that is directly competitive in nature with the Yelp Site”. Be aware of how an API’s terms of use will affect development and accessibility of a mashup, and consider the potential legal consequences of violating a provider’s terms of use.

A mashup should also take into account other legal considerations, including copyright violations (just because its accessible via an API doesn’t mean that the content may not be infringing upong copyrights), content for mature audiences, and accuracy of information; be sure to use relevant disclaimers and site policies to address these legal considerations.

There are certainly other best practices that are emerging in this space. Do you have additional thoughts on these best practices or additional ideas for mashup do’s and don’ts? Please share them in the comments section below.

Share This

292 Flickr Mashups

John Musser, December 18th, 2007   Comments(4)

The Flickr API continues to be one of the most popular Web 2.0 APIs and with a flurry of new photo mashups here lately, we now have 292 Flickr mashups listed. Overall they run a very wide range of creative applications, here are three of the most recent entries:

• flickrbackup: A Java-based Flickr backup utility that lets you back up all your photos on Flickr.



• Feel Image: Search multiple photo sharing sites. Has technology that quantifies colors like red or blue in images, making it possible to show search results by color. Search phrases like: red flower in spring.



• Custom iGoogle Skins: This new version of the skins gadget integrates Flickr and iGoogle to being you random images from a user specified Flickr group.

Share This

Man on the Street Video: What’s a Mashup?

John Musser, December 4th, 2007   Comments(1)

At last month’s Mashup Camp Europe over in Dublin, Ireland the Yahoo Developer Network (YDN) team of Chad Dickerson and Tom Hughes-Croucher brought along their Y! video cam and journeyed out to see if they could get Dublin’s citizens and visitors to answer the question “What’s a mashup?”. From the dry delivery to the semi-censored responses, the fun video “Man on the Streets…of Dublin, we hit the streets to see what a ‘mash-up’ is” gives you the answers:

Share This

Can an API Steal Data?

John Musser, October 9th, 2007   Comments(6)

In a very lively forum thread over at Flickr there’s a discussion/debate about the Flickr API, data ownership, copyright, and mashups. In a nutshell, a Flickr member, Austen Haines, noticed that some of his photos were appearing in the mashup Adactio Elsewhere even though he had flagged them All Rights Reserved (ARR). The mashup developer, Jeremy Keith, replied and noted that this was just the behavior of the API and that it “sounds like there’s a glitch in the system”. The discussion is still ongoing, and the initial thread kicked-off a second thread, this with the provacative title Flickr photos stolen by the thousands through the Flickr API. (And interesting to note that our Adactio mashup profile is one of the earliest mashups listed on ProgrammableWeb and is consistently ranked in the top 20 of our all-time most popular mashups.)

Read the rest of “Can an API Steal Data?” »

Share This

Mashup Acquired by The Weather Channel

John Musser, September 12th, 2007   Comments(7)

Demonstrating that a sufficiently talented solo mashup developer can succeed in building an application worthy of purchase by a major corporation, David Schorr’s terrific Weatherbonk was aqcuired by The Weather Channel Interactive yesterday. The mashup does a great job of integrating disparate sources like weather and traffic cams, news reports, multiple data sources, and historical averages. You can even put in a travel route and get weather forecasts that follow-along your path.

Read the rest of “Mashup Acquired by The Weather Channel” »

Share This

Twitter API Traffic is 10x Twitter’s Site

John Musser, September 10th, 2007   Comments(24)

In a very interesting interview with Twitter co-founder Biz Stone, Sean Ammirati at Read/WriteTalk asks some good questions about the role of the the Twitter API in their success and plans going forward. Two things that jumps out is that Biz’s comment that the API has 10x the traffic of the website and that of all that’s happened with Twitter in the past year that “the amount of activity around the API has been the most surprising experience”. (Our Twitter API profile here.)

Read the rest of “Twitter API Traffic is 10x Twitter’s Site” »

Share This

Monetize Your Google Maps

John Musser, August 3rd, 2007   Comments(5)

In a somewhat quiet but noteworthy move the Google Maps API now gives developers an officially sanctioned way to overlay advertisements directly onto their maps. And share the revenue. This was announced by Google’s Pamela Fox in the maps API newsgroup. You access this new functionality through an API object named GAdsManager (see the reference guide here). “A GAdsManager object fetches AdSense ads and displays them on a specified map. Ads show up as GMarkers and can be clicked on to bring up the ad within the marker’s info window. The GAdsManager selects AdSense ads based on the current viewport and the surrounding textual content on the page.” More from Pamela’s post:

At Developer Day in May, we previewed GAdsManager, a class that would place contextual ad markers for local businesses in a special layer on your map and help you monetize it. We’ve now released the ads layer with the addition of GAdsManager in Maps API v2.85, and it’s ready for early testers.

I’ve put together a demo of the ads layer here: http://imagine-it.org/google/gmaps-samples/adslayer/adslayer.html

After loading that page and waiting for a few seconds, you should see a white marker show up-this is an ad in the ads layer. If you click on the marker, you should see the text and logo of this ad, for a hotel. If you pan to New York City, you’ll probably see another hotel ad show up. Since this feature is in its very early stages, our local ad inventory is small and you might notice that only a limited number of ads show up. As we roll out this program to more advertisers, we expect to see an increase in ad inventory and thus an increase in the number of ads that show up in the ads layer on your map. Currently, the feature only shows ads for businesses in US. Apologies to our (many) international developers, who can, however, still implement the ads layer in advance of international ad inventory becoming available.

The groups conversation on this topic includes some good questions:

• Q: Do I need an AdSense account to use this? A: Yes.
• Q: Is it available in Maplets? A: Not yet, but likely.
• Q: Is there a way to ‘tell’ or ‘hint’ what you want the ads to be about? The issue is, when it’s “just a map” there’s less textual context for Google’s AdSense algorithm’s to work from. A: Not yet, part of the concern is how to prevent gaming the system.
• Q: Can AdSense Channels be used to segment across maps? A: Yes.

With services like this and Lat49 it looks like there’s going to be increasing array of options for mashup developers to monetize their applications.

Share This

Digg Mashup Contest Winners

John Musser, June 27th, 2007   Comments(1)

Digg has announced the winners of the Digg API Visualization Contest. You can read about them here on Digg. Ten finalists were selected by the contest team and then Digg community members used their diggs to vote for the winners. The winner? Digg City, shows the 10 newest popular stories. The more popular the story gets, the taller the building. When someone diggs the story a stick figure of that user is added. Figure goes inside the building he has just Dugg.

You can see the second and third place winners in our earlier coverage here and see the Digg API Profile here.

Share This