March 27th, 2008

Amazon Enhances EC2, Embraces Failure

Amazon’s web services team has just announced two enhancements to their successful EC2 cloud computing service that makes it even more desirable and useful by making it more fault-tolerant: the ability to place instances in multiple locations and “Elastic IP addresses”.

As you can see below, both of these are key pieces of the puzzle in how to build dynamic systems in the cloud that can deliver both scalability and fault-tolerance. As O’Reilly Radar’s Jesse Robbins points out “Datacenters and geographic regions are Single Points of Failure (SPOF) too. Failure Happens, and it’s far better (and cheaper) to build services that are resilient to failure than to try to prevent them from happening. This is a big step in the right direction.”

• Elastic IP Addresses are static IP addresses designed for dynamic cloud computing, and now make it easy to host web sites, web services and other online applications in Amazon EC2. Elastic IP addresses are associated with your AWS account, not with your instances, and can be programmatically mapped to any of your instances. This allows you to easily recover from instance and other failures while presenting your users with a static IP address.
• Availability Zones give you the ability to easily and inexpensively operate a highly available internet application. Each Amazon EC2 Availability Zone is a distinct location that is engineered to be insulated from failures in other Availability Zones. Previously, only very large companies had the scale to be able to distribute an application across multiple locations, but now it is as easy as changing a parameter in an API call. You can choose to run your application across multiple Availability Zones to be prepared for unexpected events such as power failures or network connectivity issues, or you can place instances in the same Availability Zone to take advantage of free data transfer and the lowest latency communication.

Amazon’s blog post points to a useful series of new articles on EC2 by the team at RightScale: DNS and Elastic IPs and how they come in to play when upgrading a server, setting up a fault-tolerant site using the Availability Zones, and now support the new Elastic IP and Availability Zone. Includes good details, walk-throughs and illustrations like below.

Share This

Posted by John Musser as Amazon, Infrastructure at 11:43 AM | 1 Comment »

March 20th, 2008

Amazon Launches Fulfillment Web Service

Today the Amazon Web Services team announced their new Amazon Fulfillment Web Service (Amazon FWS). As Amazon’s Jeff Barr explains in Our Most Fulfilling Web Service Yet this API “allows merchants to tap in to Amazon’s network of fulfillment centers and our expertise in logistics. Merchants can store their own products to our fulfillment centers and then, using a simple web service interface, fulfill orders for the products. That’s right – make a web service call, ship a product to a customer!” Our new Amazon Fulfillment API profile has an overview and more technical details.

Overall the service has two complementary sets of APIs: Inbound and Outbound:

The Inbound service gives merchants the ability to create and send shipments to an Amazon fulfillment center (FC). The getInboundShipmentPreview function is used to locate one or more Amazon FCs to receive a particular product. The service may choose to send products to a variety of FCs in order to balance supplies across the entire network. Next, the putInboundShipment function is used to inform Amazon that the merchant will be shipping the product to the indicated FCs. Once the products have been shipped, the setInboundShipmentStatus function is used to inform Amazon that the product is actually on its way.

The Outbound service gives merchants the ability to ship products from Amazon FCs to their customers. This service revolves around the concept of a fulfillment order. The order contains a destination address, a shipping speed, and a list of item/quantity pairs to be shipped. The createFulfillmentOrder function is used to initiate the shipping process. There are also functions for listing all orders, cancelling orders, and getting the status of an order.

The APIs themselves are SOAP based. In order to use them you’ll need a merchant account with Amazon because only merchants can list an item and you need an item to sell in order to have Amazon fulfill it.

With 15 different APIs available, Amazon continues to expose both more and more of their core eCommerce services as well as their hosting infrastructure, and in so doing continue to push the web service frontier.

Share This

Posted by John Musser as Amazon, Shopping at 1:35 PM | 2 Comments »

March 18th, 2008

45 Mashup Contests and Counting

If you know how to develop mashups then you may be in line to win some very big prizes. To get a sense of just how much money and how many prizes, take a look at the ProgrammableWeb Contest Guide and you’ll find that there have been more than 45 mashup contests thus far. Prizes include Xbox 360 systems, $10,000 Alien ware computer systems, $50,000 in cash, and more. The contest prizes are getting bigger and more spectacular all the time. For example, in September 2007, Adobe awarded a $100,000 “trip of a lifetime” to the winner of the AIR Developer Derby. In December, startup Ooyala won the $100,000 first prize in Amazon.com’s AWS Startup Challenge (see our earlier post about the contest).

In March the contest with the biggest prize to date gets underway. The winner of Salesforce.com’s Force.com Million Dollar Challenge will receive a $1,000,000 investment in their start-up company and a cubicle at the Salesforce.com incubator for one year. The most recent contest announcement was just last week when Zynga announced a competition for game developers using their platform.

Winning Mashups

So, what types of mashup have people developed to win these contests? With more than 45 contests already having completed, the winning entries have spanned a broad spectrum, as you’d expect. Quite a few prize-winning mashups provide services that are of everyday value to people. For example, the Home Locator mashup, winner of the Adobe Flex Developer Derby, lets you search real estate listings with photos and maps (mashup profile).

Need a doctor after hours? As we reported last year, the After Hours Doctor’s Office mashup, winner of the 2007 Etel Mashup Contest, transcribes office voice mails left by patients for doctors into text and then sends them via SMS to the doctor. It’s great demonstration on how to get in contact with your doctor when you know she or he is not in the office.

The PamFax mashup , which won the 2007 Skype Mashup Competition, lets you send a fax to any fax machine in the world, paying with your Skype Credit (our profile).

Meanwhile, if you’re wondering about campaign finance and influence in your state, take a look at the winner of the Sunlight Foundation’s Mashup Congress Contest. The Unfluence mashup will show you your state’s political contribution data (our profile).

So, what stories are others identifying as being very interesting and relevant? What better source for this information is there than Digg? To our benefit, Digg offers the Digg API to developers, and many people have taken advantage of this to develop some very useful mashups. The Digg Expose mashup takes Snap.com images from Digg and displays them in a configurable view. You can drag the images around, sort them, or change the category. The Digg Charts mashup (profile), another Digg API Contest finalist entry, is a Flex application that generates charts comparing popular stories. Additionally, a graph is generated showing a selected story’s popularity over time. See all finalists in the Digg API Contest in our earlier coverage.

And the Ooyala application that won Amazon.com’s AWS Startup Challenge is really a platform and not a simple mashup. Ooyala provides capability for improving delivery, monetization and analytics of online video, utilizing Amazon web services.

Stay Posted

Mashup and API contests are clearly a growing venue for developers and API providers. Use the ProgrammableWeb Mashup Contests Guide and the Contests blog page to keep posted on ongoing and upcoming contests. If you know of a contest that isn’t yet listed, click the “add it” link on that page to share the information with the PW community.

Share This

Posted by KevinFarnham as Amazon, Contests, Gov, Money at 2:28 AM | No Comments »

February 7th, 2008

Scale Facebook Apps With Amazon Services

Amazon and Facebook have clearly been on the leading edge of two forces driving the web as platform - cloud computing and exploitation of the social graph. Now these two companies have announced a partnership that lets Facebook developers leverage both innovations by giving developers a path to employ the Amazon cloud computing model based on the EC2 and S3 APIs. This has the ability to provide an infrastructure for developers who dream of building a wildly viral Facebook application and worry about how to handle the growth.

Amazon’s EC2 provides the metered compute capacity for creating virtual instances, and S3 the storage component. The partnership with Facebook doesn’t extend either of their respective APIs, but it offers a a set of resources including a step-by-step process to launch a Facebook app using the Amazon platform.

Facebook developers had already one solid option with one-year-free hosting from Joyent which partnered with Facebook in December and it’s likely we’ll see a variety of such app infrastructure services appearing in the next few months.

Share This

Posted by John Musser as Amazon, Facebook, Infrastructure, Social at 1:48 AM | No Comments »

January 31st, 2008

Amazon Web Services Make Earnings News

The just released Amazon Q4 2007 earnings report, besides showing that the company doubled profits this quarter, had a couple very interesting notes related to their growing suite of web services. The first is that bandwidth usage from their web services exceeds that of their web sites:

Adoption of Amazon Elastic Compute Cloud (EC2) and Amazon Simple Storage Service (S3) continues to grow. As an indicator of adoption, bandwidth utilized by these services in fourth quarter 2007 was even greater than bandwidth utilized in the same period by all of Amazon.com’s global websites combined.

This is somewhat reminiscent of the point at which Salesforce.com began to process nearly as many transactions via their APIs as through their application proper.

And the second piece of news from that division is that “Over 330,000 developers have registered to use Amazon Web Services (AWS), up more than 30,000 from last quarter.” These two details go hand in hand with more developers signed-up and more usage of the core pay-as-you-go infrastructure services.

You can read more on the Amazon results from BusinessWeek’s Rob Hof, paidContent, TechCrunch and Laurie Flynn at the New York Times.

Share This

Posted by John Musser as Amazon, Infrastructure, Money at 1:21 AM | 2 Comments »

January 2nd, 2008

Amazon DevPay: Resell Your Web Services

The Amazon Web Services team just ended an impressive year with one last innovation: Amazon DevPay. DevPay builds on Amazon’s strengths in running both online shopping services and online web services to create a business infrastructure to support developers using their web services like S3 and EC2. It helps simplify the process of billing and tracking for apps that use these pay-per-use Amazon APIs, essentially enabling a reseller model (DevPay also includes it’s own licensing API and you can see our a API profile here).

As they describe on the AWS blog:

This new service allows entrepreneurial developers to wrap their own business models around Amazon S3 and Amazon EC2, taking advantage of Amazon’s existing customer base and billing infrastructure. With DevPay, developers can focus on being creative and innovative while dispatching the less-than-glamorous aspects of dealing with bank accounts, credit cards, and so forth to us.

Developers use DevPay’s web-based registration interface to create pricing plans for their applications, monitor customer signups, and track usage. The developer’s customers use another web-based interface to sign up and enter payment information for the applications that they wish to use.

You can think of DevPay as an enabling technology for our other services.

There were some initial questions about the differences between the Amazon Flexible Payments API and DevPay, which the AWS team has clarified here.

Using Amazon Flexible Payments Service (Amazon FPS), developers can accept payments on websites. It has several innovative features, including support for micropayments.

Amazon DevPay instruments two Amazon Web Services to enable a new sort of Software as a Service. Amazon DevPay supports applications built on Amazon S3 or Amazon EC2 by allowing you to “resell” applications built on top of one of these services. You determine the retail price, which is a mark-up above Amazon’s base price. Customers pay for your application by paying Amazon. We deduct the base price plus a small commission; then deposit the rest into your Amazon account.

It’s another intriguing building block in the web services infrastructure stack. The cost? “Your customers will be billed for usage of their DevPay-powered applications on the first day of each month. We will then deduct a 3% fee plus another 30 cents, and deposit the remainder in your DevPay account.”

One of the first S3 customers to begin using this service is Amanda Enterprise, a supported version of the Amanda open source backup and recovery tool which now use DevPay along with Amazon S3 to backup, archive and retrieve customer data. See Amanda profile here.

Looks like SmugMug might start using it soon as well.

Share This

Posted by John Musser as Amazon, Infrastructure, Money at 12:28 AM | 2 Comments »

December 31st, 2007

Amazon’s EC2 Open Source Firefox Plugin

Developers using Amazon’s EC2 API might find this interesting: Amazon has created an open source project on SourceForge for ElasticFox, their Firefox extension that lets you create and manage EC2 instances from a GUI in the browser. The utility lets you launch AMIs (Amazon Machine Instances) and then find, control, replicate and shutdown running instances. A handy right-click menu lets you quick launch more like ones already running. Having the source available will let developers extend this as needed. For more see the Amazon AWS blog and at our mashup profile here.

Share This

Posted by John Musser as Amazon, Infrastructure, Source at 12:03 AM | 2 Comments »

December 17th, 2007

The Amazon SimpleDB API

Amazon has once again lead the industry by launching their latest infrastructure API, SimpleDB, a programmable database in the cloud (you can see more at our new SimpleDB API profile). It’s a forward thinking approach for a pay-as-you-go, scalable database that is very much in line with Amazon’s other popular infrastructure services like the S3 API for storage and the EC2 API for virtual computing. As they describe it:

Amazon SimpleDB is a web service for running queries on structured data in real time. This service works in close conjunction with Amazon Simple Storage Service (Amazon S3) and Amazon Elastic Compute Cloud (Amazon EC2), collectively providing the ability to store, process and query data sets in the cloud. These services are designed to make web-scale computing easier and more cost-effective for developers.

Traditionally, this type of functionality has been accomplished with a clustered relational database that requires a sizable upfront investment, brings more complexity than is typically needed, and often requires a DBA to maintain and administer. In contrast, Amazon SimpleDB is easy to use and provides the core functionality of a database - real-time lookup and simple querying of structured data - without the operational complexity. Amazon SimpleDB requires no schema, automatically indexes your data and provides a simple API for storage and access. This eliminates the administrative burden of data modeling, index maintenance, and performance tuning. Developers gain access to this functionality within Amazon’s proven computing environment, are able to scale instantly, and pay only for what they use.

Pricing is based on machine utilization ($0.14 per Amazon SimpleDB Machine Hour consumed), data transfer ($0.10 per GB - all data transfer in, $0.18 or less for data out), and structured data storage ($1.50 per GB-month).

Like S3, there’s “Simple” in the name for a reason as it’s not aiming for multitudes of features but rather focuses on performing a core infrastructure service well. For example the core data structure is like a hash or dictionary, not a full-blown relational model. There’s already been lots of discussion and debate about some of the tradeoffs here like the schemaless model, the 1024 character limit per attribute, and the need to zero-pad integers because queries are lexigraphical (see TechMeme for more). It’s likely that the outside developer community will probably build wrappers, libraries and frameworks to work around and adapt these.

It’s very clear that database storage in the cloud is a service that will eventually be offered by most of the major API providers. In order to track this we’ve now added a Database line item to our API Scorecard.

Share This

Posted by John Musser as Amazon at 12:39 AM | 1 Comment »

December 10th, 2007

Ooyala Wins $100K Amazon API Prize

Last Thursday at Amazon headquarters in Seattle, in a bid to win what might be the biggest web services-related prize to date, teams from 7 companies using Amazon Web Services APIs competed for $100K in the AWS Startup Challenge. As we covered earlier it was a tough competition with over 900 entrants and these finalists presented not just to Amazon representatives but venture capitalists from around the country including Bay Partners, Blue Run Ventures and Battery Ventures. And the winner? Ooyala, a startup focused on improving delivery, monetization and analytics of online video (see our full Ooyala mashup profile here).

As the grand prize winner, Ooyala will receive $50,000 in cash, $50,000 in Amazon Web Service credits and an investment offer from Amazon.com. You can see all good concise video summaries of each finalist on Amazon’s site. What Amazon APIs does Ooyala use? Lots. Here’s what they use and how:

• Amazon EC2 as a computing platform for video and analytics processing
• Amazon S3 for storage and conent delivery
• Amazon Flexible Payments Service for payments
• Amazon Simple Queue Service for some of their video processing
• Amazon Mechanical Turk for marketing research and even choosing their logo

Read the rest of “Ooyala Wins $100K Amazon API Prize” »

Share This

Posted by John Musser as Amazon, BestMashups, Contests, Popular at 3:19 AM | No Comments »

December 6th, 2007

Shopping Mashups for the Holidays

Of the the 2565 mashups listed here just over 9% are tagged “shopping”. That’s 338 shopping mashups. Each week we see a variety of the 33 e-commerce APIs used for interesting forms of shopping applications and this past week’s been no exception. The e-commerce API segment is so active that we’ve created a new section of ProgrammableWeb: the Shopping Mashup Dashboard. This section of the site gives you one jumping off point for what’s new in APIs, mashups and news in the e-commerce platform world.

In terms of what’s new in the past week, here are 3 of the latest shopping-related mashups that might be able to assist you in getting ready for the gift giving season:

• SearchQuilt: Search for videos, images, blogs, news, eBay auctions, and Amazon products all on one page. The homepage shows popular searches from Yahoo, AOL, Google, and Technorati, so that you can also track buzz (four different APIs used in this mashup).



• Amazon and eBay Comparison Shopping: Search both Amazon and eBay at the same time from a single search form.



• eBay Nintendo Wii Ticker Widget: If you are tired of looking for Nintendo Wiis then use this Google Gadget, a live ticker that displays Nintendo Wiis currently on sale via eBay.

Share This

Posted by John Musser as Amazon, Popular, Shopping, Site News, ebay at 1:06 AM | No Comments »