Mashup Patterns: Clipping Mashups

Michael Ogrinz, February 11th, 2009

mashuppatternsEditor’s note: Today’s post comes from Mashup Patterns author Mike Ogrinz who continues our coverage of enterprise mashup patterns.

Most mashups rely on some type of API that’s freely provided by a public web site. ProgrammableWeb tracks thousands of these resources across dozens of categories. Generally, these interfaces are SOAP or REST-based, but they may also work in cooperation with other open formats like RSS or Atom. In an enterprise setting, mashups have a more diverse set of protocols to potentially leverage including JDBC/ADO.NET (databases),  SMTP/IMAP (email), and SNMP (network monitoring). Unless you are building a data mashup, one of the participants API’s is usually focused on visually representing the data. It could be the classic Google Maps API, or perhaps some type of charting (Google Charts is a great resource).

We typically expect a mashup to involve multiple APIs. Otherwise, what’s the value-add? 1+0=1, right? Actually, there is a special case where we can mashup a resource against itself to provide interesting solutions. This single system doesn’t need to expose an API, and it often handles its own presentation. These types of mashups rely on a feature (or “Core Ability” in the parlance of Mashup Patterns) known as Clipping.

Whereas a normal mashup operates on the data that underlies presentation (either a preexisting API, or if using the API Enabler pattern, a web site’s Document Object Model or DOM), Clipping is “skin deep”. It lifts the rendered content right from a page, JavaScript, CSS, and all. Usually there is some type of intermediary layer which rewrites links, handles Ajax, etc. dynamically and re-serves the page to the consumer via a new web address. But the best part is, during this re-routing, the mashup creator has a chance to strip certain elements from the rendered content, or inject new ones.

This type of mashup is an off-shoot of traditional “portal clipping” products. Most portal vendors (IBM, Oracle, JBoss) have historically offered basic clipping capabilities. These non-mashup Clipping tools are aimed at creating JSR-168 compatible portlets. JSR-168 is a specification created under the Java Community Process (JCP) to provide a standard model for portlet development and deployment. These portlets are designed to run within an overarching portlet container.

While some portlet-centric products like the open-source portletbridge offer rudimentary abilities to change CSS rules and apply XSL transforms, mashup products that offer Clipping as either a server-side feature (Kapow, Convertigo) or on the client (MIT’s Chickenfoot Firefox plugin provides comparable features through scripting) take the technology to a new level. Imagine, for example, being able to republish a web page’s UI with your own custom enhancements as a standalone product.

For example, consider this very simple internal corporate phonebook:

phonebook-before

http://phonebook.yourcompany.com

Without access to the source code, a Clipping mashup allows you to inject the HTML for an iFrame whose src attribute is a servlet that receives the employee name harvested from another part of the page. The servlet spits out the employee’s picture from a different resource (maybe pulls it in from Flickr or LinkedIn). Your Clipping can also add links to an internal Skills database, or list what Change Requests they have pending. This is the basis for the Usability Enhancer pattern.

phonebook-after

http://betterphonebook.yourcompany.com

I should also mention that like their portal-clipping relatives, most mashups can expose JSR-168 portlets, too (Portal Enabler pattern). This allows Clipping mashups to be easily integrated into corporate portals.

Of course, if the phonebook app already exposed an API, you could accomplish all of this by duplicating its interface and wiring all this up manually; but who wants to bother doing that? Whenever the original site’s look-and-feel changed it would be remediation time! A Clipping-based mashup picks up the changes automatically.

When the reasons for using Clipping are motivated by critical bugs that need to be corrected, we move in to the realm of the Field Medic pattern. Imagine a mission-critical CRM system (hosted internally or externally) where the latest version crashes every time the “Fax #” field doesn’t contain an area code. Traditionally, you’d put up with the bug until the vendor issued a patch. This example is relatively benign. What if the bug put a firm’s data at risk or caused lost sales? A Field Medic implementation could capture the existing interface, and inject JavaScript validations where necessary to reject/correct bad inputs. In this case, the JS is normally contained within the mashup itself so technically only a single site is actually involved.

There are all kinds of productivity, risk, and reputational issues that Clipping mashups can address. One of my favorite examples is MIT’s Kangaroo, which works with Gmail. By injecting the photos of all of the people on your recipient list, you have immediate visual feedback that helps prevent embarrassing “reply-to-all” gaffes.

googleexample1

Clipping makes it possible to built powerful single-site mashups that capture updates in the source-system’s interface automatically. But be warned; this process it not without its risks. You might accidentally include ads, disruptive Javascript (for example, frame-busting scripts) or annoying applets. You can lessen the risks of the “malicious hitchhiker” anti-pattern by dynamically removing HTML tags that enable unwanted content.

Both comments and pings are currently closed.

3 Responses to “Mashup Patterns: Clipping Mashups”

February 11th, 2009
at 4:21 pm
Comment by: Data Extraction? Clipping? API? Screen Scraping??- Explained with Tutorials and Links! | WebServiceable

[...] I found Michael Oginz originally from a post about clippings on programmableweb [...]

February 20th, 2009
at 1:09 pm
Comment by: David Davies

Great explanation of one approach to mashups.

To me, the point about how you “could” duplicate the user interface and wire it up manually is a key one. Mashup builders will never want to go that route – a successful mashup requires that user interfaces are prime mashable assets, not an afterthought to a data-driven approach.

July 1st, 2010
at 11:13 am
Comment by: Lucent buys ProgrammableWeb.com

[...] membership in the New York CTO Club, and from having written/collaborated on a few articles (also here and here) for PW back when my book was released. He’s a great guy who has done a lot for this [...]

Follow the PW team on Twitter

ProgrammableWeb
APIs, mashups and code. Because the world's your programmable oyster.

John Musser
Founder, ProgrammableWeb

Adam DuVander
Executive Editor, ProgrammableWeb. Author, Map Scripting 101. Lover, APIs.