Brett Slatkin on PubSubHubbub

Continuing in the series of entries derived from my notes taken during Mozilla Labs October meetup, this post covers Brett Slatkin’s talk about PubSubHubbub. PubSubHubbub is a simple publish/subscribe protocol for turning Atom and RSS feeds into real-time streams.

The publish/subscribe paradigm is commonly found throughout software engineering. In essence, a component of a system publishes events as it performs its tasks. For example, a video encoder would post an event when it has completed encoding.

Separate components register as subscribers, and receive notice when an event occurs. In our example, a content management system could take the encoded video and publish it to a website. Meanwhile, a separate component could email the director to inform him that his video is available online.

A broker in the middle (a “hub” in PubSubHubbub terminology) is responsible for delivering published events to any interested subscribers. This decoupling creates a flexible architecture, in which separate components are isoloted and can evolve independently.

There have been many attempts to define a standard publish/subscribe protocol in the past, including the XEP-0060 extension to XMPP, AMQP, competing WS-* efforts, and RestMS (which I’m unfamiliar with and need to research). None of these protocols have achieved widespread adoption. PubSubHubbub is an attempt which initially appears like it may have a better fate.

Architecturally, PubSubHubbub is decentralized and designed to scale to the size of the web. Because of this, it doesn’t suffer from any slow down or failure at a central point. I found the touting of this benefit quite humorous given the location of the meetup at Twitter’s headquarters.

One of the potential limitation of the protocol, however, is that it is server-to-server only. It operates on the backend, and is not attempting to reach the last mile to desktops or mobile phones. I suspect that this drawback could eventually work in favor of XEP-0060, which could act as a compliment to PubSubHubbub by filling this gap.

Interestingly, Brett remarked on how slowly the web actually changes. He revealed that the egress from the reference hub on App Engine is only 24Kb/second, including all of Blogger, LiveJournal, and FriendFeed. Taken at face value, this is a remarkably low figure.

Post a comment