The problem with scaling Twitter is not the choice of framework, but the choice of architecture. In other words, abandoning Ruby On Rails probably isn't going to solve all of their problems. At the same time, I'm not sure that decentralization is necessary and comes with its own set of challenges.

The real problem is that the polling model of the Twitter API doesn't scale for real-time communications. As it is today, many Twitter clients will poll (by default) every 3-5 minutes to see if there was an update. Not only are they not really participating in real-time, they are generating an enormous number of requests that - even while each payload is small - generate a lot of overhead in aggregate just in checking and responding. As I've suggested in the past, a better solution would be to move the "real-time" API around the Jabber/XMPP client instead of HTTP.

The good news is that Twitter already has an IM presence so it's easy enough for third-party clients to tie in directly by implementing their own Jabber bot. There are countless libraries out there that make this very easy - I used the excellent Jabber.NET for the Notches Twitter bot and it was quite simple to implement. Obviously, it's still not quite as simple as building an HTTP-based client, but it's far from complex and there are certainly things Twitter could do to make it even easier. First, they could provide more structured output through an API bot to allow "easier" parsing (i.e., not tied to the language intended for users that can change). The other thing necessary to make this a viable alternative is to allow sending to phone AND IM - as it stands now, you have to choose one or the other. Instead, clients should be able to register with Twitter (based on an API key + maybe even a machine-specific identifier) and users can turn off and on specific clients to receive those events (which obviously could be done through the API as well so a client could turn itself on when it's opened).

Personally, I think Twitter should do all it can to help migrate third-party clients to this architecture. I would even suggest they consider lowering the request limit in a few months from 70 per 60 minutes to, say, 35 to further encourage use of the XMPP interface.

The other thing that prompted this post was the announcement on Friday's Gillmor Gang that Twhirl is going to roll out Jabber/XMPP support. I'm still amazed that this announcement didn't get more press. Frankly, I'm surprised more Twitter clients haven't done this already, but if they want to stay competitive with Thwirl they absolutely must react. (I suggested this already to the Witty team, and though I don't have much time I might take a crack at implementing it if no one else does). First of all, using XMPP enables true real-time interaction in a rich client and, because it's not subject to rate limiting, you can send unlimited updates as well. Second, and perhaps most importantly, it gives you access to the "track" command which is not exposed on the HTTP-based API.

My guess is that the Twitter team just didn't anticipate us using their product to have these real-time conversations, but in any case it's time to get off the HTTP API.