Ruby and Python as Cloud Lingue Franche; Ruby/Rails on 10gen

Not sure how this one slipped past me, but 10gen announced support for the Ruby language and most of the Rails framework APIs on their open-source cloud service last month.

This addition is great news for 10gen and for cloud computing (the hosted-application-platform flavor, not the hosted-hardware/datacenter flavor).

For 10gen, support for a well-known API and app model is a huge bonus, which makes it easy for people to move an app into the cloud without learning and coding to new APIs, and also lowers the perceived "lock-in" involved, should the move not work out.

Their original JavaScript platform approach, as I've written before, is problematic not only because folks are unlikely to have meaningful (for their business) apps lying around to try mounting in the cloud, but more so because there is no standard server-side JS API set. A half-dozen companies offer a JS app server or cloud and they all have different platform APIs for even the simplest things, such as reading HTTP request variables, or deleting a session.

10gen takes a big step forward, joining Stax, Heroku, and morph labs in supporting Ruby on Rails in the cloud.

This move also reinforces another emerging trend: Ruby and Python serving as lingue franche for cloud app stacks. While many cloud offerings support JavaScript or other languages, Ruby and Python seem to be emerging as the ones with broadest support: 10gen will support both; AppEngine supports Python and a language-to-be-named-later; Stax supports both; Azure will likely support IronRuby and IronPython (some Python apps can already work in Azure).

Of course, the language is only half of the battle -- there are the APIs to deal with as well, and issues will typically arise where the impedance mismatch is highest with cloud-related infrastructure. E.g., cloud databases are mostly non-relational and don't support SQL ... so an ActiveRecord or SQLAlchemy API won't work on 10gen's 'grid database' (a reasonable tradeoff for simpler scalability.)

Even so, it is starting to appear as though one could write a lot of core business logic using, say, Python, and expect it to run unmodified on most vendors' clouds. Not a bad position to be in for the Python folks.

Stax Brings more Standard App Models to the Cloud, Marginalizes Custom Platform Further

Stax, which recently launched in private beta, is a cloud hosting/deployment/ops platform based on Java appservers. The coolest thing about Stax is that it offers many flavors of JavaEE-deployable applications, including Python apps (via Jython) and Rails (via JRuby) with ready-to-roll built-in templates.

Stax has a very AppEngine-y feel, not just on the website, but in terms of the SDK interactions, local development, etc.

This is good news for all of the popular platforms ... and bad news for those rattling around the corners with non-standard APIs. As the app-hosting industry continues to mature, the emphasis will clearly be on established players like Rails, ASP.Net, JavaEE, Pylons, et al. at the expense of guys like AppJet.

It's not about the language (JavaScript) but the about learning a set of APIs, patterns/practices, and sustaining a community ... based on a roll-your-own platform.

It is true that some of these built-for-the-cloud platforms were designed from the start to default to hash-style or big-table style storage -- popular for content-oriented cloud apps because of its easy horizontal scaling -- where the "traditional" platforms focus on relational stores and have a variety of bolt-on APIs for cloud DBs.

But now that there are so many standard alternatives, it is unlikely developers will pay any custom-platform-tax no matter how elegant that platform might be.

AppEngine is Pretty Much What I Was Talking About

None of the people I know at Google had leaked me any info whatsoever about AppEngine.

It's pretty much the environment I wrote about when I talked about simplifying cloud computing. AppEngine could stand some more simplification and standardization, but it's a a rockin' first step.

Although it was tempting to be a little self-satisfied, discussing something like this initiative a month or so before launch means ... well ... discussing it many months after it was already thought of, planned, coded, and up and running inside of Google, so humility is probably a better approach.

Stored Procedures and Code in the Cloud

For modest-sized Internet applications, the allure of cloud services has two elements.

First, there's simplicity of implementation and maintenance -- the hassle of real-world ops is the sort of problem startup CEOs dream of having, while startup engineers (and engineering budgets) are ill-equipped to deal with it. Second is the promise of easy scalability -- another problem the CEOs dream of, and the engineers secretly hope will become Somebody Else's Problem.

Storage in the cloud is conceptually easy. Especially with ActiveRecord patterns that ignore (at their own peril, but that's an article for another day) 35 years' worth of learnings about data integrity in the relational model. And for those who need to write things more complicated than 37signals' latest masterpiece, true structured data services in the cloud are coming.

What about application logic, though? There's raw EC2, which works on the level of provisioned VM images, and makes you design for clustering, manage your instances (while they're up), and keep your dynamic data somewhere else. Fabulous infrastructure but non-trivial to use.

Folks like heroku have value-added application-level services above EC2, which offer the elasticity with less hassle.

But what about going even higher level, and defining a unit of work, or a service module that can be deployed into a scalable container, preferable "nearby" the data it needs?

Real world example: in a recent project, I needed to be able to run Dijkstra's algorithm on big (250,000+ nodes) graphs in a persistent store. It would be great to use SimpleDB or SSDS (Astoria) for storage, but what about running the algorithm? It's not practical to extract a representation of the graph over the network, and then run Dijkstra on it just to find some interesting nodes each and every time. Changing the algorithm or using a different one? Maybe ... But what I really wanted to do was create a small module that I could ship over to the data, and run there. Even better, I'd like to be able to compute on the data "in place" in a storage facility, rather than extract. Conceptually a bit like a stored procedure.

I believe that the solution -- and an easier way to start shipping computing into a cloud facility -- is to create a module definition that one can code to, and then just upload. I think Python or Ruby would be ideal languages as they are popular, truly cross-platform, and not encumbered with a IP issues. Plus the modules would be provided as source, so that they could be scanned for, e.g., insecure or computationally intensive uses of stuff like eval. In fact, given Google's investment in Python and in interesting tooling in general, they may already be most of the way there.

I just need a better name than "stored procedures" -- that one's not getting any points for cool.