Adobe Time-Warps Half a Decade Back, Will Still Probably Defeat MSFT

Earlier this week, I went to see a couple of folks from Adobe present their latest progress on Flash Catalyst, Flex "Gumbo," and the "Spark" UI component framework.

As someone who does a bunch of Flex work, I liked everything I saw.

Especially since it was the second time around.

No, I didn't see this stuff at MAX, I saw it at Microsoft PDC in 2003 and 2005.

It was shocking how pleased Adobe seems with itself now that it's almost ready to release a design tool that generates XML and RIA code... since everything they showed -- and more -- was part of the earliest Microsoft Expression Blend alphas that I saw years ago.

The Microsoft product was code-named "Sparkle." But we won't get this confused with Adobe's "Spark" because (1) "Spark" refers to a different bit, Adobe's re-invention of lookless, templated controls, which Microsoft implemented in WPF and shared with the world at the time (around '04 or '05), and (2) because Expression Blend is already out in a 2.0 version, so unlike the Adobe products, it doesn't need a codename anymore.

Adobe even has yet another XML dialect to facilitate moving design assets through the workflow -- it's called "FXG." And it appears to supplement MXML quite well in specific areas, so that if you take MXML and add FXG, you get XAML. Not that XAML was de novo or anything -- the XUL and Java folks (desperate to stop writing Swing code) had been creating similar XML formats for a while. The Java community was especially fond of XML with tons of imperative programming constructs mixed in alongside data objects and calling it "simple and declarative." What XAML did was provide all the necessary power, while keeping it declarative.

Anyway ... Adobe should get credit for recognizing the right way to do this when they saw it. Namely, they realized which workflow tools were needed, embraced the idea of export from Photoshop and Illustrator to a vector markup with a visual editor with timelimes, and thence to an RIA build tool with a code-oriented IDE.

Now that they're finally getting this on track, Adobe is even more likely to trounce Microsoft in the RIA world. They have penetration numbers that MSFT can only dream of, and for a company that doesn't build real developer tools they're giving it the college try.

Which is kind of sad, since I believe Silverlight is a better technology with better language and tool support ... and not any less rather more open than Flash.

Want Help With Your Startup? Let It All Hang Out on Craigslist

It's awfully easy to go looking for folks doing stuff the wrong way ... and to find it. So it's nice to be surprised by someone doing something amazingly, shockingly, frighteningly ... right!

I was greeted by a craigslist ad in my RSS reader today, one of many startups looking for folks to, essentially, work for free. I've written about why this is a bad idea before, and it's still a bad idea.

But there's a little more to this ... the poster (the company's founder presumably) posts a link to a wiki. Maybe it's genius, maybe a trainwreck -- either way I had to look.

On the other side of this link is a company wiki. An explanation of what the company is building; where they are in the process; their calendar; UI mockups with notes and the comment stream by the creators; and other items.

This is absolute genius, and it's so rare. Plus it shows the guts that most entrepreneurs fancy themselves to have, but lack when tested. I'm not commenting on their specific business/tech idea, I haven't thought much about that to be honest.

But it is so refreshing to see someone out there on the beach letting it all hang out as it were.

I work with a lot of entrepreneurs and most of them think that they're the first ones to think up some genius idea, and the best way to be successful is to either keep it stealthy and secret, or to sign reams of NDAs and non-competes with you before disclosing (cue music) their subtle and delicate brilliance.

Just writing that last paragraph, it's a struggle to keep a professional tone. These folks are usually (97%, there are a couple of specific exceptions) complete fools. And truly, they are fooling themselves, unconsciously trying to avoid exposing their idea to someone who might not think it's so good, or who might point them to the dozen other people doing the same thing. Generally speaking, the secrecy ends up being a contributing factor to their failure. Which, since startups are highly failure-prone anyway, they will deny anyway.

That's why I was so thrilled to see this post. The founder is saying, "If you want to try and 'steal' my idea, you go ahead. But if you really believe there's a bunch of money in it, wouldn't you want to work with other people who believe the same thing and who have the will to execute? And if you go off with it and succeed anyway ... you're still helping me because you're establishing the category, while I plan to work nights and sweat blood to execute better and faster than you."

The ad is reproduced below. I was going to link it, but interestingly it has been 'flagged' for removal from craigslist. It's hard to imagine why -- the whole scenario seems rather more legitimate than the typical ad in the category. Perhaps the allusion to potential full-time work disqualifies it from the free "gig" listing ... but I think a startup seeking essentially non-paid volunteers in whatever capacity they can afford qualifies as a part-time or temporary arrangement.

Technical Wizard / Web Developer Wanted | Internet Startup (sunnyvale) An internet startup is seeking a highly talented web developer If you have experience with either: PHP/MySQL, Python, or Ruby we would love to talk with you. This a very exciting startup opportunity with massive potential. At this stage, we are looking to bring aboard those who are seeking equity share in the company. We simply do not have the capital to fund salaries. For more information, please have a look at: http://wiki.kunsoom.com All of the pertinent information will be included in the wiki page. Thanks for your interest in the project! We look forward to hearing from you.

Adobe and Microsoft Get Into It Like Children on the Playground

A week or so back, Adobe exec Mark Garrett got a bunch of attention for insisting that Microsoft's Silverlight effort has "fizzled."

Microsoft promptly screamed back that it wasn't so, pointing to the inauguration video stream, and a few other factoids.

What makes this truly schoolyard funny though is what happened today when Adobe "announced" it was bringing Flash 10 to phones. This seems just as dubious as Microsoft's oft-repeated plan (since as far back as '05, when it was WPF/e) to get Silverlight onto mobile phones ... by last year ... which obviously didn't happen.

Meanwhile, for years, Adobe has been pushing a weak technology called FlashLite for mobile ... and for a variety of reasons it has never been a usable option for content providers to deploy Flash content or apps.

For both Microsoft and Adobe, for both PC and phone applications, the critical metric is current "content-ready" penetration. How many devices are ready to run new Flash/Silverlight content off of the web today.

In this 2x2, the only square that's solidly covered is Adobe's Flash on the PC. "Ready" penetration of Flash 9+ is near 100%.

On PCs, existing install base is critical because of locked-down corporate networks that won't allow end-user installs. Microsoft needs to stop talking about download numbers, or numbers of people who "can access a PC with Silverlight," and start doing anything it can to get these ready penetration numbers up.

On mobile, the barrier is user confusion over configuration. Vendors could push the updates to phones, but in nearly 10 years of smartphones, only Apple has done much of this. Windows Mobile 6 has an updater ... and in over a year I don't recall it ever updating a darned thing.

Flash Lite trumpets a large "installed base," but these are strange installs, where the runtime (but not browser integration) is baked into the phone, and there's no reasonable way to get new Flash content onto the phone, either via web pages or download.

Both of these players are big on bluster and have been for a long time. Meanwhile, developers are left with few options for all of the smartphones in the world that don't an apple on the back.

Is It Too Early or Too Late for an Open RIA Design/Dev Toolchain?

I was playing with the Raphael JavaScript graphics library (a sort of script-based, cross-browser, implementation of SVG) and started thinking how helpful this library would be in creating a browser-based (as opposed to plug-in based) RIA.

That lasted for about 15 seconds before I remembered that creating large, non-trivial RIAs generally involves designers, and most designers don't like creating vector art by coding a set of "path" statements, or animations as a collection of key-value pairs and millisecond-based transition times.

That's why tools like Microsoft Expression, and Adobe Illustrator, Catalyst, and Flash exist.

And why Adobe and Microsoft are investing so heavily in the designer-developer workflow: the ability of designers to turn graphics and animations into app skins and interaction which are immediately available to coders.

In order for an open RIA solution to be competitive and realistic -- whether it's open in the pure-browser sense, using JS via dojo.gfx, or Rapael, etc., or whether it's via an open plug-in (Java/JavaFX seems like the closest, though it's not 100% open yet and may never be) -- this full toolchain needs to exist.

We need to be able to export vector art from mainstream design programs such that they can be incorporated as assets into the RIA. It doesn't matter if this is via SVG, XAML, AI/EPS, or something else entirely. What does matter is that the import/export is robust enough that designers -- whose jobs, after all, include making stuff look just right -- are confident that what they design is what end-users will see. The Microsoft and Adobe tools can do this. To date most OSS attempts cannot.

Next up, we need a truly usable, designer-friendly authoring tool for animations and interactions. It is often argued that some standard tools (*cough* Illustrator *cough*) are not paragons of usability themselves. No matter -- it's hard enough to get converts.

Happily, there seems to be emerging some consensus among the big vendors about how these tools should work (both on-screen and in terms of in intermediate data formats). That blueprint lowers the risk and challenge for an open source contender.

The biggest obstacle remaining is a classic open-source triangle-of-trouble:

1. The toolchain/workflow will not be viable until it is quite solid, since the commercial alternatives (Flash, mainly) are so entrenched.
2. It's hard to get enough contributor man-hours against such a huge project without an active user base.
3. Since the user base is not developers, the bootstrapping for #2 that makes many OSS projects work (devs are tolerant -- even excited -- about getting up on an 0.1 release) is unlikely.

The Second Law of Thermo-Specifications

A couple of days ago, Adam Milligan posted an eponymous "law" on the Pivotal Labs blog.

It includes a corollary stating, "The full definition of correct behavior of code exists in the tests for that code."

Now, there seemed to be something fundamentally off about this proposition ... and I wanted to figure out what that was.

Surely Mr. Milligan doesn't mean this in the trivial, tautological sense (define the spec to be no more than what test cases happen to exist at a point in time) ... even though Agile dogma often borders on that view, pretending it's some kind of paradoxical path to enlightenment like a Zen koan.

The comments to his post start to touch on "spec" vs. "test," and whether 100% test coverage is practical, desirable, or conclusive.

Of course even 100% code coverage, with a missing code path and a matching missing test and behavior spec equals ... test success, and wrong functionality.

In such a case, though, the problem has been externalized from the Agile context and put onto a faceless "business person" who "doesn't get" Agile because he or she actually wants to plan ahead and describe some certain specs that persist over time.

This is a neat trick. In physics, all sorts of magical things can happen if you look only inside of one context (or frame of reference) without looking at what's happening outside, or at what's happening to the frame itself. In finance, there can certainly be a free lunch ... if you can make its cost into an externality and remove it from your model.

Once you make the tests and code-based behavior spec a part of the application you're building -- and clearly, once it's a real cost center as well as a critical deliverable part of the project, you have done so -- then you are in a sense simply externalizing that troublesome human interaction (specification, functional analysis, planning). Yes, the tests match the code under test. But seen at a different level of abstraction, it's just another flavor of interface-driven development, or debugging.

Most engineers would love to realize the dream of self-describing, self-verifying code, whether that description be some kind of formal model, or a textual DSL (as with Microsoft's Oslo), or a set of tests and code-based behavior specs. And, indeed, these systems all improve the transparency of the code, propagating requirements from the outside in, and revealing when they are not met.

But even with the most "bought-in" business stakeholders, it is impossible to escape the the outermost specification layer, the one with the humans in it.

Windows 7 Keeps Suspending in VMWare? Change Your Power Settings

I'll admit, I fell right into this one.

My Win 7 VMs kept suspending while I was off working on another machine. I'd come back to the host machine and see VMWare Workstation idling with the Win 7 instance frozen in suspend.

Hmmm... Doh!

The default power configuration for Win 7 -- even when on the machine is "plugged in" -- includes dropping into sleep mode after half an hour.

This, thanks to VMWare, translates into a "Suspend Guest" operation.

Maybe Vista shipped a default power configuration like this. I don't remember it being the case, but, to paraphrase Pulp Fiction, "[Vista] may [save power] like pumpkin pie, but I'd never know 'cause I wouldn't [run] the filthy mother****"

Think About It: Do You Really Want Your Engineering Done "For Free"?

I enjoy reading the posts on craigslist, wherein naive entrepreneurs go looking for free software engineering talent. I know this sort of ad (no pay / equity only) bothers some people -- as though it somehow has real impact on the real jobs and pay in the industry -- but I think it's just fine. I mean, if you want a service for free, there's nothing wrong with asking. And if you are in the mood to work for free, it's nice to know where to find opportunities ...

Of course I am not referring to recruiters for genuine charity / volunteer work, where the goal is to provide services to a population that isn't served by the normal market mechanisms. Rather I'm talking about real for-profit, we're-gonna-be-the-1-in-100-successful-startup-and-make-a-boatload-of-cash kinda company.

The funny thing is that it's hard to imagine these wantrepreneurs looking for any other kind of professional services for free. Not because they feel coerced by social norms to offer money, but simply because they know better than to want what the "free professionals" have to offer.

I don't see these guys saying:

• I need heart surgery and I'm looking for a doctor who wants to keep their resume fresh. No compensation, but I'll thank you if I wake up.
• Going through a brutal divorce. Looking for a sharp lawyer to keep me from losing my shirt -- no pay, but I'll buy you a drink if we come out ok.
• Need a corporate lawyer and patent attorney. Equity only.
• Startup needs business travel on the cheap. Seeking a pilot and aircraft, not so concerned with FAA licenses or airworthiness.
• Building a new facility -- need architect, structural engineers, environmental compliance, project managers, and laborers. If we make a bunch of money, we'll pay you at some point in the future.
• Love my new car, but it needs engine work -- looking for a mechanic who'll work for free. Someone with little or no experience seeking to build up a resume is ideal.

Now, it is true that in the late 90s, during the dot-com bubble, landlords, attorneys and other did take equity. But they did so in addition to -- not instead of -- cash payment. And the odds of making bank on equity back then, while still long, were stunningly better than they are today.

I know what these guys are thinking. They're thinking "I just don't have a budget to pay for real development, and anything I can get for free is better than the nothing I can afford otherwise."

Well, in some cases, perhaps. But that's setting an awfully low bar. Likely too low to allow for a real chance of success. That is, the entrepreneur is theoretically pouring all his time, money, heart and soul into starting this business. But he's willing to gamble the whole thing on the questionable code he'll get out of someone who has nothing better to do to pad a resume while unemployed? Doesn't quite add up.

Or maybe he really wants a "partner" ... only it's unlikely this developer is going to see 50% equity. And anything less likely means the developer is along for the ride while the entrepreneur follows the same hare-brained decision process that led him to advertise the non-paid gig in the first place. In any case, the developer has no real clout when push comes to shove. He's a silent partner, just along for the ride.

True, there are a few brilliant technologists out there who simply don't need any (more) money. They have cashed out of a startup, or they run another business on the side that covers their expenses. And if you can hire one of these guys, then bully for you. But I think they are more likely to spend their time where they can make an impact on a real business, product, open-source project, or non-profit.

As for the entrepreneurs who get someone to bite ... they don't know what they're in for: a few months or more down the line, they have (perhaps) some kind of a half-baked system. They need the loyalty of that unpaid assistant to modify and improve the code, while the assistant has now seen through all the "about to receive funding" promises presented at the outset. And as the limitations of the half-baked system slowly become clear, the founder may wish to replace the developer, or throw the whole thing out and start over with a new iteration.

But unless the original developer was a real sucker, all that equity means unrecoverable dilution in the cap structure. The more equity involved and the earlier the start (meaning smaller valuation), the worse this situation turns out to be. It could even come to pass that a potential investor one day passes on the deal because of these problems in the cap structure.

Is that Service Really a Scalable Cloud or Just Full-Service Web Hosting?

A lot of cloud stacks, or cloud app platforms promise scalability for your app, "With a little EC2 in every box!" (TM). There is a big catch and a little catch, though, and if your app gets big, then either or both of these may be a deal-breaker.

First, and most important: Running a vanilla RDBMS (e.g. MySQL) in a VM somewhere does not make it magically scalable. Read that sentence one more time.

Some cloud offerings integrate tightly to the traditional sort of DB instance you might attach to your web app on a single server. Examples include Heroku, which applies your Rails migrations to a PostgreSQL instance, and Stax, which offers MySQL.

The great thing about these environments is that they don't require significant changes to your standard app built on their supported platforms (mostly Rails and Java variants). Upload, minimal admin, and IJW (it just works).

That's turn-key, full-service web hosting, right there. It's beautiful -- in fact, in an OO and Rails course I wrote, I chose Heroku for deployment as a way to let students get something up and running on the web without getting into the operations/deployment/tuning aspects of Rails which deserve their own course.

But if your app gets large -- or just uses large datasets -- the database is rapidly going to be a bottleneck. Scaling out an app logic tier to a dozen EC2 instances automatically may sound good, but it won't do a thing for a DB-bound app (it may make it worse). And these databases don't scale out without a little architecture, planning, configuration -- all of the things which these cloud platforms are designed to avoid. And which, on some platforms, you cannot do at all.

For example, so far as I can tell on Heroku or Stax, there is no way to even configure multiple servers and replication, which is just a minimum starting point for scaling a DB to multiple machines. Stax may allow for a logical sharding setup, but it's not clear how one would control which VMs and disks the databases run on. Rightscale seems like the kind of firm which would specialize in the management scripts / meta-API that one would need to automate sharding, but the sharding option doesn't appear in any of the models on their website. With replication, which Rightscale does offer (though they're not exactly an app platform, more an infrastructure play), you get to this, still limited, picture:

Other cloud platforms offer datastores specifically designed to scale out, including Google App Engine, 10gen, and others. These platforms offer a non-relational or pseudo-relational datastore, with different data access APIs and a variety of restrictions relative to what you may be used to. These datastores are architected to scale easily, but there are real tradeoffs that must be considered. In fact, if you don't know these tradeoffs cold, you are not the right person to be making this platform decision. Get on craigslist and hire (or borrow) someone who knows the stuff.

The other catch is that whichever approach you choose, these vendors are offering you convenience, some outsourced operations management, and (in some tiers) elasticity and scalability ... but they are not offering cheap compute cycles. That is, if you know you'll need a large, predictable amount of raw compute time, then know also that you're paying a premium to do that computation in one of these environments.

A friend who has designed, built and operated feature film renderfarms for a number of studios confirmed that he has, on a semi-regular basis, analyzed the costs of remote VM-based datacenters (e,g. EC2) compared to their physical ones. Because the studios use these machines intensely, and are consistently consuming raw compute power, the local physical servers have always made more sense.

What does this have to do with your web app and datastore? Well, suppose you have designed your app to leverage a scalable datastore. These may not be tunable, may not perform fast, and may require you to do certain operations in code which traditionally are done in the DB. You may never see these slow queries or operations ... until they show up in your bill. That is, if the system is truly elastic and scalable, it will apply resources as needed to handle your work. If your query or sort or filter takes a lot of CPU cycles, the cycles will be made (almost) instantly available, so the user always sees your app perform well. And then you'll pay for all those cycles or instances at the end of the month.

Either way, there is no free lunch on the data persistence side. Which is not in itself a reason to avoid cloud environments. But it should be a bigger part of the conversation than it is today. And it absolutely must be part of the conversation, if larger businesses are going to move their services into the cloud.

Using AppEngine -- Or Similar Datastore -- To Integrate Complex Legacy Data Formats

I gave a lightning talk last night at the SF Bay Area App Engine Developers, showing some work I've been doing to represent gnarly legacy records in AppEngine so as to maintain source fidelity, minimize upfront analysis, and make them easy to integrate with other systems.

I had started with an XML record that I wanted to parse and represent in the datastore -- without knowing which tags and structures would be present, since this format had, ahem, evolved to obscurity over time, as often happens with real-world legacy records.

Before I talk about my approach, here's why I thought this effort might be interesting to the group: a lot of data structures have a tree structure in common with XML. From C structs and file blocks that include a header, telling which types to cast the next n bytes to (and so on inside of those) ... to mainframe "structured data" records I've encountered which consist of nested records, parsed recursively, with their meanings occasionally opaque, lost to history, or belonging to some partner company.

My approach -- which is simply to create a mapping of how to assemble and disassemble the records -- enables a record to be stored in a single App Engine record. But not as a block (or blob) -- rather with fine-grained addressable fields that are easy to talk to using the GAE Datastore API.

In my case, since my original was XML, I created a mechanism similar to a tiny subset of XPath describing the sequence of tags where a data element lived -- but with the characters changed so that it would be Python and GAE-friendly. That is, instead of "/foo/bar[2]/baz" I used _Foo_Bar__2_Baz.

This let me "flatten" the XML into a set of key-value pairs, while allowing that the XML might contain arbitrary structures injected by others ... and that I might want to inject my own extra structures. This arrangement is perfect for the Expando models in App Engine Datastore, or any similar store (e.g. Hypertable, which is modeled after BigTable, or Microsoft SQL Data Services which uses SQL 2008's sparse tables to similar effect).

So now I can store and retrieve my records. Any fields/subrecords which I understand and care about, I can easily work with from other systems, by mapping to the appropriate "key" in the stored record.

For example, if I'm storing a bunch of catalog data, and another system just cares about enumerating each "Product" with "Name" and "Price," then I can create a facade or wrapper in GAE that maps, say, Price to _Strange_Old_Way_To_Represent_Current_Price, and we're all set.

To be sure, there could be performance issues if you tried to use this to create arbitrary queries and reports against the data. That's not really the purpose and, in my experience, if there are no "shortcuts" to processing these legacy records, then the business folks are not used to being able to make an OLAP cube out of them either. (They probably have a batch or offline extraction process.)

Nonetheless, it's another tool in our chest when we need to work with systems and data that have been out in enough real-world battles to come home scarred with lots of cruft.

Twitter's Underwherlming (Former?) Architecture Problem

I recently came across this post from May 2008 comparing Twitter traffic and the Options Price Reporting Authority data feed. Needless to say, the stock market feed is many orders of magnitude larger, at 700,000+ messages per second(!)

It's also not the fairest comparison in the world on its face, for a variety of reasons: the OPRA data system was planned (Twitter met success more or less by accident), Twitter is minimally funded, etc.

A more relevant comparison, in my opinion, is that provided by newzwag, which presented its performance challenges, triumphs, and secrets at a recent SF Ruby meetup.

newzwag's site and trivia game is built on Rails, started small, and had to grow to meet traffic driven by Yahoo and the Beijing Olympics to 9 million pageviews per hour (using a total of a half-dozen machines or so). And lest you think this is a content site served out of a cache, most of the traffic consists of data writes by game players that then need to be ranked, published, etc.

As far as I can tell, that's somewhat larger than Twitter, even considering that Twitter has grown 3-4x since last May's stats.

newzwag's solutions, which they share here, are a study in sanity, reasoned problem solving, and smart efficient architecture.

Without the timelines or resources of a stock-market app, newzwag produced a nice solution that -- at least in hindsight -- appears drama-free.

Interestingly, a newzwag - Twitter comparison can be enlisted to support a variety of different startup social narratives.

One narrative is that an amateur-hour effort yields amateur-hour results, and aspiring startups shouldn't fool themselves into thinking that they won't need old-time Architecture and Sophistication to scale.

A different narrative says it doesn't matter -- if Twitter's success is your worst-case scenario, you still win. That is, build it fast, get it out there where people can try it, and you should be so lucky as to need a real re-arch to fix your scaling problems. In this model, both Twitter and newzwag played it right -- newzwag because they knew the Olympics would provide a narrower time window to showcase their system, so they managed risk against that stricter goal.

And yet another narrative says if you accept these two stories, you still wouldn't want your brokerage transaction flowing through a system built to "see what sticks," and hence Web 2.0 startup methodologies stare at mission-critical business apps from across a huge chasm.

I see this last story as persuasive but also as a big opportunity: there is a chasm, to be sure, but it needn't be quite so big. There are legacy mainframe apps that can speak webservices. Every manager in a big company wants their product to be "100% critical" even if they could create more value by admitting that a lot of nice-to-have two-nines business apps are the real bricks in the wall. If enterprises can get better at separating their Twitters from their OPRAs, they can make more money and have both.

New iPhone App Store Rules Take a Step Closer to Scriptable Apps

A lot of folks commented today on newly-approved web browsers appearing in the App Store. Or, more precisely, a handful of apps using the existing web browser widget to offer a slightly tweaked browser experience.

While iPhone apps could include the UIWebView component before -- and indeed this has proven a popular route to getting hybrid native/web apps up and running quickly -- today's change is about allowing apps that "duplicate" a built-in feature of the phone. And one of the fundamental characteristics of any web browser nowadays is that it is thoroughly scriptable.

If you build an app this way, it already includes a scripting environment ... so the question (since scripting and dynamic apps are verboten on un-jailbroken iPhones) is how far one can let the scripting go and still pass muster with the App Store overlords.

Using stringByEvaluatingJavaScriptFromString we can inject script into the browser ... including script that pulls data back out.

And although the JavaScript bridge is "one directional" compared to the OSX desktop API, there are workarounds such as registering a protocol handler to receive scripted "requests" from inside the page ... or by hooking decidePolicyForNavigationAction with a script-initiated navigation request (disclaimer: I haven't checked to see if this is in the phone API, but it seems plausible) to signal the availability of data.

So native code becomes effectively scriptable. Or, for an even less controversial but perhaps equally powerful route: just inject a bunch of JavaScript API libraries into the browser and keep the scripting (and more of your app) in Safari. That's not too different from pointing a browser at a web site (where the page loads various scripting libraries) ... except that underneath it all we are in native-caps mode ...

Unless I'm missing something here, a somewhat ambiguous situation has gotten thornier with the admission of this new class of general purpose browser apps.

Windows 7 Product Name is Missing a Feature

I didn't feel strongly one way or the other about the Windows 7 product name (i.e. "Windows 7") ... until recently when I wanted to troubleshoot the Azure SDK on Windows 7. (Apparently Azure on 7 has worked with the M3 build for at least one intrepid forum poster, but it's not behaving with the beta build for me at the moment.)

I started searching newsgroups, forums, blogs, etc., and realized that "Windows 7" is not a great search term.

On an engine like Google you can put quotes around it, specifying exact phrase, but some other full-text search systems don't seem to want to keep the Windows and the 7 together. Or perhaps they have an index by single words, and they link the results together to match your phrase later, but once you throw in other terms like SDK and Azure, the matching engine becomes a little more promiscuous, offering you a "promising" combo of Azure, SDK, and Windows ... or SDK and 7 ... as a higher-ranked match. Making it, in any case, rather harder to find what you want.

One-word product names, like "Silverlight," "Vista," and "XP" work a lot better for this kind of search.

Which is perhaps a reason that folks include the release name with the version number on products such as Ubuntu (Hardy Heron, Intrepid Ibex, etc.)

So ... what would be a good nickname to put next to Window 7?

Ruby and Python as Cloud Lingue Franche; Ruby/Rails on 10gen

Not sure how this one slipped past me, but 10gen announced support for the Ruby language and most of the Rails framework APIs on their open-source cloud service last month.

This addition is great news for 10gen and for cloud computing (the hosted-application-platform flavor, not the hosted-hardware/datacenter flavor).

For 10gen, support for a well-known API and app model is a huge bonus, which makes it easy for people to move an app into the cloud without learning and coding to new APIs, and also lowers the perceived "lock-in" involved, should the move not work out.

Their original JavaScript platform approach, as I've written before, is problematic not only because folks are unlikely to have meaningful (for their business) apps lying around to try mounting in the cloud, but more so because there is no standard server-side JS API set. A half-dozen companies offer a JS app server or cloud and they all have different platform APIs for even the simplest things, such as reading HTTP request variables, or deleting a session.

10gen takes a big step forward, joining Stax, Heroku, and morph labs in supporting Ruby on Rails in the cloud.

This move also reinforces another emerging trend: Ruby and Python serving as lingue franche for cloud app stacks. While many cloud offerings support JavaScript or other languages, Ruby and Python seem to be emerging as the ones with broadest support: 10gen will support both; AppEngine supports Python and a language-to-be-named-later; Stax supports both; Azure will likely support IronRuby and IronPython (some Python apps can already work in Azure).

Of course, the language is only half of the battle -- there are the APIs to deal with as well, and issues will typically arise where the impedance mismatch is highest with cloud-related infrastructure. E.g., cloud databases are mostly non-relational and don't support SQL ... so an ActiveRecord or SQLAlchemy API won't work on 10gen's 'grid database' (a reasonable tradeoff for simpler scalability.)

Even so, it is starting to appear as though one could write a lot of core business logic using, say, Python, and expect it to run unmodified on most vendors' clouds. Not a bad position to be in for the Python folks.

Another Windows 7 Milestone: Bounce vs. Hibernate vs. VM Suspend Times

Lately I've started running the Windows 7 Beta for some development experiments, using VMWare's fantastic dual-screen support. As I've written before, the general experience is great, even under virtualization with 1GB of RAM, and having it wall-to-wall on multiple displays makes the illusion more convincing.

An interesting thing I've noticed is that my old habit, when I want to stop working in a VM and free up the resources, is to suspend the VM. This action is roughly (but not exactly, depending on the VM you're using) equivalent to "hibernating" a laptop (S4 power state) -- memory is mapped out to a file and the device is powered off.

I usually do this because this hibernate/wake is faster than a shutdown/boot-up, not because I'm trying to save my actual work state (open apps, etc.) Especially with Windows server, but even with XP and Linux, this approach is the faster way to hop in and out of a work session. On the laptop, it's a way to save the battery power involved in a longish hard boot.

In Windows 7, VM suspend/resume (==hibernate/wake) seems to be slower than shutdown/boot. That is, even with no user apps running (which could take up an arbitrary amount of memory and thus lengthen the map-out / map-in time), boot seems faster. I say "seems" because I have only 2 machines to play with, and they are not clean images just for this test, so I won't pretend they represent absolute objective truth.

What does this mean?

It would appear that the boot process has been cleverly streamlined so that a cold machine gets to a running, usable state before all of the additional services and apps have fully loaded and gotten running, and that this is orchestrated using knowledge that a white-box VM player doesn't have.

Some folks may point out that having to reboot an OS is itself questionable ... and indeed the boot is optional -- I only reboot my XP desktop every couple of months when some security patch or other requires a restart.

But in the world of laptops and netbooks, things are different: every minute of juice is valuable, so there's always the consideration of the cost of a hibernate/wake vs. sleep vs. leaving it on with the LCD off. And that equation has just gotten a little more interesting: for Win 7 on a laptop, if you're not going to be using the machine for a while, it may turn out to be faster and use less power to do a shutdown, and just reboot later.

While this may seem like a fairly inconsequential gimmick about boot times, it is a step in the right direction as we look at the huge array of gadgets we all use and which eat a ton of phantom power. The Windows PC is kind of the holy grail for a fully-off / instant-on experience, and Win 7 appears to take a measurable step in that direction.

Embarrassment of Meetup Riches and a Suggestion for Compelling Talks

In the Bay Area, we are fortunate enough to have so many great tech meetup groups that there are frequent collisions, and it's always a bummer to pass on what promises to be a great presentation.

On January 20, the SF Java group has three sharp guys presenting on Scala.

Meanwhile, down in Mountain View, the Google App Engine group has a "hacking..." talk that with a handful of presentations including an update from Google AppEngine Product Manager Pete Koomen.

What to do? Hmmmm...

Ultimately, I decided on AppEngine.

Principally, it appeared that more new, not-readily-available-today-on-the-net material would be presented at the AppEngine group. Lightning talks give a forum to quirky thinkers, very early startups, and other interesting folks, while representation from the mothership might be able to offer a little detail or timeline on upcoming features, like large BLOB support and the next language.

The Scala talk just seems less likely to include info that I can't get from existing resources.

Which leads to the following conclusion about what makes for more compelling talks, at least for an audience of me:

A focus on information that is not readily available, and which gains from the presence and experience of the speaker.

So, for example, I've seen many talks on "my cool [insert language] library that does [insert function]."

A fine topic. Now in the execution, perhaps best to talk about the problem being solved, how you solved it, what tradeoffs were made, constraints dealt with, any magic foo inside ... rather that a bunch of examples showing how clever/elegant the external API is and examples of what one can do with it.

Not that the latter is unimportant, but that the latter is (should?!) be readily available from the online docs/examples or the presentation notes; whereas the former represents the specialized knowledge and experience of the library's creator.

On the Wide Range of Ruby Talent: Rails as a 4GL(?)

For a long time, I was puzzled by the extremely wide range of talent level among Ruby talent: more than any other language I could think of, "Ruby Programmers" seemed to range from the truly clueless to hardcore accomplished engineering veterans.

I had more or less chalked this observation up to the language and APIs themselves -- easy to get running with, highly intuitive ... and yet packing deep and powerful features that would attract the more sophisticated (easy C integration, metaprogramming, continuations, etc.)

On top of this was Rails with its famous screencasts showing database-backed websites automagically constructing themselves, a feat that got novices very excited, and reminded experts that they should be asking harder questions of any other framework they were using for the same kind of app.

But lately I've come up with another -- or perhaps stronger variant -- hypothesis.

Rails itself aspires to be a 4GL -- a DSL together with a framework, specializing in database-backed, RESTful web application development.

It appears that some programmers see (or want to see) just the 4GL parts (the DSL, the declarative bits, the "conventions" and the "magic") while others see a 4GL in close symbiosis with a 3[+]GL (Ruby) for doing procedural work.

In some sense, both groups are seeing accurately -- for certain problems. That is, apps that truly hit the "sweet spot" for which Rails was designed, and which do nothing else, can be treated as compositions of 4GL bits much of the time. Highest productivity, get your check, go home, watch a movie.

For other problems, additional logic and/or integration is required.

And here's where the two groups of programmers part company. The pure-4GL-ers want to search high and low for a plug-in, gem, or other black-box component to solve the problem. Even if the black-box component is harder to use, poorly documented or maintained, does not include a test suite verifying it does what it claims, this group of coders wants the plug-in. They'll search and argue and try different ones even if the functionality is 15 lines of Ruby code. But they won't write the functionality themselves.

The other group just writes the code. Perhaps they even bundle it into a plug-in and throw it on github. They also do things like submit patches to Rails itself.

Depending on the situation, either approach might be fine ... but: when something doesn't go right, in terms of logic or process or performance or data integrity or something else ... the first group hits the wall. All of a sudden it's a "Big Problem" ... whether or not it's really a big problem. That is the Achilles heel of all 4GLs: if you view them as "closed" systems that require pluggable components then anything outside their bounds becomes a "Big Problem."

And lately I've watched as some participants in the Rails community systematically leap tall buildings in a single bound when they encounter them, while others literally sit on their hands waiting for a plug-in to appear (with no track record or test cases), which they then gratefully throw into their app and hope for the best.

Fear of the Driver Disk

The problem of bloatware/crapware on retail PCs is well known -- to the extent that Apple makes fun of it, pointing out the absence of such software on new Macs, while PC tools exist just to clean it.

But bloatware has a less-famous, equally annoying sibling: all the garbage that brand-name hardware devices install off their driver or utility disk.

Pick up a peripheral -- printer, web cam, DVD drive -- from a major brand, and if you follow the automagical installer on the drive disk, you'll get a half-dozen apps that you may not need or like. In some cases, they're just bad apps (that have a habit of arranging to start at boot), while in other cases they can destabilize a perfectly-running system.

The problems are that

1. in some cases you do need these apps, because some hardware features require "support software" to be present, and don't fully leverage the many built-in drivers and control panels available for free in Windows ...
2. most hardware companies internally view the driver/utility software as an afterthought, writing it hastily, testing it inadequately, and staffing it with ... well ... whomever they can find.

There are two main remedies.

In many cases, getting an unbranded or "strange-branded" device is a smart idea (provided you know what you're getting). I've found these devices have straight-forward, minimalist support apps, make great use of built-in Windows drivers, and don't put any garbage on your system -- for the simple reason that they don't have the resources to write a bunch of half-baked apps, or to form "distribution partnerships" with people who do.

If I do have a brand-name product, I generally attempt to install it without its own driver disk, no matter what the instructions say. In many cases, the device is fully supported by Windows out of the box; in other cases, some features may not be available -- but I may not need them. (E.g., If I wanted to use my digital camera as a webcam, that would have required the vendor driver disk... but I have never wanted to use that feature of the device.)

And if that latter approach fails, it's pretty easy to uninstall the device or otherwise convince the PC it's never seen the device before -- so that you can go the RTFM route and use the supplied disk.

Stax Brings more Standard App Models to the Cloud, Marginalizes Custom Platform Further

Stax, which recently launched in private beta, is a cloud hosting/deployment/ops platform based on Java appservers. The coolest thing about Stax is that it offers many flavors of JavaEE-deployable applications, including Python apps (via Jython) and Rails (via JRuby) with ready-to-roll built-in templates.

Stax has a very AppEngine-y feel, not just on the website, but in terms of the SDK interactions, local development, etc.

This is good news for all of the popular platforms ... and bad news for those rattling around the corners with non-standard APIs. As the app-hosting industry continues to mature, the emphasis will clearly be on established players like Rails, ASP.Net, JavaEE, Pylons, et al. at the expense of guys like AppJet.

It's not about the language (JavaScript) but the about learning a set of APIs, patterns/practices, and sustaining a community ... based on a roll-your-own platform.

It is true that some of these built-for-the-cloud platforms were designed from the start to default to hash-style or big-table style storage -- popular for content-oriented cloud apps because of its easy horizontal scaling -- where the "traditional" platforms focus on relational stores and have a variety of bolt-on APIs for cloud DBs.

But now that there are so many standard alternatives, it is unlikely developers will pay any custom-platform-tax no matter how elegant that platform might be.

We May Look Back Fondly on Our 68% Project Failure Rate

The report [PDF] referenced here -- focusing on failure originating in the business requirements realm and offering a "68% project 'failure'" headline -- inspired two thoughts.

First, it remains a head-scratcher why nearly every project, despite talk of risk analysis and mitigation, expects to be in the 32% success area. Even if a company has the best resources (and generally it will not), many causes of project failure originate in organizational factors -- friction, process, politics -- and externalities (e.g., no budget this quarter for the tools in the original plan).

Since these issues are rarely known -- and, I would argue, actively denied in the sense that whistle-blowers and problem-spotter are systematically excluded from planning (at best) or forced down or out -- the average "expected value" of the d100 role is way less than a 68.

In startups, I've heard the excuse that the whole company is a "long shot" -- as though that justifies taking on disproportionate risk in each project implementation.

In enterprises, it seems as though management is so used to failure (in a timeline or budgetary sense), that they have simply redefined success to mean anything that they can pull off in their tenure -- and if that means a system that kinda works, but no one likes, shipped in 200% time and 400% budget, well, that's just the "reality" (which, to be sure, it is once all other possible paths have been discarded).

This redefinition of success also has the side effect of removing accountability and pretty much assuring a nice bonus and making strict accountability impossible.

Another thought inspired by the report is how the gap between enterprise development and small / startup development is widening. On the one hand, large businesses could benefit from the high-productivity tools and agile approaches popular with startups; for a variety of reasons ranging from policies to personnel, they are not exploiting the latest wave of technology, and it's costing them.

On the other hand, what they need regardless of tech is solid analysis and estimation capabilities. Analysis and estimation are possible, but hard, and the agile camp has moved mountains to try and reframe the problem so that they can advocate punting on all of the hard parts of these disciplines. That works great for the hourly agile consultants, but inside large businesses and large projects, it just doesn't cut it. A business needs to be able to attempt to estimate and plan months or even years of a project (hence the prevalence of "fake" agile in companies that purport to use it).

The fact that the business does a horrible job with the estimation today does not mean that the organization (which, generally, is run by business people not developers) won't keep planning and targeting in the large.

The result of these two pieces (different tech, different process) is that enterprise and small development are moving farther and farther apart, which is a damaging long-term trend. Ideally, these two groups should be learning from each other. They should spend more time moving in the same world.

The enterprise learns that it probably doesn't need JavaEE and Oracle and a big planning process for myriad internal utility apps that could be done with Rails at a fraction of the cost and effort. The small company learns that relational integrity, transactions, estimation, and operations management are sometimes both necessary and profitable.

Even more importantly, individual employees in the industry can cross-pollinate ideas as they move between these environments over their careers, sorting fact from fiction and getting better at determining what will work.

The farther apart these groups are, the less appealing it will be for "startup guys" to work in an enterprise or a startup to hire on an "enterprise guy."

This trend -- since it keeps useful knowledge hidden -- can only help a 68% failure rate go up.

How is Google Native Client Faster Than -- Or As Safe As -- JVM or x86 VM?

When I saw Google's proposed native-code execution plug-in earlier this week, my initial reaction was: "Don't we already have those, and they're called exploits?"

I decided to mull it over a bit, and I still don't like it. While the idea of sandboxed native x86 execution based on real-time analysis of instruction sequences makes for a great Ph.D. thesis or 20%-time project, it sounds like an awfully large attack surface for questionable benefit.

Here's what I'd like to know from a practical nuts and bolts point-of-view: how many "compute-intensive" scenarios could not be implemented effectively using either (1) the Java VM or (2) a plug-in based on widely-available open-source machine virtualization, running a tiny Linux snapshot.

While the JVM falls short of native code in some places, it can be as fast -- or even faster -- in other cases (faster because the runtime behavior provides opportunities for on-the-fly optimization beyond what is known at compile time). Yes, there are issue with clients not all having the latest Java version -- but that seems a small issue compared with the operational issue of deploying a brand-new plug-in or browser (Chrome).

Another approach is to use a plug-in that runs a virtual machine, which in turn runs a small Linux (started from a static snapshot). User-mode code should run at approximately native speed in the VM, which should cover the pure computation cases. In a rare situation where someone wants to run an OLTP file-based db (which would behave badly in a VM environment) or get access to hardware-accelerated OpenGL facililties, specific well-defined services could be created on the host, which code could access via driver "ports" installed in the VM image.

These approaches to the security boundary -- Java VM and x86 VM -- are well-known, long-tested and based essentially on a white-list approach to allowing access to the physical host. While Google's idea is intriguing, it sounds as though it's based on black-listing (albeit with dynamic discovery rather than a static list) of code. I'm not yet convinced it's necessary or safe.