Low-Hanging Fruit: a Server-Side JavaScript API (or Standards, or ...)

There's a big chunk of stuff missing from JavaScript cloud-hosting platforms (like 10gen) and as well as from JavaScript semi-app-servers (like Phobos).

It's called any kind of API or standard.

Hard to believe, but after several years of growing JavaScript influence, and a whole web culture that is tilts towards openness and standards, all of the players -- Bungee Labs, AppJet, 10gen, Phobos, and many others -- are rolling their own little server-side platform APIs.

Standards make a platform easier to learn, understand, debate, debunk, and fix. They allow a larger community to share code and ideas, and provide a small degree of lock-in-proofing and future-proofing. Standards also allow transparent competition on the basis of implementation quality, tooling, SLA, etc., rather than obscuring those things behind incompatible facades (APIs).

New platforms on new technologies with no standards behind them can be a hard sell -- especially when they do not offer any new capabilities.

According to Techcrunch, Bungee is in a "freefall." And the interesting bit is that their CEO ascribed the recent round of layoffs to 'actual vs. anticipated rates of adoption.'

Hello, if you are trying to sell the world on your server-side JavaScript programming and deployment environment, you're not helping your 'rates of adoption' by also asking people to learn and commit to your own home-brew platform API.

Now to be fair, there aren't a lot of alternatives in the absence of a standard. But ... it would make a lot more sense for all these players to get together and create some standard APIs and commit to using them. The APIs would cover all the basics: e.g., persistence (of object, key-value and relational flavors), templates, request/response handling, calls out to other web services and processing of their responses, publishing SOAP services (which still remains critical in the enterprise world), and interop with other server-side environments (Java, Python, etc.)

Overnight, there would be a single community (and acronym!) instead of a dozen fragments. Like any standard, it would generate books, conferences, training materials -- and controversy, which is never a bad thing when you need publicity. We would see real performance tests, and get a real debate over where the JavaScript-to-SomethingElse boundary should be and why.

And these vendors would gain instant legitimacy by being founding contributors to a specific platform "trend," rather than lone voices in the woods. That legitimacy (and, via the sad logic of large companies, the "legitimacy" of being printed on the top of some conference bag) would help them appear credible to customers big enough to pay them real money.

Always-On JavaScript Mildly Disturbing

Google Chrome doesn't have a switch to turn off or restrict scripts. While this might be an upcoming feature, my guess is that Chrome is about "running" web 2.0 apps, and so JavaScript is considered essential.

Well, maybe that's ok if the security model around JavaScript execution is as fantastic as the comic book suggests.

On the other hand, the launch of this browser featuring a well-known security flaw (admittedly not a JavaScript flaw) makes me a less comfortable about always-100% script execution.

Free Idea: AJAX Desktop Publishing App

If you're hankering to join the web startup circus and have everything but an idea, here's one for you: a full-on desktop publishing app ... online. The ready acquisition market for decent online productivity apps can serve as inspiration, even as the insane complexity of the undertaking urges you on toward greater heights of caffeination and maybe some anti-depressants.

Seriously, apps that can deal with complex page layouts and can produce a variety of press-ready output (InDesign, QuarkXPress) are expensive and hard to learn, and the average person rarely needs them. It would be nice to be able to hop on a web page and do a quick design. Wizards or integrated help could go a long way. All the usual business models (adds, freemium, pay-per-storage, etc.) apply.

Now, if you build this in Flash/Flex or in Silverlight (2.0), you merely have a ton of work ahead of you. While challenging, you may feel like you're not ... well ... proving just how wicked you really are, since those two platforms give you all of the technical infrastructure you need.

So why not go all the way? Do it in JavaScript, in the browser. Yep, WYSIWYG, arbitrary page sizes and layouts, in the browser. Advanced typography too. Fun times.

Here's a hint: one of the tricky bits is measuring text line layouts, since you'll need to make sure that the view matches the backing data (from which you will be generating other "views" or output formats).

JavaScript (the browser APIs really) don't give you everything you'd like. For example, one standard thing you need is to find out how many characters will lay out into a column of a particular width. Or simply what size a run of text will turn out to be.

While you can't ask these things directly, the clientHeight DOM element property can be useful: it shows the actual height that a DIV's content will lay out at ... so it will also signal when content has flowed from one line into a second line. By hiding the DIV, rigging up a while loop, and monkeying with the content (string), you can build a routine that measures text, so that you know exactly which characters are going to appear on a line.

Completing the application is left as a (possibly lucrative) exercise for the reader.

Maybe a Reason to Learn some Scheme, not a Reason to Avoid Anonymous Functions

A team member with one of my consulting projects sent an email yesterday, describing his counterintuitive run-in with the this keyword in an ActionScript 3 anonymous function. He found a fellow traveler looking at the same issue here... and concluded this might by 'yet another reason to avoid anonymous functions.'

I have a different view, which I thought might be worth reproducing here:

I'm not sure this is a reason to avoid anonymous functions. Looking at the issue and the blog post, the following points may be helpful for the newbies to ActionScript. AS3, and the Flex/Eclipse (FlexBuilder) environment especially, make ActionScript seem like a strange flavor of Java (or C#). But it's not.
ActionScript 3 is an implementation of EcmaScript 4. It has much more to do with JavaScript than with Java, although Adobe intentionally created an environment that would be familiar and comfy for Java devs.

EcmaScript (aka JavaScript) is a Lisp/Scheme family language not a C/C++/Java family language.

Unfortunately, for historical reasons, it shares some syntax constructs with the C-derivative languages, and ES4/AS3/JS2 adds more of those -- but many of them have different meanings, as the aforementioned blogger discovers the hard way.

If you're interested in how this came about, and how to think about it, I can't say enough to praise Douglas Crockford's lectures on "The JavaScript Programming Language", which you can watch from YUI theater. Brendan Eich takes issue with some of the "politics" stories in Crockford's account. But confirms that engineers were recruited to Netscape with the promise of implementing [some flavor of] Scheme in the browser.

If you expect to spend any amount of time in the future doing JS or AS, a couple hours watching Doug Crockford speak are unbelievably worth it. JavaScript (and AS) are extremely powerful languages and can work really well ... only they definitely don't work like they appear they should, if you see the syntax and come from a C/Java background...

Once we realize that we're basically running Scheme in the browser (Brendan says Self), lambda -- anonymous functions and their closures -- become first-class objects as fundamental to us as stack vars in a C language.

As a bonus, if you're new to this and have some time, MIT has published one of the classic textbooks, "Structure and Interpretation of Computer Programs" online for free under a Creative Commons License, along with instructors' manual, exercises, etc.

Actually, I'm gonna go out on a limb here and say if your company or team has a C/stack/register kind of background (Assembly/C/C++/C#/Java/Pascal/Delphi/VB.net/etc) and you're looking at getting involved in JavaScript/ActionScript/Ruby projects, make a team workshop out of doing as much of the Abelson/Sussman book as you can get away with. As a practical side effect, it'll also make your SQL code easier (core SQL is more like Scheme than C) and give you another way to look at problems.

Lessons from Porting a MS-Office-Scale App to Flex 3/Flash 9

Earlier this year, I led a consulting team which ported a large Windows client app (similar to a member of the Microsoft Office family; think something like Visio but with different functionality) to run in the Flash 9 runtime, using Flex 2 and Flex 3 Beta, in under 6 months.

I was invited to present a (very!) quick talk on some of our "lessons learned," as part of O'Reilly Ignite at Adobe MAX in October. Alas, schedule issues made it impossible to present at MAX. So here, in Ignite style (15 seconds per slide), are the key lessons from the project:

1. The learning curve for Flex/ActionScript 3 is very shallow for experienced software engineers, so a strong team can jump right in and be productive even without any Flex/Flash experience. Only two of our five developers had done any Flex or ActionScript work before at all. That said, Flex is still a “leaky abstraction” – some aspects (e.g., deferring an operation to a subsequent frame) of Flash Player still bubble up – so it’s useful to have at least one engineer who is familiar with coding for Flash.
   
2. There is no need to think about a Flex app as a Flash movie on steroids – simply think of it as a VM-based app (like a C# or Java app) from day one. Bringing standard OO pattern experience is very helpful. UML/Code generation tools for AS3 may be helpful too depending on your dev process.
   
3. Premature optimization is still the root of all evil. Most optimizations are a non-issue with the performance levels of Flash 9, and we found that a small number of relevant optimizations could be easily done toward the end of the project without significant pain in refactoring. Network operations, which were outside of our scope, were the only cause of performance issues. Nor did we compromise in terms of space: We had tens of thousands of lines of code, hundreds of features, and most of our display via custom code (the app is partly a diagramming app) … and it compiled to a .swf in the 670 KB range.
   
4. ECMAScript 4, JavaScript 2, ActionScript 3: whatever you want to call it, throwing static typing into the mix (without requiring it everywhere) rocks! Our unscientific feeling is that we enjoyed a 2-4x productivity improvement over a similar language (e.g., JavaScript) because of static analysis capabilities and compile-time error checking.
   
5. E4X on a large scale is a beautiful and terrible thing.
   
• The beautiful part is: It did not lead us to make a mess of our application with snippets of XML processing scattered everywhere (something we worried about at first), and the performance is amazing. We were doing a ton of XML for integration to existing systems so we worried at first. We measured it, and benchmarks of XML processing speed with E4X were far and away more than adequate.
  
• The terrible part is: the E4X API has some quirks and inconsistencies that are not documented, and in a number of cases we needed to cook up some ripe hacks to work around them. These issues can and should be cleaned up. E4X also misses a couple of (admittedly rarely used) aspects of the XML schema spec around attribute namespace scoping, that just happened to cause us some difficulty.
  
6. The Flex framework is in serious need of a native AS3 rich text, full [X]HTML editing widget. Workarounds currently include AJAX in the browser, or AS2-based components that can’t run immediately in an AS3 app's VM space.
   
7. For building some applications, the async-callback approach can create substantially more chaos than it eliminates. To address this, Flash/Flex needs threads, or at least continuations. In fact, this is the first time in my career I’ve really seen a problem (handling certain callbacks) where continuations would be an ideal solution and not just a clever one.
   
8. It is easy to mix open source Flash tools and commercial Adobe tools – my team did, and it worked well all the way around.

Since Flex/Flash are likely to stay key contenders for any RIA implementation project, I hope these learnings are helpful when embarking on that sort of work!

Writing Javascript on a Phone, on a Phone

I'm on vacation, so I'm writing some less, um, in-depth posts. Hopefully. Like this one:

I've gotten bored waiting in lines, on airplanes, etc., and I thought it would be a fun way to waste time if I could program my phone.

I wanted to be able to do it without a PC, just with the phone itself. And I didn't want to do it over the network (i.e., edit some code, send it off to a web page or network app to compile, run, etc., and return results, even thought that could be a fun way to tinker when you're on a good connection).

I thought there must be some kind of compiler or interpreter for my Blackberry or, now, Blackjack. And yes, I know, if I had a real Linux phone, then I could just download any tools I wanted. But Linux smartphones with QWERTY keyboards are hard to come by.

Enter Javascript.

I used to hate Javascript. A whole lot. Then I watched Douglas Crockford's fabulous lectures on Javascript, and it turned my whole attitude around. I realized I had never understood what Javascript was about in the first place. And here was a really smart guy who knew all of the problems, and didn't say "get over it," but talked about finessing your way to the true nature of Javascript. Which, incidentally, are lambda expressions. Crockford asserts that Javascript is more or less the only "lambda language" to organically receive mainstream adoption in the development industry.

Now I only hate the browser hosting environment for Javascript. And with enough plug-ins, even that isn't so bad.

Since Pocket IE has Javascript support, and I can edit text files and save them on my phone, I can build up a library of the functions that do what I want, and this library becomes the web page that I run and hack in with PIE. This is where having a real smartphone, not a crippled one whose manufacturer tries to deny me access to the local filesystem, is helpful.

Goofy? Kinda. But it keeps me out of trouble.