Here's the Deal: The Next Stack Frame

Yahoo Mail Supports Free IMAP … But Only via Mobile

2009-09-17T21:55:00.001-07:00

It’s hard to know what to say about Yahoo at this point. They have a lot of page views, but their internal organs seem to have been liquified from the inside by some flesh-eating plague.

Barely alive, they are of course responsible for their own continuing undoing. A great example is the failure to do something useful with the email user base. The purple crew has missed every opportunity to exploit the email product, from Google-style ads to Facebook-style social networking (back when they had a chance). And there’s the infamous CEO with her Napoleonic airs.

But I digress.

See, here’s the thing. On the Internet, unless you’re lucky enough to be a telco, you actually have to compete. Which means when Google gives away IMAP-enabled mail, and Windows Live gives away POP access and 25+ GB of online storage, you need to rethink your strategy of making your mail product harder to use while praying that the nine people on earth who haven’t heard of Google will sign up for your deluxe mail service (now with undercoating!)

But Yahoo has never been able to think one thing at a time. It’s not so much that they excel at multitasking than that they are schizophrenic. So while one group is committing ritual suicide with the email product, another realizes that to reach the mobile market, exposing mail via open protocols might help.

And hence they offer endpoints called something like imap.mail.yahoo.com and smtp.mobile.yahoo.com. Before you get all excited and ask me to check the hostnames and port numbers, though, I’ll drop the punchline: it appears Yahoo filters access to these services based on IP, and opens the service to mobile carrier address blocks (and carrier proxy addresses).

After seeing my phone successfully configure itself for these services, I couldn’t resist trying them from a desktop client. No joy. I guess if I were really hardy I could grab the public-net-facing IP address for my phone (by having it connect to my own server) and pretend to be that IP. But … really … is it worth it? No… with Yahoo still living in 2002, I’m afraid it’s just not.

Work 2 Different Logons (or Sessions) with Private Browsing Modes

2009-08-31T23:51:00.001-07:00

Here’s a trick that’s fairly obvious to web developers but probably not so for everyone else.

Private browsing modes – such as Incognito (in Google Chrome) or InPrivate (IE 8) clear out locally stored browser cookies (snippets of tracking data) each time they are run, and do not share cookies with the main (non-private) browser tabs.

Since these cookies are the core mechanism by which web sites associate independent browser tabs/windows with a single user session, the private browsing modes can be used to interact with multiple different logons or accounts at the same time.

How does this work in practice?

Let’s say you want to interact with two totally different accounts (logons) at, say, Gmail, Yahoo, eBay, Orbitz, or some other site. You’ve probably noticed that if you are logged in to, e.g., Yahoo with one browser tab, and you open another window or tab in the same browser, and go to Yahoo, it will “know” who you are and allow you to interact with the same account.

In many cases, this is a desired behavior. But if you want to work with two different Yahoo accounts at the same time, it is quite tricky.

Here’s where the private browsing mode comes in.

Open a private browsing window/tab, and log on to Yahoo with the second account. Since the private tab doesn’t share cookies with the main tab, you now have two interactive sessions with the two different accounts, and they “stay separate.”

There are three catches though:

First, since the private browsing tabs are designed to discard cookies on shutdown, they will “forget” your logon when you close the browser even if you select “stay logged in” on a specific web site.

Second, it is possible (though unlikely) that this mechanism could fail if the site uses “Flash cookies” apart from regular cookies.

Third, while this trick works perfectly well, do not assume that the service you are connected to has no idea what you are doing. They probably don’t care. But if they do care, assume that they could guess that the same user was on both sessions (through IP addresses, NAT port assignment patterns, etc.) at the same time.

Two More Small Tales from the IP Front

2009-08-24T17:10:00.001-07:00

There are a small number of areas in which I am an expert, and intellectual property is not one of them. For expertise, I like to refer folks to Mike Masnick and Techdirt.

Nonetheless, it’s hard to move through the technology world and not get bonked on the head by IP absurdities and incongruities calling out for some solution aside from lawsuits and Nancy-Reagan-style denials.

First up is the tale of two Comcasts, and two identical HD streams of Battlestar Galactica. One of these streams comes in via the HD DVR, and the other comes in via Bittorrent.

Actually, I’m leaving a bit of information out in the setup. See, the two HD streams from SciFi channel should be more or less identical modulo commercials. But they’re not.

Due to bandwidth allocation and network management issues, the “legitimate,” paid-for SciFi channel stream is full of bitrate-spike artifacts reminiscent of late ‘90s web video. So the HD channel, the HD DVR, the HDTV gets you … a pretty awful picture the second characters start fighting, running, or blowing things up.

Then you have the Bittorrent stream, ripped from SciFi, and carried in via the same Comcast coax line. This stream looks great, and makes a customer glad to have Comcast and a HDTV. In some strange respect, this latter stream may represent a problem in the “industry’s” opinion, even though it’s clearly what the customer wants and is paying to view, and even though the bandwidth (both instant and aggregate) is a small fraction of that required for the “broadcast” stream.

Next strangeness is Grooveshark, a free, crowdsourced, on-demand streaming service that appears to live on the razor-thin edge of legality if it has a claim on legality at all.

That said, the ability to send (or post/tweet/blog/…) a link that goes directly to a particular song is an extremely potent way to virally spread the music you like. And more music loved by more fans is the core asset base for any music industry, whether it resembles the 20th-century “legacy” record industry or not.

Without Grooveshark, there is always YouTube for sharing instant, no-membership, no-login-required tracks. But unlike YouTube, Grooveshark is structured in a way that encourages more exploration of an artist, album, etc.

So anyone wanna throw into a pool for how long ‘til this service gets shuttered? Wait, that’s not quite legal either. We’ll “just say no” and it’ll all just go away.

Suck Us All Into the Machine: Build FluidDB on Twitter and #Hashtags

2009-08-18T21:23:00.000-07:00

I was reading today about a fairly amorphous, tag-based, public "database" concept called FluidDB.

To avoid the usual distractions, I will assume my audience knows all about relational and non-relational datastores, implications of tag-oriented metadata, etc. In fact, since my friends know about this stuff and they are most of my audience, I think I'm not too far afield making this assumption.

One reason I found the FluidDB concept interesting is vanity: about four weeks ago, I spent some time considering building just this kind of database ... on top of Twitter.

Why would one want to do that?

Precisely because Twitter already has a lot of meaningful data curated by humans and tagged with a well-known metadata scheme (hashtags).

In exchange for having very small or hypernormalized data records (since each atomic entry is limited to 140 characters minus the tags and any indexed keys), we get a strange merging of human- and machine-readable data.

Humans could read (and follow, search, etc.) data entities of interest.

And clearly the "goal" of automated (machine) participants (clients) would be to understand as much of the human content as possible, treating it as objects, tuples, logical inferences, or knowledge base "facts."

Moreover, the originator of a tweet, as well as any @-referenced recipients, are critical metadata. They are, actually, tags themselves in way which is linearly independent of the hashtags. That is, a from-@adbreind tweet (entity) marked #database is different from a to/ref-@adbreind tweet marked #database, while #database must be considered to (possibly) have a different sense than it does in, say, a from-@headius tweet. However, the same tools and semantic analyzers can be applied, essentially treating the writer and target of a tweet as special tags.

In this way, our twitter discourse, short enough to make machine understanding tempting even when the packed cultural references make such understanding impractical, merges us into the database and makes us "just another part of the machine."

AT&T “Fixes” My Phone by Downgrading It

2009-08-06T20:48:00.001-07:00

I don’t want to be another corpse in the AT&T pile-on. AT&T has many problems, and they impact me, but let’s look at another technical clue about the network trouble:

I stopped in Tuesday at an AT&T store to see if they had any advice or fixes for the network issues plaguing my phone. The only thing the service rep said he knew of was to replace the SIM card. Sure, why not? Can’t hurt anything, can it?

Well the answer to that question apparently depends on whether ditching 3G access counts as “hurting.”

Since I got the new SIM, the phone spends most of its time on the EDGE and GPRS service, and only rarely gets a fix on the 3G service even when I’m in known-good 3G areas.

For a fleeting moment, I thought about taking the phone back and asking AT&T to do something to restore my 3G goodness.

But I quickly realized that the network’s inability to keep the phone on 3G was a big part of the problem. In other words, as so many iPhone users have discovered, staying on EDGE is better than flitting between EDGE and 3G and getting nowhere. Likewise in the case of EDGE and GPRS.

So for now, I’ll take my retro GPRS and its 50kbps or so of throughput. Since the setup and teardown of the data connection, as well as the latency in repeating a request, are the slowest part of the process, I’d rather use dependable 50kbps connection over an unstable, unusable 500kbps one.

Until I get a network that actually works, that is.

Link: An Illustrated Brief History of Augmented Reality

2009-08-06T20:37:00.001-07:00

Check it out.

Plus who knew that Philippe Kahn invented the camera phone?

Apparently, a lot of people, just not me. The guy is so awesome.

Microsoft’s Real Punishment is Having to Fight with the Gloves On

2009-07-13T16:58:00.001-07:00

By the time the Microsoft anti-trust cases wound down, it could be argued that the alleged damage was long since done to the industry.

In a curious reversal, it appears that Microsoft’s direct penalties from the case were only a tiny beginning.

The real penalties Microsoft pays are not in money, nor in shipping “K” and “N” SKUs. They are in product strategies unpursued because they would be too provocative. In other hands, such strategies might be reasonable if aggressive, but for Microsoft they might look like “relevant conduct.”

Today’s announcements about Office 2010 (“14”) were … well … let’s just say if you didn’t read about it, the most interesting thing you missed was seeing critics point out that Microsoft, inventor of AJAX (literally, for the Outlook web client), is only bringing other apps online 10+ years later.

The reasoning here is not about selling client OS licenses. Microsoft could have moved to the cloud richer and faster and more profitable if it could take the gloves off.

Need to get Silverlight penetration up from its abysmally low numbers to where it can really compete with Flash and become a meaningful platform? Just ship it with Windows and make it a priority update to every Windows box in the world. Problem solved. Now we can get down to the real work writing apps. Or at least Adobe could actually face some competition. But Microsoft doesn’t dare do this.

Why not stitch cloud storage directly into the OS? I hate leaving files “on the other machine.” Right out of the box, anytime I see an “Open…” or a “Save As…” dialog box on Windows 7, I would like the default destination to be a secure folder on Windows Live SkyDrive. Using the provider pattern, other vendors could offer a similar service, and the end user could choose. But Microsoft doesn’t dare with this either.

It is quite possible that the long-term benefit to the industry of having Microsoft thus restrained far outweighs the “lost” value we could have had from Redmond. But let no one fool himself into believing that what we see from Microsoft these days is everything they have to offer.

VncSharp Rocks for Programmatic and Interactive Remote Access from .NET

2009-07-09T15:59:00.001-07:00

VncSharp is a C# implementation of the VNC protocol together with a handy visual Remote Desktop .Net widget.

At first, the open-source VNC remote-access solution might seem like a surprising item to need in a Microsoft-based solution.

But once he gets you in the little room, VNC starts telling you that, in exchange for lower performance (than Microsoft’s own RDP), he can get you more flexibility, more features, no licensing issues, and access to remote Macs or *nix hosts. After these arguments (or is it the heat?) VNC looks a lot more persuasive.

Throw in the fact that you can run your VncSharp-enabled apps on Mono, and … well, it would be cool if that bought you a lot. But actually if you aren’t already focused on a Linux solution then the Mono angle is just another bullet.

VncSharp itself, though, works extremely well straight out of the gate.

When you see the documentation page – where the author essentially invites you to read the source to figure out how to drive it – you may be concerned. Or even start to form silent curse words with your lips.

Do not let that stop you. There are demo/sample apps that will show you what you need for basic use cases (e.g., popping open a “remote help” window will only require a line or two). And the source, if you need it, is elegant and straightforward to navigate.

A programmable VNC client is, perhaps, a niche product. VNC on Windows maybe more so. So it’s gratifying to see such a mature and streamlined OSS effort.

Can iPhone et al Drag Augmented Reality Into Non-Augmented Reality?

2009-07-07T20:47:00.001-07:00

I’ve been pitching augmented reality apps in startup circles for a few years now, so it was exciting to see the AR startup crop grab some press coverage this week (VentureBeat and more VentureBeat and …even SF ABC 7).

With the iPhone hardware suite (and comparable devices like the Android and Pre), there’s no shortage of hardware for the core AR tasks:

capture decent-resolution images
recognize “target” areas in images
contextualize the targets if necessary, by adding GPS data, solid-state compass data, and/or accelerometer (angle) data
lookup augmentation data suitable to the target and the end-user via suitable web services
employ “billboarding” or 3D rendering to composite a representation of the augmentation data on top of the target
repeat as fast as possible without draining the battery (yeah, right)

Now for part two of the plan: this facility needs to run through a cool looking visor (a.k.a. "head-mounted display” or HMD). And neither the $6,000+ price tag nor the Silence-of-the-Lambs-night-vision look is appealing on these traditional high-tech units.

Happily, there are mass-market headsets designed for the gaming or personal entertainment market which are almost ready to go. A couple are even within striking distance of cool factor. Maybe an Apple logo would be enough to do it, at least for the Bay Area.

Even better, leaders such as Vuzix recognize the need to provide video and accelerometer data from the POV of the headset (vastly reducing the amount of computation needed contextualize the image). They appear to be planning these capabilities as optional clip-on modules to their newest “Fall ‘09” model visor.

Note the word “planning.”

Like smartphones themselves, we’ve been here before … a lot of times. The iPhone was easily the industry’s 10th attempt at a commodity handheld computer, so it’s not like the writing is on the wall. Unless it’s AR writing:

Workaround for Some Instances of Win 7 WiFi Problem

2009-06-13T20:34:00.001-07:00

One of the very few real broken bits in the Win 7 RC is a WiFi problem. The current thread on Microsoft TechNet is here.

On at least a few laptops, the following is a dependable if annoying workaround.

Turn off the WiFi with the laptop hardware switch (“airplane mode”)
Open Start –> Computer –> Manage –> Device Manager –> Network Adapters
Right-click and disable the wireless adapter
Wait a bit and verify the OS has completely lost the wireless adapter (by watching the icon in the tray)
Turn on the WiFi with that hardware switch
Right-click and re-enable the wireless adapter
Wait … potentially a couple of minutes for Windows to find the appropriate network, connect, and recognize the route (if applicable) to the Internet

This maneuver is a hassle, to be sure. But it seems to work 100% of the time on some laptops (including mine) and it is much more convenient than rebooting, which is the only other dependable solution.

iPhone and Palm Pre – the Obligatory Post

2009-06-10T20:40:00.001-07:00

I’ve had my paws on the Pre, and while I have not, of course, gotten hold of a 3GS, it doesn’t really matter.

See, getting my hands on a 3GS might convince me it has a better hardware/software experience. And since the 3G already has a better hardware/software experience than the Pre, I’m going to call it a “gimme” for the new 3GS.

The Pre, for all of its clever conceits compared to most phones, is still clunky, hiccup-y, and jittery next to even the current iPhone model. The graphics aren’t as smooth, the UI is harder to use, the physical keyboard is marginal, and on and on.

On top of that, it is hard to overstate how important the app ecosystem is to this “competition,” and Palm doesn’t even seem to be trying (they’re still saying “real soon now” on the SDK).

No matter how many apps in the App Store are just fart apps, and no matter how beautiful the bundled apps on the Pre are, there is no contest because these guys are playing different games.

Apple has succeeded in making the phone a general computing platform in the mind of the public – something I argued for 3 years ago – and you judge a platform not by its internal specs but by what you can run on it. Palm doesn’t seem to get that. They’ve got a decent bundle of specs but there’s nothing to run on it and there may never be much.

So with Apple still killing in the UX department, and Palm leaving their A-game at home (if they ever had one) as far as the app/platform/dev community goes … is there anything positive to be said for Pre in this contest?

Only this: AT&T’s network is so egregiously ill-behaved in so many prime metro areas that Sprint could actually pull a few people across the line.

I am one these last folks: I would much rather replace my current phone with an iPhone, but the thought of another two years of dropped calls, missed calls, bars-but-no-coverage, data connection unusable half the time … and I’m seriously considering the Pre.

Say what you will about Sprint (I’ve used every major carrier and none is perfect), where they have coverage, the devices just work. You can make or take a phone call. Which, ironically given that smartphones are bordering on augmented reality nowadays, is still the sine qua non for a phone.

Windows 7 RC, One Month In

2009-06-09T17:35:00.001-07:00

A month ago, not long after the RC was released, I wiped my ol’ Server 2003R2 notebook (yes), and installed Win7 x64. Time to try a consumer OS on the machine again, finally go 64-bit, and jump without a net (I had a big client demo coming up and decided it would be sink-or-swim with 7).

First, a word about this laptop. Bought in the holiday sale period of ‘07 just for client demos and occasional web surfing, it would take a whole new, um, “lower end” category of Microsoft “Shopper” commercials to capture the spirit of this baby: for $299 I got a mobile Celeron (1.6 GHz, one core, no HT) and 512MB of RAM. Billed as “Vista Basic Ready,” it was being discontinued due to … not really being Vista Basic Ready. Well, duh. I had given up on Vista after my 3.6GHz desktop choked on it, so I put another GB in the laptop and loaded Server 2003R2 (using unofficial XP drivers I grabbed off the net).

So … Windows 7.

Upon install, all of the hardware was supported perfectly, which isn’t a huge surprise since 7 uses Vista-era drivers and this hardware ensemble was originally targeted for Vista.

Except for a compatibility issue with AVG Anti-Virus (which I’ve written about before), it has worked almost perfectly with everything I’ve thrown at it – ranging from Office and Visual Studio 2008 to Alfresco Enterprise (yes, a Java server app), Google Earth, and Ruby. Overall performance has been excellent and better than I would have expected. No, I wouldn’t play games on this machine, and the 5400-rpm hard drive can be a drag just as it is on most laptops.

The two big negatives I’ve seen are as follows and will hopefully be fixed by RTM time.

First, there is a well-documented issue with WiFi. On a cold boot, Windows does just what you want it to do with the WiFi. But, after waking from sleep or hibernate, or trying to switch networks a few times, it just cannot seem to sort itself out. Most of the time you are forced to reboot to get a working connection. This bug is all over Microsoft’s Win 7 feedback forums, I’m just too lazy to look up a link right now. So hopefully it will get a fix.

Second, the integration with other default browsers (I use Chrome) is broken in a bunch of places. Many of the cases where the shell is called to supply a browser to a specific URL do not work (e.g., menu options in twhirl, connections to the web from Office Live). In addition, Windows doesn’t want to associate local .html (or .htm) files with Chrome. Sometimes I can get these things fixed, but then they revert (perhaps partly a result of Chrome’s auto-updates).

These cases all work fine on XP and don’t require in-process loading (suggesting it’s not a x86/x64 issue). It’s inconvenient, and the anti-trust folks will be back after Microsoft if they don’t make other browsers first-class citizens soon. So this bug should get squashed.

Aside from those two admittedly very annoying issues, this is a really solid, fast, and elegant operating system. How serious is the “Microsoft tax” issue on OEM PCs? I don’t know, but if I do pay such an implicit tax on a new machine, I’ll be a heck of a lot happier if I can get a Win 7 license out of the deal and not a Vista license for my trouble.

Mark Process for x86 to Use Skype API or Topaz SigPlus API

2009-05-29T10:35:00.000-07:00

If you happen to be integrating your Windows app with Skype's COM bridge, or with Topaz' SigPlus API for their biometric digitizing signature pads, those libraries won't play nice if loaded from an x64 process.

The easy fix -- assuming you don't have dependencies that mean you need x64 -- is to create a build configuration for your main app that specifies a target platform of x86 (instead of "Any CPU"). The resulting app will run in WoW (the 32-bit shim for 64-bit Windows) and works fine with these libraries.

The hard fix would appear to be using IPC to coordinate two different processes: your main app in x64, and a proxy/helper 32-bit process running in WoW that works these DLLs for you.

Those Little Shape Widgets in VS2008 Require Deployment

2009-05-29T10:02:00.001-07:00

The Visual Studio 2008 SP1 control palette has a few "shape" widgets -- Rectangle, Oval, Line ... which come from a "Visual Basic PowerPack" library: Microsoft.VisualBasic.PowerPacks.Vs.dll

If you use these, you need to deploy the library manually with your application. That is, an up-to-date .Net 3.5+ machine will not have the library available globally, and Visual Studio will not mark this DLL to be copied into your output folder.

I haven't checked to see whether the VS Installer project will determine the dependency and include it -- I'm guessing it will. But if you are doing xcopy deploy, or just testing, you will need to bring this library along.

There is also an earlier version of this same library, that shipped on VS2008 but will not work (it is missing some components that were added to the namespace later). So if you need to build against the current version (e.g. if you are working on code that uses the shape components), make sure you update to VS2008 SP1, which updates this library.

Windows 7 and AVG == Unhappy CPU Usage

2009-05-14T16:12:00.000-07:00

The title pretty much says it all.

Backstory is that I ran Windows 7 in a VM without any anti-virus while doing some evaluation, and I was impressed by how sparing it was of resources.

Then I installed the RC on my (underpowered old) laptop, figuring it would be a good replacement for Server 2003 (which is resource efficient but not exactly designed for laptops).

The RC used somewhat more memory -- it appears to be able to adjust its memory footprint depending on the host hardware, which is cool -- and a ton more CPU.

The CPU usage was suspicious because it's rare to find software that consistently pins a CPU with actual work, and because the usage was high priority -- coming from kernel space or a library tied tightly to kernel hooks. It was also suspicious because Microsoft plans to sell a version of 7 on netbooks, which are even more underpowered than my two-year-old bargain-o-matic laptop.

Turns out AVG, generally a fine anti-virus product, struggles with Windows 7 and often insists on every available CPU cycle, while the end user sits there wondering why context menus won't even open anymore.

After a bit of Googling, I removed AVG and installed Avast, and 7 is screaming along on the laptop now.

Note the retroactive Windows 7 hardware subsidy: this laptop was discontinued by the manufacturer in '07 because it shipped with Vista Basic but proved so slow as to be completely unusable in that configuration. The new OS basically injects value into the old machine.

Bonus Security Credit for Google Chrome’s Strange Install

2009-04-25T11:10:00.001-07:00

Last fall, many folks including myself commented about Google Chrome’s unusual install behavior. Our best guess at the time was that it represented an attempt to accelerate adoption, by allowing non-administrative users to install Chrome.

It also allowed lower-privileged domain users in corporate environments to install and use Chrome unless their IT specifically blocked it.

With this recent Chrome vulnerability and rapid patch cycle, though, I’ve come to see the install in a new light.

Firefox requires an admin to initialize an update. This can be done through programmatic remote admin or right on the console, but still requires intervention. IE can be updated via Windows auto-update, but if auto-update isn’t set to run or if a specific patch needs to be applied, it requires intervention. Chrome, on the other hand, will update itself on the fly for each user’s install (it does require a restart, but only of the Chrome app) unless the installer is cracked to remove the GoogleUpdater component.

Given the cost of having an out-of-date browser version versus the risk of having Chrome updated without admin knowledge … I have to say I like this approach.

Notebarn Update

2009-04-25T10:46:00.001-07:00

Notebarn, my Windows Mobile / Exchange sync notes app, definitely looks like an archaeological relic these days. Dating from early ‘07, before the iPhone era, and being a simple text utility, it is almost comic how it doesn’t resemble modern mobile apps.

That said, I still use it, and it turns out a lot of other people have been using it too. So when a user helped me reproduce a tricky timing bug that could cause data loss under certain circumstances on app initialization, I hopped back into the old (and quite small) codebase to fix it.

There is a little more info on the notebarn project page. Or if you just want to install the app you can install it over-the-air from here. If you already have the app it will automatically install in-place over your existing version. And since the “notes” are actually stored in an Outlook/Exchange Task, the install won’t affect existing data.

A word about backups: notebarn doesn’t have its own data backup mechanism. There are two main approaches to backing up and recovering data if you should lose it for any reason (e.g. problem with notebarn, problem with ActiveSync, accidentally deleting a note you needed, etc.)

One is to lean on whatever backup solution protects all of your Outlook/Exchange data, since notebarn data is really Outlook data. If you can go back to a backup snapshot of this data, even temporarily, you can simply grab the notes data from there. If that’s not practical, you can either manually or via a script back up the “My Notes” item from Outlook tasks, into another place in Outlook, the filesystem, etc.

Google “Similar Images” Roadmap

2009-04-20T15:57:00.001-07:00

Ok, it’s not their roadmap, it’s my roadmap.

I was psyched to see the Similar Images announcement today, but I was underwhelmed by the results. That’s ok, it’s helpful, it’s free, and here is the post where I explained how to build the rest of it.

Oracle and Sun: Cui Bono?

2009-04-20T15:53:00.001-07:00

Well, here’s a hint: it’s not Oracle, “Sun,” Java, or MySQL in the long run.

I’m thinking the Ruby, Python, and PostgreSQL worlds just got a shot in the arm, as this is minor calamity (at least) for Java, and a major one for MySQL. Ironic, since Java maturing like a fine wine and recovering from early-decade blunders; MySQL was already in trouble thanks to Sun.

As for benefits, it’s also not Google, who relies heavily on Java but could eventually find itself in an adversarial relationship with Oracle as enterprise computing moves to the cloud. Google does have enough sheer wo/manpower to exploit the OSS licensing on Java to take it in its own direction if necessary … but is that really a desirable way to go? or one the investors can live with?

I don’t think Microsoft minds this one bit either … since there was nothing that that Java, Oracle, and their communities (and users) couldn’t do before that they can now, while a number of scenarios (Java and open source databases/appservers in the enterprise) suddenly become just a bit murkier.

Atalasoft: Another Example of Gnarly DRM == Lost Sale

2009-04-15T20:43:00.001-07:00

I’m working on a project that involves semi-automated document imaging. Scan, deskew, crop, re-arrange …

It’s on Windows, where every modern scanner hooks into both TWAIN and WIA out of the box, often without even needing a vendor driver, so I just needed a library/toolkit to do the lifting on the app logic side.

Enter Atalasoft DotImage imaging libraries. Does everything you need, works fairly well. Established presence in the market. We start heading in that direction. The Atalasoft bits we needed turn out to be pricey as components go, and we would need a runtime license as well as the development license – but this is a commercial project the success of which would not be diminished by the software costs. So we didn’t blink at the price.

We downloaded the dev SDK, implemented a few features … and we needed to show them to customers. In other cities on other machines. Well, the dev SDK is crippled and doesn’t allow that.

Atalasoft’s sales department generated a 30-day license for me, and sent me the instructions to install and deploy it. And … it half worked. Some machines could run the deployed app. Other machines, the app would crash when the relevant DLLs tried to load, despite deployment of the magic binaries, license files, and other DRM voodoo.

For a brief moment, I thought maybe my app is just broken … but, upon attaching a debugger, I saw that all of these crashes threw the same error. And, since it was .Net, the error was in plain English: Atalasoft’s licensing module was barfing and taking the whole app down.

At that point I could have spend more critical hours trying to navigate around these problems (I’m guessing their pre-sales tech support would have tried) … but … wouldn’t you know it, here is another company offering a similar library, much more agreeable terms, 30-day trial and a seemingly foolproof license key mechanism.

Download, type type build deploy. Success. Haven’t looked back.

Now it’s also convenient that this other product seems to work a little better, has more agreeable legal terms and costs less. But those were not dealbreaker criteria at this stage.

I would never have even gone down the list to this other vendor if Atalasoft’s DRM hadn’t broken my tight-deadline customer demos.

Facebook “Private” RSS Feeds Probably Don’t Leach Data…

2009-04-14T22:32:00.001-07:00

Last year I experimented with private group microblogging systems via authenticated feeds. Didn’t go anywhere, because many of the biggest newsreaders don’t properly support authenticated feeds. And “obscure but public” feeds get indexed by aggregators like Bloglines, by design, making sensitive content much less obscure.

Enter feed access control, a several- (3-?) year-old RSS/ATOM extension that tells Bloglines, and anyone else who is listening, that this feed should be treated as private, even though it’s public.

Facebook’s feeds are intended to support this protocol:

Which seems reasonable enough.

There are a couple of issues though. First, this approach is based on a third-party’s positive action to prevent or “opt-out” of publishing and indexing, in a system that normally defaults to syndication, indexing, etc. So it’s easier for a glitch to expose data.

Second, the whole “fac” extension is a gentlemen’s agreement among parties that couldn’t even agree on making authenticated feeds work well. Perhaps they all make a best effort to isolate the marked content. But tomorrow, a startup with a rocking aggregator could simply ignore “fac” and expose all of the feeds it has.

In some sense, the same vulnerability exists with other systems – if you signed up with some random webmail provider, who’s to say they don’t expose your mail. But because RSS is public by nature, almost all feeds live utterly unprotected, and this extension is one vendor’s hack, it’s not quite the same.

All in all, probably not a big reason for concern. But when people tell me how private things can be on facebook (where you can sneeze and end up revealing your data because the IxD is tilted so heavily toward sharing everything) it always seems worth noting how your data (via your friends’ feed subscriptions) can slowly leach out into the open ocean of the indexed net.

Random Bit: Sysprep Re-Writes Boot.ini … Not Always Correctly

2009-04-13T10:26:00.001-07:00

I discovered the hard way that Microsoft’s sysprep tool (for configuring machine images) re-writes (at least some of the time) the boot.ini file, the file which tells the Windows initial bootloader which OSes are installed on which devices and partitions.

The new boot.ini contains the same OSes as the old one, but it specifies a different default, and a zero timeout for the user to choose what to boot.

I can imagine some reasons why sysprep might want to do this, based on speculating how I might deploy enterprise images.

Only the thing is, if I were going to re-write boot.ini, I would at least check to see which OS was currently running and maybe make that the default. As it is, sysprep made a different OS the default – it picked the “first” OS in the device tree even though that is not the OS I was trying to sysprep. This behavior seems more like a bug than a feature.

In any case, if this happens to you, there was no long-term damage done -- you can just reconfigure the boot.ini file by hand and restart.

Enable “Modern” (Themed) Common Controls in Hybrid WPF/WinForms Apps

2009-04-06T16:19:00.001-07:00

Here is a quick hint to save someone from a bunch of Googling:

If you are building a WPF app, you may find that you need or want to also use some Windows Forms windows. In my case, I was adding a form just to host a WinForms control, so there was no point in creating a WPF form just to host the WinForms Host container in order to add the control. A more common scenario is you want to invoke a built-in Windows dialog box, which is not natively a WPF object.

If you do this, it will work, but you will notice that some controls are rendering their old-fashioned look and behavior – you’ll be zapped back to the era of Win 2000 or the earliest .Net apps that lacked the benefit of comctl32.dll version 6. Square edges, no mouse-hover behaviors, etc.

The short answer for how to fix this is that you need to add a call to System.Windows.Forms.Application.EnableVisualStyles().

Add it once, somewhere early on. It’s ideal (though not always necessary) to do this before you start instantiating the WinForms objects.

Apparently the template code for WinForms projects contains this line, and depending on your POV, that’s either “low level boilerplate that an app developer shouldn’t have to care about” or “the kind of thing that kids nowadays just take fer granted with their magical IDEs and WYSI-whatnot, virtual memory and lazy programming habits.”

I was also particularly motivated to write this post because the most accurate (and earliest) Google hit I found on this topic was to one of those scam programmer support boards, where they wanted me to sign up for a trial with a credit card just to see the discussion thread on this issue.

Which is half insane if they could persuade me that had the right answer inside, but 100% insane since there was no way for me to know that their “answers” weren’t way off topic from clueless n00b who thinks a HWND is what you pull to keep the rain out of your office.

Visual and Context-Cued Semiotic Search Opportunity

2009-04-03T13:26:00.001-07:00

Want a hardcore problem to work on? to fund? to stay-up-nights-only-to-see-Google-do-it? or maybe get-bought-by-Google?

We need a search engine that searches based on visual and contextual clues about the appearance of objects -- especially of signs and symbols -- rather than just based on words that (perhaps) describe them.

For example, if I see a bumper sticker around town, with a green star on a blue field, I might want to see if this represents some well known organization or cause. I could search for “green star" and “bumper sticker” or something similar. But I probably won’t find anything.

Moreover, when the elements of the design don’t have names (“star”, “stripe,” “field”), properly describing a complex design in a single search gets difficult. Imagine you saw the new Pepsi logo:

You don’t know what it is; for the sake of the argument, imagine you don’t have any cultural Pepsi associations to work from either. What do you type in to the search box? Circle? red? stripe?

Good luck.

How do we solve this problem?

I envision a search that consists of several stages. At the first stage, you can add descriptive words, or you can import a similar image, or even draw/sketch some cues right on the page. That may sound unlikely for less design-oriented folks, but many unknown visual designs consist of largely straight lines, simple geometry, etc. So it’s not unreasonable that I could sketch in a simple design, or even take a swing at the Pepsi logo above, with just a circle and 3 straight lines using an AJAX or Flash inline drawing tool.

From these inputs, the search engine draws a set of possible results – but it also generates a set of context-narrowing options that I can use.

It presents options to choose where I saw this design: web, billboard, tv, clothing, museum, public building (e.g., a capitol or courthouse), manhole cover, etc.

Perhaps knowing material is useful: was this printed? embroidered? leather? denim? engraved metal?

This is a challenging but eminently creatable piece of software.

I’ve actually had a lot of instances where I would have liked to use something like this – but, if it’s never happened to you, consider: when computer vision progresses beyond working with the local environment, objects and known patterns (people), the machine will need to take the next step. It will want to dereference symbols to find data and meanings in order to solve problems. And, in order to do this, it will need benefit from this kind of visual-semiotic search heuristic, which starts with a visual-context search like the one we are discussing.

Good Results So Far For Google RAM

2009-03-31T16:59:00.001-07:00

A week ago I had 4GB of RAM die (well, part of matched pair anyway) in my main desktop PC.

I’m currently awaiting replacement under warranty from Corsair, but meantime it’s hard to run dev tools and big virtual machines with the measly amount of memory I have left. So I thought it was time to give the new Google network-attached RAM a try.

I had to flash the motherboard BIOS of course and upgrade the chipset driver and the on-board network controller firmware. Google RAM, just like wake-on-LAN, has to interact with the network card at a hardware/BIOS level. In this case, the purpose is to ensure that any OS I boot sees the new space just like local memory.

Then I rebooted and … nothing.

Where is my free 4GB of storage?

Then I remembered that Google’s revenue model for this product requires you to run a Windows service that in turn interacts with a Google-provided kernel patch for PAE.

In addition to providing checks in real time – as my machine accesses RAM – for any security threats, this service displays Google ads as 5 new icons on my desktop.

Apparently they are context-based, and determined by Google’s analysis of what I have in RAM at the time.

And they are surprisingly accurate. I had a picture of a Corvette open in Photoshop, and the G-RAM icons turned into links to car dealerships, new-car financing, and a discount oil change.

Google’s FAQ insists that it does not look at my clicks or the image file metadata – instead, its server analyzed the image in real time (since the network RAM is in their datacenter) and determined I was looking at a new Corvette.

The only downside was that my cable modem signal dropped out for a couple of minutes, and the local service warned me not to touch any processes using G-RAM until it could sync back up, or those apps would immediately crash.

No matter, overall it’s great technology, and I think my RAM replacement will arrive from Corsair tomorrow.

Quick Hit and a Deep Hit on Social Nets and Identity

2009-03-29T10:18:00.001-07:00

This article from the WSJ is neither deep nor particularly novel, but I like it because if focuses a laser on propagation of identities and the history of identities in popular social networks. This is the most important metatopic for social networks.

If you have a few more minutes, this article by MIT Media Lab prof Judith Donath gets a lot clearer on the signals that make up identities online, and how the mechanics of those signals can function.

Harm Reduction in Windows 7

2009-03-27T17:02:00.001-07:00

Guest mode … kid mode … whatever you want to call it, is brilliant.

But more than that, it’s an interesting admission that (1) you can’t fight the power of the darknet and (2) you might as well empower people to behave in a way that minimizes the damage, whether or not you approve of what they’re doing.

If I had a dollar for every individual who ever swore they never go near warez or pr0n or questionable media downloads, and ended up with a mucked up machine … or worse, a machine that transmits their passwords and SSN to a bad guy …

Even with an older OS, like XP, one can achieve a fair degree of isolation and protection by using a patched up Firefox or Chrome on top of a plain user (not admin) account. There are still holes by design; e.g., a user could fill up the hard drive or install software that persists in certain places. And I’m sure there are serious security flaws that allow code downloaded as user to escalate itself to admin … perhaps even coming from a “drive-by” Javascript source via Firefox/Chrome … but such threats seem to be pretty darned rare if everything is patched up and prophylactic protections are applied (e.g. Spyware S&D’s “immunization”).

Guest mode (and IE 8 “In Private” browsing) appears to close many of the remaining holes.

What we need now is an education campaign to convince people to segregate their online activities. But besides not knowing how to create these low-privilege accounts, a lot of people I know refuse to admit they ever visit the darknet. Or the visits are rare and they “hope for the best.”

Let’s pre-configure – by default -- a second account for ever power user (or admin) on a machine. At login time, offer the guest (more protected) account along with some description of when it might be a good idea to use it.

I’m not sure the best way to label the buttons, because it’s a bit hard to explain how the more secure, more protected mode is paradoxically for the more anonymous, more dangerous behavior; while the “less protected” mode is for normal operation which might involve vital personal data. I’ll let the UX wizards sort this part out.

(Semi) Portable Comet Framework

2009-03-23T22:52:00.001-07:00

I meant to reblog this at the time of the announcement: Sun has released a first alpha of their atmosphere project. The project was about extracting useful comet-y bits from Grizzly and making a standalone pluggable kit for comet.

It is its own small framework, and autodetects where you drop it.

Here’s a nice article. It looks to be Jean-Francois Arcand’s project – follow his blog and the project’s twitter.

How Much Would You Pay to “Learn to Pitch Big [Failing] Newspapers”

2009-03-23T17:58:00.001-07:00

If you’re in SF next week, and you don’t mind paying $15-20 for the privilege, you can come hear some people from the SF Chronicle and NY Times talk about how to pitch your (probably tech) company to them.

So they’ll write a glowing and informed article about you.

Wait, wait, wait … this is all wrong.

First, these are “reputable” newspapers, meaning they won’t necessarily write anything good about you. At most, they’ll theoretically assemble a balanced story, interviewing your competitors, talking to customers, maybe even your employees … or ex-employees.

Oh, wait, I’ve got this wrong again.

They aren’t going to anything like that … unless, maybe, you were already a big news-section story already. Else they will write something that’s like a watered-down blog post, without any specific expertise or authority, but with a couple of quotes. Newspapers like to quote because they can’t link. They’ll also mention twitter in the story, they can’t help themselves.

These papers do have a big circulation though, maybe that’s the appeal.

But it’s hard to tell their attention reach, or the “effective circulation” of your story buried in the tech or lifestyle section. How many people really read that? Are they influencers? Customers? Relevant at all to you?

It’s hard to tell. I can tell you that the people who are really interested might find the story … when it comes to them through the backdoor via some RSS feed or Google alert. But if they care enough to do the RSS thing and find you, then they’ll also have all the other, better, material about you that comes from all of the experts in your field who blog about you and also turn up in RSS and Google alerts. Ironic.

Next, these two newspapers are in dire financial straits. At this point, $20 probably keeps the Chron publishing for another couple of days. And why are they in trouble? Not just because people can read their content online for free – rather, it’s because in most areas of reporting, the big organs have no specific interest, capability, or credibility, and so no one cares what they write. The one thing they can do is send a foreign correspondent to Iraq or the White House, and maybe the correspondent has some credibility…

Wait, there I go again, the Times sold out on Iraq years ago, by their own admission, and so did most of the rest of the traditional press.

Ok, I give up.

I’m going to sponsor a meetup where newspapers can send people, who will each pay me $15-20, buying my attention long enough to tell me why I should care.

Silverlight Sound And Fury (You Know the Rest)

2009-03-20T18:34:00.001-07:00

So as not to bore regular readers, I’ll skip the jeremiad.

Bottom line: despite the hoopla at MIX over Silverlight 3 – which is an incredible platform – there were still no meaningful penetration numbers presented.

And while it’s great to see the platform revving and maturing, the various version make a development decision that much harder. If you have a new idea for a Silverlight app, and you imagine your target audience will have the plugin or is able to install it, do you aim for v3? v2? v1?

I also haven’t heard a word about any explicit program to drive Silverlight client installs.

According RIAStats.com – perhaps not the best source of detected install info, but … wait, I guess if it’s the only source of information, that automatically makes it the best – Silverlight is on 22.3% of their observed clients.

Microsoft SDS Change Eerily Reminiscent of WinFS Fate

2009-03-16T13:00:00.001-07:00

Last week Microsoft announced that they would be abandoning the ACE and dynamic entity (“property bag”) model for the SQL Server Data Services cloud data storage system. They would also switch from their REST data API (used in ADO.Net Data Services) to the old-school “Tabular Data Stream” wire protocol.

While Microsoft’s promise of more relational support was always a distinguishing feature of their cloud DB service, and while they tried to spin the news in that direction, it feels a lot more like when they abandoned WinFS and announced that, really, everything you could do with WinFS would work fine using NTFS and a whole heck of a lot of indexing. Maybe sorta true … but feels like a big step back.

Of course, big customers – large enterprises with SQL Server databases and lots of SQL code – would not want to see a change in their data layer and would prefer this move. But accommodating them is assuming that they are ready to become first-version customers of the data cloud at all. And I doubt this for two reasons.

First, any move to the cloud involves a trade-off of control which some companies are loath to make even if they are confident the system will work. Which is problematic because:

Second, anyone who has dealt with big databases knows that there is no magic. Despite the quest for automagic autoscaling self-tuning databases, no one, so far as I know, has made one that does all of this for really large enterprise applications. There are just too many application specific variables, not to mention poorly written app code that can cause trouble in proportion to the amount of resources you give it access to.

I do believe Microsoft has the engineering brainpower to try the problem, and are as likely as anyone to succeed. It’s just that I haven’t seen any evidence of a specific strategy or technology. Maybe if I were a bigger customer … but seriously, if Redmond had this problem solved (and it’s one of the biggest out there), they would either patent it or publish lots of white papers. Either way, it would be publicized and reviewed. A trade secret? maybe, but which Fortune 500 CIO is going to jump on that bandwagon and the cloud and the outsourced data stuff all at the same time?

To the extent that these large database apps could be made to behave without human intervention, there is likely to be a tradeoff in resources, and when you’re paying per GB or per compute-cycle, that equals a side order of more cost to go along with the entree of new greater risk.

The point is that the ACE/dynamic entity/REST model is well understood, performs, utilizes resources in a known manner. Not appropriate for every app. Not relational in the formal sense if at all. Not easy to migrate to. But it goes like the devil. So you’re getting something concrete in exchange for your risk and your dollars. Unlike a magical SQL Server instance in the sky.

Maybe there is magic in there, and I’ll be proven wrong. Or maybe 99% of the customers’ database needs are so small that it’s a non-issue, and Microsoft is really just competing with the thousands of hosting providers that will host actual individual SQL Server instances for you on a large server. But this change still seems to raise more questions than it answers.

Way to Compete, Guys…

2009-03-12T16:47:00.001-07:00

Microsoft’s app store … sure, why not? But that’s not the bit they need to take on Apple.

Microsoft and the smartphone is really a funny/ironic/sad story depending on who you are.

They had a true next-generation mobile OS starting back in ‘01 … It was really easy to code for – like GUI-builder, point-and-click web services, run-your-regular-.Net-code easy. And they were outselling pretty much everyone in total device count a couple of years later. By ‘06 they even had consumer friendly devices, in the Moto Q series and then the Samsung Blackjack. They were poised to challenge RIM for the big shiny belt.

And then Apple came along and wiped the smirk off everyone’s faces. What’s surprising is that no ‘softie seems to have circulated an “Internet Tidal Wave” memo about mobile. Or, if they did, no one paid any attention.

In the last two years, we’ve seen a continuing proliferation of Windows Mobile devices, but no fundamental change – or even speed-up – on platform evolution. If anything, we’ve seen a slowdown, as Mobile 7 devices seem to be at least a year away, and the “app store” is going to launch on Mobile 6.5

In case anyone didn’t already notice, v 6.5 is a great OS if it’s 2005, but a non-entity in the iPhone era. An app store? well, maybe … but a store by itself has never been the magic sauce in mobile (remember Verizon’s “vending machine”).

And with a “logo validation” scheme for each app? Developers violating the logo cert guidelines is not the problem. The problem is that there are too many different form factors for Win Mo devices. Used to be, practically anything could run the OS. Around the 5.0 era, they reduced the number of supported screen configurations, and a few other things.

But there appears to be little escape from the compromise Microsoft made to be successful on the enterprise side: it’s really easy to code a simple utility/productivity/line-of-business app that will run great on almost any Windows Mobile device. And it’s equally hard to write anything really cutting edge, because there is simply too much variation in device capability and performance, and that genie's not going back in the bottle.

Perhaps Microsoft’s best chance lies in forking a “consumer” mobile OS, with stricter controls over the handsets. On the other hand, Apple is clawing into the enterprise, so an artificial separation of consumer vs. enterprise offerings may be hopeless at this point.

Adobe Time-Warps Half a Decade Back, Will Still Probably Defeat MSFT

2009-02-27T22:49:00.001-08:00

Earlier this week, I went to see a couple of folks from Adobe present their latest progress on Flash Catalyst, Flex "Gumbo," and the "Spark" UI component framework.

As someone who does a bunch of Flex work, I liked everything I saw.

Especially since it was the second time around.

No, I didn't see this stuff at MAX, I saw it at Microsoft PDC in 2003 and 2005.

It was shocking how pleased Adobe seems with itself now that it's almost ready to release a design tool that generates XML and RIA code... since everything they showed -- and more -- was part of the earliest Microsoft Expression Blend alphas that I saw years ago.

The Microsoft product was code-named "Sparkle." But we won't get this confused with Adobe's "Spark" because (1) "Spark" refers to a different bit, Adobe's re-invention of lookless, templated controls, which Microsoft implemented in WPF and shared with the world at the time (around '04 or '05), and (2) because Expression Blend is already out in a 2.0 version, so unlike the Adobe products, it doesn't need a codename anymore.

Adobe even has yet another XML dialect to facilitate moving design assets through the workflow -- it's called "FXG." And it appears to supplement MXML quite well in specific areas, so that if you take MXML and add FXG, you get XAML. Not that XAML was de novo or anything -- the XUL and Java folks (desperate to stop writing Swing code) had been creating similar XML formats for a while. The Java community was especially fond of XML with tons of imperative programming constructs mixed in alongside data objects and calling it "simple and declarative." What XAML did was provide all the necessary power, while keeping it declarative.

Anyway ... Adobe should get credit for recognizing the right way to do this when they saw it. Namely, they realized which workflow tools were needed, embraced the idea of export from Photoshop and Illustrator to a vector markup with a visual editor with timelimes, and thence to an RIA build tool with a code-oriented IDE.

Now that they're finally getting this on track, Adobe is even more likely to trounce Microsoft in the RIA world. They have penetration numbers that MSFT can only dream of, and for a company that doesn't build real developer tools they're giving it the college try.

Which is kind of sad, since I believe Silverlight is a better technology with better language and tool support ... and ~~not any less~~ rather more open than Flash.

Want Help With Your Startup? Let It All Hang Out on Craigslist

2009-02-17T10:30:00.001-08:00

It's awfully easy to go looking for folks doing stuff the wrong way ... and to find it. So it's nice to be surprised by someone doing something amazingly, shockingly, frighteningly ... right!

I was greeted by a craigslist ad in my RSS reader today, one of many startups looking for folks to, essentially, work for free. I've written about why this is a bad idea before, and it's still a bad idea.

But there's a little more to this ... the poster (the company's founder presumably) posts a link to a wiki. Maybe it's genius, maybe a trainwreck -- either way I had to look.

On the other side of this link is a company wiki. An explanation of what the company is building; where they are in the process; their calendar; UI mockups with notes and the comment stream by the creators; and other items.

This is absolute genius, and it's so rare. Plus it shows the guts that most entrepreneurs fancy themselves to have, but lack when tested. I'm not commenting on their specific business/tech idea, I haven't thought much about that to be honest.

But it is so refreshing to see someone out there on the beach letting it all hang out as it were.

I work with a lot of entrepreneurs and most of them think that they're the first ones to think up some genius idea, and the best way to be successful is to either keep it stealthy and secret, or to sign reams of NDAs and non-competes with you before disclosing (cue music) their subtle and delicate brilliance.

Just writing that last paragraph, it's a struggle to keep a professional tone. These folks are usually (97%, there are a couple of specific exceptions) complete fools. And truly, they are fooling themselves, unconsciously trying to avoid exposing their idea to someone who might not think it's so good, or who might point them to the dozen other people doing the same thing. Generally speaking, the secrecy ends up being a contributing factor to their failure. Which, since startups are highly failure-prone anyway, they will deny anyway.

That's why I was so thrilled to see this post. The founder is saying, "If you want to try and 'steal' my idea, you go ahead. But if you really believe there's a bunch of money in it, wouldn't you want to work with other people who believe the same thing and who have the will to execute? And if you go off with it and succeed anyway ... you're still helping me because you're establishing the category, while I plan to work nights and sweat blood to execute better and faster than you."

The ad is reproduced below. I was going to link it, but interestingly it has been 'flagged' for removal from craigslist. It's hard to imagine why -- the whole scenario seems rather more legitimate than the typical ad in the category. Perhaps the allusion to potential full-time work disqualifies it from the free "gig" listing ... but I think a startup seeking essentially non-paid volunteers in whatever capacity they can afford qualifies as a part-time or temporary arrangement.

Technical Wizard / Web Developer Wanted | Internet Startup (sunnyvale)

An internet startup is seeking a highly talented web developer

If you have experience with either: PHP/MySQL, Python, or Ruby we would love to talk with you. This a very exciting startup opportunity with massive potential. At this stage, we are looking to bring aboard those who are seeking equity share in the company. We simply do not have the capital to fund salaries.

For more information, please have a look at: http://wiki.kunsoom.com

All of the pertinent information will be included in the wiki page. Thanks for your interest in the project! We look forward to hearing from you.

Adobe and Microsoft Get Into It Like Children on the Playground

2009-02-16T19:45:00.001-08:00

A week or so back, Adobe exec Mark Garrett got a bunch of attention for insisting that Microsoft's Silverlight effort has "fizzled."

Microsoft promptly screamed back that it wasn't so, pointing to the inauguration video stream, and a few other factoids.

What makes this truly schoolyard funny though is what happened today when Adobe "announced" it was bringing Flash 10 to phones. This seems just as dubious as Microsoft's oft-repeated plan (since as far back as '05, when it was WPF/e) to get Silverlight onto mobile phones ... by last year ... which obviously didn't happen.

Meanwhile, for years, Adobe has been pushing a weak technology called FlashLite for mobile ... and for a variety of reasons it has never been a usable option for content providers to deploy Flash content or apps.

For both Microsoft and Adobe, for both PC and phone applications, the critical metric is current "content-ready" penetration. How many devices are ready to run new Flash/Silverlight content off of the web today.

In this 2x2, the only square that's solidly covered is Adobe's Flash on the PC. "Ready" penetration of Flash 9+ is near 100%.

On PCs, existing install base is critical because of locked-down corporate networks that won't allow end-user installs. Microsoft needs to stop talking about download numbers, or numbers of people who "can access a PC with Silverlight," and start doing anything it can to get these ready penetration numbers up.

On mobile, the barrier is user confusion over configuration. Vendors could push the updates to phones, but in nearly 10 years of smartphones, only Apple has done much of this. Windows Mobile 6 has an updater ... and in over a year I don't recall it ever updating a darned thing.

Flash Lite trumpets a large "installed base," but these are strange installs, where the runtime (but not browser integration) is baked into the phone, and there's no reasonable way to get new Flash content onto the phone, either via web pages or download.

Both of these players are big on bluster and have been for a long time. Meanwhile, developers are left with few options for all of the smartphones in the world that don't an apple on the back.

Is It Too Early or Too Late for an Open RIA Design/Dev Toolchain?

2009-02-13T11:39:00.001-08:00

I was playing with the Raphael JavaScript graphics library (a sort of script-based, cross-browser, implementation of SVG) and started thinking how helpful this library would be in creating a browser-based (as opposed to plug-in based) RIA.

That lasted for about 15 seconds before I remembered that creating large, non-trivial RIAs generally involves designers, and most designers don't like creating vector art by coding a set of "path" statements, or animations as a collection of key-value pairs and millisecond-based transition times.

That's why tools like Microsoft Expression, and Adobe Illustrator, Catalyst, and Flash exist.

And why Adobe and Microsoft are investing so heavily in the designer-developer workflow: the ability of designers to turn graphics and animations into app skins and interaction which are immediately available to coders.

In order for an open RIA solution to be competitive and realistic -- whether it's open in the pure-browser sense, using JS via dojo.gfx, or Rapael, etc., or whether it's via an open plug-in (Java/JavaFX seems like the closest, though it's not 100% open yet and may never be) -- this full toolchain needs to exist.

We need to be able to export vector art from mainstream design programs such that they can be incorporated as assets into the RIA. It doesn't matter if this is via SVG, XAML, AI/EPS, or something else entirely. What does matter is that the import/export is robust enough that designers -- whose jobs, after all, include making stuff look just right -- are confident that what they design is what end-users will see. The Microsoft and Adobe tools can do this. To date most OSS attempts cannot.

Next up, we need a truly usable, designer-friendly authoring tool for animations and interactions. It is often argued that some standard tools (*cough* Illustrator *cough*) are not paragons of usability themselves. No matter -- it's hard enough to get converts.

Happily, there seems to be emerging some consensus among the big vendors about how these tools should work (both on-screen and in terms of in intermediate data formats). That blueprint lowers the risk and challenge for an open source contender.

The biggest obstacle remaining is a classic open-source triangle-of-trouble:

The toolchain/workflow will not be viable until it is quite solid, since the commercial alternatives (Flash, mainly) are so entrenched.
It's hard to get enough contributor man-hours against such a huge project without an active user base.
Since the user base is not developers, the bootstrapping for #2 that makes many OSS projects work (devs are tolerant -- even excited -- about getting up on an 0.1 release) is unlikely.

The Second Law of Thermo-Specifications

2009-02-11T16:32:00.001-08:00

A couple of days ago, Adam Milligan posted an eponymous "law" on the Pivotal Labs blog.

It includes a corollary stating, "The full definition of correct behavior of code exists in the tests for that code."

Now, there seemed to be something fundamentally off about this proposition ... and I wanted to figure out what that was.

Surely Mr. Milligan doesn't mean this in the trivial, tautological sense (define the spec to be no more than what test cases happen to exist at a point in time) ... even though Agile dogma often borders on that view, pretending it's some kind of paradoxical path to enlightenment like a Zen koan.

The comments to his post start to touch on "spec" vs. "test," and whether 100% test coverage is practical, desirable, or conclusive.

Of course even 100% code coverage, with a missing code path and a matching missing test and behavior spec equals ... test success, and wrong functionality.

In such a case, though, the problem has been externalized from the Agile context and put onto a faceless "business person" who "doesn't get" Agile because he or she actually wants to plan ahead and describe some certain specs that persist over time.

This is a neat trick. In physics, all sorts of magical things can happen if you look only inside of one context (or frame of reference) without looking at what's happening outside, or at what's happening to the frame itself. In finance, there can certainly be a free lunch ... if you can make its cost into an externality and remove it from your model.

Once you make the tests and code-based behavior spec a part of the application you're building -- and clearly, once it's a real cost center as well as a critical deliverable part of the project, you have done so -- then you are in a sense simply externalizing that troublesome human interaction (specification, functional analysis, planning). Yes, the tests match the code under test. But seen at a different level of abstraction, it's just another flavor of interface-driven development, or debugging.

Most engineers would love to realize the dream of self-describing, self-verifying code, whether that description be some kind of formal model, or a textual DSL (as with Microsoft's Oslo), or a set of tests and code-based behavior specs. And, indeed, these systems all improve the transparency of the code, propagating requirements from the outside in, and revealing when they are not met.

But even with the most "bought-in" business stakeholders, it is impossible to escape the the outermost specification layer, the one with the humans in it.

Windows 7 Keeps Suspending in VMWare? Change Your Power Settings

2009-02-06T15:26:00.001-08:00

I'll admit, I fell right into this one.

My Win 7 VMs kept suspending while I was off working on another machine. I'd come back to the host machine and see VMWare Workstation idling with the Win 7 instance frozen in suspend.

Hmmm... Doh!

The default power configuration for Win 7 -- even when on the machine is "plugged in" -- includes dropping into sleep mode after half an hour.

This, thanks to VMWare, translates into a "Suspend Guest" operation.

Maybe Vista shipped a default power configuration like this. I don't remember it being the case, but, to paraphrase Pulp Fiction, "[Vista] may [save power] like pumpkin pie, but I'd never know 'cause I wouldn't [run] the filthy mother****"

Think About It: Do You Really Want Your Engineering Done "For Free"?

2009-02-04T16:46:00.001-08:00

I enjoy reading the posts on craigslist, wherein naive entrepreneurs go looking for free software engineering talent. I know this sort of ad (no pay / equity only) bothers some people -- as though it somehow has real impact on the real jobs and pay in the industry -- but I think it's just fine. I mean, if you want a service for free, there's nothing wrong with asking. And if you are in the mood to work for free, it's nice to know where to find opportunities ...

Of course I am not referring to recruiters for genuine charity / volunteer work, where the goal is to provide services to a population that isn't served by the normal market mechanisms. Rather I'm talking about real for-profit, we're-gonna-be-the-1-in-100-successful-startup-and-make-a-boatload-of-cash kinda company.

The funny thing is that it's hard to imagine these wantrepreneurs looking for any other kind of professional services for free. Not because they feel coerced by social norms to offer money, but simply because they know better than to want what the "free professionals" have to offer.

I don't see these guys saying:

I need heart surgery and I'm looking for a doctor who wants to keep their resume fresh. No compensation, but I'll thank you if I wake up.
Going through a brutal divorce. Looking for a sharp lawyer to keep me from losing my shirt -- no pay, but I'll buy you a drink if we come out ok.
Need a corporate lawyer and patent attorney. Equity only.
Startup needs business travel on the cheap. Seeking a pilot and aircraft, not so concerned with FAA licenses or airworthiness.
Building a new facility -- need architect, structural engineers, environmental compliance, project managers, and laborers. If we make a bunch of money, we'll pay you at some point in the future.
Love my new car, but it needs engine work -- looking for a mechanic who'll work for free. Someone with little or no experience seeking to build up a resume is ideal.

Now, it is true that in the late 90s, during the dot-com bubble, landlords, attorneys and other did take equity. But they did so in addition to -- not instead of -- cash payment. And the odds of making bank on equity back then, while still long, were stunningly better than they are today.

I know what these guys are thinking. They're thinking "I just don't have a budget to pay for real development, and anything I can get for free is better than the nothing I can afford otherwise."

Well, in some cases, perhaps. But that's setting an awfully low bar. Likely too low to allow for a real chance of success. That is, the entrepreneur is theoretically pouring all his time, money, heart and soul into starting this business. But he's willing to gamble the whole thing on the questionable code he'll get out of someone who has nothing better to do to pad a resume while unemployed? Doesn't quite add up.

Or maybe he really wants a "partner" ... only it's unlikely this developer is going to see 50% equity. And anything less likely means the developer is along for the ride while the entrepreneur follows the same hare-brained decision process that led him to advertise the non-paid gig in the first place. In any case, the developer has no real clout when push comes to shove. He's a silent partner, just along for the ride.

True, there are a few brilliant technologists out there who simply don't need any (more) money. They have cashed out of a startup, or they run another business on the side that covers their expenses. And if you can hire one of these guys, then bully for you. But I think they are more likely to spend their time where they can make an impact on a real business, product, open-source project, or non-profit.

As for the entrepreneurs who get someone to bite ... they don't know what they're in for: a few months or more down the line, they have (perhaps) some kind of a half-baked system. They need the loyalty of that unpaid assistant to modify and improve the code, while the assistant has now seen through all the "about to receive funding" promises presented at the outset. And as the limitations of the half-baked system slowly become clear, the founder may wish to replace the developer, or throw the whole thing out and start over with a new iteration.

But unless the original developer was a real sucker, all that equity means unrecoverable dilution in the cap structure. The more equity involved and the earlier the start (meaning smaller valuation), the worse this situation turns out to be. It could even come to pass that a potential investor one day passes on the deal because of these problems in the cap structure.

Is that Service Really a Scalable Cloud or Just Full-Service Web Hosting?

2009-01-26T11:26:00.001-08:00

A lot of cloud stacks, or cloud app platforms promise scalability for your app, "With a little EC2 in every box!" (TM). There is a big catch and a little catch, though, and if your app gets big, then either or both of these may be a deal-breaker.

First, and most important: Running a vanilla RDBMS (e.g. MySQL) in a VM somewhere does not make it magically scalable. Read that sentence one more time.

Some cloud offerings integrate tightly to the traditional sort of DB instance you might attach to your web app on a single server. Examples include Heroku, which applies your Rails migrations to a PostgreSQL instance, and Stax, which offers MySQL.

The great thing about these environments is that they don't require significant changes to your standard app built on their supported platforms (mostly Rails and Java variants). Upload, minimal admin, and IJW (it just works).

That's turn-key, full-service web hosting, right there. It's beautiful -- in fact, in an OO and Rails course I wrote, I chose Heroku for deployment as a way to let students get something up and running on the web without getting into the operations/deployment/tuning aspects of Rails which deserve their own course.

But if your app gets large -- or just uses large datasets -- the database is rapidly going to be a bottleneck. Scaling out an app logic tier to a dozen EC2 instances automatically may sound good, but it won't do a thing for a DB-bound app (it may make it worse). And these databases don't scale out without a little architecture, planning, configuration -- all of the things which these cloud platforms are designed to avoid. And which, on some platforms, you cannot do at all.

For example, so far as I can tell on Heroku or Stax, there is no way to even configure multiple servers and replication, which is just a minimum starting point for scaling a DB to multiple machines. Stax may allow for a logical sharding setup, but it's not clear how one would control which VMs and disks the databases run on. Rightscale seems like the kind of firm which would specialize in the management scripts / meta-API that one would need to automate sharding, but the sharding option doesn't appear in any of the models on their website. With replication, which Rightscale does offer (though they're not exactly an app platform, more an infrastructure play), you get to this, still limited, picture:

Other cloud platforms offer datastores specifically designed to scale out, including Google App Engine, 10gen, and others. These platforms offer a non-relational or pseudo-relational datastore, with different data access APIs and a variety of restrictions relative to what you may be used to. These datastores are architected to scale easily, but there are real tradeoffs that must be considered. In fact, if you don't know these tradeoffs cold, you are not the right person to be making this platform decision. Get on craigslist and hire (or borrow) someone who knows the stuff.

The other catch is that whichever approach you choose, these vendors are offering you convenience, some outsourced operations management, and (in some tiers) elasticity and scalability ... but they are not offering cheap compute cycles. That is, if you know you'll need a large, predictable amount of raw compute time, then know also that you're paying a premium to do that computation in one of these environments.

A friend who has designed, built and operated feature film renderfarms for a number of studios confirmed that he has, on a semi-regular basis, analyzed the costs of remote VM-based datacenters (e,g. EC2) compared to their physical ones. Because the studios use these machines intensely, and are consistently consuming raw compute power, the local physical servers have always made more sense.

What does this have to do with your web app and datastore? Well, suppose you have designed your app to leverage a scalable datastore. These may not be tunable, may not perform fast, and may require you to do certain operations in code which traditionally are done in the DB. You may never see these slow queries or operations ... until they show up in your bill. That is, if the system is truly elastic and scalable, it will apply resources as needed to handle your work. If your query or sort or filter takes a lot of CPU cycles, the cycles will be made (almost) instantly available, so the user always sees your app perform well. And then you'll pay for all those cycles or instances at the end of the month.

Either way, there is no free lunch on the data persistence side. Which is not in itself a reason to avoid cloud environments. But it should be a bigger part of the conversation than it is today. And it absolutely must be part of the conversation, if larger businesses are going to move their services into the cloud.

Using AppEngine -- Or Similar Datastore -- To Integrate Complex Legacy Data Formats

2009-01-21T19:24:00.001-08:00

I gave a lightning talk last night at the SF Bay Area App Engine Developers, showing some work I've been doing to represent gnarly legacy records in AppEngine so as to maintain source fidelity, minimize upfront analysis, and make them easy to integrate with other systems.

I had started with an XML record that I wanted to parse and represent in the datastore -- without knowing which tags and structures would be present, since this format had, ahem, evolved to obscurity over time, as often happens with real-world legacy records.

Before I talk about my approach, here's why I thought this effort might be interesting to the group: a lot of data structures have a tree structure in common with XML. From C structs and file blocks that include a header, telling which types to cast the next n bytes to (and so on inside of those) ... to mainframe "structured data" records I've encountered which consist of nested records, parsed recursively, with their meanings occasionally opaque, lost to history, or belonging to some partner company.

My approach -- which is simply to create a mapping of how to assemble and disassemble the records -- enables a record to be stored in a single App Engine record. But not as a block (or blob) -- rather with fine-grained addressable fields that are easy to talk to using the GAE Datastore API.

In my case, since my original was XML, I created a mechanism similar to a tiny subset of XPath describing the sequence of tags where a data element lived -- but with the characters changed so that it would be Python and GAE-friendly. That is, instead of "/foo/bar[2]/baz" I used _Foo_Bar__2_Baz.

This let me "flatten" the XML into a set of key-value pairs, while allowing that the XML might contain arbitrary structures injected by others ... and that I might want to inject my own extra structures. This arrangement is perfect for the Expando models in App Engine Datastore, or any similar store (e.g. Hypertable, which is modeled after BigTable, or Microsoft SQL Data Services which uses SQL 2008's sparse tables to similar effect).

So now I can store and retrieve my records. Any fields/subrecords which I understand and care about, I can easily work with from other systems, by mapping to the appropriate "key" in the stored record.

For example, if I'm storing a bunch of catalog data, and another system just cares about enumerating each "Product" with "Name" and "Price," then I can create a facade or wrapper in GAE that maps, say, Price to _Strange_Old_Way_To_Represent_Current_Price, and we're all set.

To be sure, there could be performance issues if you tried to use this to create arbitrary queries and reports against the data. That's not really the purpose and, in my experience, if there are no "shortcuts" to processing these legacy records, then the business folks are not used to being able to make an OLAP cube out of them either. (They probably have a batch or offline extraction process.)

Nonetheless, it's another tool in our chest when we need to work with systems and data that have been out in enough real-world battles to come home scarred with lots of cruft.

Twitter's Underwherlming (Former?) Architecture Problem

2009-01-19T13:22:00.001-08:00

I recently came across this post from May 2008 comparing Twitter traffic and the Options Price Reporting Authority data feed. Needless to say, the stock market feed is many orders of magnitude larger, at 700,000+ messages per second(!)

It's also not the fairest comparison in the world on its face, for a variety of reasons: the OPRA data system was planned (Twitter met success more or less by accident), Twitter is minimally funded, etc.

A more relevant comparison, in my opinion, is that provided by newzwag, which presented its performance challenges, triumphs, and secrets at a recent SF Ruby meetup.

newzwag's site and trivia game is built on Rails, started small, and had to grow to meet traffic driven by Yahoo and the Beijing Olympics to 9 million pageviews per hour (using a total of a half-dozen machines or so). And lest you think this is a content site served out of a cache, most of the traffic consists of data writes by game players that then need to be ranked, published, etc.

As far as I can tell, that's somewhat larger than Twitter, even considering that Twitter has grown 3-4x since last May's stats.

newzwag's solutions, which they share here, are a study in sanity, reasoned problem solving, and smart efficient architecture.

Without the timelines or resources of a stock-market app, newzwag produced a nice solution that -- at least in hindsight -- appears drama-free.

Interestingly, a newzwag - Twitter comparison can be enlisted to support a variety of different startup social narratives.

One narrative is that an amateur-hour effort yields amateur-hour results, and aspiring startups shouldn't fool themselves into thinking that they won't need old-time Architecture and Sophistication to scale.

A different narrative says it doesn't matter -- if Twitter's success is your worst-case scenario, you still win. That is, build it fast, get it out there where people can try it, and you should be so lucky as to need a real re-arch to fix your scaling problems. In this model, both Twitter and newzwag played it right -- newzwag because they knew the Olympics would provide a narrower time window to showcase their system, so they managed risk against that stricter goal.

And yet another narrative says if you accept these two stories, you still wouldn't want your brokerage transaction flowing through a system built to "see what sticks," and hence Web 2.0 startup methodologies stare at mission-critical business apps from across a huge chasm.

I see this last story as persuasive but also as a big opportunity: there is a chasm, to be sure, but it needn't be quite so big. There are legacy mainframe apps that can speak webservices. Every manager in a big company wants their product to be "100% critical" even if they could create more value by admitting that a lot of nice-to-have two-nines business apps are the real bricks in the wall. If enterprises can get better at separating their Twitters from their OPRAs, they can make more money and have both.

New iPhone App Store Rules Take a Step Closer to Scriptable Apps

2009-01-14T18:22:00.001-08:00

A lot of folks commented today on newly-approved web browsers appearing in the App Store. Or, more precisely, a handful of apps using the existing web browser widget to offer a slightly tweaked browser experience.

While iPhone apps could include the UIWebView component before -- and indeed this has proven a popular route to getting hybrid native/web apps up and running quickly -- today's change is about allowing apps that "duplicate" a built-in feature of the phone. And one of the fundamental characteristics of any web browser nowadays is that it is thoroughly scriptable.

If you build an app this way, it already includes a scripting environment ... so the question (since scripting and dynamic apps are verboten on un-jailbroken iPhones) is how far one can let the scripting go and still pass muster with the App Store overlords.

Using stringByEvaluatingJavaScriptFromString we can inject script into the browser ... including script that pulls data back out.

And although the JavaScript bridge is "one directional" compared to the OSX desktop API, there are workarounds such as registering a protocol handler to receive scripted "requests" from inside the page ... or by hooking decidePolicyForNavigationAction with a script-initiated navigation request (disclaimer: I haven't checked to see if this is in the phone API, but it seems plausible) to signal the availability of data.

So native code becomes effectively scriptable. Or, for an even less controversial but perhaps equally powerful route: just inject a bunch of JavaScript API libraries into the browser and keep the scripting (and more of your app) in Safari. That's not too different from pointing a browser at a web site (where the page loads various scripting libraries) ... except that underneath it all we are in native-caps mode ...

Unless I'm missing something here, a somewhat ambiguous situation has gotten thornier with the admission of this new class of general purpose browser apps.

Windows 7 Product Name is Missing a Feature

2009-01-12T20:15:00.001-08:00

I didn't feel strongly one way or the other about the Windows 7 product name (i.e. "Windows 7") ... until recently when I wanted to troubleshoot the Azure SDK on Windows 7. (Apparently Azure on 7 has worked with the M3 build for at least one intrepid forum poster, but it's not behaving with the beta build for me at the moment.)

I started searching newsgroups, forums, blogs, etc., and realized that "Windows 7" is not a great search term.

On an engine like Google you can put quotes around it, specifying exact phrase, but some other full-text search systems don't seem to want to keep the Windows and the 7 together. Or perhaps they have an index by single words, and they link the results together to match your phrase later, but once you throw in other terms like SDK and Azure, the matching engine becomes a little more promiscuous, offering you a "promising" combo of Azure, SDK, and Windows ... or SDK and 7 ... as a higher-ranked match. Making it, in any case, rather harder to find what you want.

One-word product names, like "Silverlight," "Vista," and "XP" work a lot better for this kind of search.

Which is perhaps a reason that folks include the release name with the version number on products such as Ubuntu (Hardy Heron, Intrepid Ibex, etc.)

So ... what would be a good nickname to put next to Window 7?

Ruby and Python as Cloud Lingue Franche; Ruby/Rails on 10gen

2009-01-12T09:46:00.001-08:00

Not sure how this one slipped past me, but 10gen announced support for the Ruby language and most of the Rails framework APIs on their open-source cloud service last month.

This addition is great news for 10gen and for cloud computing (the hosted-application-platform flavor, not the hosted-hardware/datacenter flavor).

For 10gen, support for a well-known API and app model is a huge bonus, which makes it easy for people to move an app into the cloud without learning and coding to new APIs, and also lowers the perceived "lock-in" involved, should the move not work out.

Their original JavaScript platform approach, as I've written before, is problematic not only because folks are unlikely to have meaningful (for their business) apps lying around to try mounting in the cloud, but more so because there is no standard server-side JS API set. A half-dozen companies offer a JS app server or cloud and they all have different platform APIs for even the simplest things, such as reading HTTP request variables, or deleting a session.

10gen takes a big step forward, joining Stax, Heroku, and morph labs in supporting Ruby on Rails in the cloud.

This move also reinforces another emerging trend: Ruby and Python serving as lingue franche for cloud app stacks. While many cloud offerings support JavaScript or other languages, Ruby and Python seem to be emerging as the ones with broadest support: 10gen will support both; AppEngine supports Python and a language-to-be-named-later; Stax supports both; Azure will likely support IronRuby and IronPython (some Python apps can already work in Azure).

Of course, the language is only half of the battle -- there are the APIs to deal with as well, and issues will typically arise where the impedance mismatch is highest with cloud-related infrastructure. E.g., cloud databases are mostly non-relational and don't support SQL ... so an ActiveRecord or SQLAlchemy API won't work on 10gen's 'grid database' (a reasonable tradeoff for simpler scalability.)

Even so, it is starting to appear as though one could write a lot of core business logic using, say, Python, and expect it to run unmodified on most vendors' clouds. Not a bad position to be in for the Python folks.

Another Windows 7 Milestone: Bounce vs. Hibernate vs. VM Suspend Times

2009-01-11T12:56:00.001-08:00

Lately I've started running the Windows 7 Beta for some development experiments, using VMWare's fantastic dual-screen support. As I've written before, the general experience is great, even under virtualization with 1GB of RAM, and having it wall-to-wall on multiple displays makes the illusion more convincing.

An interesting thing I've noticed is that my old habit, when I want to stop working in a VM and free up the resources, is to suspend the VM. This action is roughly (but not exactly, depending on the VM you're using) equivalent to "hibernating" a laptop (S4 power state) -- memory is mapped out to a file and the device is powered off.

I usually do this because this hibernate/wake is faster than a shutdown/boot-up, not because I'm trying to save my actual work state (open apps, etc.) Especially with Windows server, but even with XP and Linux, this approach is the faster way to hop in and out of a work session. On the laptop, it's a way to save the battery power involved in a longish hard boot.

In Windows 7, VM suspend/resume (==hibernate/wake) seems to be slower than shutdown/boot. That is, even with no user apps running (which could take up an arbitrary amount of memory and thus lengthen the map-out / map-in time), boot seems faster. I say "seems" because I have only 2 machines to play with, and they are not clean images just for this test, so I won't pretend they represent absolute objective truth.

What does this mean?

It would appear that the boot process has been cleverly streamlined so that a cold machine gets to a running, usable state before all of the additional services and apps have fully loaded and gotten running, and that this is orchestrated using knowledge that a white-box VM player doesn't have.

Some folks may point out that having to reboot an OS is itself questionable ... and indeed the boot is optional -- I only reboot my XP desktop every couple of months when some security patch or other requires a restart.

But in the world of laptops and netbooks, things are different: every minute of juice is valuable, so there's always the consideration of the cost of a hibernate/wake vs. sleep vs. leaving it on with the LCD off. And that equation has just gotten a little more interesting: for Win 7 on a laptop, if you're not going to be using the machine for a while, it may turn out to be faster and use less power to do a shutdown, and just reboot later.

While this may seem like a fairly inconsequential gimmick about boot times, it is a step in the right direction as we look at the huge array of gadgets we all use and which eat a ton of phantom power. The Windows PC is kind of the holy grail for a fully-off / instant-on experience, and Win 7 appears to take a measurable step in that direction.

Embarrassment of Meetup Riches and a Suggestion for Compelling Talks

2009-01-07T22:57:00.001-08:00

In the Bay Area, we are fortunate enough to have so many great tech meetup groups that there are frequent collisions, and it's always a bummer to pass on what promises to be a great presentation.

On January 20, the SF Java group has three sharp guys presenting on Scala.

Meanwhile, down in Mountain View, the Google App Engine group has a "hacking..." talk that with a handful of presentations including an update from Google AppEngine Product Manager Pete Koomen.

What to do? Hmmmm...

Ultimately, I decided on AppEngine.

Principally, it appeared that more new, not-readily-available-today-on-the-net material would be presented at the AppEngine group. Lightning talks give a forum to quirky thinkers, very early startups, and other interesting folks, while representation from the mothership might be able to offer a little detail or timeline on upcoming features, like large BLOB support and the next language.

The Scala talk just seems less likely to include info that I can't get from existing resources.

Which leads to the following conclusion about what makes for more compelling talks, at least for an audience of me:

A focus on information that is not readily available, and which gains from the presence and experience of the speaker.

So, for example, I've seen many talks on "my cool [insert language] library that does [insert function]."

A fine topic. Now in the execution, perhaps best to talk about the problem being solved, how you solved it, what tradeoffs were made, constraints dealt with, any magic foo inside ... rather that a bunch of examples showing how clever/elegant the external API is and examples of what one can do with it.

Not that the latter is unimportant, but that the latter is (should?!) be readily available from the online docs/examples or the presentation notes; whereas the former represents the specialized knowledge and experience of the library's creator.

On the Wide Range of Ruby Talent: Rails as a 4GL(?)

2008-12-30T15:23:00.001-08:00

For a long time, I was puzzled by the extremely wide range of talent level among Ruby talent: more than any other language I could think of, "Ruby Programmers" seemed to range from the truly clueless to hardcore accomplished engineering veterans.

I had more or less chalked this observation up to the language and APIs themselves -- easy to get running with, highly intuitive ... and yet packing deep and powerful features that would attract the more sophisticated (easy C integration, metaprogramming, continuations, etc.)

On top of this was Rails with its famous screencasts showing database-backed websites automagically constructing themselves, a feat that got novices very excited, and reminded experts that they should be asking harder questions of any other framework they were using for the same kind of app.

But lately I've come up with another -- or perhaps stronger variant -- hypothesis.

Rails itself aspires to be a 4GL -- a DSL together with a framework, specializing in database-backed, RESTful web application development.

It appears that some programmers see (or want to see) just the 4GL parts (the DSL, the declarative bits, the "conventions" and the "magic") while others see a 4GL in close symbiosis with a 3[+]GL (Ruby) for doing procedural work.

In some sense, both groups are seeing accurately -- for certain problems. That is, apps that truly hit the "sweet spot" for which Rails was designed, and which do nothing else, can be treated as compositions of 4GL bits much of the time. Highest productivity, get your check, go home, watch a movie.

For other problems, additional logic and/or integration is required.

And here's where the two groups of programmers part company. The pure-4GL-ers want to search high and low for a plug-in, gem, or other black-box component to solve the problem. Even if the black-box component is harder to use, poorly documented or maintained, does not include a test suite verifying it does what it claims, this group of coders wants the plug-in. They'll search and argue and try different ones even if the functionality is 15 lines of Ruby code. But they won't write the functionality themselves.

The other group just writes the code. Perhaps they even bundle it into a plug-in and throw it on github. They also do things like submit patches to Rails itself.

Depending on the situation, either approach might be fine ... but: when something doesn't go right, in terms of logic or process or performance or data integrity or something else ... the first group hits the wall. All of a sudden it's a "Big Problem" ... whether or not it's really a big problem. That is the Achilles heel of all 4GLs: if you view them as "closed" systems that require pluggable components then anything outside their bounds becomes a "Big Problem."

And lately I've watched as some participants in the Rails community systematically leap tall buildings in a single bound when they encounter them, while others literally sit on their hands waiting for a plug-in to appear (with no track record or test cases), which they then gratefully throw into their app and hope for the best.

Fear of the Driver Disk

2008-12-29T22:20:00.001-08:00

The problem of bloatware/crapware on retail PCs is well known -- to the extent that Apple makes fun of it, pointing out the absence of such software on new Macs, while PC tools exist just to clean it.

But bloatware has a less-famous, equally annoying sibling: all the garbage that brand-name hardware devices install off their driver or utility disk.

Pick up a peripheral -- printer, web cam, DVD drive -- from a major brand, and if you follow the automagical installer on the drive disk, you'll get a half-dozen apps that you may not need or like. In some cases, they're just bad apps (that have a habit of arranging to start at boot), while in other cases they can destabilize a perfectly-running system.

The problems are that

in some cases you do need these apps, because some hardware features require "support software" to be present, and don't fully leverage the many built-in drivers and control panels available for free in Windows ...
most hardware companies internally view the driver/utility software as an afterthought, writing it hastily, testing it inadequately, and staffing it with ... well ... whomever they can find.

There are two main remedies.

In many cases, getting an unbranded or "strange-branded" device is a smart idea (provided you know what you're getting). I've found these devices have straight-forward, minimalist support apps, make great use of built-in Windows drivers, and don't put any garbage on your system -- for the simple reason that they don't have the resources to write a bunch of half-baked apps, or to form "distribution partnerships" with people who do.

If I do have a brand-name product, I generally attempt to install it without its own driver disk, no matter what the instructions say. In many cases, the device is fully supported by Windows out of the box; in other cases, some features may not be available -- but I may not need them. (E.g., If I wanted to use my digital camera as a webcam, that would have required the vendor driver disk... but I have never wanted to use that feature of the device.)

And if that latter approach fails, it's pretty easy to uninstall the device or otherwise convince the PC it's never seen the device before -- so that you can go the RTFM route and use the supplied disk.

Stax Brings more Standard App Models to the Cloud, Marginalizes Custom Platform Further

2008-12-23T15:25:00.001-08:00

Stax, which recently launched in private beta, is a cloud hosting/deployment/ops platform based on Java appservers. The coolest thing about Stax is that it offers many flavors of JavaEE-deployable applications, including Python apps (via Jython) and Rails (via JRuby) with ready-to-roll built-in templates.

Stax has a very AppEngine-y feel, not just on the website, but in terms of the SDK interactions, local development, etc.

This is good news for all of the popular platforms ... and bad news for those rattling around the corners with non-standard APIs. As the app-hosting industry continues to mature, the emphasis will clearly be on established players like Rails, ASP.Net, JavaEE, Pylons, et al. at the expense of guys like AppJet.

It's not about the language (JavaScript) but the about learning a set of APIs, patterns/practices, and sustaining a community ... based on a roll-your-own platform.

It is true that some of these built-for-the-cloud platforms were designed from the start to default to hash-style or big-table style storage -- popular for content-oriented cloud apps because of its easy horizontal scaling -- where the "traditional" platforms focus on relational stores and have a variety of bolt-on APIs for cloud DBs.

But now that there are so many standard alternatives, it is unlikely developers will pay any custom-platform-tax no matter how elegant that platform might be.

We May Look Back Fondly on Our 68% Project Failure Rate

2008-12-18T13:26:00.001-08:00

The report [PDF] referenced here -- focusing on failure originating in the business requirements realm and offering a "68% project 'failure'" headline -- inspired two thoughts.

First, it remains a head-scratcher why nearly every project, despite talk of risk analysis and mitigation, expects to be in the 32% success area. Even if a company has the best resources (and generally it will not), many causes of project failure originate in organizational factors -- friction, process, politics -- and externalities (e.g., no budget this quarter for the tools in the original plan).

Since these issues are rarely known -- and, I would argue, actively denied in the sense that whistle-blowers and problem-spotter are systematically excluded from planning (at best) or forced down or out -- the average "expected value" of the d100 role is way less than a 68.

In startups, I've heard the excuse that the whole company is a "long shot" -- as though that justifies taking on disproportionate risk in each project implementation.

In enterprises, it seems as though management is so used to failure (in a timeline or budgetary sense), that they have simply redefined success to mean anything that they can pull off in their tenure -- and if that means a system that kinda works, but no one likes, shipped in 200% time and 400% budget, well, that's just the "reality" (which, to be sure, it is once all other possible paths have been discarded).

This redefinition of success also has the side effect of removing accountability and pretty much assuring a nice bonus and making strict accountability impossible.

Another thought inspired by the report is how the gap between enterprise development and small / startup development is widening. On the one hand, large businesses could benefit from the high-productivity tools and agile approaches popular with startups; for a variety of reasons ranging from policies to personnel, they are not exploiting the latest wave of technology, and it's costing them.

On the other hand, what they need regardless of tech is solid analysis and estimation capabilities. Analysis and estimation are possible, but hard, and the agile camp has moved mountains to try and reframe the problem so that they can advocate punting on all of the hard parts of these disciplines. That works great for the hourly agile consultants, but inside large businesses and large projects, it just doesn't cut it. A business needs to be able to attempt to estimate and plan months or even years of a project (hence the prevalence of "fake" agile in companies that purport to use it).

The fact that the business does a horrible job with the estimation today does not mean that the organization (which, generally, is run by business people not developers) won't keep planning and targeting in the large.

The result of these two pieces (different tech, different process) is that enterprise and small development are moving farther and farther apart, which is a damaging long-term trend. Ideally, these two groups should be learning from each other. They should spend more time moving in the same world.

The enterprise learns that it probably doesn't need JavaEE and Oracle and a big planning process for myriad internal utility apps that could be done with Rails at a fraction of the cost and effort. The small company learns that relational integrity, transactions, estimation, and operations management are sometimes both necessary and profitable.

Even more importantly, individual employees in the industry can cross-pollinate ideas as they move between these environments over their careers, sorting fact from fiction and getting better at determining what will work.

The farther apart these groups are, the less appealing it will be for "startup guys" to work in an enterprise or a startup to hire on an "enterprise guy."

This trend -- since it keeps useful knowledge hidden -- can only help a 68% failure rate go up.

How is Google Native Client Faster Than -- Or As Safe As -- JVM or x86 VM?

2008-12-12T15:45:00.001-08:00

When I saw Google's proposed native-code execution plug-in earlier this week, my initial reaction was: "Don't we already have those, and they're called exploits?"

I decided to mull it over a bit, and I still don't like it. While the idea of sandboxed native x86 execution based on real-time analysis of instruction sequences makes for a great Ph.D. thesis or 20%-time project, it sounds like an awfully large attack surface for questionable benefit.

Here's what I'd like to know from a practical nuts and bolts point-of-view: how many "compute-intensive" scenarios could not be implemented effectively using either (1) the Java VM or (2) a plug-in based on widely-available open-source machine virtualization, running a tiny Linux snapshot.

While the JVM falls short of native code in some places, it can be as fast -- or even faster -- in other cases (faster because the runtime behavior provides opportunities for on-the-fly optimization beyond what is known at compile time). Yes, there are issue with clients not all having the latest Java version -- but that seems a small issue compared with the operational issue of deploying a brand-new plug-in or browser (Chrome).

Another approach is to use a plug-in that runs a virtual machine, which in turn runs a small Linux (started from a static snapshot). User-mode code should run at approximately native speed in the VM, which should cover the pure computation cases. In a rare situation where someone wants to run an OLTP file-based db (which would behave badly in a VM environment) or get access to hardware-accelerated OpenGL facililties, specific well-defined services could be created on the host, which code could access via driver "ports" installed in the VM image.

These approaches to the security boundary -- Java VM and x86 VM -- are well-known, long-tested and based essentially on a white-list approach to allowing access to the physical host. While Google's idea is intriguing, it sounds as though it's based on black-listing (albeit with dynamic discovery rather than a static list) of code. I'm not yet convinced it's necessary or safe.

Yes, VC May Be Irrelevant if it Continues Focusing on Weekend Projects

2008-12-09T14:34:00.001-08:00

Where are Web 2.0's Amazon.com, PayPal, Google, or Travelocity? They were never funded.

Paul Graham and now Eric Schonfeld are extending the several-year-old "VCs don't know what to do with Web 2.0's super-low-cost startups" meme. The extension has to do with the recession, arguing that it will accelerate the decline of VC relevance, as VCs become more reluctant to fund these shoestring startups, and more entrepreneurs pull a DIY anyway and ignore VC.

Well... maybe. The VC problem with micro-cap micro-startups is real, in the sense that it's a real math problem where the VC fund size divided by the proposed investment size equals too many portfolio companies to interact with, and a kind of interaction that is a big break away from the old model.

But the easiest fix is for VCs to simply invest in more expensive businesses. The current predicament is a classic chicken/egg: after the dot-com crash, VC money was hard or impossible to get, so the businesses people started were these undergraduate-level 1 man-year (or couple-of-weekends!) efforts.

Small product, small team, small money. We got things like 'Remember the Milk.' Cute, sure. Useful, sure. But nothing too hard or too ambitious. Nothing a tiny shop -- or a bored college student -- couldn't hack out in their spare time. Indeed many were spare time or side projects.

Some of these apps got a lot of users, especially where network effects were involved, and eventually VC wanted nothing that wasn't viral, network-effect, social. Never mind that there was never big challenge or big value in those. Facebook is the biggest thing in Web 2.0 at the moment, and it's nothing but network effect and questionable monetization. "Not that there's anything wrong with that..." It's a fine, useful application. But in the absence of any real substance the model became more Hollywoodesque, more personality- and connection-dominated than it should have.

So where is the Amazon? PayPal? Google? Travelocity? Ariba? Netflix? Danger (maker of the Sidekick devices)? Those big projects that take tens of man-years, maybe hundreds? The projects that can't "launch in beta" after a few months and acquire tens of thousands of fanatical users because they're more than a glossy AJAX UI on a local database?

Where are the startups that drag whole industries whining and screaming into the 21st century and liberate billions in value, trapped in transactions that real people make every day?

Those big projects haven't been A-round darlings for a long, long time. VCs, terrified of risk, moving in a tight pack, loved the new ethos: let the entrepreneur build a product, get it launched ("beta"), get customers, get mirror-hall PR (blogosphere), then later drop in a few bucks. Less risk. But not easy money. Count the exits. And the ad-only valuation has no more magic today than it did in 1998.

And no one even notices when hundreds or thousands of these little pownces and iwantsandys hit the deadpool.

It's no coincidence that many of the step-by-step tutorials for new frameworks and tools teach you to clone a blogger, a flickr, or a wiki farm in a sitting. After all, those are trivial undertakings, lacking only for a network-effect mob signing on. And we'd all have a good laugh about widgets one day... if we weren't laughing so hard already.

Can you imagine a tool vendor of the dot-com era giving an hour tutorial that produces a working Travelocity clone? a working PayPal clone? It's sketch comedy, or something sadder and more disturbing. Frankly, the most ambitious projects in all of web 2.0 are the tooling and infrastructure plays that are largely open source.

Investors, entrepreneurs, engineers and end users might all do well by hunting some bigger game.

Approaches to the Silverlight Dilemma: Cashback? Vista SP? Win 7?

2008-12-04T18:26:00.001-08:00

The jury's still out on Windows Live Search Cashback -- apparently there were some issues on Black Friday and, overall, things aren't up to goal ... but that could change.

In light of this cashback program, though, my proposal from last year that Microsoft simply pay people to install Silverlight seems astonishingly realistic.

The problem with Silverlight is that it doesn't have wide enough deployment -- so there is a chicken-and-egg problem deploying apps on the platform. Microsoft has released stats about how many people 'have access to a PC running Silverlight' -- but that's an odd definition of penetration (if a state university has 15,000 students, and 10 computers in a library lab have Silverlight, then presumably all 15,000 'have access to a PC with Silverlight'.)

So ... why not just pay folks a small amount to install? Maybe a $2 coupon(Starbucks/Amazon/PayPal/etc.) to install the plug-in in the user's default browser.

The WGA or XP key-checking components could be used to keep track of machines that have participated, making it impractical to game the system by performing extra installs. I'm sure a suitable equivalent could be used to verify installs on Macs and Linux boxes.

Another approach is to wait for a future Vista SP or Windows 7 upgrade cycle. Since Silverlight will presumably be included for IE, the trick is to hook the default browser selection and attempt to install a suitable version of Silverlight into the target browser (Firefox, Chrome) when the user selects it.

This move would surely be controversial -- even if there were a way to opt out -- and would bring back memories of the Microsoft of the 90s for some. But, since the web page itself decides what plug-ins to use, and no content is "forced" to render via Silverlight, it's only moderately different from making WMP or any other dynamically loadable object available.

And for those "open web" advocates who have nothing good to say about proprietary plug-ins ... consider that Silverlight does not really compete with the open web, but rather with Flash, which otherwise has a de facto monopoly on any RIA that cannot easily or cost-effectively be implemented in HTML/JavaScript. Breaking that monopoly could actually help the open web cause by creating a real platform debate in a space that doesn't really have any debate right now.

Microsoft Begins Posting "Proceedings" Docs for PDC Sessions

2008-12-02T15:36:00.001-08:00

Microsoft has started posting an accompanying document for each 2008 PDC session, containing a detailed content summary, links, and indexes (into the video stream) for demos.

These docs, called "proceedings," had been part of the plan to publish the PDC content this year, and it's nice to see them showing up alongside the slide decks and full-length session video streams that had already been posted.

While not full transcripts, or even detailed outlines, the information value of these docs is far higher than the full-length videos (per minute of attention investment), so I highly recommend them if you have interest in any of the newest Microsoft technology.

Docs, transcripts, and outlines seem to be a more efficient way to present content for multicast consumption than live video. While full-length web video is cheap and easy to produce, it wastes an enormous amount of time on the viewer side. Instead of reading/scanning the content as fast as the reader likes, he or she must sit through the lower-information-density spoken content and -- what's worse -- is forced to do so in a linear fashion. The time loss is then multiplied by all of the viewers ...

If an employer is paying for your time watching videos -- whether they be technical sessions, product demo videos (all the rage now), or so-called webinars -- then perhaps it's nice to sit back, have a cup of tea, and go into watcher mode.

But if you're investing your own time/resource/productivity into acquiring some knowledge, it's nice to (1) be able to do it at your own reading pace and (2) be able to skim/scan sections of less value at high speed and pop out into 'careful reading mode' as necessary.

Most Laser Printers are Razors, not Cars

2008-11-30T16:50:00.001-08:00

Once upon a time, buying a small or midrange laser printer was like buying a car. Big upfront expenditure, lots of sweating the details, and a moderate amount of thought about gas mileage and scheduled maintenance, er, toner and fusers and all that.

Now, however, it's more like buying a razor. The core features are mostly reasonable, each "size" printer has a speed/duty cycle that determines its suitability for an installation, and the cost is so small that it's all about the consumables (blades).

So why won't vendors -- or manufacturers -- print the cost-per-page of consumables right next to the dpi, ppm, idle power consumption, and time-to-first-page?

It's easy to find 10x differences between otherwise similar printers in the cost-per-page based on the manufacturer's price for toner cartridges and their intended yield.

Big companies, of course, have IT purchasing folks who perform these calculations, factor in the discount they get because the CIO plays golf with the right people, and order the gear. In the case of printers, large companies are typically buying high-volume printers that are among the cheapest per page anyway.

But startups, professional practices (think doctors, accountants), small to midsize businesses -- they rarely calculate the TCO for each device. It would be helpful to have the consumables price per page listed right on the sticker, like MPG.

For Individuals, Top Laptop Values May Be At Low-End Prices

2008-11-29T13:56:00.001-08:00

About a year ago, when I started my latest stint doing 100% contracting, I realized I would need a laptop for meetings, client presentations, etc. Not for coding -- I've written about my views on that, and I'll never trade away my nuclear aircraft carrier of a dev box for an inflatable dinghy just so I can hang with hipsters at the coffee shop.

Since I wouldn't be using the laptop much, and historically the many company-provided laptops I've used have turned out to be poor performers, malfunction-prone, and costly to deal with, I resolved to get the cheapest new laptop I could find. (Laptops have a strange depreciation curve, with the result that a cheap new laptop is often a much better device than a used one at the same price.)

In addition to the usual holiday sales (you can guess what prompted this article now), the whole Vista-Basic-capable-not-really thing was going on, with the result that many machines were being cleared out for good, considered illegitimate for Vista and not saleable.

I snagged a Gateway floor model at BestBuy for under $300, put another gig of RAM in ($30?), snagged XP drivers, and installed Windows Server 2003 R2, since I had found that Server 2003 was very light on the hardware while offering more than all the benefits of XP.

At this price point, I figured the laptop borders on disposable, and if I could prevent the TCO from getting high come what may.

Well, a year or so on I have some results to report.

It has performed far beyond my expectations (and as a developer my expectations tend be unreasonably high).

The only negative is the comically poor build quality -- this is a machine that one must literally 'Handle With Care' as it's built about as well as those tiny toy cars out of a quarter-vending-machine. I think I could snap it in half if I tried, and a careless twist could rip the drive door off or crack the case right open. The keyboard rattles a bit on some keys.

I have a padded briefcase and the machine was never intended for "heavy duty," so that wasn't a big deal for me. And, in any case, it seems more of a reflection on Gateway than on the price point, since, e.g., Acer offers rock-bottom laptops with much higher build quality.

That issue aside, the machine has performed flawlessly. No problems with any part of it, despite being a display model. And performance adequate to some real programming.

The ergonomics are poor for programming (single monitor, low-ish resolution, etc.) -- but it snappily runs NetBeans/Ruby/Java; Eclipse/Java plus various mobile device emulators (e.g., Android) which I needed for a course I taught this summer; even Visual Studio 2008. I do run MySQL rather than SQLServer, in part to keep the load down.

Let's see ... what else has run well on here? ... the Symbian S60 emulators (along with dev tools) ... Sun's VirtualBox virtualization software, with an OS inside. All the usual productivity stuff (Office 2007, MindManager 8) ... Microsoft's Expression design/UI tool suite ... video encoding software and high-bitrate x264 streams from hi-def rips ... often many of these at the same time. Everything I've asked it to do, it does seamlessly.

My conclusions are that sturdier laptops may well be worth it, especially for corporate IT departments -- I'm thinking about products like the ThinkPad and Tecra lines, where the price doesn't just reflect the specs but also a sturdy enclosure, standard serviceable components, slow-evolution/multi-year-lifecycle per model etc.

But for an individual, unless you have a very specific hard-to-fill need (e.g. you want to do hardcore 3D gaming on your laptop or capture DV, a bad idea with a 4200 RPM HDD), the top end of the value equation for laptops appears to be at or near the bottom of the price range. When one considers that higher-end peripherals (e.g., a BlueRay writer) can easily be plugged in via USB, and a faster hard drive will snap right into the standard slot, the value-price equation seems to get seriously out of whack for those $1200-$2500 machines.

That's not to say these higher end machines are not great ... they just don't represent value at the pricepoint. Just as a Mercedes E-class is a fine car, but the radio commercials that try to make it out to be some kind of value purchase are downright funny, I think the same applies for the high-end VAIOs, MacBook Pros, etc. Those machines are a style and brand statement for people who care about making such a statement.

This possibility is interesting because, in most products, the "optimum value" position is somewhat above the bottom end of the price range ... that is, the least expensive products are missing critical features, making them a poor value, while the high-end ones charge a "luxury premium." If laptops are different, that seems worth noting.

The usability of an ultra-cheap laptop also suggests a response to folks who commented on my earlier article, saying that companies are loath to buy a desktop and a laptop, so if an employee needs any mobility at all, they get a laptop. It appears a good solution might be to provide a high-end desktop and an ultra-cheap laptop. At these prices, the employee's time costs more than the laptop, and my experience suggests little productivity (given a remote scenarios such as a training class or client demo) is sacrificed.

Do FlexBuilder and MXMLC Really Feature Incremental Compilation?

2008-11-25T21:27:00.001-08:00

I use FlexBuilder in my work, and, overall, it's a decent tool. Eclipse gets a lot of points for being free; Flex SDK gets a lot of points for being free. FlexBuilder doesn't get points because it's basically the above two items glued together along with a GUI builder, and it costs real cash.

Wait, I'm off track already. The price isn't the issue for me. Rather, I want to know why FlexBuilder doesn't feature incremental compilation.

Hold up again, actually, I guess I want to know how Adobe defines incremental compilation since they insist that it is present and switched on by default in FlexBuilder.

Now, if I make any change (even spacing) to any code file -- or even a non-compiled file, like some html or JavaScript that happens to be hanging out in the html-template folder -- FlexBuilder rebuilds my entire project. And it's a big project, so, the job even on a 3.6GHz box means a chance to catch up on RSS or grab more coffee.

Interesting take on incremental compilation. See, I thought the whole idea was to allow compilation of some, ah, compilation unit -- say a file, or a class -- into an intermediate format which would then be linked, stitched or, in the case of Java .class files, just ZIPped into a final form.

Besides allowing compilation in parallel, this design allows for an easy way to only recompile the units that have changed: just compare the date on the intermediate output file to the date on the source file. If the source file has changed later, then recompile it. It does not appear that this is how the tool is behaving.

Perhaps this logic is already built into FlexBuilder -- mxmlc, really, since that's the compiler -- and the minutes of build time are spent on linking everything into a SWF. Since Adobe revs Flash player regularly, and many movies are compiled with new features to target only the new player, it should be possible to update the SWF format a bit in the next go-around, so that linking doesn't take egregiously long.

Apparently, at MAX this year, Adobe has started referring to the Flash "platform" -- meaning all of the related tools and tech involved around the runtime. Fair enough, it is a robust ecosystem. But "platform" kind of implies that the tools support writing real -- and big -- applications, not just a clone of Monkey Ball or another custom video player for MySpace pages.

Software Discipline Tribalism

2008-11-23T14:31:00.001-08:00

Unfortunately, people seem rarely able to stop at a reasoned preference -- e.g., "I like X, since X may offer better outcomes than Y" ... and too often end up, at least whenever group persuasion is involved, somewhere more dramatic, personal, extreme, and narrow -- e.g., "I'm the X kind of person who rebels against Y, since Y can offer worse outcomes."

This is as true in cultures of software development as anywhere else.

While many people have been guilty of this unproductive shift in attitudes, it seems many of the Agile development, dynamic languages, small-tools/small-process/small-companies crowd has long since fallen prey to it.

To be sure, there was much temptation to rebel.

At the start of the decade, proprietary Unix was strong; many processes came with expensive consultants, training, books, and tools ... and overwhelmed the projects they were meant to guide; many tools were expensive and proprietary. Web services were coming onto the scene and large players, with licenses and consulting hours to sell, created specs that were unwieldy for the small, agile, and less-deep-pocketed.

When the post-dot-com nuclear winter set in, small companies had no money to pay for any of that, and we got LAMP, Agile, TDD, REST, etc. Opposition to OO, which had been strong in many quarters, suddenly faded as OO was no longer identified (rightly or not) with certain problematic processes. Ironically, many new OO language fans had been ignoring the lightweight, free (speech and beer) processes that some OO advocates had been producing for years.

These have all proven to be useful tools and techniques and have created whole companies and enormous value ... but somewhere along the line, instead of being in favor of these tools and techniques because in some cases they produced better outcomes, either the leaders or the converts started thinking they were the rebels against anything enterprise, strongly typed, thoroughly analyzed, designed and well tooled.

This shift in attitudes does not help the industry ... nor even the clever consultants who lead the charge, deprecating last year's trend for a new one which they just happen to have written a book about.

We desperately need a broader perspective that integrates all of these pieces. There are things manually-written tests just won't do -- tools like Pex can help immensely, even if (or because... )they are from a big company.

Analysis and design are not bad words, while Agile can get dangerously close to simply surrendering to the pounding waves of change (and laughing at goals all the way to the bank) rather than building against the tide, and trying manage to a real outcome on a real budget.

Static languages can get hideously verbose for cases with functor-like behavior (Java and C# [pre-3.0], I'm looking at you). At the same time, go talk to some ActionScript developers -- who have had dynamic and functional for years -- and you'll see an amazing appreciation for the optional strict typing and interfaces in AS3.

REST is great, but in playing at dynamic, it turns out to be rather like C -- it's as dynamic as the strings you pipe into the compiler, and no more. Absent proper metadata, it cannot reflect and self-bind, so it sacrifices features that dynamic language developers love in their day-to-day coding.

Ironically, most of the critical elements of this "movement" -- along with open source -- are being subsumed into the big enterprise software companies at a prodigious pace. Sun owns MySQL and halfway owns JRuby; Java servers may serve more Rails apps than Mongrel/Ebb/Thin/etc. soon, Microsoft is all over TDD, IronRuby, IronPython...

I suppose the sort of tribalizing we see here is at least partly inevitable in any field. But it would serve the entire industry if that "part" could be made as small as reasonably possible. As a young industry with a poor track record and few rules, we ought to be more interested in better software outcomes than in being rebellious.

Microsoft Pex Moves the Needle Bigtime on Software Testing and Correctness

2008-11-20T17:14:00.001-08:00

Over two years ago, I wrote about how neither the assurances of static compiler technology nor the ardent enthusiasm and discipline of TDD (and its offshoots) represent major headway against the difficulty and complexity of large software projects.

At the time, this issue came up in the context of static languages versus dynamic languages. There still exists a political issue, although today it is more transparently about different organizations and their view of the computing business. I will revisit the successor debate in my next post.

For now, however, I want to talk about a tool. In my post of two years ago, I suggested that significantly better analysis tools would be needed in order to make real progress, regardless of your opinion languages.

So I've been excited to see the latest tools from Microsoft Research fast-tracking their way into product releases -- tools which can really move the ball downfield as far as software quality, testing, productivity, and economy.

The most significant of these is called Pex, short for Program Explorer. Pex is a tool that analyses code and automatically creates test suites with high code coverage. By high coverage, it attempts to cover every branch in the code and -- since throwing exceptions or failing assertions or contracts count as branches -- it will automatically attempt to determine all of the conditions which can generate these occurrences.

Let me say that again: Pex will attempt to construct a set of (real, code, editable) unit tests that cover every intentional logic flow in your methods, as well as any exceptions, assertion failures, or contract failures, even ones you did not code in yourself (for example, any runtime-style error like Null Pointer or Division by Zero) or which seem like "impossible-to-break trivial sanity checks" (e.g. x = 1; AssertNotEqual(x, 0))

Moreover, Pex does not randomly generate input values, cover all input ranges, or search just for generic edge cases (e.g., MAX_INT). Instead, it takes a complex theoretical approach called "abstract interpretation" coupled with a SMT (satisfiability modulo theories) constraint solver to explore the space of code routes as it runs the code and derives new, significant inputs.

In addition, so far as I can understand from the materials I've seen, Pex's runtime-based (via IL) analysis means that it should work equally well on dynamic languages as on static ones.

To get an idea of how this might work, have a look at this article showing the use of Pex to analyze a method (in the .Net base class library) which takes an arbitrary stream as input.

For those of you who are inherently skeptical of anything Microsoft -- or anything that sounds like a "big tool" or "big process" from a "big company" -- I'll have more for you in my next article. But for now keep in mind that if Microsoft can show that the approach can work and is productive, user friendly, and fun (it is), then certainly we will see similar open source tools. After all, it appears the same exact approach could work for any environment with a bytecode-based runtime.

Last, I do recognize that even if this tool works beyond our wildest expectations, it still has significant limitations including

reaching its full potential requires clarity of business requirements in the code, which in turn require human decision making and input and
for reasons of implementation and sheer complexity this tool operates at the module level, so you can't point it at one end of your giant SOA infrastructure, go home for the weekend, and expect it to produce a report of all the failure branches all over your company.

That said, here are a couple more great links:

"Contract Checking and Automated Test Generation with Pex" from PDC 2008
"Pex: Automated White Box Testing for .NET" on MS DevLabs

BizSpark Shows Wider Microsoft View Around SaaS Innovation

2008-11-12T21:39:00.000-08:00

Depsite a lot of reporting about Microsoft's new BizSpark program, one interesting bit wasn't featured in the coverage:

Microsoft's existing nearly-free-licenses-to-help-developers-get-going-on-the-platform program has been around for a while and has always required that a company plan to distribute a "packaged and resalable" application targeting a Microsoft platform.

It could be an app on Vista or Server, an Office add-in, a Windows Mobile app, or one of a few other options. But it had to be packaged, at least in the sense that it was digitally bundled into an installable set of files even if it never got put in a physical cardboard box.

This requirement made some sense as far as promoting the client OS ecosystem but it disqualified any online offering. An online service had to work around the restriction: for example, by offering a small Windows Mobile app that has some interaction with the service.

But the language makes a statement -- Empower was largely about helping ISVs new or old develop apps on the platform, thus making the client platform stronger. Nevermind that an online service that targets browsers and the iPhone might lead to Server license sales later.

BizSpark, on the other hand, takes another approach. There is much talk of online solutions -- the program is meant to dovetail with hosting providers (or Microsoft's Azure platform) to offer the server-side muscle a solution will need after its incubation period. The program is aimed exclusively at new companies -- if a firm has been in business 3 years or more, it does not qualify.

Both programs exist today and will presumably continue. So I'm not suggesting there is a big move from one view of the world to another. But there does seem to be a conscious broadening of horizons in terms of seeing where innovation is taking place and how Microsoft can be part of it.

On the Knocking at the Gate, VCs, and a Math Problem

2008-11-11T15:08:00.001-08:00

Bang!

This economy may be the wakeup to VCs (and CEOs alike) that their future isn't what they want. But the murder happened years ago, and the "I don't want yes men, but strangely I don't listen to much else'" hivemind has just woken up.

The IPO window isn't shutting or recently shut -- it never re-opened after the dot-com meltdown. A handful of IPOs (including a Google) doesn't make a "window." Whether you blame SarbOx, or a trend of investing in companies with a wink-wink style of sustainable competitive advantage, that could not produce high enough valuations to warrant a public offering, there were to be few IPOs.

The startup and VC world started getting this idea a couple of years ago, when they realized that at the smaller exit values (for the exits that were to be had), in order to get a high multiple return, the initial investments would have to be so small that the venture fund couldn't afford to service the quantity of investments. That is, the investment would have to be too small to be worth the firm's time. Uh-oh. A few innovative programs came out of that realization. But for the most part everyone acted like this was just a bad dream.

Moreover, the vaunted "get acquired" exit that appeared to be the next-best exit option, has been rather overrated. The real acquisition numbers for the most part are not what investors (or founders) would like. Not to mention the acquisition could well mean the end of the road for the business (Google is the most famous for this) which is the opposite of what founders should want, and so produces some strange incentives. Yes, YouTube and Skype ... but the curve falls off quickly.

The bad news is that this pile of trouble has been sitting in the corner stinking up the room for years.

The good news is that it's not a sudden crisis, and may well be correctable by VCs who are willing to, um, take some risk (this means getting out of their comfort zone in terms of rituals and assumptions, or expanding said zone) which is, ironically, what they are supposed to be doing for their investors.

But what about all of that advertising? Isn't there money in all of those targeted ads? Or, at least, wasn't there supposed to be until the advertising market started downhill?

In the short term, maybe ... but in the long term, the model doesn't work at the macro level and here is some math that suggests why.

Showing ads is kind of like printing money. You can show as many as you like up to a function of your pageviews. In order for the ad-economy to grow, the attention economy has to grow. That is, the aggregate amount of attention-hours spent against ad-supported pages needs to grow. Ok, there's definitely evidence for that (GMail, etc.)

But what about the ratio of the ad growth rates to attention growth? Attention growth has real-world limits (number of people, amount of time, ad-blockers, desensitization to ads) while theoretical ad supply does not. The limit on attention growth does not limit real-world ad growth. For example, if I view 50 GMail pages where I used to view 15, it's entirely possible that the same amount of attention is now divided across more ads -- or that the total spent attention is even smaller.

In the long run, the ad-value growth is smaller than the application-value growth. So the deficit of uncaptured value for businesses relying on ad revenue grows larger over time.

We can check this analysis by looking at the numbers from another point of view. Start with the raw resource itself -- a piece of a data center that includes a unit of compute power, storage, bandwidth, and hosting.

Hosting and displaying ads is relatively less expensive (in units of the resource) than hosting application functionality. So the very resource which makes the ad-supported model plausible will supports growth on the ad side that is at least as strong as growth in hosted functionality, which is the attention-harnessing product. That is, the resource availability supports growing the supply of units for spending attention as fast or faster than the supply of units for capturing attention. Again, over time, in the aggregate, more ads are powering less features. The value of each ad goes down.

This has nothing to do with the overall economy, consumer spending, etc. It's simply a side-effect of the coupling between the monetization mechanism and the product.

If, at the same time, the "real-world" spending that is driven by even successful ads is flat or in decline, you have an even bigger problem.

Windows 7 Puts on a Good Show

2008-11-06T15:00:00.001-08:00

I took Windows 7 for a quick test drive yesterday. My main goal was to see whether the performance would be so brutally bad as to make me relive my Vista experience.

For those who haven't read my Vista posts, the short version is: early Microsoft developer releases had unbearably bad performance; Microsoft made excuses ("debug build" etc.); turns out the RTM was nearly as bad. I made an honest attempt to run Vista but, as a developer, I just couldn't bear the excruciating waiting, knowing that I could be screaming along in XP. I run XP (or Server) to this day for my development.

Just the Vista-like look of the early Windows 7 bits made me anxious -- I wasn't expecting a lot. I set it up in a VMWare VM with 768 MB of RAM and no VMWare tools (= minimal video perf) in order to torture the OS. Naturally, things would be better running on the metal in a new machine designed for Win 7.

Install was very fast and seamless. I could see a difference right away in perf: even the shell in Vista runs slow, and this one was snappy. I saw the new-and-removed UAC, and I liked.

To add some more load, I installed Visual Studio 2008, which is a fairly heavy app. In addition, it was Visual Studio that had made me give up on Vista in 2007, so I thought it was fitting to try it again.

Inside Visual Studio, I opened a WPF windows project. Mucked around with the GUI editors, built, debugged ... and it cruised along nicely in the VM. Next, I set up an ASP.Net web project, and got that going in debug mode with the integrated server. Finally I started to feel some minor slowdown -- but it appeared I was running out of RAM with my 768MB VM. This was not a huge shock, since my install of Win 7 was consuming about 450MB RAM at idle, with no user apps running.

The 450MB RAM usage is a little disturbing, but, hey, even fast RAM is cheap. And my Server 2008 setup was idling at about 350MB with few services enabled, so I suppose this usage is to expected.

Overall, I was very happy with my Win 7 preview. I could see myself actually using this as my OS and not cursing all the time, which was a pleasant surprise.

The big unanswered, and unanswerable, question is: how similar will this experience be to the final RTM of Windows 7?

On one hand, Microsoft might have released this preview "stripped down" -- either to make it run better on today's hardware, or just because the additional components with which it will ship are not yet ready for public consumption. In that case, future builds might be slower.

On the other hand, still smarting from Vista, Microsoft might adjust the previews in the opposite direction -- a sort of "under-promise, over-deliver" thing -- lest anyone see a later build and say anything except "wow this is fast."

On BASIC and JavaScript and 25 Years of Coding

2008-11-04T15:51:00.001-08:00

I realized I've been putting programs together on a regular basis for 25 years now. I distinctly remember some of the earliest programs I worked on, around October 1983, and the fourth-grade teacher who let me get away with a lot more access to my school's computers than was the norm. When I somehow convinced may parents later that year to get me a machine ... things got even more out of control.

I worked with a bunch of the classic 80s "home computers": Tandy Color Computers (1, 2, and 3) and Apple II (and II+, IIe, IIc, IIgs, ...), and some not-so-classics like a CP/M machine that ran via an add-in board (with its own Z80) inside an Apple.

The languages and tools were primitive, performance was always a concern, most serious programs required at least some assembly-language component for speed and hardware access and, even if they didn't, compatibility across computer types was nearly impossible.

A lot like programming in JavaScript nowadays (I guess replace assembly language and hardware with native browser plug-in [e.g., gears] and OS APIs).

I could flog the 80s-BASIC / JavaScript analogy to death, but (1) if you read this blog, you likely can fill in the blanks yourself and (2) my goal isn't to bash JavaScript, which would be a side-effect of the drawn-out analogy.

What I find interesting is the question of why these things seem similar, and I have a hypothesis.

I have noticed that many members of my peer group, who started programming at a young age on these early microcomputers, have an affinity for tools, structured languages, and to a lesser extent models and processes. I wonder whether this affinity is not some kind of reaction against the difficulties of programming these early microcomputers in what can only be called a forced-agile approach, where debugging and testing took an inordinate proportion of the time, "releases" were frequent, and where the only evidence of working software was ... working software.

I will be the first to admit I am quite conscious that my experiences in the years before the C/DOS/Windows/C++/Mac era make me appreciative of and (depending upon your perspective) perhaps overly-tolerant of -- process, tools, models, definitions, infrastructure, etc. as a kind of reaction.

Let's stretch the hypothesis a little further: Gen-Y, who missed this era in computing (I would say it really ended between 1987 and 1989) will have typically had their first experience with coding in a structured, well documented, "standardized" ecosystem -- whether that was C on DOS, or Pascal on a Mac, or for those still younger perhaps C++ on Windows or even Java on Linux.

Depending on their age today, and the age at which they first took up coding, this generation had compilation, linking, structured programming, OS APIs, perhaps even OO and/or some process from the beginning. For them, it is possible to imagine, the overhead of the structure and process was something to rebel against, or at least a headache worth questioning.

Hence their enthusiasm for interpreted languages and agile approaches, and the general disdain for processes with formal artifacts, or extensive before-the-fact specs.

That's the hypothesis.

A simple study could validate at least the correlation -- by gathering data on age, age when someone started coding, the language/machine/environment they were using, perceived benefits or disadvantages in those years, and their interests today. And even the bare correlation data would be fascinating.

Considering that these approaches are often "best fits" for rather different kinds of development projects, knowing what sort of prejudices ("everything looks like a nail") individuals bring to the debate might add a lot of context to decisions about how to make software projects more successful, and how to compose the right teams and resources to execute them.

Great Business Model: Pseudo-Business, Pseudo-Freemium Online Software

2008-10-31T16:34:00.001-07:00

I can't come up with a better name for this model, but, not to worry, you'll recognize it right away. In this period of renewed discussion of "how to make money," I'm trotting out my favorite -- perhaps the best one for a startup today.

A couple of examples are YouSendIt.com and JotSpot.com (prior to its acquisition by Google).

Let's take the explanation in two parts. Pseudo-Business Software is software used to conduct business, but which is not necessarily sold directly to businesses. Put another way, it is priced and and offered in such a way that individuals and small work groups inside of businesses can buy and use the software directly, without larger purchasing approval and without IT department approval. It offers a businesslike function so that it is easy to justify on an individual expense report -- and it's cheap enough that some folks may happily pay for it themselves just to be more effective at work, the same way they might shell out $25 for a DayRunner or $50 for a nice portfolio without thinking twice.

YouSendIt fits this model: it offers an oft-needed business function -- transferring large files. It lets employees bypass the tortuous and unwinnable debates with IT over why and whether attachments fail, how to share files with others, etc. Pay a few bucks and if you can get to the web at work, you're good to go. Easy to expense or even pay for on your own. It's enterprise software sold cheaply and one user at a time.

JotSpot (now Google Sites) -- in its original freemium wiki form -- fits as well. Easy to justify as project groups scale and each monthly increment is a small charge. Instantly bypass all the broken collaboration infrastructure your company can't get right.

At the same time, these products are Pseudo-Freemium software. If freemium is software that offers one level of functionality or resources for free, with more available for a price, then pseudo-freemium is like freemium except that the free version is not terrifically usable as a business solution except to make the customer comfortable with the product.

YouSendIt has, and JotSpot had, free versions. Unlike many consumer freemium use cases where many users happily use the free version and never need to upgrade, these pseudo-freemium products are were specified so as to be more like a mini-free-trial.

IIRC, the JotSpot free account was limited to 5 users and 20 wiki pages, while YouSendIt free limits the file size to just shy of what I always seemed to need. There are countless other examples, ranging from single-user hosted source code services to online storage that offers only minimal free space.

The free version gives you the "warm fuzzy" of a free never-goes-away account and lets you see that the product works as advertised and won't embarrass you with some crazy or unprofessional aspect when you're asking your boss to sign for it on your expense report. These have always been the easiest-sell, no-brainer for-pay services that I've subscribed to. In general, they appeal to that certain purchasing area in people's minds -- next to the planners, a beer in the airport restaurant, or a nice tie -- as modest costs of being a professional that are either expensable or should just be paid for oneself.

Azure -- and the Other Clouds Players -- Should Lean Forward

2008-10-27T15:27:00.001-07:00

Since I covered Azure pretty well two weeks ago, there's not much to add except the name and the open question of which parts of the platform can be run in-house, on AMIs, or anywhere outside of MSFT data centers (via a hosting partner). And Microsoft hasn't really addressed that either (I have questions in at PDC) so the answer appears to be "not yet, stay tuned."

Now that the semi-news is out of the way, I am a little disappointed that all the cloud players haven't leaned in more, in terms of providing added-value capabilities beyond scaling. Elastic scaling is valuable, but it's a tradeoff. You are paying significantly more to be in the cloud than you would be to host equivalent compute power on own machines, or on VMs or app server instances at a consolidated host.

If you have reasonable projections about your capacity, then you're wasting money on the elasticity premium. You do get some nice operations/management capabilities ... but for apps that really need them, you still need to bring a bunch of your own, and you're taking on someone else's ops risks too.

For some businesses, these costs make sense. Here are some value-added features that would make the price persuasive for more people outside that core group:

Relational and transaction capabilities. Microsoft does get the prize here, as they are the only ones offering this right now. Distributed transactions and even joins are expensive. So charge as appropriate. It's a meaningful step beyond the $/VM-CPU-cycle model that dominates now.
Reverse AJAX (comet and friends). Here is a feature that is easy to describe, tricky to get right and multiplies the value of server resource elasticity. It's a perfect scenario for an established player to sell on-demand resources, and could be a differentiator in a field sorely lacking qualitative differentiation.
XMPP and XMPP/BOSH (leveraging the reverse AJAX capability above). XMPP is clearly not just for IM anymore, and may evolve into the next generation transport for "web" services. Not to mention, having a big opinionated player involved may help at the next layer in the stack, namely how a payload+operation gets represented over XMPP for interop.

Those are just a couple of ideas that spring to mind -- I'm sure there are much better ones out there. To make the cloud more of a "pain killer" than a "vitamin" for more people, some new hard-to-DIY features are the way to go.

hquery: Separate data from HTML ... without templates!

2008-10-20T13:51:00.001-07:00

Out of a bunch of cool presentation at least week's San Francisco Ruby meetup, my favorite was Choon Keat's demo of a working implementation of his hquey project -- a lib that lets you use Ruby, CSS, and jQuery patterns to bind data to views ... in the DOM, on the server side.

He had previously blogged about the motivation -- the core idea is that 10-15 years into the web application era, we are largely using template languages for integrating data into HTML. We've come a long way toward avoiding procedural code in our templates, enforcing MVC, etc. But aside from collectively agreeing to avoid a set of 'worst practices,' we're still inserting data into HTML in a manner reminiscent of 1998.

For well designed pages, with CSS classes and/or IDs, it should be possible to specify the data binding using CSS on the server, without any special binding tokens or markup. hquery does just that. So a designer can create full mockups with dummy data, and hquery can swap in the live data

The current demos and syntax might be touch verbose, but this is just an intial proof-of-concept and Ruby lends itself to easy re-arrangement an API if you need to.

This sort of radical division of dynamic data from HTML, while still using standards and not introducing yet-another-meta-templating-scheme reminds me a little of the old idea of using XML+XSLT to create pages. We all know how popular that ended up.

The hquery approach seems about fifty times more accessible to the present community of developers ... so the question is: how do we make this more popular and build community interest?

Listen to Early Windows 7 Feedback, Even From Developers

2008-10-16T15:22:00.001-07:00

At the upcoming 2008 Professional Developers Conference, Microsoft will be showing Windows 7 to developers.

My bet is that the 160GB portable hard drives they are handing out to distribute preview bits will actually contain Virtual PC images of Windows 7 in various states or configurations. Such a setup will be more convenient to try, even if it does narrow what aspects of the OS can be seen.

In any case, Microsoft would do well to pay attention to the feedback it receives from these developers.

We know all of the reasons why geeks can make poor proxies for "real end users." Nonetheless, I recall the 2005 PDC, when Microsoft gave us the latest beta of Windows Vista. A chorus of complaints arose from many who tried the new OS. It's way too slow; it doesn't work with the hardware we have; we can't explain the 10-odd different SKUs to our customers.

Do these sound familiar? They should, because they're uncannily similar to the problems "real end users" found -- and continue to find -- with Vista.

At the time, the 'softies at the conference, who are generally open, approachable, and humble with regard to technical matters, didn't want to hear these complaints about Vista. I was rebuffed more than once: the SKUs haven't been ironed out yet; the beta build is a checked debug build, so of course it's slower. Well, maybe. But I found it to be little slower than the release build on the same hardware. Either way it was unusable.

I think everyone's learned from the Vista experience -- and that includes Microsoft, ISVs, consumers, PC builders ... and Apple.

Let's try it differently this time around, starting with feedback from PDC.

One last thing: it would make sense to release the Windows 7 preview to the general public at the same time. Why? It'll be on the file-sharing networks instantly, where there is a greater chance of folks downloading a trojaned image, etc. So it will help everyone to have an official distro from Redmond instead.

What's In Microsoft's "Strata"[?] Cloud OS

2008-10-13T18:20:00.001-07:00

(Probably)

Just for fun, let's do a little educated speculation on Microsoft's "cloud os" initiative. It's not too hard to make some good guesses -- Microsoft's existing and unreleased products telegraph a lot about what they are likely assembling. For example, the semi-well-known "COOL" and Visual J++/WFC gave you most of what needed to know to imagine the real .Net platform.

There are lots of pieces out there -- certainly enough to comprise a pretty interesting cloud stack and application model.

Since Microsoft -- and platform vendors in general -- like to go all out, let's imagine this stack reaching from real hardware up through virtualized hardware up to application servers and then to client components and the end-user's browser or alternative on the other end.

Let's start in the middle of this stack and work our way out.

What would the "middle" look like? Well, what makes a hosted ASP.net account different from a cloud platform? Some answers: storage and bandwidth may not be elastic; clustering the app is neither automatic nor declarative, but requires programmatic and operational work; the database is typically a SQL Server instance (perhaps a mirrored failover cluster) with all of the usual capabilities and scaling constraints.

So imagine a hosted ASP.net account with a few changes that address these limitations.

First, swap in an alternative implementation of sessions, that supports clustering, proper caching, etc., with zero config. Add a lint-like tool to warn about code that isn't properly stateless. And an asynchronous worker service for any long-running, background, or scheduled tasks that could be "fudged" with threads or events in a controlled Windows Server environment, but won't work that way in the cloud.

Next, replace the datastore with something like ... SSDS, and a LINQ provider so that in many cases code won't need to be changed at all. The interesting thing about SSDS, of course, is that unlike other non-relational cloud datastores, Microsoft has said the roadmap will offer more relational capability (subject to constraints, no pun intended). So ASP.net apps that need real relational behavior might have an easier time moving to this new datastore.

So, without much new, we have a flavor of ASP.net that is more cloud-centric and less server-centric.

Now on the hardware and VM end of the stack, bear in mind also that -- to add value and sell the Server product, as well as to service enterprises which would like cloud architecture but need parts of the "cloud" to stay inside the firewall -- the whole enchilada is likely to be available as a service (or its own SKU) on Windows Server.

In fact, a number of Microsoft products related to modeling data centers, virtualization, and automated migration of services and machine images suggests that a key thrust of the "cloud os" might be that a customer can easily move services from individual servers up to a private cloud implementation and on to one (or more -- perhaps an opportunity for the hosting partners) public cloud data centers... provided they are coded to conform to the API.

ADO.Net Data Services (aka Astoria) already supports AtomPub, the format Microsoft is using or moving to for all of its Live services, so minimal wrappers (not to say minimal effort in the API design) could turn this into a platform API. A simple using directive brings in a File object and API that works with Skydrive instead of My Documents.

Last, look at the client end of things. Right now, we have ASP.net serving web pages, and we have web services for Silverlight clients. There is also a project (named "Volta", and which has just recently gone offline while a "new version" is in the works) aimed at dynamic tier splitting and retargeting apps in terms of the client runtime. Hmmm... Sounds like a critical part of the front end of the cloud os stack.

In order to provide a RIA experience via Silverlight (or even desktop experience for a cloud-os edition of office), promote the client os product by offering a best-of-breed experience on Windows clients, and at the same time offer a legitimate cross-platform web-browser-consumable app, a piece like Volta is critical, and makes complete sense.

Microsoft tends to hunt big game, and I doubt they are interested in a me-too web app environment. They really intend to offer a cloud os, allowing developers to code libraries and GUIs that are outside of the web paradigm. These bits can run as .Net on Windows ... as .Net in Silverlight on Mac or (one day) Linux ... and as Javascript apps in non-.Net-capable browsers.

The big question in my mind is timing -- how far along are they on the supportable, RTM version of this stuff. Whether this is relevant -- or even becomes a reality -- will depend on how fast they can get this out of beta.

It seems that when Microsoft is quite close to production with a platform they can grab enormous mindshare (recall the release of the .Net platform). If this is an alpha look, with no promised timeline, things are a lot more tenuous. If there is a 1.0 planned before mid 2009, this could make things interesting.

Time for a Wireless Coverage Map that Shows Utilization, Not Just "3G"

2008-10-12T18:21:00.001-07:00

It's easy to get distracted with mobile protocol (HSDPA vs. EVDO) or "generational" system (GSM-3G vs. EDGE) speed claims. In fact, that's the most common conversation that mobile operators, hardware manufacturers, and customers have.

But it's only a piece of the puzzle. The other big ones include latency (the time required to establish a connection and get the packets flowing) and congestion (the instantaneous demand for bandwidth relative to the current capacity in a location).

Not to downplay the value of 3G+ data speeds, it is still instructive how well a "slow" connection can work when congestion is low, and how badly 3G can work when congestion is high.

Apple iPhone customers have complained about the high congestion experience. The other day I had an interesting low congestion experience.

I was in a corner of San Rafael where my phone could only negotiate GPRS -- but I had the air to myself. Subjectively, the browsing experience was better than a typical EDGE connection on the same hardware, and similar (for modest amounts of data) to the 3G experience.

Demand on the network changes constantly as users do different things, and the effective capacity changes due to everything from weather to RF interference to upstream network congestion. So it's not easy for an operator to make a priori statements about actual speeds or actual congestion ... hence they talk about the protocols they offer and their "optimal throughput."

But congestion/capacity issues are a first-order concern in many areas, so I propose some mechanism be created so that customers and operators can have an informed negotiation about service.

I'd like to see a coverage map, for example, that doesn't just show "3G" areas in a certain color -- but also color codes the average real utilization over the past 90 days. Sort of like shopping for an airline ticket and looking at that column that shows "on-time percentage." It lets a customer separate the hypothetical performance from what can actually be expected.

That information would also motivate operators to invest in capacity and infrastructure where the demand is, rather than trying to extend that "patch of orange" on their coverage map to one more town for sales purposes.

Engineering Departments Should Take Their Medicine Too

2008-10-08T21:44:00.001-07:00

Without getting into gloomy predictions, it's clear a lot of companies will be feeling some pain soon, if they aren't already feeling it.

Software development groups at tech companies should take some medicine too -- cutting costs, becoming more competitive, improving morale, and attracting good talent at the same time.

How is that possible? Here's one way:

Unless your company is brand new, and you've got geniuses agile-ly coding your hot frog-sticker-collectors' social network, your project has a bunch of history, cruft, and mistakes in it.

It's nothing to get upset about, that's just the way of the world when developing software over the course of years in an organizational setting.

Yet many companies don't make any effort -- or actively resist any effort -- to identify those legacy problems and mitigate them. Perhaps it's fear of blame (why didn't you see this before? why didn't you say something? aren't you guys supposed to be experts?) or fear of appearing backward-looking rather than forward-looking (we'll make do with all that stuff; now can we shrink wrap and ship your latest proof-of-concept and call it next quarter's product?)

Suppose that, instead, your development group listed all the crufty bits that bug them. Stuff that maybe made sense at the time, but just isn't right any more -- wrong code, wrong architecture, wrong network protocol, wrong database, wrong format, whatever. Suppose the team got to rank these in order of annoyance factor, and impediment to productivity. Then, picking the top handful, they got to decide how to use the latest and greatest (and in many cases less expensive) technology to refactor and fix those modules.

A shortsighted manager might complain that 'if it ain't broke, don't fix it' -- it's a waste of resources.

But we know better than that.

In many projects a significant percentage of resources (up to two-thirds in extreme cases, based on my research and experience) can be spent wrestling with these "legacy cruft" issues. So, from a simple economics standpoint, it's definitely "broke" if you're spending $1 million per year on a dev group that could theoretically deliver the same functionality on $333,000, or 3x as much for the same $1 million.

A project that removes these old bits becomes more competitive. Why? The competition, particularly if it's a newer company or on a newer platform, isn't running with these parking brakes holding them back. Why should your team? If you can release the brake, you can deny your competitors something they definitely view as an advantage against you.

Moreover, these moves can boost morale and make your company more attractive to prospective employees in several different ways.

Getting rid of old morale-busting code makes everyone feel good.

Using a newer technology -- not as a novelty but because it's solving a real problem better than existing code -- is appealing to developers who want to learn new skills.

Doing a refactor of critical code without breaking existing code, wrecking release schedules, or introducing excessive ops downtime is a challenging and rewarding skill, kind of like working on the engine while plane is flying -- and top developers relish this kind of "black-diamond" assignment.

Finally, it tells everyone, inside the company and out, that this isn't Initech, where an engineer will have to work on 15-year-old tech for the next 15 years.

TextMarks and AppEngine Make Building SMS-Enabled Webapps Simple

2008-10-07T22:29:00.001-07:00

TextMarks has been on my radar for a little while. It's a company offering free (ad-supported) access to SMS shortcode 41411, via a secondary keyword of your choice. They also have pro options with no ads, and shorter (or reserved) keywords.

Not having a great idea for a SMS-based app (I think I'm biased from having web-enabled, push-email PDAs for too long), but wanting to kick the tires, I decided to build an iteration of the now-classic "to-do list" app.

Meanwhile, I've been spending a little time with the Google AppEngine. When a hosting account came due for renewal, I decided to see if I could replicate it using AppEngine. AppEngine covers all the basics, and makes it easy to stream BLOBs out of the datastore as well. But if you host files (anything over 1MB), those are not allowed in AppEngine, either as static files or as datastore objects. Too great a magnet for bandwidth or data hogs I suppose. So those files live somewhere else.

But AppEngine makes a fine place to put a small SMS-driven app (assuming you don't need the background processes or scheduled processes that AppEngine doesn't allow).

Registering a TextMark is simple. Set the URL to the one you want called when 41411 receives your keyword, and add dynamic data from the SMS into the URL request GET params using escape sequences (e.g., \0 means "the whole message after my keyword"). So your URL might look something like http://foo.appspot.com/sms?text=\0).

Go into the "Manage" tab, pick "Configuration" and turn off the various options related to "posting messages" -- these aren't necessary for your app. If you aren't planning to asynchronously send messages out to your users (hard to do with AppEngine, as mentioned above), you can also turn off the subscription options.

Now you just need a handler on AppEngine to receive those calls and do something. Whatever the handler renders will be truncated and sent back to the user as a reply SMS, so you get a semi-web-like mechanism for confirming that you're done an operation, or returning results.

Create your AppEngine app, a handler, and yaml that ties 'em together, per the tutorial. Now you can just modify the basic HelloWorld to do your bidding instead.

Here is my little memo handler -- it files memos under the phone number of the person sending the text, and forwards a list of all the memos to an email address if it receives the command "export (email address)." Don't put anything private in there, since there's no PIN/authentication (although it would be easy enough to add) ... so anyone who knows your phone number and can build a URL by hand can just ask for all your notes :)

Of course, it's easy to test out your handler with just the browser address bar, but TextMarks also provides a nice emulator so you can send messages to your handler -- and see what the response will look like with their 120 char (free account) limit and their ad tacked on. And there are a bunch of other neat things you can do, like create short term stateful contexts, where a user can just text back Y/N or a response to numbered "menu" and the messages will get to your app automagically.

zBoost Cell Extender: Refinement Post

2008-10-03T15:44:00.001-07:00

This is a refinement, or follow-up, to my first post about cell signal boosting with zBoost:

While the base station allows decent communication over a specific area once a data or voice call is established, it seems to have little or no ability to propagate the signaling part of the GSM protocols when there is no connection established.

This symptom means that the network is unaware of a handset, and a handset thinks there is no service. As a result, it often will not even try to make a call.

Once the handset decides there's a network -- usually by picking up a very faint signal from a real tower -- it will attempt the call and then discovers a near-perfect connection.

zBoost may not be responsible for this behavior -- the product docs specifically say that you have to have some real signal level in order to use the repeater. I had imagined this restriction was solely due to the zBoost device needing a tower to talk to (it cannot bridge to another backhaul) -- but in fact it may also be due to zBoost's inability to simulate the idle signaling between the handset and the tower. That is, you need a tower to make or get a call, but the zBoost can boost the data in the call stream.

So what does this all mean?

Well basically that if your RF visibility to a tower is marginal, as mine is, zBoost may still be able to help. But you don't want to rely on this to reach 911 or make any other kind of need it right now phone call, as it might take you a few minutes of moving around or monkeying with the phone to get it to realize there's any service.

Likewise, if you absolutely positively cannot miss an incoming call, this may not be the solution for you, although, interestingly, incoming signaling (calls and SMS) seems to be stronger/prioritized traffic from the tower than idle updates, and so gets to the handset more frequently, without any kind of booster present.

Credit Crisis: At Least One Great Thing Already Happening for Silicon Valley

2008-09-30T22:35:00.001-07:00

The unfolding credit crisis may have severe implications for the region -- or the world -- a bit down the line. But it is already helping chip away at one of the Bay Area's biggest long-term problems: housing (un)affordability.

I'm going to keep with my narrow technology focus here and, fairly or not, make an analysis specifically of the region's viability as a technology and innovation hub, its ability to route investment into businesses that attract top engineers and churn out world-changing products.

Every year when the region's large business leaders meet, housing is at or near the top of their list of concerns. It's hard to hire in a place people cannot afford to live. Especially when potential employees are talented and often mobile, with a lot of options in front of them.

Owning a house in the Bay Area has been expensive for some time. But -- in the space of the last eight years or so -- it has crossed the line from "painful but affordable" to "not realistically affordable" for the technology professionals smart enough not to take one of those wild-'n'-wacky subprime loans with a handful of bogus low payments up front.

I'm going to be concrete here, and use specific numbers... numbers which may leave you gasping or slapping your head if you live elsewhere in the world, but numbers which are real.

In the wake of the dot-com crash, a large portion of the area's "starter house" stock was available for $350,000 to $500,000. These were often 2-bedroom tract homes, not always in the most desirable locations, but not in the worst either. They were no bargain, but they were affordable for the "senior software engineers" with a decent education, a half-dozen years of experience, and the willingness to save for a real down payment all that time instead of going wild with the Visa card. These engineers had seen their earning power top $100,000 (partly due to the dot-com boom), and if they avoided Aqua when the company wasn't paying, they could sock away some serious cash.

A widely held rule of thumb regarding housing affordability suggests that at most one-third -- perhaps a little more -- of gross income can go to housing. More than that, and the probability of the buyer experiencing financial hardship or outright defaulting goes up sharply.

So the senior engineer, with say six or seven years of professional experience, making a little over $100,000 -- exactly the "heart of the batting order" in a growing tech company's talent pool -- could afford a $400,000-$500,000 house, at around 6%, with the traditional 20% down payment. If they had a significant other who was also earning money, they could afford a little more. But not too much more if they didn't want to be dependent on those full dual incomes indefinitely.

Almost all of the folks in my "cohort" (say, within four years or so of my age) whom I know and who bought houses in the Bay Area did so in this way, at this time, and with this sort of cost.

Fast forward three years.

Easy money and bogus mortgages have caused prices to balloon. Those $400k-$500k houses become $700-$800k houses.

Salaries have drifted up a touch as the worst "crash" years pass, but not much beyond the rate of inflation, maybe $5k-$10k per year for these mid-level positions. Certainly not enough to cover the change in housing prices.

And -- just like that -- those engineers, the early-to-mid-career core of any tech company trying to scale, the folks who know enough to use a little process and not re-invent the wheel, while still working hard and willing to take chances to innovate and make something happen, have no access to the stock of starter homes.

Beyond the larger down payment, they discover that making a "real" (i.e., based on traditional mortgage terms) monthly payment on this $750,000 house means making over $150,000. And that's well beyond the typical salary for a senior engineer or even a lead engineer / architect type role.

And there's the story: that bubble cut off the up-and-coming generations of engineers from homeownership. Indefinitely if not permanently. If the Silicon Valley business leaders roundtable thought housing was an issue before, they are in a whole new landscape now.

But this is exactly where the credit crisis is starting to help. It was never practical to build our way out of the housing shortage because not only is buildable land scarce here, but ready credit meant each housing unit gets bid up based on what lenders are in the mood to invest.

Now that the brakes are on, we've already seen the median home price in the Bay Area drop by nearly a third from its high in '06. We need a couple more years of this -- together with lenders that want to see payments under that one-third of income mark, and a solid down payment.

When it all shakes out, perhaps some of my friends who missed that narrow window early in the decade and so, despite working hard, excelling, and making serious incomes, missed a chance to own any kind of house, will get their opportunity.

And, if it's too late for them -- after all, families grow, the kids get bigger, and that worn-down starter house won't look so attractive when we're all middle-aged -- at least the next generation of geeks and whiz kids will have a reason to work in Silicon Valley.

Low-Hanging Fruit: a Server-Side JavaScript API (or Standards, or ...)

2008-09-20T21:53:00.001-07:00

There's a big chunk of stuff missing from JavaScript cloud-hosting platforms (like 10gen) and as well as from JavaScript semi-app-servers (like Phobos).

It's called any kind of API or standard.

Hard to believe, but after several years of growing JavaScript influence, and a whole web culture that is tilts towards openness and standards, all of the players -- Bungee Labs, AppJet, 10gen, Phobos, and many others -- are rolling their own little server-side platform APIs.

Standards make a platform easier to learn, understand, debate, debunk, and fix. They allow a larger community to share code and ideas, and provide a small degree of lock-in-proofing and future-proofing. Standards also allow transparent competition on the basis of implementation quality, tooling, SLA, etc., rather than obscuring those things behind incompatible facades (APIs).

New platforms on new technologies with no standards behind them can be a hard sell -- especially when they do not offer any new capabilities.

According to Techcrunch, Bungee is in a "freefall." And the interesting bit is that their CEO ascribed the recent round of layoffs to 'actual vs. anticipated rates of adoption.'

Hello, if you are trying to sell the world on your server-side JavaScript programming and deployment environment, you're not helping your 'rates of adoption' by also asking people to learn and commit to your own home-brew platform API.

Now to be fair, there aren't a lot of alternatives in the absence of a standard. But ... it would make a lot more sense for all these players to get together and create some standard APIs and commit to using them. The APIs would cover all the basics: e.g., persistence (of object, key-value and relational flavors), templates, request/response handling, calls out to other web services and processing of their responses, publishing SOAP services (which still remains critical in the enterprise world), and interop with other server-side environments (Java, Python, etc.)

Overnight, there would be a single community (and acronym!) instead of a dozen fragments. Like any standard, it would generate books, conferences, training materials -- and controversy, which is never a bad thing when you need publicity. We would see real performance tests, and get a real debate over where the JavaScript-to-SomethingElse boundary should be and why.

And these vendors would gain instant legitimacy by being founding contributors to a specific platform "trend," rather than lone voices in the woods. That legitimacy (and, via the sad logic of large companies, the "legitimacy" of being printed on the top of some conference bag) would help them appear credible to customers big enough to pay them real money.

BYOA (Bring Your Own Analogies)

2008-09-11T10:01:00.001-07:00

I found this brilliant label on the side of an industrial-strength wood chipper/eater/pulverizer.

There are so many other places -- especially in software development -- where a label like this (including both explicit content and implicit assumptions about the attitude of the reader) would be appropriate.

I won't ruin your fun by babbling on about all the specific cases; instead I'll leave you the pleasure and satisfaction that will come as you begin thinking of them.

Presentations from SF Flash Hackers August '08

2008-09-09T20:55:00.001-07:00

A couple of weeks ago I gave two mini-presentations to the SF Flash Hackers group, on topics I've talked about here before ... but I figured I'd post the slides to slideshare.

Here are slides about porting a large Windows app (Mindjet MindManager 7.2 with Connect) to run in the browser via Flash (developerd with Flex):

Porting a large Windows app to Flex

View SlideShare presentation or Upload your own.

And here are slides on generating ActionScript 3 code from UML class diagrams using my VASGen tool -- which is really a contribution building on two existing tools, the Violet UML modeler, and the Metaas ActionScript 3 meta-library (in Java):

As3 Code Gen from Uml

View SlideShare presentation or Upload your own.

Citibank Needs to Get Their PKI Act Together

2008-09-04T23:25:00.001-07:00

While I'm thinking about security ... there has been plenty of debate over whether Firefox 3's hostility toward self-signed certs is a good idea.

Either way, this should be a non-issue in the banking world, which ought to have proper certs on any public facing machines.

So I was more than a little surprised when I went through a workflow with Citi, where a number of links, widgets, etc., triggered the Firefox self-signed-cert blockade/warning. The problem resources were loading from subdomains like foo.citibank.com or bar.citimortgage.com.

When I did some work with [insert other extremely large bank here], everything was SSL, even internal web service and app communications, and usually (not for external customers) mutual authentication. The bank had an entire PKI department, which controlled numerous separate CAs corresponding to dev, test, production, different business units/functions, etc.

Hooking up to most things meant sorting out the right kind of cert to present, and working the proper cert chain for whatever the server gave you. In some situations -- e.g., programmatically sorting this out in Java -- it was a major hassle. Not to mention that the PKI group was cooperative but busy, so there could be delays. And the certs were set to expire in not-so-long, so a whole mechanism was necessary to make sure you didn't have system failures in your department due to not getting a new cert placed in time.

It would have been much easier to use self-signed certs all over the place, but the bank wanted some extra protection even against rogue calls from inside the network. The policy made sense and, even if it didn't to you, your alternative was to box up your stuff and leave the building.

Of course intentionally deploying a production service to external customers with a bogus cert was so unimaginable it wouldn't have even been funny ... in order to be funny there would've had to have been some molecule of possibility in it, and there wasn't.

Can Citibank really not have controls that prevent this?

Chrome's Unusual Installation Location: Good, Bad, or Ugly?

2008-09-04T23:06:00.001-07:00

I -- and many other folks -- have noticed that Google Chrome installs only for a single user, and does so in a way that does not require administrative privileges to run the installer.

Basically, it just drops its files into a subdirectory of the user's home directory, places its shortcuts in the user's specific Start Menu folder, Desktop folder, etc., and arranges for its GoogleUpdate.exe helper app to launch from Windows/CurrentVersion/Run under HKEY_CURRENT_USER, rather than HKEY_LOCAL_MACHINE.

This is an unusual pattern for a Windows installer, almost certainly rigged in order to allow minimal-privilege user accounts on corporate networks to install and run Chrome ... under the radar of IT or management policy, if need be.

The question is whether this is inherently a security problem.

On one hand, I've read posts pointing out that this setup leaves the executable vulnerable to other executables that run with the user's permissions. This means another app could replace Chrome with a compromised Chrome, and the user would never know. At the same time, if Chrome can install, then any other malware could install itself the same way -- set itself up to launch under HKCU/.../CurrentVersion/Run, and sit in the background doing anything it wanted (like listen to keystrokes for another HWND). Then again, being in the user's browser might make snarfing credentials and scripting their use (or taking advantage of an auth cookie being present) a lot easier. The point is that a traditional executable under Program Files should be less vulnerable -- a nonprivileged user account can't rewrite those files.

On the other hand ... this is not terribly unlike the install/run routine on *nix servers. If I'm a "regular" user, I'm not installing to /usr/bin, I'm just untarring in a local directory, possibly building, and then running the binary. Of course a user doing this is likely more sophisticated than general Windows users, and fewer *nix end users means less malware at the moment.

Always-On JavaScript Mildly Disturbing

2008-09-03T16:25:00.001-07:00

Google Chrome doesn't have a switch to turn off or restrict scripts. While this might be an upcoming feature, my guess is that Chrome is about "running" web 2.0 apps, and so JavaScript is considered essential.

Well, maybe that's ok if the security model around JavaScript execution is as fantastic as the comic book suggests.

On the other hand, the launch of this browser featuring a well-known security flaw (admittedly not a JavaScript flaw) makes me a less comfortable about always-100% script execution.

Amazon EC2 to Support Windows Server AMIs

2008-09-03T16:08:00.001-07:00

Jeff Barr, Amazon Web Services Senior Evangelist, just finished giving a talk at "The AWS Start-up Event – San Francisco" in which he showed slides listing upcoming plans at AWS, including support for Windows Server.

This is an exciting development, as Windows Server / ASP.Net make for a fantastic if potentially expensive platform. Now Microsoft has to step up to the plate and come up with a pay-as-you-go, per-cycle or per-cpu-hour licensing scheme.

One of the things that makes ASP.Net interesting is that it lives in a nice middle ground between Java, which is extremely fast (in EC2, this means less expensive per transaction) and has great "enterprise" capabilities but is cumbersome to develop with, and, say, Rails, which is quite slow and has poor enterprise app cred but is very pleasant and lightweight to develop with. ASP.Net, especially with the MVC framework and forthcoming support for Python and Ruby in addition to C# and the other .Net languages, seems to combine extreme performance, easy development, and access to as much "enterprise" as you need while offering lightweight alternatives like LINQ and SSDS.

My point isn't to make a commercial for ASP.Net, but to point out that if Microsoft can get their licensing in order, they might catch up in the cloud world through fast, cheap development cycles plus faster (and hence cheaper) runtime operation on a given machine instance than some competing platforms.

Just to be fair, cloud vendor enomalism has run Windows Server on EC2 before, by virtue of the Qemu emulation software (on top of Linux). But if we're talking about maximizing efficiency, wasting cycles on another layer of emulation (EC2 instances are of course virtual to begin with) doesn't sound like the way to go.

Om:Yes, Gillmor:No ... Comcast's Data Transfer Cap

2008-09-01T14:53:00.001-07:00

Om Malik nailed it at the end of one his posts on the 250GB cap. He lists Comcast's stilted usage examples for email, music, etc., and asks "How many HD movies will fit under the cap?"

That's the right question, but not because he is worried he'll bump the cap watching video.

The reason it's the right question is that the cap has more to do with Comcast's long-term business plans -- and the cable/telco industry's plans -- than about slapping down a handful of network hogs.

One fact before we continue. As I've written before, while everyone squabbled over BlueRay or HDDVD, the filesharing world settled on the x264 implementation and a container format for it. In that compressed format, a high-def film averages 10-15 GB.

But here's the real deal:

Comcast's cable TV and on-demand video business competes directly with Internet video (both legal and gray-market). By making the cap high enough that it won't immediately impact YouTube watchers but -- over time -- will put a serious dent in customers' ability to get HD video over the Internet, Comcast is simply crippling one product so that customers will have to buy another product from them. Don't even think about mentioning broadband competition -- if you define competition as more than one entity offering substitutable goods (i.e. similar bandwidth, QoS, etc), then most of the U.S. has zero competition.

The 250GB cap is a way to push customers to buy the same content twice. To be specific, you'll pay to see a movie broadcast on cable TV in HD, and then pay again for some special plan that lets you watch the same content later on Hulu. Or you'll pay for Comcast "on-demand" to watch a show on your TV. But if you want to place-shift that show to your laptop or the gym (on your phone), you'll have to pay to get enough headroom to download the same video.

This move is not just about trying to squeeze out more money. It's yet another attempt by network operators to neglect their core business and instead try to get into a content business, so they can do two things badly instead of just one.

Meanwhile, Steve Gillmor writes that bandwidth caps will mean the end of BitTorrent downloading and a shift to streaming. Steve's a smart guy, but streaming is one of those perennial non-trends like speech recognition and thin-client computing. It's only conceivable to someone who has a computer connected to their TV or who lives on their laptop.

Having played a key role in several home-entertainment / media-tech startups, two of which have been successfully acquired, I feel confident saying streaming is only a little piece of a solution. For the foreseeable future, streaming will never be more than that. It neglects too many consumer needs and desires, is too inflexible, and is too warped by DRM and platform-lock-in issues.

Moreover, the latest round of DRM around HDMI and high-def media playback in general means that even in-home "streaming" -- i.e., from a PC to a TV or from one PC to another -- is facing setbacks just as Joe tech-savvy consumer finally starts to realize how to use a home server or network attached hard drive to get and watch shows.

And the set-top boxes, increasingly tied in tightly with content delivery (Comcast DVR, DirecTV TiVo expiring your pay-per-view movies, etc.) are more closed than ever. Instead of encouraging an ecosystem of streaming, they force customers into an ecosystem of downloading.

I think we'll see more people gaming the 250GB cap ... using scheduling software so they can maximize the utility of the 250GB and hoard downloads. Not to mention leaving the torrent client running so that every time I visit someone else's network -- an office, a coffeeshop, etc. -- I'll keep the downloads slowly but surely coming in.

One final note: the marginal cost of moving a kb of data around a big network -- that is, the cost of the power and cooling for the switches and the amortized cost of the cables and gear and ops guys divided over their lifetime data transfer -- is more or less zero when the network isn't already near peak bandwidth somewhere. So even if there is a problem, the volume-based cap is the answer to a different problem.

Extending a Cell Signal With zBoost

2008-08-25T12:42:00.001-07:00

Although I'm curious about the AIRAVE, I'm not on Sprint and not convinced it's worth switching just for the AIRAVE capability.

But I do have a signal problem at my house, and I'm currently testing out the zBoost Wi-Ex signal booster to see how much of a dent it can make.

Unlike AIRAVE, which creates a local CDMA network and bridges it to another backhaul, the Wi-Ex is an RF repeater/amplifier. So you need to feed it a signal -- from somewhere nearby that has a signal -- and it extends the signal via a new base station and antenna.

So far I'm inconclusive on the quantitative results. Certainly, in my house, the coverage area it creates is closer to 600 square feet than the 2500 square feet advertised. But as with any RF setup, there are so many variables that this doesn't prove a heck of a lot.

I can report positive results on a qualitative measure.

Normally, the signal is so weak inside my house that, without this unit, I can receive a call signal (e.g., ring or text message) but as soon as I press "talk" it drops. 100% of the time. Also, my Blackjack had never successfully retrieved a web page from inside the house. With the zBoost unit, there is qualitative difference: The data connection can retrieve web pages -- sometimes slower, sometimes faster, but pretty much successfully. And if I receive a call, I can answer without the call dropping instantly. Again, depending on various things the call quality may be better or worse -- but there is always a call there, struggling to get through. Which despite being reminiscent of 1995 is a major improvement.

What I'm wondering is ... if, as zBoost claims, the device is FCC legit and is designed not to create trouble for the existing cell network, why don't the cell carriers themselves sell this device? It seems like it could help them extend their reach geographically and satisfy a few more customers.

And before you suggest that they don't offer it because doing so would be tantamount to admitting their existing network footprint isn't solid, note that both T-Mobile and Sprint are already selling personal femtocells and rumors are that AT&T isn't far behind. And anyway, spotty coverage inside buildings has never been any kind of secret.

Airave and Network Sniffing for Fun and Profit

2008-08-23T08:31:00.001-07:00

Sprint's AIRAVE BYOB (bring-your-own-backhaul) femtocell went on sale nationwide this week. Reviews seem mildly positive; it kind of costs a bunch considering you're subsidizing Sprint's infrastructure, but it's marketed toward people who have little other choice.

I don't have one (I'm not on Sprint at the moment). But if I did, the second thing I'd do with it is to start sniffing the traffic it sends back up the wire.

But all that data would be encrypted, right? Well, maybe ... and maybe not. Since Excel files seem to be an enterprise data store of choice these days for companies handling sensitive personal information, and since telcos are not the science-fair winners of the class but more like that kid in the back that just giggles all day keeps getting his lollipop stuck in his own hair, it wouldn't surprise me if they think ADPCM is encryption.

Even if some or all of the call content is encrypted, though, one could probably learn a lot about the signaling layer of the system -- i.e., how the network identifies and talks to the phones to indicate presence or session initiation.

Why would this be useful?

Well, for one thing, it would be interesting to use the AIRAVE as a presence detection mechanism. Once I can determine that the base station has recognized a known cell phone ID, I can automate all sorts of things that are supposed to happen when I'm nearby.

Applications include home automation (open locks, turn off the alarm); media (access and/or license to all of the media I own "appears" on any connected device when I'm nearby with my cell phone, and goes away when I leave); or for commerce: my hotel reservation, preferences at a restaurant, or online search history for a retailer can cue up automagically as I enter range.

Tripeedo Brings Travel Search Full-Circle

2008-08-15T13:52:00.000-07:00

I couldn't help but crack up when I read a post about Tripeedo, an AJAX-enabled "command line for travel," since of course all this reservation searching stuff was on a command line to begin with, on a terminal a generation older than this hideous beast. And in fact while the hardware has mercifully been emulated away, the protocols are still in wide use under the hood in the travel industry.

I had to use a command line like this as recently as 2006 for a project. Unlike tripeedo's version, I recall the search started with a strange lozenge character that doesn't seem to have a proper Unicode name, but was part of the 6-bit EBCDIC charset these terminals used when they were deployed in 1969. Then came a letter or a delimiter of some sort, and then a line like 15AUGSFODFW to look at flights on August 15th from SFO to DFW.

I thought I'd try it in Tripeedo. The site prefers the month first, and it likes spaces. And, unlike the old system, it has lowercase letters too!

So I type "15 aug sfo dfw" and get some flights to look at. In 1969, this was the future.

iPhone Brings New Attention to AT&T's Existing 3G Problem

2008-08-14T15:31:00.001-07:00

The wave has been building for a few days now, with consumers, Apple, and AT&T sparring over the problems with the iPhone 3G staying connected (and on 3G).

Today seems to be a local maximum and the mainstream media are starting to look.

If you read this blog regularly, you're probably expecting me to blame Apple for the problem, then blame Apple again for denying there's a problem, then blame Apple fanboys for putting up with it all for so long then suddenly acting betrayed.

Surprise!

As much as that might be a valid argument -- especially the part about Apple denying there's a problem -- this one is really not their fault.

It's kind of AT&T's fault -- and kind of GSM's fault -- and kind of just mediocre engineering.

A while back, I wrote about similar experiences with my Samsung Blackjack on AT&T. I've also seen the same thing with the AT&T 8525 (Cingular 8525 / HTC TyTn), as this was one of the first multi-protocol, multi-band 3G devices on that network.

Most people don't have any idea what this is all about, since their devices cannot experience this specific sort of problem because they don't support 3G, and so never have to resolve between 3G, EDGE, GPRS, etc.

In this case, AT&T argues that one suspect, the Infineon chipset, is not to blame because it's the same one in Samsung devices, with which they have had little problem. I don't think that's their strongest argument: others appear to be having as much trouble as I do with the Blackjack/3G. We just never made as much noise as the iPhone folks.

Perhaps a better argument would be that it affects all the EDGE/UMTS/HSDPA devices on the network and then maybe blame GSM for having such a funky data transport technology roadmap over the last 10 years.

I'm not saying AT&T could easily do better -- maybe they need more capacity and a tune-up of switching algorithms. A CNET article points to GSM carrier T-Mobile Netherlands having the same problem. And even though CDMA is worlds better as a technology, it's nice to have a little competition out there: two CDMA carriers for the U.S. out of four majors is enough.

So what does all this mean?

First, I'll tip my hat for once to the iPhonosphere and the obsessive media that love them: this is a real issue and we all know it takes a lot of volume to be heard talking to a telco.

Second, even Steve Jobs' strong-arming of the carriers worldwide (and good for him on that one!) doesn't mean everything is really under his control. According to this Sydney Morning Herald story, Apple didn't provide any test units to some carriers until the day before the product release, hoping it would all "just work."

To paraphrase someone I used to work with, in technology "hope is not a strategy."

Non-compete Ruling May Indirectly Help Broken Organizations

2008-08-14T09:09:00.000-07:00

Last week's California court ruling against non-competes is interesting, not least because I've had one client who was more interested in a getting me to sign a specially crafted non-compete that he hoped to use in a lawsuit than in having me actually deliver work.

Actually I'm not sure this ruling would even affect independent contractors like me; the view of employment law towards contractors (where it applies at all) is very different from that toward employees.

This ruling may play a small role spurring innovation by making it easier for employees to quit and then compete with their former employer.

I'm also hopeful that the newly prominent threat of such competition may encourage dysfunctional tech organizations to renew their efforts at achieving better group dynamics. (Is there such a thing as therapy for whole organizations? No, I don't mean those motivational or consulting clown-schools that involve offsites and create a distraction ... at least until the check is cashed.)

In addition to the ominpresent threat that one's best employees might leave, there is now renewed emphasis on the fact that a group of them might well leave together and "do it right" if they can.

While the ruling does not of course give anyone license to loot protected IP from the former company, history shows that's rarely the issue in Silicon Valley anyway.

Every development manager should view this as a wake-up call to think about keeping the team on board.

Format Shifting

2008-08-13T20:17:00.000-07:00

The dead-tree-publishing world is slowly starting to worry about online, possibly illegal, distribution of books. Not nearly enough to get smart about it though.

To me, it seems brain-dead obvious that if I buy a paper copy of a book for anywhere from $20 to $50, I ought to be able to use an electronic copy. I don't know the law on this, but I don't have pangs of conscience downloading a gray-market PDF of a book I've already bought, because the publisher wants to charge me again for an "e-book bundle" or because they make the content available in an annoying format.

But that's just the tip of the blade. In a short while, things like Kindle will take off, and the next generation -- motivated to leap in because of expensive textbooks -- will start assuming all books will be free.

Publishers should be experimenting with their models right now in attempt to adapt.

Here's one idea: the whole hardcover-softcover release cycle makes about as much sense as releasing a movie in the U.S. on Wednesday and not expecting it to be an xvid in homes in Thailand by Friday.

I like paper books, but I don't like hardcover books for anything non-classic. They're overly large, awkward to carry, just plain silly. So what happens when a publisher releases a new book I want to read in hardcover? I'm not inclined to buy it -- heck if I wanted to carry the hardcover around I could just get it out of the library.

I could download it as soon as someone puts it online, but I would want to print and bind it ... so I'd have to submit it as a print-on-demand job somewhere, hope they don't complain about the copyrighted material, and get it sent to me. I guess Kinko's is an option.

Anyone else see the problem/opportunity here?

For now, anyway, the hassle and cost of print-on-demand makes it a cost-neutral issue; I simply don't have the option getting a bound paper copy of the book for free. So given that I'm willing to -- and indeed must -- spend money to get the book, why won't the publisher sell it to me in the format I want?

Publishers ought to be taking the risks and making the friends now, before e-book readers make a big dent in the marker. Otherwise, it will be game over soon enough.

And before anyone suggests that the publishers play some critical role in blessing publications, I would point out that is just an artifact of the traditional physical distribution mechanism (paper, shelf space, etc.) ... the Internet already allows vastly more content than could be run through a press and tossed up at Borders. It's not all "good" ... but the net does a fine at creating search, moderation, and recommendation networks that allow one to find the good stuff, in a way that 3x5 "recommendation" cards tacked under a bookshelf at the local store cannot.

Rails and .Net Had One Thing in Common Before ASP.Net-MVC: Magic

2008-08-13T12:48:00.001-07:00

It's possible now to build a very Rails-like website via ASP.Net MVC. And at RailsConf, John Lam showed IronRuby starting to run Rails.

But long before all this, these two very different platforms had one thing in common: "magic."

By magic, I mean the sort of implicit, we-solve-problems-you-didn't-even-know-you-had kind of magic that makes it easy for people who don't know the details, don't want to know the details, or know but don't want to be bothered by the details to build an app.

From ASP.Net's first iteration, where you could drop an item on a "web form," wire a server-side event handler, and talk to items on the page through a .Net server-side object interface, ASP.Net has been heavy on the magic. Rails has been the same way, from scaffolding up through ActiveRecord, you point Rails in the direction you want to go and it drives.

Of course we all know about the leaky abstractions... in .Net, you used to get some funky HTML and Javascript, plus a server-based page lifecycle that didn't always match the model of what you wanted to do with the DOM. In Rails, we've seen perf issues, and discussions of the "worst practices" [pdf] that can result from letting the autopilot do too much and your brain too little when writing for Rails.

But in any case -- as opposed to a framework like Django (which included a big feature push called 'removing the magic' to make functionality more explicit) -- Rails and .Net give you as much magic as they can.

[How] Will Microsoft Weather a Perfect Storm?

2008-08-06T19:27:00.001-07:00

Last week's article What Is Microsoft So Afraid Of? is meant to be provocative ... but it is also real reporting.

Thinking about the situation Microsoft is in, I believe the firm has, at a high level, quite a lot to be worried about ... a sort of perfect storm of high-level business trends, many of which transcend any individual product or feature. Big trends are harder to debug and then patch on Tuesdays.

Here are the issues:

Declining importance of the desktop OS. As people spend more and more time in their browsers, the OS underneath it matters less. The trend is definitely toward more browser-based apps, even if they employ Flash or Silverlight or other acceleration technologies. Offline support is moving slowly but together with cloud (Mesh?) storage, will make the local filesystem less important.
Rise of OS X as a real competitor -- between publicity, 'time to sink in,' and the decreasing premium that Apple charges for a Mac over a similarly equipped Dell/Acer/Toshiba laptop, OS X is a real and growing threat to Windows.
Bill's off to save the world. For a while now, Bill has been the good cop (hey, stop laughing, I'm serious) to Steve Ballmer's bad cop. Now that Gates has retired to work on philanthropy, and Ballmer's the CEO, we're seeing more dumb-ass bad-cop stuff like the Mojave experiment, and less brilliant product strategy. Microsoft needs a good cop to pair off with Steve.
Vista failure. The .Net platform was one of the smartest, most successful things Microsoft has ever done, an enormous accomplishment that brought enterprise IT shops and other developers along. Following that up with Vista isn't just bad at the retail (license selling) level, it shakes enterprise confidence in MSFT, and ticks off developers. From a developer POV, most of the cool stuff in Vista was gutted, or moved somewhere else ... or never really worked very well (WPF), after years of build-up. Now MSFT desperately needs devs to buy into "the next thing," and they're hesitant.
Silverlight stuck in transition: Silverlight 2 is an awesome technology... but it has two problems: First, small penetration, unknown/unpublished penetration numbers, and no specific numeric commitment by MSFT to create penetration on PCs. Second, folks are sceptical that Silverlight on Mac really has a future. If that's in doubt, it seriously impacts the choice to use Silverlight because of #2 above. I can say that I've already had more than one client interested in doing serious work on Silverlight 2, but they insist on installed base projections (not download numbers) to make the business decision, and I don't blame them.
Rise of console gaming: PC gaming was always a key part of the "latest and greatest" PC/OS/component ecosystem, plus it helped press down Mac adoption. As consoles (including Microsoft's own Xbox 360) gain, this relative benefit of the Wintel client platform goes down... and yet MSFT still has to spend a ton on R&D if they intend to keep the PC gaming client (DirectX and hardware integration) decent. So less bang for the buck there.
Rise of USB. Now that practically everything runs over USB, it's easier for device makers to offer Linux, MacOS, etc. drivers (the upper layers can often be just user-space apps). So on the peripheral front there is less Windows differentiation in terms of hardware choices, and less lock-in.
Reluctance to innovate with server licensing model. Most startups dream of going big, and they assume they'll do so on an open-source stack -- not because Windows Server isn't a killer product ... but because the current licensing for Server makes it a non-starter for cloud services or for architectures designed around cheap, flexible horizontal scaling. An easy fix is to add auditing to the core WMI counters and create a Windows Server SKU that costs $0 per CPU and $0 per client ... but to stay in compliance you average your ASP.net transaction count, or SQL size/complexity + operation count and then cut a license renewal check. Make it free (as in beer, with online-only support) for the first few thousand page views per month for a typical app, and it'll rapidly start taking over a big piece of the startup and cloud/on-demand computing world.

All this said, big companies tend to have nine lives (even if Yahoo! seems intent on burning through all nine of them). So I'm not about to count Microsoft out, or suggest an "over-the-hill" tipping point is at hand. Just some big decisions, both strategic and tactical, that ought to be made well and soon.

So What's the Deal With ... ?

2008-08-02T15:43:00.001-07:00

... those tech videos where the topic sounds interesting but I have to watch 3 guys chat about it in real time for 45 minutes? Like I have nothing better to do than watch people talk? Give me the slide deck, the bullets, a transcript, something I can scan through and read, since I know it's going to take me 4 minutes max to sort through the content.
... 10gen and the folks that want to write yet another server computing platform ... in JavaScript, and then compete with Google using their $1.5 million in VC? Ever since Douglas Crockford schooled me in the true nature of JavaScript, I've had little bad to say about the language. But why yet another platform for server-side computing? 10gen's argument that Google's AppEngine is too closed and locks apps in seems like a stretch: you can run Python/Django apps on AppEngine ... or somewhere else. Yes, the Google data API is their own, but it's simpler than a relational database API -- if you really need to use Google's data API over your own data store, it won't be hard to set up. There's also Rails, which is open and has the advantage (over 10gen) of being well known already ... If you don't think Rails can work as a cloud platform, talk to Heroku.
... Symbian's S60 emulator triggering DEP faults, so that Windows has to kill it? I can imagine how one might, when writing an emulator, run afoul of DEP ... But this is my first run-in with an app fatally triggering it, and I've already used Microsoft's device emulators, Palm's emulators, VMWare, and Sun's xVM VirtualBox (a VM is not the same as an emulator ... but some parts of the VM function via emulation) ... and had no problems.
... all the enmity toward Windows from the Rails guys? First, Ruby and Rails run fine on Windows, there's no need for all the "we specifically won't say anything about working on Windows, I mean, if you're crazy enough to want to do that" stuff that keeps popping up in otherwise respectable books. Second, many (most?) of the top Rails hackers are so young, they probably don't even remember Microsoft's real "bad old days." Yes, Microsoft has done some shady things worth complaining about. The fact that IE 7 doesn't render your CSS the way you wanted is not one of them.
... those posts where an unknown wantrepreneur with no valley connections or financing seeks a successful, experienced, genius hacker to build his mediocre but top secret idea for equity only? The posts crop up constantly on craigslist and all the SF tech meetup mailing lists. If this kind of post represents your thought process, why not just skip step 1 and 2, and post for people to send you a burlap sack full of cash?

No, CherryPal Will Not be the First (or any) Mass-Market Cloud Computer

2008-07-24T15:05:00.000-07:00

Notwithstanding this predictable VentureBeat article about a thin-client device with a somewhat anatomically suggestive logo.

I've previously written about why thin-client is a not-now and not-soon solution.

The $249 "CherryPal" box reminded me of another twist on the problem ... one that comes into play when the thin client isn't just software (like Skyfire's screen-scraping browser) but a new hardware device.

See, here's the thing about building your own new hardware in smallish quantities: it's really expensive on a per-unit basis. Or, another way, you can't offer a fraction of the capability per dollar that a Dell or Sony can. Your $249 thingamabob is going up against other $249 devices that have way more stuff (e.g., entire laptops) because they are produced in high-volume orders of established designs/modules/parts.

To be more precise, there is a spreadsheet you can put together that maps your bill of materials, plus any special physical design issues into a cost per fabricated unit. It includes multipliers that will make you sad, like your $2 can't-live-without-it chip might end up adding $20 to the finished product cost... depending on where it fits in, affects other parts, quantities, and other stuff.

It's tempting though, especially since the last 5-10 years have brought the ability to fabricate in China for lower cost and in much smaller quantities than would have been practical before. (Of course the costs aren't really lower, they're just externalized into a bunch of other areas ... but those are politics/economics/policy areas more than tech, so I'll leave them be for now.)

So, whereas before you might have needed to sell 700,000 units to break even on something, now 20,000 units will do it: increased temptation.

But if you're CherryPal, you still need to convince someone that a $249 thin-client is more useful than a $299 laptop with 10x the horsepower, a screen, storage, and all the rest.

Free Idea: AJAX Desktop Publishing App

2008-07-23T13:10:00.001-07:00

If you're hankering to join the web startup circus and have everything but an idea, here's one for you: a full-on desktop publishing app ... online. The ready acquisition market for decent online productivity apps can serve as inspiration, even as the insane complexity of the undertaking urges you on toward greater heights of caffeination and maybe some anti-depressants.

Seriously, apps that can deal with complex page layouts and can produce a variety of press-ready output (InDesign, QuarkXPress) are expensive and hard to learn, and the average person rarely needs them. It would be nice to be able to hop on a web page and do a quick design. Wizards or integrated help could go a long way. All the usual business models (adds, freemium, pay-per-storage, etc.) apply.

Now, if you build this in Flash/Flex or in Silverlight (2.0), you merely have a ton of work ahead of you. While challenging, you may feel like you're not ... well ... proving just how wicked you really are, since those two platforms give you all of the technical infrastructure you need.

So why not go all the way? Do it in JavaScript, in the browser. Yep, WYSIWYG, arbitrary page sizes and layouts, in the browser. Advanced typography too. Fun times.

Here's a hint: one of the tricky bits is measuring text line layouts, since you'll need to make sure that the view matches the backing data (from which you will be generating other "views" or output formats).

JavaScript (the browser APIs really) don't give you everything you'd like. For example, one standard thing you need is to find out how many characters will lay out into a column of a particular width. Or simply what size a run of text will turn out to be.

While you can't ask these things directly, the clientHeight DOM element property can be useful: it shows the actual height that a DIV's content will lay out at ... so it will also signal when content has flowed from one line into a second line. By hiding the DIV, rigging up a while loop, and monkeying with the content (string), you can build a routine that measures text, so that you know exactly which characters are going to appear on a line.

Completing the application is left as a (possibly lucrative) exercise for the reader.

Packaging Outlook Scripts in Word Documents

2008-07-21T17:28:00.001-07:00

Microsoft Office includes (unless you exclude it at install time) a flavor of ye olde Visual Basic (i.e. VB-COM not VB.Net) that is designed to facilitate Office automation.

If you're willing to put up with the VB syntax and conventions, you can put together mini-programs (including GUIs, menu options, toolbar buttons, etc. if you like) in even less time than it would take to boot up Visual Studio and run the Office add-in wizard or add some Office .Net assembly references.

Interestingly, distributing the resultant "macros" can be tricky... especially for Outlook. There are all sorts of suggestions depending on how widely you want to deploy your script, and how many "Are you really really sure you want to run this script?" dialogs you expect your end users to click through. Some of the approaches are quite hokey... for example copying over a client script file that will delete any other macros your user has. Eeesh.

It's possible that .vbs files and Windows Scripting Host are actually the "right" answer ... but here's another I used:

Write and debug your Outlook scripts in Outlook using the built-in VBS IDE.
Copy the bits you need into the dev environment in Word (this assumes your user has Word, and I'm guessing if they run Outlook they probably run Word as well).
Save the word document as a word-doc-with-executable-content (*.docm). Even if you haven't defined any GUI elements, any quick and dirty VB procedures you've written will show up as macros in the View->Macro dialog.
Bonus: write any instructions you'd like to pass along right in the word doc.
Give the file to your users.
COM-based scripting is COM-based scripting, so even though a user runs the macro in a Word doc, it can find and manipulate their Outlook items.

Not to worry, Word (and Outlook) prompt the heck out of the users when they open and try to run the code. And saying yes to your file will not require the user to grant access to any other script-bearing files.

It's not the slickest procedure in the world, but it's wicked fast and it serves the basic scripting purpose: Here's a doc with a script I wrote to convert Outlook notes into a single Task item for syncing with my Windows Mobile notes app.

Pardon the Glare: I Think My Tinfoil Hat is Catching the Sun

2008-07-18T16:55:00.001-07:00

Last week, it became clear that Kevin Martin and the FCC are getting closer to laying the smackdown on Comcast for the ISP's BitTorrent "management" policies (or was it because Comcast keeps lying about it? or because Comcast has tried to game the hearings about it? so hard to keep track of now).

And now all of a sudden I'm getting 4x more BitTorrent throughput on my Comcast cable line than I ever did before.

I'm tempted to imagine a connection.

But I also suspect that positing a connection is tinfoilhattery on my part ... in particular because the bandwidth/connection patterns I see don't look like the TCP reset packet approach that Comcast had been using, and which gave them away.

I'm getting more throughput in the middle of transfers, where there had never been any resets.

So maybe I should just be happy.

Or maybe that's just how Comcast wants me to feel.

On the Other Hand...

2008-07-17T12:57:00.001-07:00

Yesterday I wrote about the potential of the iPhone to break the longstanding wireless app logjam in the U.S.

Today the Free Software Foundation blogs about the other logjam we might be getting into with the device, namely that the app development and publishing community is just as locked down and controlled as the DRM on iTunes tracks. Wait, actually the app side is even more locked down (since there's a low-tech path to extract songs by way of audio CDs, whereas the app infrastructure requires jailbreak).

I'm curious whether this passage from the FSF post will have the fanboys plugging their ears and whining "I can't hear you" or just shrugging their shoulders and saying "who cares":

"Apple, through its marketing and visual design techniques, is manufacturing an illusion that merely buying an Apple makes you part of an alternative community. But the technology they use is explicitly chosen to divide people into separate digital cells, and to position Apple as sole warden. When your business depends on people paying for the privilege of being locked up, the prison better look and feel luxurious, and the bars better not be too visible."

If not a prison, it's still a walled garden. Like Verizon's mobile app vending machine (odd are you've never even hard of that one) or the old America Online. Maybe it can work, but there are better alternatives.

iPhone App Store: Let's Hope It's Not All Fun and Games

2008-07-16T15:44:00.001-07:00

The iPhone phenomenon seems to be a kind of Rorschach test, where you can find whatever you're looking for ...

There are all sorts of posts on data from the App store, looking at, for example, pricepoints in the app catalog, or the kinds of apps available.

These data are far from perfect (though worth a look), and in any case I hope that VentureBeat's post on the prevalence of Game and Entertainment apps turns out to be wrong.

Why? Mobile apps and mobile net usage have been waiting to "break out" in the U.S. for nearly 10 years. It's tempting to agree with the sentiment that mobile net use is reaching critical mass thanks to iPhone, but everyone in the industry has made that mistake many times before.

Successful off-deck software on phones in the U.S. has been more or less limited to games. So when I read the headline "Fun! Nearly half of all the iPhone App Store apps are games or entertainment," all I could think was "let's hope that it doesn't stay that way!"

I have no problem with Super Monkey Ball per se. It's just that if all the top app activity is in games and entertainment, then the mass psychology around the iPhone / App Store ecosystem is in danger of slipping irretrievably in the direction of "diversion" and not the enormous opportunity it really is.

Before you say I should just chill out about the Monkey Ball, consider:

Blackberry, because of its email device pedigree, was pigeonholed as a "corporate email/organizer device" years ago, even though it could do many other things. Despite RIM's attempt to sex up the brand, boost the hardware, cut prices and improve the design, it looks now like the window is closing for RIM to make any big gains. The enormous deployed base of Blackberries never translated into a general platform for mobile net-based computing.

At the moment, all of the top 10 -- and 18 of the top 20 -- paid App Store apps (over the last 24 hours) are Game/Entertainment apps. So right now, when the frenzy is at a peak, with mass media coverage of sold-out stores and consumers showing acquaintances their hot new gadget, the eyeballs are on the Super Monkey Balls.

To a lot of people, who have never seen a 3rd-party app on a phone, and who don't totally get what this is all about, the demonstration is going to look more like a Nintendo DS than a handheld computer.

Which raises a possibility: perhaps Apple was wise to restrict iPhone 1.0 to Safari-based apps, thus forcing consumers to view the device as a mini-web-tablet, rather than as a portable entertainment device.

Refactoring: Bittersweet if it Could Easily Have Been Avoided

2008-07-16T10:48:00.001-07:00

I've been doing some refactoring on a decent sized app lately. Since there are lots of other people doing stuff and the app is close to a supported 1.0 release, it's like working on the engine of a car while it's cruising down the freeway. Not just tightening the cap on the coolant reservoir, more like converting the engine to run on hemp seed oil and corn cobs instead of gasoline.

On one hand, it's fun: mundane tasks that might feel academic become a lot more interesting when the task is part of the Rubik's cube of after-the-fact architectural change.

On the other hand, a lot of the refactor is an attempt to deal with unnecessary complexity in less complex way. The underlying complexity -- not mediocre code or an initial pass at red-green development -- is really the enemy.

And here's the rub: this app has some network protocol requirements that are (1) outrageously baroque; (2) opaque and undocumented; and (3) completely unnecessary for the app to function for its users.

As a result, it's already eaten over 120 man-months (and a staggering amount of money, let's just say over 8x what you're probably thinking the labor cost was).

At least 2/3 of that time and money (maybe 75%) went to nothing the user will ever see or care about, and nothing that offers any inherent business value or competitive advantage for the owner. That 2/3 is all expended in dealing with the bizarro protocol requirements ... and the issues and complexity it spawns: the design compromises, the testing, the bug analysis and remediation, the refactoring. I'm not even counting the actual delays and feature cuts in this cost.

Where did I get the 2/3 number? Well, the project was initially scoped without this network protocol issue. The estimates were written down. And, in the initial iteration -- about 30 man-months of work over 5 calendar months -- we tracked resource usage and found that core functionality and QA was reliably 1/3 and closely matched the estimates, while accommodating these "other" issues was 2/3. Week in, week out. No surprises. Things haven't changed since.

It's ironic because this pattern is something that often comes up when developing against a true legacy system -- the formats, protocols, and systems in place simply cannot be changed right away for the convenience (or productivity) of a new team. And yet this initiative was green-field, not legacy. It could have used any protocol in the world. There was 0 (zero) install base that needed compatibility with this particular system.

I almost wish I didn't know this information ... It's the one thing that puts a real damper on the refactoring fun: knowing that with even a tiny bit of management planning and the willingness to deal with team issues, it would be unnecessary.

Yet Another Microblogging App ... With a Twist

2008-07-05T16:50:00.000-07:00

Fun as it is to argue about whether the latest Twitter outage will be the death of it, or whether identi.ca has legs, there's a whole other world of RSS/Microblogging possibilities that hasn't been exploited. Unfortunately, as I'll describe in a bit, there's a reason for that ... and an implicit call to action to fix it.

Microblogging -- or, really, the feed-oriented delivery mechanism -- could be huge for all kinds of private-domain problems that are stuck in Web 1.0 mode of e-mail updates and web-page dashboards.

Whether it's a group trip; a job search (that my good friends but not my coworkers know about); the latest status on project, issue, or change request; whether my cat is in or out (neighbor cats beware!); what my son is up to; a surprise party ... and on and on ... there are microblogging feeds I'd like to publish and consume, that are not public. They are not 1-1 either. They are published to a group. I trust members of that group, and they can bring in others. Perhaps many people can update or write to the feed as well.

Seems awfully useful in personal and professional contexts. But there's no app that exactly does this right now. The closest things are Tumblr's "groups" -- which are cool, but don't offer feeds -- and FriendFeed's "rooms" -- which are also cool, but don't offer authenticated feeds.

So I hacked together an app during a few spare hours one weekend to do what I wanted.

You can try it if you like at http://www.statusmonster.com or its real location, http://stat.heroku.com (have I mentioned how totally freaking awesome heroku is? well, that's a post for another day, but take me word for it, they are ninja rockstars those guys).

This app makes it super easy to create a multitude of private RSS (atom, really) status feeds, and share them with people. It has a mini-dashboard to watch the latest on a bunch of feeds. But I'm thinking folks don't need Yet Another Web Page to visit all the time. They need authenticated feeds, which statusmonster offers.

Wow, I'm gonna get rich and famous.

No, I'm not.

Here's the problem:

If a feed is not authenticated (like the basic feeds from statusmonster, or the "room" feeds at FriendFeed) then midstream aggregators like Bloglines or NewsGator may index these feeds for search, and/or offer them to others for subscription. This is the essential difference between syndication and publication -- a.k.a. the answer to, "Hey, how is a blog different from a homepage?" This is great for my blog, but really bad for my candid job search notes.

Ok, so we'll offer authenticated feeds. According the Bloglines FAQ, for example, authenticated feeds are not indexed for search or exposed to other users.

I did that, and started trying it out with NewsGator's desktop and mobile products, Outlook's RSS reader, Bloglines, Google Reader, etc.

The power of feeds lies in the fact that the end user gets to decide how to consume and/or process the content. That is, to get the full power/potential of feeds, they need to work with pretty much every major reader/aggregator service.

And that's where the trouble starts. Reading authenticated feeds with readers is a completely hit-and-miss affair.

The best behavior I found actually came from Outlook 2007, which not surprisingly treats the feed almost as an email account. It takes your credentials one time, and then when it polls -- or when you click "send/receive" -- it supplies them behind the scenes and updates the view of read/unread items. Pretty much exactly what you want.

But it was all downhill from there.

Bloglines processed my authenticated feed, but seemed to take forever to reflect updates (much longer than other feeds), and it eventually lost sync with the backing feed. The feed still exists, same location, same credentials, but eventually Bloglines started showing its little "[!]" marker meaning "problem with feed" and never updated again.

NewsGator kind of half worked. Google reader doesn't do authenticated feeds. And so on down the line.

So doing all this cool stuff on the existing infrastructure is not gonna happen. Big bummer because I'm convinced there's major value in these use cases, so we need to figure out how to make them a reality in a way people can actually use (and will want to use).

What are the constraints?

We don't need super end-to-end crypto any more than email does. Most folks do their email in the clear, figuring the content is not top secret, but also assuming that their employer/ISP/email provider is not gonna go and publish the email in a Google-able way.

I think that's the standard we're aiming for -- basically a better way to do stuff that nowadays might be handled by a whole lot of one-to-many emails.

A set of standards around handling authenticated feeds might be all we need. But how to enforce that, since anyone can cook up their own aggregator and ignore the standards?

Force the user agent (browser or client app) to supply some kind of secret to decode the feed? Maybe, but this is only strong in proportion to the strength of the key material, and having lots of high-entropy key material per feed makes this cumbersome and hard to use.

Something about confidential and syndication don't really mix. But we don't need another walled garden email/messaging system, and we don't need more web pages to visit (like Tumblr's groups).

I'm working on it. Meantime, what do you think?

When Corner Cases Attack

2008-07-04T15:51:00.001-07:00

Notwithstanding the credit given to MTV's Real World, I think reality TV took off with that Fox special that featured a deer knocking a hunter on his rear. That guy just never saw it coming.

Same thing seems to happen too often when we start making architectural adjustments around a corner case in a software application.

The real problem is how to tell when an exception to the initial model -- the corner case or edge case -- is truly a strange bird, versus when it is a big piece of the model that was missed the last time through.

Sounds like something that should be easy to resolve with proper analysis, whether of the thick-stack-of-paper variety, OO variety, or an agile story-based style.

But it's not.

I've recently been doing some work on a project where this mistake was made. At some point in the past, this team looked at an odd duck, thought hard, and realized it was a big piece of their world. Big enough that the underlying infrastructure -- database, network services, etc. -- should be designed with sufficient generality that this corner case would become a mainstream case.

I understand how the analysis went, and why it made sense.

Only problem is, with 20/20 hindsight we can now see it was wrong. Bringing this edge case into the tent made the system an order of magnitude more complex, and it should have been left as an odd hack in a dozen places ... but as development happened over time, on several different module teams, the problem of the added complexity wasn't clear until it was too late.

To be a little more specific, I'll offer an analogy: you're writing a driving game or simulation. A big part of the fun is the spots where the players catch air, jumping their vehicles over obstacles to get ahead. But how to model the behavior of the cars as they fly through the air? A quick hack? or ... integrating elements of flight simulation or 3-D physics simulation?

Well, this team went the equivalent of the flight simulator route, and their hard-enough-to-code 2-D driving game has become a brutal 3-D simulation. All for a few jumps that in retrospect should have been hacked.

My point is not to complain about a historical decision that probably didn't seem bad given the info available at the time. It's to ask about how to avoid these problems in the future.

If you grant my assertion that, in some cases, no straightforward analysis reveals the problem at the moment the "how do we handle this case?" question is asked, then it seems like we need a set of milestones for a posteriori analysis.

We make a design call when we need to make a call, and then we need some specific questions -- and times or situations to ask them -- that will tell us if we're on the right track.

Specifically: What is the best and earliest way to ascertain if we're

monkeyhacking in an edge case that really demands a proper refactor
OR
doing a refactor that way overengineers a solution that really just needs a monkeyhack

What do you think?