Opera Mini as Abstraction Layer

Opera Mini is a free J2ME-based browser from the folks at Opera. This FAQ tells you how it works and this demo lets you try it. I've spent some time using it on my old Blackberry 7250 and it's a pretty impressive browser &em; far more capable than the built-in browser on the Blackberry (although that's not saying a whole lot). How does it work its magic? The J2ME client handles user I/O and screen rendering (including a fabulous small antialiased font that is built into Opera); the page rendering, on the other hand, is offloaded to Opera's servers.

Since latency is the bane of even the fastest 3G networks, having extra proxying in the path makes the actual go-to-page browsing way slower than usual for a mobile. On the other hand, you have that warm feeling that practically any page, even moderately AJAX-y ones, will actually render attractively and usably when they do show up. Contrast this to the Blackberry browser experience, where the data starts coming down very quickly, but the device spend 30 seconds thinking about how to render it, and then ultimately produces something barely usable. Since subsequent navigations seem to perform better than initial ones, I suspect the Opera server is chasing forward links and preparing them while you read the first page. Not a bad plan.

I started wondering: Would Opera Mini serve well as a "base platform" abstraction layer for mobile web apps? If I code to O.M., and I can expect to be able to run unmodified on handsets all over the world, there's significant value in that.

What are the costs of such an approach? Hmm... let's see...

• Like all abstraction layers (Win32/POSIX/.net/...), the end user has to get it up and running. Sounds easy, but ask anyone who's ever wanted to deploy a "lightweight" .net or Java app and assumed users would already have the runtime ready to go...


• Opera has a done a lot of heavy lifting, but what if they change their browser capabilities, and now there's another fragmented set of clients, just a layer up the stack?


• Like all abstraction layers, there's a performance hit. In this case it's twofold: the memory and CPU usage of the Java runtime hits some devices pretty hard, and the proxy-through-Opera network access is painful even on EV-DO
  

In the end I think I'll take the weaselly way out for now. I wouldn't go all in, build an app, and say to the user "Opera Mini is a pre-requisite for this app. If you get it, you're golden; if not, no support for you!" I'd target a base of browser set with some kind of adaptive rendering. But I might well make Opera Mini the first-tier tech support response for any native browser problems.

The Eagle has Landed at 601 Townsend St.

I spent last night at Adobe's Apollo Camp. The Apollo Camp was about exposing a group of developers to the (almost) latest build of Apollo; introducing some partner apps and the insights and clever tricks these apps have already spawned; and letting us meet, chat, question, and occasionally argue with the guys who build Flash, Flex, and Apollo.

Since this is a commentary blog, I'll let the tech news blogs cover all the great content that was presented last night. Much of it is probably in the blogosphere by now; a lot more is due to be released by Adobe via Labs very soon.

Apollo is going to be an important and prominent platform. It's not perfect (Adobe certainly doesn't claim it is right now either), and the truth is it doesn't actually do all that much. But it is a kind of fabulous connective glue for the front end. Metaphorically it reminds me of OLE/COM/ActiveX in its desktop ambitions. It's not defined to be a desktop object bus per se, but it's way easier to use, and it's cross platform. It is also reminiscent of web services and RSS in the middleware realm, in that it provides a connective mechanism simple enough but expressive enough to throw the door open for all sorts of creative mixing and mashing and integrating fun.

It's interesting that we're at this juncture with Adobe/Macromedia poised to leap into the RIA lead with this tech. Credit is due to some long-range strategic thinking. For example, we were told that Flash player has for years now been designed as an express install vector for Apollo. And moving to from AS2 to AS3 reminds me of the bold but necessary move from VB6 to VB.net that Microsoft took in '01.

More than a little is owed to luck, or lucky timing, as well: without the evolution of Agile and TDD, leading to effective development practices with dynamic languages (and popular day-to-day AJAX, Ruby, and Flash apps as a result), it would be hard to imagine a bright future for desktop apps built on ActionScript. But these things have come to pass, and AS3/Flash9/Flex/Apollo/Tamarin has shown up at just the right place, and just the right time.

Making Mobile Browsers Work Better with ASP.NET

ASP.NET is architected with a robust scheme for adaptive page rendering. This scheme allows "server controls" or metamarkup to be rendered appropriately to various browsers, and becomes particularly important when using the mobile web controls.

For an overview of how this works, take a look at ASP.NET Web Server Controls and Browser Capabilities (from MSDN/Visual Studio '05 docs)

There are also numreous ways of overriding this framework, from code (Page.ClientTarget) to XML config files, to defining new browsers, or altering attributes of the browser configurations that ship with the framework.

Modification is critical for mobile web development, because the device data that ships with ASP.NET 2.0 doesn't include any of the wireless devices shipped in the past two years or so. (For a list of what is included, check out this list and walk down memory lane)

Since most users of mobile applications will be carrying a device built in the last 2 years the built-in browser profiles will not cover current customer devices. The good news is that the browsers in these newer devices are more advanced than the earlier browsers, and make a credible attempt to process xhtml and javascript. So less divergence from a "basic desktop browser" profile is needed to address these clients.

Here is what I did to handle some problems with newer Motorola and Nokia devices:

If the framework cannot come up with a positive match for the User Agent string to either an individual browser definition or a family of browsers, it is bound to the "default" profile and identified by the name "Unknown"

All major desktop browsers are correctly identified, whereas many (maybe most) late-model mobile browsers are not identified. So if the browser is identified as "Unknown", we'll assume it's a new mobile device.

The framework will render for a base desktop browser, which works pretty well. But certain postbacks cause a processing error which can be circumvented by altering the "default" profile and adding the attribute "requiresPostRedirectionHandling" right into the default definition. Rather than editing the files that ship with ASP.NET (which may be impossible in a shared hosting environment anyway), the preferred approach is:

1. Add the special "App_Browsers" folder to your ASP.NET 2.0 app
2. Add .browser file to that folder -- mine is called Supplement.browser
3. Add the attribute into an XML element that specifies this should modify the existing default definition. Here's how the whole file looks:

<browsers>
 <browser refID="Default">
  <capabilities>
   <capability name="requiresPostRedirectionHandling" value="true" />
  </capabilities>
 </browser>
</browsers>

The refID attribute refers to the Default definition that you can find in [WINDIR]\Microsoft.NET\Framework\[VERSION]\CONFIG\Browsers

For my application, this fix got my app running great on the MOTO RAZR devices, where I had experienced a number of problems before.

4. I also wanted to target newer Nokia phones. They have a family of browsers which are identified even when the individual phone model is not. So I added this element inside the 'browsers' tag to fix the Nokias:

<browser refID="Nokia">
 <capabilities>
  <capability name="cookies" value="true" />
  <capability name="preferredRenderingMime" value="application/xhtml+xml" />
  <capability name="preferredRenderingType" value="xhtml-basic" />
  <capability name="isColor" value="true" />
  <capability name="requiresPostRedirectionHandling" value="true" />
 </capabilities>   
</browser>

As you can see, I noticed that these devices are xhtml capable, but the Nokia default profile was sending them WAP content. I also pointed out to ASP.NET that these devices take cookies, are all color, and handle redirects differently from desktops.

I'm sure I'll find more issues and more fixes, but these have made a big difference once I figured them out.

Incidentally, save yourself a lot of pain and do not rely on either Motorola's ADK emulator or Cingular's online (ActiveX) emulators to test web content on these devices. Many difficult bugs appeared with these emulators, but on the actual devices, everything worked great. Nokia's emulator/SDK on the other hand was pretty solid and very helpful.

I Have a New 6-lb. Cellphone

I was traveling last week with a pretty decked-out Thinkpad T60p, and decided it would make a great smartphone. It's got a huge screen, a 512MB 400MHz ATI graphics card, 2 GB of RAM, happily runs a few VMs, development tools, databases, and DVD-quality movies all at the same time. It runs 5 hours on a charge while doing that, can idle in sleep mode for days, and stays on the network with integrated WiFi and EV-DO.

Last time I checked, smartphones don't do any of that stuff, and if you continually use them to access data, their batteries will not even last 5 hours.

So crank up Skype or Gizmo Project and it's a heck of a phone.

And yet ... at 6.5 lbs, and a foot across, I'll get some odd looks when I clip it to my belt.

Where am I going with all this? there are two things we need here:

First, a new BIOS-level powersave mode that lets me receive calls -- routed to a Bluetooth headset -- when the laptop is sleeping, maybe even with a satellite UI ... wait, someone had this idea, it's called Vista Sideshow. So I'm asking for a Vista Sideshow VoIP phone gadget.

Second, a more flexible way to link multiple telephony devices to a single phone line. I want my Sideshow PC phone and I want a $49 cellphone that I can carry around when I don't want to carry the PC -- and I want them on the same number. And I'd like to link my Blackberry onto that number too, so I have my choice of high productivity, high voice quality, or a compromise. I'll bring the hardware and the subscriptions. Wait! My second wish is already coming true. It's called GrandCentral.

Now let me get this all rigged up and I'll report back.

BitTorrent's paid movie rentals are just silly

Not the concept. But these offerings are so predictable, and so predictably bad, in their pricing that each one is just another big delay before a service can finally be produced that offers real value to consumers, studios, content creators, and publishers/distributors.

Here are a couple of examples. First up, this Bittorrent service: from them, for $4, I get a 24-hour viewing period on a media file playable in some devices, with Windows Media Player.

Now, behind door #2, I have a video store where for a little over $3, I can get a physical disk that will play in far more devices, for a period of several days, most likely with better video and sound quality.

Maybe this demo (males 15-35) is too cool to go to a video store? I doubt it, but you also have Netflix. A conservative calculation (cycling 2 disks per week on a 3-at-a-time sub) yields about $2 per rental. Plus I can keep the disk as long as I like, and I have the luxury to not watch it in 24 hours if I’m busy. Although I do pay another convenience cost in that I am dealing with the Netflix queue, not an on-demand selection, this cost is essentially paid to Netflix; that is, it is a virtual subsidy of the Netflix operational model. The studios get no benefit from that at all, since the relative physical scarcity of disks is not in their model (they fix the original disk price and press as many as they can sell), but only enters into Netflix’ model (Netflix can only reasonably acquire, use, and then dispose of a modest number of disks for each film).

There’s also on-demand films from satellite/cable. Cost: $4. Terms of use? basically old school TV on VHS rules: I can record the show to my TiVo and make suitable personal copies (e.g. with my DVD recorder), which last indefinitely and which I am allowed to watch whenever I want. Downside? NTSC quality video.

Bittorrent is selling me a strictly inferior good at a higher price. Economics says they’re not going to succeed, and the studios will claim it’s because viewers are all crooked.

Not to pick on Bittorrent in particular – without doing a rundown of all of these work-alike online stores, let’s look at one more: Amazon Unbox made headlines for its onerous Terms of Service as well as its implausible pricing. Here are a few couple of typical price matchups all from amazon.com:

The Departed:
Unbox restricted download: $14.99;
actual DVD, widescreen with extras: $15.99;
BluRay high def 1080p disk $23.95

The Devil Wears Prada:
Unbox restricted download: $14.99;
DVD new $13.89;
DVD used ("very good condition") $8.76

Babel:
Download: $14.99;
DVD new: $14.24;
HDDVD or BluRay: $27.95

In some cases there are shipping charges, but there are numerous ways to avoid shipping charges on Amazon. For reference, iTunes new releases are around $12.99 and are also heavily restricted.

The point here is that not only do these ventures refuse to concede that more restrictions on media make it less valuable to the consumer, but they actually imagine that they are somehow innovating in a way that will let them charge more than the baseline cost (physical DVD/CD and accompanying rights are the baseline).

Nothing here suggests that the media should or must be "free" – only that Bittorrent president/cofounder Ashwin Navin and the studios are all yanking our chains when Navin says, "We're really hammering the studios to say, 'Go easy on this audience' ... We need to give them a price that feels like a good value relative to what they were getting for free."

MSFT apologia

Ok, it's not a huge secret that I'm a Microsoft apologist (that is to say defender). Not that Microsoft hasn't made its share of mistakes and done some things wrong. But yesterday a friend, not a developer but a power user, lightheartedly referred to Bill et al. as "software bozos" and I felt obliged to point out a few things...

Microsoft produces great products under an unbelievable set of constraints. Customers want Microsoft stuff to work seamlessly on everything from cell phones to PCs to set-top boxes to web servers to XBox 360s; they want it to make sense to everyone from CEOs to doctors to my mom; they want it to be localized (support local language, culture, currency, calendar, phones) everywhere in the world, and to be accessible to the handicapped and to be secure even when an extremely unsophisticated user tries to do really dumb things.

They also want it to be inexpensive and to work on any cheap hardware you buy off the 'net and install it on (unlike, say, Apple, where the OS is only legal and supported on the hardware they're in the mood to offer this month); oh and besides being a general purpose operating system, customers like it that Windows is one of the most advanced 3D gaming platforms, competing with dedicated gaming consoles that cost just as much to build as a PC and need do nothing except play games...

Oh, and also, unlike pretty much any other OS I'm familiar with, customers (especially business customers) need it to be perpetually backward compatible, so that when they put a new Vista machine together today it'll still run line-of-business apps that were written for DOS 4.01 in the 80s, and somehow magically these old apps will print reports on the new color laser printers attached to the computer, that were never even dreamt of when the apps were written. And mostly this actually works.

Now let's say you live in America and you buy a new/upgrade copy of Windows every 4 years for about $200, and a new copy of Office for about $400. You're paying about $12.50 per month. And you get the security updates, and browser updates, media player, Virtual PC, development tools (if that's your thing) and all kinds of other stuff for free (or included in your $12.50 per month admission price if that's the way you want to think about it.)

I'm not sure there's anything else I pay $12.50/month for that even tries to think about solving problems on this kind of scale, let alone succeeds.

Lastly, someone will be tempted to point out that Microsoft's enormous presence in the client OS and office productivity space may inhibit all kinds of other software ecosystems from flourishing. There are a number of open questions about this. First, it is reasonable to believe that standardization at one level in a stack enables massive innovation at the next level up the stack, which would otherwise have been impractical. This goes for any platform piece -- Ethernet, Windows, *nix, Java, HTTP...

More importantly, do not assume for a minute that the open PC architecture would even exist without the dominating historical presence of Microsoft Windows. The fact that you can even sit down with an assembler and start hacking a boot image and work your way up to running literally whatever you want on a readily available PC has never been a given. Considering the attitudes of more closed OS and hardware makers in other ecosystems (like cell phones), it is entirely possible that without Microsoft and the need for backwards compatibility, just running code on a cheap mass-produced box would long ago have required signed code, a crypto key from some industry licensing group, and more cash for membership and fees than any small company is ever going to have.

Grepping in PowerShell

I originally wrote this for the company wiki the other day, and thought it might be useful to a wider audience. The context is parsing and processing an iTunes library.xml file (just a one-off task), which I thought might a be a fun and educational opportunity to slice, dice, and ... how does that Ron Popeil commercial go? ... with PowerShell.

PowerShell is the new shell for Windows. New, and supported, but not "the official" in the sense that it doesn't ship with Vista, although I'm guessing it will ship in the Longhorn Server rev.

If you're used to Unix shells, then you'll probably be floored by the power of PowerShell and somewhat annoyed by the syntax, which, despite liberal aliases to familiar things like ls takes some getting used to.

.net framework integration means you can easily access any object in the .net base class library, and there are some special tricks that do some of this for you too. The canonical example seems to be this one, a quickie rss reader:


$wc = new-object System.Net.WebClient
$rssdata = [xml]$wc.DownloadString(‘http://foo.bar/rss.xml’)
write-host $rssdata.rss.channel.title
$rssdata.rss.channel.item | foreach { write-host $_.title }


Since the source file is xml, I had thought the XML parsing would come in handy, but it turned out that there was no real data model to the XML. Basically, there is just a big nested map structure (key-value pairs in blocks) in the item list. Sort of XML for the "takes-void*-returns-void*" crowd. So then grep looked promising because the keys and values (and their tags) were grouped on individual lines.

Grepping is a little counterintuitive with PowerShell because the pipeline between commandlets in PowerShell is filled with full-on objects not strings. If you just want text, you can use Get-Content, which provides its output as a bunch of string objects, 1 per line, which is convenient. Here's an example I came up with after struggling a little bit to get a grep type of functionality. I throw a sort and unique on here for fun:


Get-Content Library.xml | ForEach-Object { if ($_ -match [regex]"(?<=Artist\<.{13}).*(?=\<\/)" ) { $matches[0] }} | Sort-Object | Get-Unique | Out-File lib.txt


Many of these things can be abbreviated too, so if you want your script to read a little tighter, you can use


gc Library.xml | % { if ($_ -match [regex]"(?<=Artist\<.{13}).*(?=\<\/)") { $matches[0] }} | sort | unique

Isn't that sweet?

Since the regex uses zero-width lookahead and lookbehind assertions instead of extracting a marked subexpression, I'm curious if anyone has input on whether one approach is faster / better / shinier than the other.

My first guess is that they are similar, since my first cut at implementing lookahead + lookbehind would probably be to match the whole outer expression while naming the non-zero-width-bit in the middle, and assigning the value of that to the expression.

Yipes it's Y! Pipes

Super cool: there is no reason that a human should need to handwrite HTTP/XML/mashup/filtering logic for simple cases. Even with the highest-level toolkit, it still requires time, introduces bugs, needs to be hosted...
Systems like this are about moving toward a declarative specification for extracting semantics from web services (in this case RSS).

This particular implementation is a bit fancy on the graphics, which makes it run slowly, and it seems like it needs to extract data from RSS only. That is, if you try it out, it expects every URL "fetch" result to look like an RSS formatted collection of "somethings" ... which is nice, but it would be cool if you could also process XML from REST queries, or build SOAP queries as well. My first inclination was to ask for some kind of RegEx widget, but perhaps the Y! Pipes team intentionally doesn't want to allow us to go down that route ... over time they want more structure, not less structure in the data. They probably feel like RegEx has already been done in the HTML scraping world, although there is certainly lots more work to do there.

If you are interested in this stuff, check out some other approaches and flavors of this notion too:

- Dapper which tries to build web services on top of any web page as a data source. These guys have a "virtual browser" which lets you point and click your way through existing pages to build a service

- Kapow and OpenKapow -- enterprise and "free online" design tools for scraping, mixing, mashing and republishing the web

- QL2 an "old-school" enterprise software product used for industrial strength scraping, it implements a query language so that you can treat the web data sources that are being used as a virtual database (!) (frighteningly enough for an "unstructured data" query tool, this system is used in some large mission critical apps)

- YubNub: this souped-up version of wget lets you define "commands" (aka abbreviations) for issuing web queries, can substitute parameters, and pipe things together. It's arbitrarily extensible since you can always write a servlet/ashx/&c. to provide any data access or transformation you might want. On the other hand, it's more about plaintext (or human readable anyway) than XML

A Little Less Code, A Little More Action

I've been working on a side project, with a very web 2.0 flavor, partly serious (I really want to use this product, so I'm building it myself) and partly tongue-in-cheek (includes many free cliches, from the GEN-U-INE web-2.0-logo-generator masthead to a name that ends in -r).

In trying to maximize my productivitah and agilitah, I've been forcing myself to write absolutely as little code as possible, and to lean heavily on framework pieces that let me get it running now, and refine/refactor/redo later:

1. ASP.net 2.0 -- the built in support for users, roles, master pages, data binding to arbitrary objects, integrating SOAP web services, and mobile web pages is both well documented and fantastic. Been around for a few years, hardly news. Well understood and can scale like a mofo if I'm ever so lucky.

2. ASP.net AJAX and the AJAX Control Toolkit -- these are newer and totally rock out. In my book, there are only two long-term high-productivity high-sustainability conceptual approaches to AJAX. One is the ASP.net AJAX approach (there are also libraries on other platforms that use this same method), where simple declarative markup causes automatic generation of relevant client side code, server side endpoints, object marshalling, etc.

This allows really neat tricks like the following: start with an asp:calendar tag for a calendar control. Bind any sort of logic you want to its ASP.net post-back driven events on the server -- as simple as changing the UI or as sophisticated as booking a reservation on the selected date. Now just wrap the tag inside an asp:updatepanel like this

<asp:UpdatePanel ID="UpdatePanel1" runat="server">
   <ContentTemplate>
      <asp:Calendar ID="Calendar1" runat="server"/>
   </ContentTemplate>
</asp:UpdatePanel>

...and now your post-back is done, server code run, and results rendered all in an AJAX call.

The other sane approach is the Google Web Toolkit / Script# approach, of writing the client app in Java or C# with static typing, interactive debugging, refactoring, etc., and then compiling to Javascript as a build step. Of course I did succumb to one use of the "unsustainable" approach -- hand-coding a particular AJAX feature I wanted -- but I think that most typical AJAX cases can be handled by one of the above structured approaches.

3. SubSonic, an ActiveRecord implementation for .net -- I'm a big fan of object-oriented design, which usually means a heavier OR/M layer to a substantially distinct RDB schema. But for a simple project with a tiny straightforward domain model, I wrote the schema and went with ActiveRecord instead. I am extremely impressed with SubSonic. It does just what you want, code is easy to debug in the rare case something blows up, and poof! automagic simple data-accesss layer with beautiful generated classes, and no code.

Using these three framework pieces, I've managed to put together a site with users, roles, profiles, a dozen pages, a half-dozen mobile pages with automatic device-specific rendering, data access for the actual domain objects, mashing up a couple of external services, AJAX where appropriate ... in probably less than 250 lines of hand coded C#, and less than one man-week of effort.

Time (and some free analytics data) will tell whether it's useful to anyone besides myself, and whether I should expand it with more sophisticated functionality. But of course the beautiful thing (and yet another web 2.0 cliche) is that with this little effort to build, and a few bucks a month to host ... it's a fun time and who cares if no one else wants to use it ?

Moving On but not Skipping Out

In the interest of full disclosure, yesterday was my last official day at Skip Interaction. I'll be moving on to new projects which I hope to discuss here soon.

Most likely I will continue to support Skip's efforts in one or another fashion as the company seeks to move up to the next level of larger distribution channels, additional funding, and the fun stuff: more one-click travel functionality.

A Computer that You Can Program: How Novel

In a big-boxy toy store I noticed what might be called kids' computers. These are laptop form-factor devices made for kids, that include age-appropriate pseudo-educational games.

These devices are sophisticated, with full keyboards, card slots, USB connectivity, mice, touch screens and touch pads in some cases... and benefiting from overseas production in mass quantities, they range in price from around $30 to $90. This page, if you scroll all the way to the bottom and look at the last three rows, shows the devices I'm talking about.

I'm standing there thinking, "Sweet!" ... How cool would it have been to have one of these to work on when I was a kid, programming a Color Computer 2 that cost over $500 inflation-adjusted (about $250 at the time). I learned more from programming the CoCo than from any software I could have run on it. And I believe that any child will learn more from creating with a computer than from some "shape drill" or "math drill" software. Just as I give my son blocks and Lego bricks to build with, I wondered if these inexpensive laptop wonder toys had a code mode, where the child could write a program, in Logo, BASIC, Squeak or anything else.

So far as I can tell, after reading through the manual (published online) for VTech's top-of-the-line Color Blast Notebook, there is no opportunity for programming this device. The manual for the Touch Tablet notebook (which supports PC connectivity) also shows a huge list of interesting built-in programs and utilities (including a whole category of "math and logic"), but no programming language.

With Logo and Squeak, programming can be as fun and easy as using Lego blocks. Making kids' computers impenetrable objects featuring only software that is published to them, rather than created by them, makes about as much sense as teaching kids about shapes but never giving them a crayon and letting them draw.

P#: Yet Another Reason I Love .net and the CLR

When Microsoft said that the CLR was designed to support many languages that might be right for many tasks, I don't think they meant fundamentally similar languages like VB and C# (modulo the syntax).

What's really cool is being able to cruise along in C# ... and then let Prolog go to work on your logic resolution with this Prolog implementation.

Here's a partial list of .net languages from Ada to Zonnon.

Bruce Tate's Gettysberg Address

Bruce Tate is a consultant probably best known for working on Java, and for publishing the book Beyond Java.

I recently read Bruce's more recent From Java to Ruby, while standing in the tech section of the San Rafael, California Borders store. Sorry, Bruce, that you won't get the royalties for this, but I just couldn't put it down.

This book has a lot of great content (not that I don't have a small bone to pick here or there ... if you really think .net is still Java's separated-at-birth-twin in 2006, you need to spend more time using it). But what really struck me was the elegance, brevity, comprehensiveness, and precision of the book. A few more writers like Bruce, and we could happily live without about 90% of the IT press.

In precise, minimalist language Bruce systematically looks at the where Java and Ruby came from; where they each excel; where they fall short; how they compare to their predecessors and contemporaries in various dimensions -- real, no nonsense dimensions that enterprise architects think about; and how to get started with Ruby (if you want to) with a fabulous insight into the organizational dynamics that would foster different kinds of Ruby trials.

You get the immediate and lasting impression that this is a guy who really has a valuable perspective on the long-term evolution of software engineering practice, from the Mythical Man Month through today's productivity push with things RoR, and on into the future.

DIV is the new TABLE

Ok, I'm not a web designer by any stretch of the imagination, let alone an expert in building layouts using CSS. On the other hand, I've been using HTML and its offshoots since before it was the standard for the WWW. So I'm puzzled that designers spend pages debating the best way to get, say, a 3-column layout to work well in various browsers using CSS, and that there isn't a straightforward declarative way to do it.

I used to do "desktop publishing" (remember that term?) with a Mac SE, 1 MB of RAM and PageMaker. PageMaker had no problem working with all sorts of page layouts: first creating columns and then adding blocks that lived in, around, across, or through those columns, with and without "flow"... on more or less any size piece of paper ... and it could do WYSIWYG on the Mac SE's tiny monochrome screen while delivering mathematically precise PostScript for prepress work. A different but equally effective view of the world was that of QuarkXPress, where content blocks asserted their own column layout, rather than just snapping to column guides on a page. So it's really not such a hard problem in 2007.

There is a CSS3 draft published over a year ago on multi-column layout. The next step in that process appears to be another draft. I have to believe that someone is overthinking the problem. Start with column count and gutter, all uniform, and go from there. Then go back and add uneven columns, blocks that span multiple columns, flow of content between elements.

Never mind that CSS allows (or could allow) you to flip every switch on every element in a page -- good design should let you do first things first. And the first thing for many page templates is a header-multicolumn-footer layout. CSS without core layout declarations is like a sophisticated office phone where you can't find the number pad.

Defenders of the CSS process may blame the browsers, and there is blame to go around on some well-known bugs. But an overly complex (dare I say overengineered?) spec for how CSS must work doesn't make it easy for browser makers to comply. Instead, it gives them cover. Meanwhile Microsoft and Adobe are edging in on "standard XHTML/CSS" with extremely appealing flow document alternatives. I love Flash, WPF, and WPF/E. But I'd hate to see proprietary runtimes become the only way to deliver top quality design experiences on the web.

Obligatory Windows Vista Post (™), Part 2

Improved high-density display support is a big part of Vista. But what about those of us with low-DPI displays?

Ok, is that supposed to be a joke? Who would have a low resolution display?

Anyone who has a LCD panel larger than 17" and which runs at a native resolution of 1280x1024. Which is to say, anyone who has bought a nice desktop 18" or 19" panel in the last couple of years (excluding some widescreens, and the 19" panels at 1600x1200). The Vista "baseline" resolution is 96 DPI, which turns out to be just right for typical 17" flat panels (at 1280x1024). On these displays, UI elements which are sensitive to DPI render beautifully. In particular, the ClearType font smoothing technology which is widely used in Vista, even for legacy apps.

That same ClearType logic on 18" panels (87 DPI or thereabouts) or larger produces text so blurry it's distinctly uncomfortable to read.

The display customization box where a user can specify the DPI of the display has another complementary quirk: it allows you to choose higher-than-standard pixel densities, but not lower ones. If one types in a lower pixel density (as a percentage), it "snaps" back to the default setting. Moreover, if you have two monitors with two different pixel densities (I can't be the only one in the world in this situation), there's no way to specify that. It may be the case that a video card can supply this additional option, but it's not available in the latest nVidia drivers. And what if you have two different video cards (so that a single driver is not managing both of them)? The latter possibility suggests that the OS needs to provide this option...

Any ideas out there? Am I missing the "big red button" somewhere that fixes this? (By fix, I mean offer a place to enter the precise DPI of each display, so that the OS can inform apps, and tune its own services [ClearType] accordingly.)

Finally, lest anyone wonder why a large lower-res display (such as 19" monitor at 1280x1024 native resolution) would ever be desirable:

1. Viewing HD media content from a distance. Neither a smaller monitor, nor more pixels on a bigger monitor is a significant help here.
   
   
2. Gaming. Running a game at 1280x1024 on a larger screen is a great way to get a rockin' immersive experience without the substantial additional system load that would be required to render 1600x1200 (or widescreen) frames.
   

Anyway, I'm looking for answers. In the meantime I think I'll downgrade to a pair of matching 17" displays for regular Windows work.

Obligatory Windows Vista Post (™), Part 1

Ok, what's to say about Vista RTM amid the flood of coverage...

I installed Vista on my wife's laptop to do some usability testing. She is a relatively strong "typical Windows user", but not a "power user." Uses Office, the file system, corrects redeye in photos, knows what not to click on on the web; has no idea how to add user accounts, update a driver, or monkey with the control panel; doesn't know why anyone would want to do it.

As a geek, the parts of Vista I love are the parts not immediately visible. Mainly, the out-of-the-box support for .net 3.0, especially WPF and XPS. I'm always asking, "Have you seen the New York Times Reader?" I'm still mourning the loss of WinFS, and the Vista shell experience struck me as just-another-graphical-shell, with a couple of new things to see/find/learn/adjust to. My wife, on the other hand, had no problem with any of the changes (e.g. in the filesystem Explorer windows). She loves Vista. She said she found it way easier to use than XP: more intuitive and more productive (not to mention more visually attractive).

My point here is not to make a commercial for Windows Vista; this experiment compares Vista only to XP, and says nothing of how, say, OS X would stack up. Instead, what I really learned was how differently usability design / engineering / testing in Vista comes across to a "normal" user than to a geek. Every time I "discover" this usability fact, I am surprised anew, even though I shouldn't be.

I'll never stop being amazed by (1) how completely wrong we geeks can get it when we try to talk about what is usable and appealing to average tech consumer personae; (2) how brilliant the great usability engineers and designers are who can get some of this stuff right up front; and (3) how much of a mistake it is, when developing technology products, perpetually to defer serious usability study.

Mauve has the Most RAM

Here's a recent post on the 37signals Job Board. I've seen a number of these postings lately, and I wonder: why would you specify a particular technology or platform when you haven't prototyped (let alone built out) a product yet?

When I've posted ads for jobs at Skip, notwithstanding our existing platform and the advantages of extensive experience in the relevant technologies, I've always emphasized that talent and attitude count way more than a particular language or platform. Experience with core CS concepts and perennial software development patterns/anti-patterns comes next, but is still more important than a platform or framework. Because anyone really talented knows that just as "Java is the new Cobol," .net will someday be the new MFC, and RoR will be the new ... OK, you get the point. And anyone with the proper foundation, inclined to ask the right questions, can learn new frameworks -- even new practices/methodologies (like an agile approach) -- and put them to use.

So why are Web 2.0 founders performing what I would call an extreme and misguided attempt at premature optimization?

First guesses:

• It's boom time again in the Silly Valley, they've read a Business Week article that talks about platform X, and decided the way to become the next Google is find some X hackers
• They don't understand the differences between the open source options, but like the idea of open source, so they've picked one... that is, they really mean "I believe open source affords my startup some opportunities or advantages, and I need an architect who is an expert in the relevant options"
• They asked their geekiest / most successful techie friend, who said, "Just go with X nowadays, don't worry about [insert important but subtle tradeoff here]"

I have another idea: the entrepreneurs have gone a bit behind the hype to get a feel for the practices or philosophy associated the best-known practitioners on that platform. They then imagine that the best way to find folks who subscribe to that philosophy is to decide on the platform, then hire for the platform.

So when someone insists on LAMP, for no particular reason, maybe they really mean they are hoping to engage an engineer or team that embraces agile practices or at least agile concepts. When the entrepreneur has got his mind set on Ruby, he really means "the Ruby way" (least surprise, don't use 200 lines where you could use 10, etc.) If he or she says Ruby on Rails, it probably means get-it-up-quickly, include-some-AJAX, and ActiveRecord-will-do-it-for-now.

These folks would do well to distill out their methodology desires from premature platform commitments that cascade into hiring filters. After all, with the hardware virtualization and bytecode interpreters prevalent (and coming), and web services as de facto server-side object bus, we can do ActiveRecord on .net on Mono on Linux, or Java interoperating with Ruby on OS X, Agile development with IronPython on IIS... There is less reason than ever to close off options before any code has even been written.

Travel Right with Skip 1.2 for Blackberry

It's been about a year since the first version of the Skip client became available for testing. Targeting connected business travelers, and inspired by the interface metaphor of classic killer Blackberry apps like email, the Skip Blackberry client was heavy on text and light on visual sophistication.

As Blackberry users migrated to the high-density color devices like the 87xx and now the Pearl, Skip started looking a lot like opening Notepad on Windows Vista. But the 1.0 release suffered from the curse of being good enough. No major bug was ever reported for the Blackberry client, and the biggest usability problem by far turned out to be folks having trouble keying in their id and password on the device (not just SureType -- even QWERTY folks have had a lot of trouble, strangely enough).

Meanwhile, 2006 turned out to be a busy year for Skip's 1.5-FTE engineering division: to get the ball rolling we needed to deliver the core Skip server app, minimal web UI, travel industry integration, Java phone client (twice), Plam/Treo client (twice), Windows Mobile (it's in test, so try it now with this OTA install; additional bits you might need install OTA from here and here)... and the Blackberry client never got a proper polishing because it was just good enough never to get to top of the "urgent" list.

So I'm glad to finally offer this substantially touched up Blackberry client to all of our loyal early adopters. Most of the changes are small bug fixes, ergonomics improvements, and UI features which will be self-explanatory to readers of this blog. The only non-obvious change is a power-user mechanism that lets you use the same client to work with a production (www.goskip.com) or test system (peridot.goskip.com) account: in the username field, simply prepend "test!" to your email address (e.g. instead of foo@bar.com, test!foo@bar.com), and the client will operate with the test system.

You may notice some new UI features (like the icons) will only appear via the test server for now, as we'll be doing some additional testing before moving the server code changes into production. ETA is probably a week. Also, the first time the client syncs, it will load up the icons into its local cache, so that first sync may take a little longer than usual, especially over EDGE.

Let me know what you think, and thanks for your patience this, er, um, whole year.

Security Questions Considered Harmful

I went to sign on to the citibank site today, and before I could complete the sign-on, I was presented with the following required step:

This whole approach is just so darned awful.

This data isn't secret. Some is public record, like where I was born. Other things, like my favorite pet's name, are not pieces of data one normally protects. It's much easier to amass a collection of unprotected facts, and use those to pose as someone, than to compromise an actual password, an encryption scheme, etc. Moreover, most of these questions are used by many sites for the same purpose. So if I know someone's nickname, street address growing up, city of birth, mother's maiden name -- all readily available in the U.S. -- I'm that person on a lot of websites.

Personally, I use Mr. Schneier's approach of typing in random gibberish, thereby protecting myself at the cost of some convenience if I do lose my password.

While I'm on the topic -- and I'm sure someone has written authoritatively on it before, but nevertheless ... -- I am surprised how rarely people realize that their unprotected email account is the weakest link for all of their "secure" online activity.

Many sites (including Skip) email a new password to a user upon request. The gmail account that they stay logged in to, or log in to from a questionable computer in a hotel lobby, thinking, "My life is so boring, hey if someone wants to read my gmail, let them" ... or a business email account accessed on the road in the clear ... these become the easy way to get passwords.

There's nothing inherently wrong about using email to do a password reset (reset -- that means only a temporary password is sent via email; the user must then change it to a more protected one). Folks just need to realize that the email account -- if it is registered with sites that send passwords -- needs to be protected. Complex password, routine changes, all that...

The Transactional Web

Ok, maybe a double-Z-list blog like this one ought not to start coining phrases.

On the other hand, I started thinking a little about the taxonomy of the (programmable | read-write | 2.0) web.

Three categories came to mind right away:

• Classic mashups take two or more sources comprising different kinds of data, and combine them in a new UI or tool. This group includes things like HousingMaps (housing from craigslist + Google Maps), or BroadwayZone, which pulls in show info, hotels, transportation, etc., and uses Google Maps to provide a substrate and a UI for working with all of this data.
  
  
• Agent apps take multiple sources of related data, and bring them together to perform an action on behalf of a user. Some are goal-seeking agents, which search a wide space and narrow it down to something tractable based on a heuristic like minimizing a price. Kayak, which interacts with a huge number of travel websites and vendors, and Ugenie, which searches e-commerce sites, are agent apps.
  
  
• Proxy apps take one or more sources of data and bring it into a new context. 411Sync's Kayak queries use Kayak's own API to get raw data that can be formatted for the SMS service. This category also includes RSS (client) widgets, and mobile smart clients like Abidia (mobile auction), Mobio (movies) and, yes, Skip.

Being at Skip, it's probably no surprise that I find the last category the most interesting personally. This category of apps is spawning what I've started calling the transactional web -- it's a flavor of mashed up app that makes real commits against external services. Not all proxy apps are involved in the transactional web, it's just that mobile proxy apps are where a lot of the transactional excitement is right now. Unlike the other categories above, these apps are doing things like buying movie tickets out of a finite inventory, checking you in to a flight, bidding in an auction.

Other kinds of apps can be "transaction mashup apps" of course -- I can imagine a travel web site that uses airline seat change APIs together with, say, SeatGuru, to automatically get your family the combination of seats you want to sit in.

But to do that, and all sorts of other amazing things, more "transactional APIs" need to be opened up beyond the B2B world they're trapped in now. Companies -- and old economy companies in particular -- have been hesitant to open up these services as transactional APIs to the general public. There are some financial and security concerns, but they are not insurmountable as PayPal's successful payment API has shown.

What we need now is for folks like United Airlines to offer the same things via API that it has already put onto its web site (check in, cancel, standby, upgrade, seat change, flight change, status). Each web check-in helps the airline, and saves it money; the service should be published as widely as possible.

OpenTable blazed an early path with web-service-standards-as-B2B-infrastructure. But, hey, there are some great apps in the heads of people outside their small group of strategic partners.

Web 2.0 has shown if nothing else that there are more good ideas out there for what can be done with a data set than there are in here. And that the value of the data often increases the more freely it can be used by other applications in unexpected ways. I'm suggesting that the notion of a web-service-accessible data set be expanded to include the real-time seat map of an aircraft, the reservation book at a restaurant, the transaction history and current status of a Visa account, a doctor's appointment schedule ...

All the hard work for this stuff has already been done. When this switch finally gets flipped on, you're gonna see some real fireworks.

More API ... Build Your Own Interaction

An updated API is now available that starts letting anyone bring the pieces together to create mobile interactions. As before, the web service API is on the test server at https://peridot.goskip.com/skip.asmx and you can get a dev key by emailing me. The secure connection which was originally optional, is now required.

The API adds three new operations. Two of them ("MemberExists" and "CreateAccount") can be used to programmatically create new Skip accounts. Why would you want to do this? Suppose you had, say, a site oriented around a conference. You could give users an option to add conference events to their Skip account... and if they don't have a Skip account, but you have their email addresses, you could automatically generate a Skip account for them and insert the new activities.

But here's where it gets really fun: suppose you want to create interactions for these users, meaning real clickable menu items on their mobile device, associated with the conference activities you've just created for them. We've implemented a system that lets you do this in the easiest possible way. Using the "PublishAction" API, you can specify a display name (what shows up on the mobile device menu) and a URL that Skip will call back when the person clicks the item.

Technically, Skip will issue a HTTP GET to the URL you specify, will append ?user=xxx or &user=xxx as appropriate, where xxx is the user's email address, and whatever text you return from the GET (probably best to keep it short!) will become the "result" document from the user's Skip action invocation. When you call PublishAction, the API returns a bind key (a GUID) that you insert into the Publisher Data field of any item you create and that you want to feature your new menu item. The syntax is [bind:xxx] where xxx is the bind key. You can include more than one action on an item as well.

So here's a quick example. You're organizing this conference, and you want to offer your users event data through Skip. In addition, for conference keynote sessions you want to offer a quick "Thumbs Up" or "Thumbs Down" rating menu item. First, create the Thumbs Up and Thumbs Down actions, with as much specificity as you like (e.g., different URLs per keynote). Let's say you publish Thumbs Up, asking for a callback at http://www.tempuri.org/rating.ashx?keynote=monday

You then create the Monday keynote event for your users, and pass the bind key expression in the publisher data. When the user views the keynote event, she will now be offered a clickable menu item "Thumbs Up" -- and if she clicks it, Skip will make a call to http://www.tempuri.org/rating.ashx?keynote=monday&user=jane@janedoe.org in real time.

Where you go from here is up to you, but the possibilities are huge -- these callbacks are versatile enough that with a little creativity you can use them for restaurant reservations, ordering food, meetings, conferences ... they are pretty much a generic mobile front end for any one-click simple transaction that you might currently make available via the web.

Two final notes...

First, these APIs will continue to expand, allowing you to update items you've published in real time, and allowing you to create more complex interactive menu items with "wizard" type workflows and graphical elements. We already use these techniques at Skip for travel industry transactions like airline check in, and we want to make them available for general use.

Second, you might have read this far and are thinking "This is news? I publish a label and a URL, a user clicks something and your system issues a HTTP GET to the URL and returns a document ... isn't this called the web, and the technique is like 15 years old?" If you're asking this questions, then the answer is "Yes, exactly."

Windows and Linux Avoided a Big Java Mistake

As we wind down toward releasing our first dedicated Windows Mobile Skip client, I reflected on how easy it was to port the core logic from a Windows desktop test implementation. I compared this to a time when I wanted to port a modest J2SE library (which did not require any "fancy" APIs) to J2ME. And to another time when I needed a similar port for a Linux library.

It seems silly if not dumb to have to say this, but the fact that the Windows APIs for mobile are so similar to the full-on Win32 APIs (including the .net compact framework) makes it a no-brainer to convert many apps to run on mobile. Ditto with Linux, which of course represents an even purer case since "mobile Linux" is generally just another distro on another hardware platform.

So with Windows libraries, we can build, make a couple of changes, and off we go. Linux rules for porting, for obvious reasons, and ./configure; make; make install gets us "mobile."

Then there's J2SE to J2ME. Ouch. What were these guys thinking? Yes, things like this. And this.

But the bet was against Moore's law, and I believe it was not a smart bet for the platform. If Java has a future on devices, it'll be J2SE aka "Java SE Embedded"

Try Skip's First Public API

Skip's first public API is live on our test server. Part of our "keep it real" philosophy involves allowing public access to our test server almost all of the time, where interested parties can see and play with things we haven't quite finished.

The web service is located at https://peridot.goskip.com/skip.asmx, and opening the link in a browser provides the .net-generated descriptive and sample data for each operation, as well as a link to the WSDL.

Email me to get a free developer key, which is a string you pass in the devkey field.

The member field is the email address (Skip login) of the Skip member in whose itinerary you wish to create an item. publisherdata is the initial descriptive text corresponding to an item (which appears in the "annotation" box on the web). In the final release, this data will have its own independent field, which you will be able to update via the item handle that is returned from the Create operations.

Ignore the groupkey field for now, as it is not implemented yet -- but it will be used in a final release to allow you to associate multiple items so that the end user can move or manage them together as a group.

For flight segment items, the record locator and pax names are optional, but must be present if you want to enable the end user to do a one-click check-in via his mobile device.

These operations only affect accounts on the test server (peridot.goskip.com), not the production www.goskip.com site. The test server also hosts versions of the mobile client which run against test, if you would like to play with the full end-to-end travel publishing and travel consuming experience.

What Exactly are you Testing Anyway?

Static/dynamic language debates continue ... spurred on by the continued success of LAMP and RoR. These discussions often evolve into an admission that the real issue is "What is being tested?, What is doing the testing?, and When?". For example, Bruce Eckel (a C++ and Java guru among other things) wrote a canonical post in 2003 asserting that dynamic languages seemed to him to offer greater programmer productivity, and although certain things were not being checked before runtime, they were in any case being checked. Cf. the whole RuntimeException and strongly typed exception debate in the Java world.

The corollary to this conclusion of course is that you don't want your users finding your bugs at runtime, so something of a test-driven or at least strongly test-supported approach is necessary. Testing and TDD are great. Just as great as compile-time checking. The problem is that the two are not testing the same things, or with the same coverage.

The specifics: static compile time checking typically covers 99% of the code for most applications (it normally excludes casts, dynamically generated code, and reflection among other things). From a user/feature point of view, it covers, well, 0%. So it covers a whole lot of things that are helpful, but not immediately relevant to the feature set.

Now look at a solid TDD-inspired test suite. Assume it adheres to best practices meaning coverage of all (functional) features (the core of TDD), multiple failure cases, non-trivial tests, etc. So we have coverage of 100% of features (again I'm excluding non-functional requirements like security, performance, design for the time being). And it will, through sheer exercise of code, verify a significant percentage of the formal code correctness, including type compatibility.

At this point, the problem and the solution are obvious (if hard to implement). PROBLEM: The test engine, including test suites, UI test runners like Selenium, components that generate random input data or events, load testing facilities, security analysis probes, etc. are still limited to producing an a posteriori analysis of a small set of cases. But these are only the positive cases (where positive includes success and defined failure modes) plus a tiny fraction of the negative (unspecified failure) cases. Type checking in strongly-typed languages provides a comforting a priori data set over nearly all cases.

SOLUTION: We need to create better analysis tools for all languages. We need a big step beyond the formal syntax checking, type checking, and lint analysis of code. Some more sophisticated approaches are going to be necessary. At the minimum, it would likely involve recursive runtime use (GUI design tools have "cheated" like this for years), dictionaries of data types, ranges, precision, etc., and a probability engine that can handle breaking up workflows into blocks in order to keep permutations within the compute capability of the hardware, and then recombining them along with a risk analysis of what's thereby been left out.

A lot of work has already gone into finding the boundaries of this problem, especially in academia and where languages (e.g. Scheme) with heavy academic use are concerned. Traditional static analysis and more importantly some human user hints will need to set up the constraints, so that the testing does not devolve into "testing any string that can be passed to eval"

Mini Manifesto

I wrote the points below to a colleague this morning, and I realized that they do a pretty good job of articulating the Skip "stake in the ground" on mobile software in general. Perhaps this a is "duh, no kidding" post. But I hope it's useful to someone who may be thinking about either introducing a new mobile product, or mobilizing an existing one:

Two-Screen Model

We’ve patterned our system after a two-interface system, one on the web, and one on the device, where both interfaces work against the same account, but are tailored to the web or mobile context respectively. We looked at how Microsoft Outlook and the Blackberry suite complement each other in offering access to the Microsoft Exchange account and we also looked at the Apple iTunes desktop client along with the iPod player software. Our belief is that those models work really well for users – a powerful management interface on the web or on the desktop, and then a custom-designed mobile client that picks up those settings and accesses the same data while on the go.

Smart Client Architecture

Although every phone (more or less) has a WAP/[X]HTML-lite browser, we believe that compelling user experiences will require at least a minimal client on the device. We like the smart client architecture as articulated by Microsoft: namely, an app that has access to local resources and privileges (local storage, control over the screen, security access for dialing, etc.) while gaining much of its leverage from remote network-based services. The app is offline capable, while the remote services make the client light and flexible across devices/platforms.

Client Device Development has Turned a Corner

In the past, it was not practicable for most companies to target a wide range of devices successfully. The devices were too disparate, and even the standards (famously J2ME) were absolutely not standard across devices. This environment is changing as we speak, and good progress has been made, so it is no longer prohibitive to target many devices. Skip is proof of that, with over 100 devices supported, on several platforms, and we’re a 5-man shop with other work to do too! Some specifics: Sprint has standardized all of their MIDP 2.0 phones on a single specific Java implementation, guaranteeing wide compatibility on Sprint with minimal effort (our tests have shown this is true in the real world, not just a Sprint PR world). Verizon and Adobe announced an agreement earlier this year to support a version of Flash on many of the new Verizon phones in North America. Mobile Flash has a track record in Asia/Pacific, so it’s not a “here’s my untested new world-conquering technology” play, and availability in the US should make it relatively easy to bring even sophisticated interfaces onto a whole bunch of phones using designer-friendly (rather than just coder-friendly) authoring tools.

Consumers

Consumers need a little marketing/education, but are hungry for mobile productivity tools. Especially the younger (16-35) demographic. These folks either assume their phone is capable of doing a lot, or they are easily persuaded. There is a real desire for anything that’s both useful (i.e. does something practical) and easy to use.

IT Procurement 2.0

I posted a little while back about Zenni Optical, which sells eyeglasses starting at about $16/pair complete. The other day I was looking at a pile of generic ink cartridges I've gotten from Digi4Me.com, at a cost of about $1-2 each (the name brand versions can run anywhere from $10-$30 each in stores). We test a lot of phones at Skip, and sometimes we need extra chargers. I have a bunch of USB-type AC chargers (e.g. for Moto RAZR or Blackberry) -- $6 each at BargainCell.com. Price for "official" equipment at other sites? $25-$30 each. And miscellaneous hardware that we use for desktops, laptops, etc. in the office has come from a whole host of great vendors that all have two things in common:

1. The vendors offered the best or near best (within 20%) price for a generic product on pricewatch, or they were matching that price range
   
2. If they advertised on pricewatch, then they had 4+ stars from their reviews

Once upon a time, Dell disrupted all of the PC builders. They were so successful at addressing every market segment (home, small biz, enterprise) that even startups would get their equipment through a big ol' Dell account. And it was a reasonable choice.

That era is over, for startups anyway. With the half-life of technology running as short as ever, there is no justification to pay extra for a supportable or supported configuration. (Mind you, production servers are a bit of a different story, but like many startups, Skip only has a few of those and they don't change much.) If a laptop breaks in 2 years, the user would probably need a better performing one anyway, and Dell doesn't look like they're too eager to support their own gear regardless.

And memory modules, monitors, hard drives, everything else? Bring on pricewatch! I haven't been burned yet, and more importantly I do not believe that any "established vendor" is going to do better by my small startup at any affordable price. I've seen enterprise vendors' support when I worked at a major bank; we had a virtually unlimited support budget for one vendor and yet no amount of green could make them show up, get things working, and keep them working.

So I've realized that IT Procurement 2.0 is about TigerDirect and Fry's and a myriad other vendors that are selling white-label gadgets or last-quarter's products for pennies, and are getting my dollars. Just as success put the lie to a priori dismissals of the LAMP stack, Ruby, and the cluetrain, a thousand startups today are controlling their burn by buying "great cheap stuff that works" from great vendors that want to sell it to them.

Moto + Linux = Real Progress in Phone as Platform

Motorola has followed up its aggressive stance on Java development at JavaOne (and I mean this in a good way, as any vendor's aggressive position is helpful in the wishy-washy world of mobile development platforms). At LinuxWorld today, Motorola exhibited the community and tool projects that go along with its plans to develop all mid-range and higher phones on a Linux OS. Exhibits included work on an Eclipse-based SDK for native app development. They plan to take on Symbian and Windows Mobile head-to-head in this arena of supporting native apps.

Greg Besio's keynote emphasized that Motorola is opening their platform (including publishing code for the core phone apps, in compliance with GPL requirements) and hoping that in return they will benefit from innovation in third-party apps that will flourish on their devices. Sounds like a plan. I asked a number of Moto folks about the balance they will be striking between carrier desires for control, and developer desires to, well, be able to deploy software on the phones. After all, innovative and compelling user experiences only go so far if you need a sync cable and bunch of cracked toolkits to get an app on your phone.

It sounds like a clear position statement has not yet been formulated. One individual said that infrastructure is being prepared in the SDK for various levels of code-signing to control deployment to devices. This could be a "cover all bases" move though, rather than a clear indication that code would need to be signed. Others emphasized that Motorola is committing resources toward helping developers move along the deployment path.

For my part, I emphasized two points to everyone with whom I spoke:

1. Any unlocked GSM phone that a user owns should be wide open. If a SIM card / network affiliation wants to restrict some net or phone traffic, fine. But the core device and the decision about what runs on it needs to be the owner's.
   
   
2. Even very restrictive carriers (Verizon, Nextel) have historically allowed owners of high-end devices carte blanche with app installation, provided the devices were designated and marketed as smartphones -- e.g. Q on Verizon, Blackberry on Nextel. Plenty of apps generate fatal exceptions on both of these devices, and the earth hasn't stopped rotating, so app robustness isn't the issue. And since smartphone users pay big bucks for their data plans, loss of revenue isn't either. Therefore, it would seem reasonable for Moto to position at least the high-end Linux phones as smartphones, and make clear that they are being promoted as devices which let users move data on and off, install apps, etc.

In any case, this is a great development for too many reasons to blog here. (Ok, so just one: J2SE anyone? ... yeah, it's about time ...)

Work at Skip!

Skip is hiring, and Mr. Malik said it best, just last week:

"the job boards don’t seem to have the necessary impact or perhaps get the right kind of users"

So first ... check out our ad on CrunchBoard.

Then, tell me if you don't think these are some of the issues that render the "big traditional" job boards less-than-useful for Skip and other startups:

1. They have a poor job taxonomy - many of these sites serve every industry from sanitation to aviation, and yet instead of having a category like "technology," they split tech up into all sorts of categories and subcategories. A startup-type employer has a hard time deciding where to post, and a candidate has a hard time narrowing the search.
   
   
2. Likely, #1 is a symptom of a heavy bias toward Fortune-500 org chart / HR type hiring. All of the tech categories and subcategories look like they came from some post-re-org bad dream at a very large company. The bigco bias doesn't just make it hard for the long tail (like Skip) to get involved, it drives away the audience we want (folks who love startups).
   
   
3. They foster job descriptions (and concommitantly attract resumes) built out of tech acronym lists and bullet lists of "years of" (you know what I mean). If you have "8 years of C#" you'd better have worked for Microsoft. Of course, resumes like this are a desperate response to foolish HR-composed job ads asking for 8 years of J2EE
   
   
4. For us, attitude counts. A whole lot more than the difference between "3 years of C#" and "4 years of C#" -- so these job boards that scream: "the candidates are bored, the hiring companies are boring, and we're both" are not going to draw the right folks.
   
   
5. Lastly, these boards are packed with jobs posted by recruiting agencies (staffing companies). Many of these jobs are widely believed to be bogus. That is, the ads exist to draw resumes, from which the recruiters try to find clients. Now, even if this isn't always true, the appearance of this situation drives away the serious candidates. Dice.com is a great example of a tech job board that used to be (in 1999 or so) rock solid. Then the staffing companies moved in with "jobs" that I've yet to hear of anyone actually getting/taking, and now ... well, let me just say I'm not spending money to advertise there.
   

Long live CrunchBoard and the rest of the niche sites!

First Impressions, Second Impressions

Ok, I'm not aiming specifically at the quality or related issues of the Moto Q or Treo 700w here. But this Amazon review of the Q is just brilliant: http://www.amazon.com/gp/cdp/member-reviews/A10U3Z35FCPXF2/

Synopsis (if you haven't read it yet or are too lazy to click through):

1. Smartphone-savvy customer is delighted to begin working with his brand new Motorola Q.
2. Customer discovers some minor issues and remains enthusiastic and forgiving.
3. Customer has multiple hardware failures. Service rep suggests an alternate device with a better quality track record (in the opinion/experience of the rep).
4. Customer buys this alternate device and regrets giving the 5-star review to the Q in the first place.

Footnote: This all takes place in a week, and remember this reviewer seems to be a sophisticated PDA/phone user.

Question: When (Apple) did firms pushing the "high-end" wannabe-Lexus products (Apple) start acting like used car salesmen (Apple) hyping something shiny (Apple) that barely drives off the lot (Apple) with the engine still running?

Someone set me straight here ... meanwhile, I guess this is why I drive a Honda and build my own whitebox PCs...

Relational Databases: What are They Good For?

I'll answer the question: they're typically good for a whole lot, provided a whole lot looks something like:

• Extreme concurrency support (if handled right)
• Solid transaction support, with well-known deterministic tradeoffs agains concurrency
• Deterministic (in both time and output), standardized (mostly) and accessible query capability
• Proven logging, failover, and recovery mechanisms
• Standardized management and operations tools
• Exposure of data to standard, existing reporting and OLAP tools
• Declarative management
• Declarative data cleanliness constraints
• And a few other things
  

Now, nowhere in that list does it say "great data structure for every problem domain" -- or even "great data structure for many problem domains." In fact, the relational model is inherently a good structure for a few domains, but by no means a large number.

And yet somewhere along the way, in the last 20 years or so, it's become a not-entirely-embarrassing-and-career-limiting-move to design a software system by initially modelling entities in a relational model, then slapping a wrapper above the SQL (maybe even auto-generated), tossing a UI on there, and declaring "Mission Accomplished." Wait! Where's the "business layer"? Well, that's where things start to go off track. Actually it's usually above the DB wrapper piece and below the UI -- but more importantly it's usually almost empty in the beginning, since the designer has an entity model that is tightly coupled to the current flavor of business rules. The business layer becomes a real monster ... er I mean tier ... later, when each case or change that isn't supported by the data schema earns a hack-around in the business layer. And at the end of the day all of these systems end up looking and acting the same way: like the 3-tier DB apps they are.

What's wrong with that? The fact that the apps end up expressing this architecture more strongly than they express solutions to their problem domain. These apps feel like fields and records -- they don't feel like what they actually are (e.g. a cell phone provisioning app, a flight management app, etc.) So they're mediocre and they're disliked by their users, disliked by their maintainers... they're "good enough" ... "until we have a chance to do it right."

For most apps, a database is a particular way to persist data, with some handy aspects, that's all. Don't design like it's anything else! Design like you're building a video game, like the experience is everything and only the user interaction needs to be right (and 100% right). The reality of business apps may be dirtier in the details (not to mention you'll rarely have the resources to make your UX dreams come true), but that's the point: if you're smart you can keep it in the details. Which means designing a whole application not just a database.

Remember "Disintermediation"

That was a hot word for the way some online businesses were reshaping value chains in the late 90s by eliminating various links. One might think that arbitrage and free markets being what they are, all of the legitimate disintermediation opportunities were exploited years ago.

But here's a new one -- well, it's not new anymore, but it's a lot more recent. Anyone who has ever bought a pair of eyeglasses retail can't help but wonder what kind of scam is being perpetrated. In an era of $19 MP3 players and $300 PCs, we have $250 for an eyeglass frame that just screams "Made in China, $20 per dozen" Not to mention that in many developing nations, people do buy eyeglasses, and the $250 pricepoint would be unimaginable. Then you pay $40+ for lenses (which are pre "ground" and come out of an envelope), and extra for all sorts of protective magical coatings.

About a year ago I googled for discount eyeglasses, thinking that surely out there on the web someone is importing these $1 frames in bulk and would sell them to me for a modest 1000% markup of $10. No luck. Lots of bogus sites, and the legitimate ones were just selling the same "designer" frames at the same price ($1 manufacturing + $5 licensing + $244 American sucker tax = $250)

Fast forward to today: Google for eyeglasses, and you find a dozen sites selling complete glasses (i.e. frames + prescription lenses) online for anywhere from $15 to $50. I tried a pair from a San Rafael company called Zenni Optical (a.k.a. 19dollareyeglasses.com) because at that price, really what was there to lose? I had the glasses in a week, they look great, the optics are fabulous, done. Price including shipping: $28. These guys rock out!

The only mystery left is: how can a business make any money selling a pair of glasses at, say, $25. Unlike the commodity MP3 players, these glasses do need a little bit of handling and custom work (cutting the lenses, inserting into frames, packing, etc.) The fast shipping and one-off ordering suggests that the custom work is being done in the U.S., not overseas. Let's say it takes 20 minutes to process a pair, and there is $20 of margin in the $25 glasses. That's $60/hour. Which isn't bad if you've got a couple of employees, low rent, and your production pipeline is full. But if the pipeline has some slack, or you have a manager in there somewhere or any kind of real estate ... that's brutal! Even Wal-Mart's absolute cheapest non-branded glasses were about $65 (frames + lenses) last time I checked...

ByRef moreDetails As String

Quiet down, you'll get the joke when you follow the links.

My last post talks about what's not there as far as developing for the Palm. Jamie Flournoy, who has done a ton of rocking contract work for Skip on our mobile clients, has written a post discussing the solution the we did choose for building a new Palm/Treo client.

Even better, he's going to be contributing to the community a test framework he built, and some hard numbers that can be used to look at relative productivity for different tools. Thanks, Jamie!

Waiting for Godot ... er, Cobalt

At Skip, we're creating a native (well, 68K, so native vis-a-vis Java) client for the Palm. In particular, we're targeting the many Palm Treo smartphone users, who deserve a more robust Skip experience than Java has been able to deliver.

In the process of building this client, we've been stunned by the huge gaping hole that is the Palm high-level app development platform.

Since ours is a high-level business/productivity type app, since the client is not complex, and since we support many platforms, some sort of RAD tool or at least high-level API is ideal for client development.

The amazing thing is that the space is a gaping void, where only a handful of tiny ISVs with proprietary tools play.

IBM hasn't updated its Java implementation for Palm in several years ... it's not great, not free, and has no roadmap into the future. At the other end of the spectrum, PalmOS has a broad C API, but this is no high-productivity environment (it's more like C for Win 3.1). Not to mention that using many third-party C libraries requires assembly hacking and linker foo, not ./configure and make.

Adobe could push the Build button and release a Flash player for the Treo that would instantly become the high-level environment of choice for 21st-century connected apps. And the great UI possibilities would be a bonus!

The lack of a top-notch development system is even more mysterious given the large community of Treo users and thriving (by PDA/phone standards) ISV app ecosystem. In other words, the Palm Treo area is one of the most successful and least risky places to invest in infrastructure, with the highest potential return.

Presumably this whole situation stems from the prolonged wait for imaginary Cobalt devices. That's going to make things all better. Real soon now.

Disappointed but not Surprised

Over the weekend I noticed that my DirecTV TiVo DVR wouldn't let me pause or "replay" XM radio stations. Clicking pause or flashback brought the "error beep" and an on-screen message that this functionality was unavailable for audio-only programming.

I am not sure if that is a new change subsequent to their switch to XM for music, or if it has always been part of the TiVo software on their boxes, but I couldn't help laughing at how ridiculous it was. Most likely they are preventing "digital recording" of the audio channels as some sort of protection against imagined piracy.

The reasons I laughed are these -- top reasons why a DirecTV TiVo DVR is not a piracy threat:

1. Although the box has a S/PDIF digital audio output, the audio is compressed in transmission to the satellite receiver, so it's not as if some magical studio-pristine copy of the audio is there on the box.
2. DirecTV TiVo units (unlike Series 2 TiVos) do not allow an ethernet adapter to be added on, or saved files to be removed from the unit. So pirating anything off of the box would have to be done by playing the file back in real time!
   
3. Every DirecTV satellite receiver (at least every legal one) has a smartcard with a unique ID and account information. Since activity on the DirecTV box doesn't even pretend to be anonymous or unobserved, it's not the most appealing channel for doing anything illegal or inappropriate.
4. Since DirecTV owns the box, they could always restrict the unit if it were being used excessively for audio recording and replay in a way that looked suspicious (although, as already explained, it's hardly a practical or appealing way to do anything like that)
5. In the last case, they could always watermark the media on the unit and allow customers to do whatever they want, figuring that (a) most customers wouldn't do anything remotely inappropriate if they know the media is traceable to their account (and bill!) and (b) anyone sophisticated enough to remove watermarking would be getting their media somewhere else in the first place.
6. Unlike the TV and movie channels, the audio channels do not publish a time-based "guide" listing each program or song that will be played. So it would not be possible to plan a recording of a particular song. Basically, the would-be pirate is like a 1980s teenager making a mix tape off an FM stereo. This guy (or gal) would have to either sit there waiting for the song to come on, and Quick! Press record! or else they'd have to record hours of content and then play it back in real time and chop it up on a computer.

I love this whole scenario because it shows just how silly the whole thing is. My DVR has specific features and UI in it that basically amount to guaranteeing that when my 21-month-old hears banjo music he likes, I can't press pause or rewind. Welcome to the 21st century.

A Change in Context Does not Require Rebuilding the World

I love YubNub.

Besides just being very cool in a geeky way, it's got a lot in common with Skip. The interface that YubNub exposes to the user (a "command line for the Internet") and the Skip interface (a "click interface for getting things done with a phone") are diametrically opposed. YubNub is about typing out commands, while the Skip interface avoids typing at all costs. So it's not that part.

What Skip and YubNub have in common is the idea of adapting some of the great applications out there for a new context -- without rebuilding massive parts of those applications from the ground up.

YubNub rocks when your context involves keeping both hands on the keyboard. Maybe you're working in various console windows, or you're a writer or a reporter. I'd love to have this under a hotkey in Microsoft Word (maybe someone's already written the plugin). I type in a command like "gim porsche 911" and *bam* pictures of Porsche 911s. YubNub is extensible and has the notion of pipes and filters. So maybe I want to insert a picture of a 911 into my Word doc -- I could execute the search in such a way as to return just the first image, copy, paste, done. In the keyboard/console context, this is a powerful way to access anything that lives on the other end of a URL.

Skip does something similar, for the no-keyboard, no-typing context you're in while sitting at a stoplight or running from airport parking to catch a plane. We don't want to rebuild all of the airline checkin applications, limo reservation systems, or fast food point-of-sale systems in the world just to make them "mobile."

We'd rather build a context-specialized application that knows just enough about what you're trying to do to make it a whole lot easier, and lets companies get the transaction systems they've already built and operationalized literally into their customers' hands.

It's Not a Tumor ... er Toaster

HCI specialists have metaphors they like to use to describe how products can be made more usable. One of these metaphors is the toaster. They may explain how a complex copy machine, or a PC printer, should behave more like a toaster. You should be able to walk up to one you've never seen before, be able to tell what goes in what hole, press a lever, and reasonable default behavior should pop out a minute later.

Now this is fabulous Crossing the Chasm advice. But the early adopters don't buy toasters -- they buy electrical thermal-transfer bread processing systems. They are interested in the options in the print dialog box -- or to use a literal example, my friend's ancient toaster (an early adopter toaster, as it were) had a tuning screw on the bottom which could be adjusted to calibrate the toaster such that the color of the toasted bread exactly matched the color on the enamel toast darkness selector.

So what's my point?

I recently got to try a Roomba home vacuuming robot. The robot, an interesting piece of technology from a company that clearly makes some hard-core gear, came with about a 4-page instruction booklet. The instructions try to follow a toaster model: simple, clean, "nothing complicated here" -- push "Clean" and go. But it's not a toaster, it's a domestic robot! Like in the Jetsons! This isn't like buying a lamp (and it's not cheap either), so I want to know exactly what the heck it does. It's probably best if I don't have to read a 50-page manual to use it, but I would certainly appreciate one that answers some basic questions.

I'm the guy who's going to be recommending (or not) this device to the people on the other side of the chasm. So when it fails to manage its battery charge and fails to return to its charging station like it's advertised to, I'm the guy who will be very sympathetic if only I can get some understanding of what it's trying to do and why it fails.

A device with great docs is a Samsung VCR-style DVD recorder which I bought a year or two ago, when those devices were a little more raw. It's a solid piece of equipment, and easy to use without reading the manual. But it came with 50+ pages of docs so that I could understand exactly what it could and would do, and where some less familiar recording formats (like DVD-VR) came into play. Because I understood what it could and couldn't do, I felt comfortable recommending it to less technically inclined people. Those people in turn don't want the 50 pages -- they want the toaster, and good for them!

As for iRobot, they score massive points for putting a serial port on the outside of the device and encouraging people to reprogram it if they want! I love seeing a company that isn't afraid to say, "Hey! You bought it, it's yours, go ahead and hack! We're mature enough not to worry about managing perceptions or fighting imaginary lawsuits if you make it do something stupid." That alone is enough to make me forgive if it mysteriously misses a swath of carpet.