Amazon EC2 to Support Windows Server AMIs

Jeff Barr, Amazon Web Services Senior Evangelist, just finished giving a talk at "The AWS Start-up Event – San Francisco" in which he showed slides listing upcoming plans at AWS, including support for Windows Server.

This is an exciting development, as Windows Server / ASP.Net make for a fantastic if potentially expensive platform. Now Microsoft has to step up to the plate and come up with a pay-as-you-go, per-cycle or per-cpu-hour licensing scheme.

One of the things that makes ASP.Net interesting is that it lives in a nice middle ground between Java, which is extremely fast (in EC2, this means less expensive per transaction) and has great "enterprise" capabilities but is cumbersome to develop with, and, say, Rails, which is quite slow and has poor enterprise app cred but is very pleasant and lightweight to develop with. ASP.Net, especially with the MVC framework and forthcoming support for Python and Ruby in addition to C# and the other .Net languages, seems to combine extreme performance, easy development, and access to as much "enterprise" as you need while offering lightweight alternatives like LINQ and SSDS.

My point isn't to make a commercial for ASP.Net, but to point out that if Microsoft can get their licensing in order, they might catch up in the cloud world through fast, cheap development cycles plus faster (and hence cheaper) runtime operation on a given machine instance than some competing platforms.

Just to be fair, cloud vendor enomalism has run Windows Server on EC2 before, by virtue of the Qemu emulation software (on top of Linux). But if we're talking about maximizing efficiency, wasting cycles on another layer of emulation (EC2 instances are of course virtual to begin with) doesn't sound like the way to go.

Om:Yes, Gillmor:No ... Comcast's Data Transfer Cap

Om Malik nailed it at the end of one his posts on the 250GB cap. He lists Comcast's stilted usage examples for email, music, etc., and asks "How many HD movies will fit under the cap?"

That's the right question, but not because he is worried he'll bump the cap watching video.

The reason it's the right question is that the cap has more to do with Comcast's long-term business plans -- and the cable/telco industry's plans -- than about slapping down a handful of network hogs.

One fact before we continue. As I've written before, while everyone squabbled over BlueRay or HDDVD, the filesharing world settled on the x264 implementation and a container format for it. In that compressed format, a high-def film averages 10-15 GB.

But here's the real deal:

Comcast's cable TV and on-demand video business competes directly with Internet video (both legal and gray-market). By making the cap high enough that it won't immediately impact YouTube watchers but -- over time -- will put a serious dent in customers' ability to get HD video over the Internet, Comcast is simply crippling one product so that customers will have to buy another product from them. Don't even think about mentioning broadband competition -- if you define competition as more than one entity offering substitutable goods (i.e. similar bandwidth, QoS, etc), then most of the U.S. has zero competition.

The 250GB cap is a way to push customers to buy the same content twice. To be specific, you'll pay to see a movie broadcast on cable TV in HD, and then pay again for some special plan that lets you watch the same content later on Hulu. Or you'll pay for Comcast "on-demand" to watch a show on your TV. But if you want to place-shift that show to your laptop or the gym (on your phone), you'll have to pay to get enough headroom to download the same video.

This move is not just about trying to squeeze out more money. It's yet another attempt by network operators to neglect their core business and instead try to get into a content business, so they can do two things badly instead of just one.

Meanwhile, Steve Gillmor writes that bandwidth caps will mean the end of BitTorrent downloading and a shift to streaming. Steve's a smart guy, but streaming is one of those perennial non-trends like speech recognition and thin-client computing. It's only conceivable to someone who has a computer connected to their TV or who lives on their laptop.

Having played a key role in several home-entertainment / media-tech startups, two of which have been successfully acquired, I feel confident saying streaming is only a little piece of a solution. For the foreseeable future, streaming will never be more than that. It neglects too many consumer needs and desires, is too inflexible, and is too warped by DRM and platform-lock-in issues.

Moreover, the latest round of DRM around HDMI and high-def media playback in general means that even in-home "streaming" -- i.e., from a PC to a TV or from one PC to another -- is facing setbacks just as Joe tech-savvy consumer finally starts to realize how to use a home server or network attached hard drive to get and watch shows.

And the set-top boxes, increasingly tied in tightly with content delivery (Comcast DVR, DirecTV TiVo expiring your pay-per-view movies, etc.) are more closed than ever. Instead of encouraging an ecosystem of streaming, they force customers into an ecosystem of downloading.

I think we'll see more people gaming the 250GB cap ... using scheduling software so they can maximize the utility of the 250GB and hoard downloads. Not to mention leaving the torrent client running so that every time I visit someone else's network -- an office, a coffeeshop, etc. -- I'll keep the downloads slowly but surely coming in.

One final note: the marginal cost of moving a kb of data around a big network -- that is, the cost of the power and cooling for the switches and the amortized cost of the cables and gear and ops guys divided over their lifetime data transfer -- is more or less zero when the network isn't already near peak bandwidth somewhere. So even if there is a problem, the volume-based cap is the answer to a different problem.

Extending a Cell Signal With zBoost

Although I'm curious about the AIRAVE, I'm not on Sprint and not convinced it's worth switching just for the AIRAVE capability.

But I do have a signal problem at my house, and I'm currently testing out the zBoost Wi-Ex signal booster to see how much of a dent it can make.

Unlike AIRAVE, which creates a local CDMA network and bridges it to another backhaul, the Wi-Ex is an RF repeater/amplifier. So you need to feed it a signal -- from somewhere nearby that has a signal -- and it extends the signal via a new base station and antenna.

So far I'm inconclusive on the quantitative results. Certainly, in my house, the coverage area it creates is closer to 600 square feet than the 2500 square feet advertised. But as with any RF setup, there are so many variables that this doesn't prove a heck of a lot.

I can report positive results on a qualitative measure.

Normally, the signal is so weak inside my house that, without this unit, I can receive a call signal (e.g., ring or text message) but as soon as I press "talk" it drops. 100% of the time. Also, my Blackjack had never successfully retrieved a web page from inside the house. With the zBoost unit, there is  qualitative difference: The data connection can retrieve web pages -- sometimes slower, sometimes faster, but pretty much successfully. And if I receive a call, I can answer without the call dropping instantly. Again, depending on various things the call quality may be better or worse -- but there is always a call there, struggling to get through. Which despite being reminiscent of 1995 is a major improvement.

What I'm wondering is ... if, as zBoost claims, the device is FCC legit and is designed not to create trouble for the existing cell network, why don't the cell carriers themselves sell this device? It seems like it could help them extend their reach geographically and satisfy a few more customers.

And before you suggest that they don't offer it because doing so would be tantamount to admitting their existing network footprint isn't solid, note that both T-Mobile and Sprint are already selling personal femtocells and rumors are that AT&T isn't far behind. And anyway, spotty coverage inside buildings has never been any kind of secret.

Airave and Network Sniffing for Fun and Profit

Sprint's AIRAVE BYOB (bring-your-own-backhaul) femtocell went on sale nationwide this week. Reviews seem mildly positive; it kind of costs a bunch considering you're subsidizing Sprint's infrastructure, but it's marketed toward people who have little other choice.

I don't have one (I'm not on Sprint at the moment). But if I did, the second thing I'd do with it is to start sniffing the traffic it sends back up the wire.

But all that data would be encrypted, right? Well, maybe ... and maybe not. Since Excel files seem to be an enterprise data store of choice these days for companies handling sensitive personal information, and since telcos are not the science-fair winners of the class but more like that kid in the back that just giggles all day keeps getting his lollipop stuck in his own hair, it wouldn't surprise me if they think ADPCM is encryption.

Even if some or all of the call content is encrypted, though, one could probably learn a lot about the signaling layer of the system -- i.e., how the network identifies and talks to the phones to indicate presence or session initiation.

Why would this be useful?

Well, for one thing, it would be interesting to use the AIRAVE as a presence detection mechanism. Once I can determine that the base station has recognized a known cell phone ID, I can automate all sorts of things that are supposed to happen when I'm nearby.

Applications include home automation (open locks, turn off the alarm); media (access and/or license to all of the media I own "appears" on any connected device when I'm nearby with my cell phone, and goes away when I leave); or for commerce: my hotel reservation, preferences at a restaurant, or online search history for a retailer can cue up automagically as I enter range.

Tripeedo Brings Travel Search Full-Circle

I couldn't help but crack up when I read a post about Tripeedo, an AJAX-enabled "command line for travel," since of course all this reservation searching stuff was on a command line to begin with, on a terminal a generation older than this hideous beast. And in fact while the hardware has mercifully been emulated away, the protocols are still in wide use under the hood in the travel industry.

I had to use a command line like this as recently as 2006 for a project. Unlike tripeedo's version, I recall the search started with a strange lozenge character that doesn't seem to have a proper Unicode name, but was part of the 6-bit EBCDIC charset these terminals used when they were deployed in 1969. Then came a letter or a delimiter of some sort, and then a line like 15AUGSFODFW to look at flights on August 15th from SFO to DFW.

I thought I'd try it in Tripeedo. The site prefers the month first, and it likes spaces. And, unlike the old system, it has lowercase letters too!

So I type "15 aug sfo dfw" and get some flights to look at. In 1969, this was the future.

iPhone Brings New Attention to AT&T's Existing 3G Problem

The wave has been building for a few days now, with consumers, Apple, and AT&T sparring over the problems with the iPhone 3G staying connected (and on 3G).

Today seems to be a local maximum and the mainstream media are starting to look.

If you read this blog regularly, you're probably expecting me to blame Apple for the problem, then blame Apple again for denying there's a problem, then blame Apple fanboys for putting up with it all for so long then suddenly acting betrayed.

Surprise!

As much as that might be a valid argument -- especially the part about Apple denying there's a problem -- this one is really not their fault.

It's kind of AT&T's fault -- and kind of GSM's fault -- and kind of just mediocre engineering.

A while back, I wrote about similar experiences with my Samsung Blackjack on AT&T. I've also seen the same thing with the AT&T 8525 (Cingular 8525 / HTC TyTn), as this was one of the first multi-protocol, multi-band 3G devices on that network.

Most people don't have any idea what this is all about, since their devices cannot experience this specific sort of problem because they don't support 3G, and so never have to resolve between 3G, EDGE, GPRS, etc.

In this case, AT&T argues that one suspect, the Infineon chipset, is not to blame because it's the same one in Samsung devices, with which they have had little problem. I don't think that's their strongest argument: others appear to be having as much trouble as I do with the Blackjack/3G. We just never made as much noise as the iPhone folks.

Perhaps a better argument would be that it affects all the EDGE/UMTS/HSDPA devices on the network and then maybe blame GSM for having such a funky data transport technology roadmap over the last 10 years.

I'm not saying AT&T could easily do better -- maybe they need more capacity and a tune-up of switching algorithms. A CNET article points to GSM carrier T-Mobile Netherlands having the same problem. And even though CDMA is worlds better as a technology, it's nice to have a little competition out there: two CDMA carriers for the U.S. out of four majors is enough.

So what does all this mean?

First, I'll tip my hat for once to the iPhonosphere and the obsessive media that love them: this is a real issue and we all know it takes a lot of volume to be heard talking to a telco.

Second, even Steve Jobs' strong-arming of the carriers worldwide (and good for him on that one!) doesn't mean everything is really under his control. According to this Sydney Morning Herald story, Apple didn't provide any test units to some carriers until the day before the product release, hoping it would all "just work."

To paraphrase someone I used to work with, in technology "hope is not a strategy."

Non-compete Ruling May Indirectly Help Broken Organizations

Last week's California court ruling against non-competes is interesting, not least because I've had one client who was more interested in a getting me to sign a specially crafted non-compete that he hoped to use in a lawsuit than in having me actually deliver work.

Actually I'm not sure this ruling would even affect independent contractors like me; the view of employment law towards contractors (where it applies at all) is very different from that toward employees.

This ruling may play a small role spurring innovation by making it easier for employees to quit and then compete with their former employer.

I'm also hopeful that the newly prominent threat of such competition may encourage dysfunctional tech organizations to renew their efforts at achieving better group dynamics. (Is there such a thing as therapy for whole organizations? No, I don't mean those motivational or consulting clown-schools that involve offsites and create a distraction ... at least until the check is cashed.)

In addition to the ominpresent threat that one's best employees might leave, there is now renewed emphasis on the fact that a group of them might well leave together and "do it right" if they can.

While the ruling does not of course give anyone license to loot protected IP from the former company, history shows that's rarely the issue in Silicon Valley anyway.

Every development manager should view this as a wake-up call to think about keeping the team on board.

Format Shifting

The dead-tree-publishing world is slowly starting to worry about online, possibly illegal, distribution of books. Not nearly enough to get smart about it though.

To me, it seems brain-dead obvious that if I buy a paper copy of a book for anywhere from $20 to $50, I ought to be able to use an electronic copy. I don't know the law on this, but I don't have pangs of conscience downloading a gray-market PDF of a book I've already bought, because the publisher wants to charge me again for an "e-book bundle" or because they make the content available in an annoying format.

But that's just the tip of the blade. In a short while, things like Kindle will take off, and the next generation -- motivated to leap in because of expensive textbooks -- will start assuming all books will be free.

Publishers should be experimenting with their models right now in attempt to adapt.

Here's one idea: the whole hardcover-softcover release cycle makes about as much sense as releasing a movie in the U.S. on Wednesday and not expecting it to be an xvid in homes in Thailand by Friday.

I like paper books, but I don't like hardcover books for anything non-classic. They're overly large, awkward to carry, just plain silly. So what happens when a publisher releases a new book I want to read in hardcover? I'm not inclined to buy it -- heck if I wanted to carry the hardcover around I could just get it out of the library.

I could download it as soon as someone puts it online, but I would want to print and bind it ... so I'd have to submit it as a print-on-demand job somewhere, hope they don't complain about the copyrighted material, and get it sent to me. I guess Kinko's is an option.

Anyone else see the problem/opportunity here?

For now, anyway, the hassle and cost of print-on-demand makes it a cost-neutral issue; I simply don't have the option getting a bound paper copy of the book for free. So given that I'm willing to -- and indeed must -- spend money to get the book, why won't the publisher sell it to me in the format I want?

Publishers ought to be taking the risks and making the friends now, before e-book readers make a big dent in the marker. Otherwise, it will be game over soon enough.

And before anyone suggests that the publishers play some critical role in blessing publications, I would point out that is just an artifact of the traditional physical distribution mechanism (paper, shelf space, etc.) ... the Internet already allows vastly more content than could be run through a press and tossed up at Borders. It's not all "good" ... but the net does a fine at creating search, moderation, and recommendation networks that allow one to find the good stuff, in a way that 3x5 "recommendation" cards tacked under a bookshelf at the local store cannot.

Rails and .Net Had One Thing in Common Before ASP.Net-MVC: Magic

It's possible now to build a very Rails-like website via ASP.Net MVC. And at RailsConf, John Lam showed IronRuby starting to run Rails.

But long before all this, these two very different platforms had one thing in common: "magic."

By magic, I mean the sort of implicit, we-solve-problems-you-didn't-even-know-you-had kind of magic that makes it easy for people who don't know the details, don't want to know the details, or know but don't want to be bothered by the details to build an app.

From ASP.Net's first iteration, where you could drop an item on a "web form," wire a server-side event handler, and talk to items on the page through a .Net server-side object interface, ASP.Net has been heavy on the magic. Rails has been the same way, from scaffolding up through ActiveRecord, you point Rails in the direction you want to go and it drives.

Of course we all know about the leaky abstractions... in .Net, you used to get some funky HTML and Javascript, plus a server-based page lifecycle that didn't always match the model of what you wanted to do with the DOM. In Rails, we've seen perf issues, and discussions of the "worst practices" [pdf] that can result from letting the autopilot do too much and your brain too little when writing for Rails.

But in any case -- as opposed to a framework like Django (which included a big feature push called 'removing the magic' to make functionality more explicit) -- Rails and .Net give you as much magic as they can.

[How] Will Microsoft Weather a Perfect Storm?

Last week's article What Is Microsoft So Afraid Of? is meant to be provocative ... but it is also real reporting.

Thinking about the situation Microsoft is in, I believe the firm has, at a high level, quite a lot to be worried about ... a sort of perfect storm of high-level business trends, many of which transcend any individual product or feature. Big trends are harder to debug and then patch on Tuesdays.

Here are the issues:

1. Declining importance of the desktop OS. As people spend more and more time in their browsers, the OS underneath it matters less. The trend is definitely toward more browser-based apps, even if they employ Flash or Silverlight or other acceleration technologies. Offline support is moving slowly but together with cloud (Mesh?) storage, will make the local filesystem less important.
2. Rise of OS X as a real competitor -- between publicity, 'time to sink in,' and the decreasing premium that Apple charges for a Mac over a similarly equipped Dell/Acer/Toshiba laptop, OS X is a real and growing threat to Windows.
3. Bill's off to save the world. For a while now, Bill has been the good cop (hey, stop laughing, I'm serious) to Steve Ballmer's bad cop. Now that Gates has retired to work on philanthropy, and Ballmer's the CEO, we're seeing more dumb-ass bad-cop stuff like the Mojave experiment, and less brilliant product strategy. Microsoft needs a good cop to pair off with Steve.
4. Vista failure. The .Net platform was one of the smartest, most successful things Microsoft has ever done, an enormous accomplishment that brought enterprise IT shops and other developers along. Following that up with Vista isn't just bad at the retail (license selling) level, it shakes enterprise confidence in MSFT, and ticks off developers. From a developer POV, most of the cool stuff in Vista was gutted, or moved somewhere else ... or never really worked very well (WPF), after years of build-up. Now MSFT desperately needs devs to buy into "the next thing," and they're hesitant.
5. Silverlight stuck in transition: Silverlight 2 is an awesome technology... but it has two problems: First, small penetration, unknown/unpublished penetration numbers, and no specific numeric commitment by MSFT to create penetration on PCs. Second, folks are sceptical that Silverlight on Mac really has a future. If that's in doubt, it seriously impacts the choice to use Silverlight because of #2 above. I can say that I've already had more than one client interested in doing serious work on Silverlight 2, but they insist on installed base projections (not download numbers) to make the business decision, and I don't blame them.
6. Rise of console gaming: PC gaming was always a key part of the "latest and greatest" PC/OS/component ecosystem, plus it helped press down Mac adoption. As consoles (including Microsoft's own Xbox 360) gain, this relative benefit of the Wintel client platform goes down... and yet MSFT still has to spend a ton on R&D if they intend to keep the PC gaming client (DirectX and hardware integration) decent. So less bang for the buck there.
7. Rise of USB. Now that practically everything runs over USB, it's easier for device makers to offer Linux, MacOS, etc. drivers (the upper layers can often be just user-space apps). So on the peripheral front there is less Windows differentiation in terms of hardware choices, and less lock-in.
8. Reluctance to innovate with server licensing model. Most startups dream of going big, and they assume they'll do so on an open-source stack -- not because Windows Server isn't a killer product ... but because the current licensing for Server makes it a non-starter for cloud services or for architectures designed around cheap, flexible horizontal scaling. An easy fix is to add auditing to the core WMI counters and create a Windows Server SKU that costs $0 per CPU and $0 per client ... but to stay in compliance you average your ASP.net transaction count, or SQL size/complexity + operation count and then cut a license renewal check. Make it free (as in beer, with online-only support) for the first few thousand page views per month for a typical app, and it'll rapidly start taking over a big piece of the startup and cloud/on-demand computing world.

All this said, big companies tend to have nine lives (even if Yahoo! seems intent on burning through all nine of them). So I'm not about to count Microsoft out, or suggest an "over-the-hill" tipping point is at hand. Just some big decisions, both strategic and tactical, that ought to be made well and soon.

So What's the Deal With ... ?

• ... those tech videos where the topic sounds interesting but I have to watch 3 guys chat about it in real time for 45 minutes? Like I have nothing better to do than watch people talk? Give me the slide deck, the bullets, a transcript, something I can scan through and read, since I know it's going to take me 4 minutes max to sort through the content.
• ... 10gen and the folks that want to write yet another server computing platform ... in JavaScript, and then compete with Google using their $1.5 million in VC? Ever since Douglas Crockford schooled me in the true nature of JavaScript, I've had little bad to say about the language. But why yet another platform for server-side computing? 10gen's argument that Google's AppEngine is too closed and locks apps in seems like a stretch: you can run Python/Django apps on AppEngine ... or somewhere else. Yes, the Google data API is their own, but it's simpler than a relational database API -- if you really need to use Google's data API over your own data store, it won't be hard to set up. There's also Rails, which is open and has the advantage (over 10gen) of being well known already ... If you don't think Rails can work as a cloud platform, talk to Heroku.
• ... Symbian's S60 emulator triggering DEP faults, so that Windows has to kill it? I can imagine how one might, when writing an emulator, run afoul of DEP ... But this is my first run-in with an app fatally triggering it, and I've already used Microsoft's device emulators, Palm's emulators, VMWare, and Sun's xVM VirtualBox (a VM is not the same as an emulator ... but some parts of the VM function via emulation) ... and had no problems.
• ... all the enmity toward Windows from the Rails guys? First, Ruby and Rails run fine on Windows, there's no need for all the "we specifically won't say anything about working on Windows, I mean, if you're crazy enough to want to do that" stuff that keeps popping up in otherwise respectable books. Second, many (most?) of the top Rails hackers are so young, they probably don't even remember Microsoft's real "bad old days." Yes, Microsoft has done some shady things worth complaining about. The fact that IE 7 doesn't render your CSS the way you wanted is not one of them.
• ... those posts where an unknown wantrepreneur with no valley connections or financing seeks a successful, experienced, genius hacker to build his mediocre but top secret idea for equity only? The posts crop up constantly on craigslist and all the SF tech meetup mailing lists. If this kind of post represents your thought process, why not just skip step 1 and 2, and post for people to send you a burlap sack full of cash?

No, CherryPal Will Not be the First (or any) Mass-Market Cloud Computer

Notwithstanding this predictable VentureBeat article about a thin-client device with a somewhat anatomically suggestive logo.

I've previously written about why thin-client is a not-now and not-soon solution.

The $249 "CherryPal" box reminded me of another twist on the problem ... one that comes into play when the thin client isn't just software (like Skyfire's screen-scraping browser) but a new hardware device.

See, here's the thing about building your own new hardware in smallish quantities: it's really expensive on a per-unit basis. Or, another way, you can't offer a fraction of the capability per dollar that a Dell or Sony can. Your $249 thingamabob is going up against other $249 devices that have way more stuff (e.g., entire laptops) because they are produced in high-volume orders of established designs/modules/parts.

To be more precise, there is a spreadsheet you can put together that maps your bill of materials, plus any special physical design issues into a cost per fabricated unit. It includes multipliers that will make you sad, like your $2 can't-live-without-it chip might end up adding $20 to the finished product cost... depending on where it fits in, affects other parts, quantities, and other stuff.

It's tempting though, especially since the last 5-10 years have brought the ability to fabricate in China for lower cost and in much smaller quantities than would have been practical before. (Of course the costs aren't really lower, they're just externalized into a bunch of other areas ... but those are politics/economics/policy areas more than tech, so I'll leave them be for now.)

So, whereas before you might have needed to sell 700,000 units to break even on something, now 20,000 units will do it: increased temptation.

But if you're CherryPal, you still need to convince someone that a $249 thin-client is more useful than a $299 laptop with 10x the horsepower, a screen, storage, and all the rest.

Free Idea: AJAX Desktop Publishing App

If you're hankering to join the web startup circus and have everything but an idea, here's one for you: a full-on desktop publishing app ... online. The ready acquisition market for decent online productivity apps can serve as inspiration, even as the insane complexity of the undertaking urges you on toward greater heights of caffeination and maybe some anti-depressants.

Seriously, apps that can deal with complex page layouts and can produce a variety of press-ready output (InDesign, QuarkXPress) are expensive and hard to learn, and the average person rarely needs them. It would be nice to be able to hop on a web page and do a quick design. Wizards or integrated help could go a long way. All the usual business models (adds, freemium, pay-per-storage, etc.) apply.

Now, if you build this in Flash/Flex or in Silverlight (2.0), you merely have a ton of work ahead of you. While challenging, you may feel like you're not ... well ... proving just how wicked you really are, since those two platforms give you all of the technical infrastructure you need.

So why not go all the way? Do it in JavaScript, in the browser. Yep, WYSIWYG, arbitrary page sizes and layouts, in the browser. Advanced typography too. Fun times.

Here's a hint: one of the tricky bits is measuring text line layouts, since you'll need to make sure that the view matches the backing data (from which you will be generating other "views" or output formats).

JavaScript (the browser APIs really) don't give you everything you'd like. For example, one standard thing you need is to find out how many characters will lay out into a column of a particular width. Or simply what size a run of text will turn out to be.

While you can't ask these things directly, the clientHeight DOM element property can be useful: it shows the actual height that a DIV's content will lay out at ... so it will also signal when content has flowed from one line into a second line. By hiding the DIV, rigging up a while loop, and monkeying with the content (string), you can build a routine that measures text, so that you know exactly which characters are going to appear on a line.

Completing the application is left as a (possibly lucrative) exercise for the reader.

Packaging Outlook Scripts in Word Documents

Microsoft Office includes (unless you exclude it at install time) a flavor of ye olde Visual Basic (i.e. VB-COM not VB.Net) that is designed to facilitate Office automation.

If you're willing to put up with the VB syntax and conventions, you can put together mini-programs (including GUIs, menu options, toolbar buttons, etc. if you like) in even less time than it would take to boot up Visual Studio and run the Office add-in wizard or add some Office .Net assembly references.

Interestingly, distributing the resultant "macros" can be tricky... especially for Outlook. There are all sorts of suggestions depending on how widely you want to deploy your script, and how many "Are you really really sure you want to run this script?" dialogs you expect your end users to click through. Some of the approaches are quite hokey... for example copying over a client script file that will delete any other macros your user has. Eeesh.

It's possible that .vbs files and Windows Scripting Host are actually the "right" answer ... but here's another I used:

1. Write and debug your Outlook scripts in Outlook using the built-in VBS IDE.
2. Copy the bits you need into the dev environment in Word (this assumes your user has Word, and I'm guessing if they run Outlook they probably run Word as well).
3. Save the word document as a word-doc-with-executable-content (*.docm). Even if you haven't defined any GUI elements, any quick and dirty VB procedures you've written will show up as macros in the View->Macro dialog.
4. Bonus: write any instructions you'd like to pass along right in the word doc.
5. Give the file to your users.
6. COM-based scripting is COM-based scripting, so even though a user runs the macro in a Word doc, it can find and manipulate their Outlook items.

Not to worry, Word (and Outlook) prompt the heck out of the users when they open and try to run the code. And saying yes to your file will not require the user to grant access to any other script-bearing files.

It's not the slickest procedure in the world, but it's wicked fast and it serves the basic scripting purpose: Here's a doc with a script I wrote to convert Outlook notes into a single Task item for syncing with my Windows Mobile notes app.

Pardon the Glare: I Think My Tinfoil Hat is Catching the Sun

Last week, it became clear that Kevin Martin and the FCC are getting closer to laying the smackdown on Comcast for the ISP's BitTorrent "management" policies (or was it because Comcast keeps lying about it? or because Comcast has tried to game the hearings about it? so hard to keep track of now).

And now all of a sudden I'm getting 4x more BitTorrent throughput on my Comcast cable line than I ever did before.

I'm tempted to imagine a connection.

But I also suspect that positing a connection is tinfoilhattery on my part ... in particular because the bandwidth/connection patterns I see don't look like the TCP reset packet approach that Comcast had been using, and which gave them away.

I'm getting more throughput in the middle of transfers, where there had never been any resets.

So maybe I should just be happy.

Or maybe that's just how Comcast wants me to feel.

On the Other Hand...

Yesterday I wrote about the potential of the iPhone to break the longstanding wireless app logjam in the U.S.

Today the Free Software Foundation blogs about the other logjam we might be getting into with the device, namely that the app development and publishing community is just as locked down and controlled as the DRM on iTunes tracks. Wait, actually the app side is even more locked down (since there's a low-tech path to extract songs by way of audio CDs, whereas the app infrastructure requires jailbreak).

I'm curious whether this passage from the FSF post will have the fanboys plugging their ears and whining "I can't hear you" or just shrugging their shoulders and saying "who cares":

"Apple, through its marketing and visual design techniques, is manufacturing an illusion that merely buying an Apple makes you part of an alternative community. But the technology they use is explicitly chosen to divide people into separate digital cells, and to position Apple as sole warden. When your business depends on people paying for the privilege of being locked up, the prison better look and feel luxurious, and the bars better not be too visible."

If not a prison, it's still a walled garden. Like Verizon's mobile app vending machine (odd are you've never even hard of that one) or the old America Online. Maybe it can work, but there are better alternatives.

iPhone App Store: Let's Hope It's Not All Fun and Games

The iPhone phenomenon seems to be a kind of Rorschach test, where you can find whatever you're looking for ...

There are all sorts of posts on data from the App store, looking at, for example, pricepoints in the app catalog, or the kinds of apps available.

These data are far from perfect (though worth a look), and in any case I hope that VentureBeat's post on the prevalence of Game and Entertainment apps turns out to be wrong.

Why? Mobile apps and mobile net usage have been waiting to "break out" in the U.S. for nearly 10 years. It's tempting to agree with the sentiment that mobile net use is reaching critical mass thanks to iPhone, but everyone in the industry has made that mistake many times before.

Successful off-deck software on phones in the U.S. has been more or less limited to games. So when I read the headline "Fun! Nearly half of all the iPhone App Store apps are games or entertainment," all I could think was "let's hope that it doesn't stay that way!"

I have no problem with Super Monkey Ball per se. It's just that if all the top app activity is in games and entertainment, then the mass psychology around the iPhone / App Store ecosystem is in danger of slipping irretrievably in the direction of "diversion" and not the enormous opportunity it really is.

Before you say I should just chill out about the Monkey Ball, consider:

Blackberry, because of its email device pedigree, was pigeonholed as a "corporate email/organizer device" years ago, even though it could do many other things. Despite RIM's attempt to sex up the brand, boost the hardware, cut prices and improve the design, it looks now like the window is closing for RIM to make any big gains. The enormous deployed base of Blackberries never translated into a general platform for mobile net-based computing.

At the moment, all of the top 10 -- and 18 of the top 20 -- paid App Store apps (over the last 24 hours) are Game/Entertainment apps. So right now, when the frenzy is at a peak, with mass media coverage of sold-out stores and consumers showing acquaintances their hot new gadget, the eyeballs are on the Super Monkey Balls.

To a lot of people, who have never seen a 3rd-party app on a phone, and who don't totally get what this is all about, the demonstration is going to look more like a Nintendo DS than a handheld computer.

Which raises a possibility: perhaps Apple was wise to restrict iPhone 1.0 to Safari-based apps, thus forcing consumers to view the device as a mini-web-tablet, rather than as a portable entertainment device.

Refactoring: Bittersweet if it Could Easily Have Been Avoided

I've been doing some refactoring on a decent sized app lately. Since there are lots of other people doing stuff and the app is close to a supported 1.0 release, it's like working on the engine of a car while it's cruising down the freeway. Not just tightening the cap on the coolant reservoir, more like converting the engine to run on hemp seed oil and corn cobs instead of gasoline.

On one hand, it's fun: mundane tasks that might feel academic become a lot more interesting when the task is part of the Rubik's cube of after-the-fact architectural change.

On the other hand, a lot of the refactor is an attempt to deal with unnecessary complexity in less complex way. The underlying complexity -- not mediocre code or an initial pass at red-green development -- is really the enemy.

And here's the rub: this app has some network protocol requirements that are (1) outrageously baroque; (2) opaque and undocumented; and (3) completely unnecessary for the app to function for its users.

As a result, it's already eaten over 120 man-months (and a staggering amount of money, let's just say over 8x what you're probably thinking the labor cost was).

At least 2/3 of that time and money (maybe 75%) went to nothing the user will ever see or care about, and nothing that offers any inherent business value or competitive advantage for the owner. That 2/3 is all expended in dealing with the bizarro protocol requirements ... and the issues and complexity it spawns: the design compromises, the testing, the bug analysis and remediation, the refactoring. I'm not even counting the actual delays and feature cuts in this cost.

Where did I get the 2/3 number? Well, the project was initially scoped without this network protocol issue. The estimates were written down. And, in the initial iteration -- about 30 man-months of work over 5 calendar months -- we tracked resource usage and found that core functionality and QA was reliably 1/3 and closely matched the estimates, while accommodating these "other" issues was 2/3. Week in, week out. No surprises. Things haven't changed since.

It's ironic because this pattern is something that often comes up when developing against a true legacy system -- the formats, protocols, and systems in place simply cannot be changed right away for the convenience (or productivity) of a new team. And yet this initiative was green-field, not legacy. It could have used any protocol in the world. There was 0 (zero) install base that needed compatibility with this particular system.

I almost wish I didn't know this information ... It's the one thing that puts a real damper on the refactoring fun: knowing that with even a tiny bit of management planning and the willingness to deal with team issues, it would be unnecessary.

Yet Another Microblogging App ... With a Twist

Fun as it is to argue about whether the latest Twitter outage will be the death of it, or whether identi.ca has legs, there's a whole other world of RSS/Microblogging possibilities that hasn't been exploited. Unfortunately, as I'll describe in a bit, there's a reason for that ... and an implicit call to action to fix it.

Microblogging -- or, really, the feed-oriented delivery mechanism -- could be huge for all kinds of private-domain problems that are stuck in Web 1.0 mode of e-mail updates and web-page dashboards.

Whether it's a group trip; a job search (that my good friends but not my coworkers know about); the latest status on project, issue, or change request; whether my cat is in or out (neighbor cats beware!); what my son is up to; a surprise party ... and on and on ... there are microblogging feeds I'd like to publish and consume, that are not public. They are not 1-1 either. They are published to a group. I trust members of that group, and they can bring in others. Perhaps many people can update or write to the feed as well.

Seems awfully useful in personal and professional contexts. But there's no app that exactly does this right now. The closest things are Tumblr's "groups" -- which are cool, but don't offer feeds -- and FriendFeed's "rooms" -- which are also cool, but don't offer authenticated feeds.

So I hacked together an app during a few spare hours one weekend to do what I wanted.

You can try it if you like at http://www.statusmonster.com or its real location, http://stat.heroku.com (have I mentioned how totally freaking awesome heroku is? well, that's a post for another day, but take me word for it, they are ninja rockstars those guys).

This app makes it super easy to create a multitude of private RSS (atom, really) status feeds, and share them with people. It has a mini-dashboard to watch the latest on a bunch of feeds. But I'm thinking folks don't need Yet Another Web Page to visit all the time. They need authenticated feeds, which statusmonster offers.

Wow, I'm gonna get rich and famous.

No, I'm not.

Here's the problem:

If a feed is not authenticated (like the basic feeds from statusmonster, or the "room" feeds at FriendFeed) then midstream aggregators like Bloglines or NewsGator may index these feeds for search, and/or offer them to others for subscription. This is the essential difference between syndication and publication -- a.k.a. the answer to, "Hey, how is a blog different from a homepage?" This is great for my blog, but really bad for my candid job search notes.

Ok, so we'll offer authenticated feeds. According the Bloglines FAQ, for example, authenticated feeds are not indexed for search or exposed to other users.

I did that, and started trying it out with NewsGator's desktop and mobile products, Outlook's RSS reader, Bloglines, Google Reader, etc.

The power of feeds lies in the fact that the end user gets to decide how to consume and/or process the content. That is, to get the full power/potential of feeds, they need to work with pretty much every major reader/aggregator service.

And that's where the trouble starts. Reading authenticated feeds with readers is a completely hit-and-miss affair.

The best behavior I found actually came from Outlook 2007, which not surprisingly treats the feed almost as an email account. It takes your credentials one time, and then when it polls -- or when you click "send/receive" -- it supplies them behind the scenes and updates the view of read/unread items. Pretty much exactly what you want.

But it was all downhill from there.

Bloglines processed my authenticated feed, but seemed to take forever to reflect updates (much longer than other feeds), and it eventually lost sync with the backing feed. The feed still exists, same location, same credentials, but eventually Bloglines started showing its little "[!]" marker meaning "problem with feed" and never updated again.

NewsGator kind of half worked. Google reader doesn't do authenticated feeds. And so on down the line.

So doing all this cool stuff on the existing infrastructure is not gonna happen. Big bummer because I'm convinced there's major value in these use cases, so we need to figure out how to make them a reality in a way people can actually use (and will want to use).

What are the constraints?

We don't need super end-to-end crypto any more than email does. Most folks do their email in the clear, figuring the content is not top secret, but also assuming that their employer/ISP/email provider is not gonna go and publish the email in a Google-able way.

I think that's the standard we're aiming for -- basically a better way to do stuff that nowadays might be handled by a whole lot of one-to-many emails.

A set of standards around handling authenticated feeds might be all we need. But how to enforce that, since anyone can cook up their own aggregator and ignore the standards?

Force the user agent (browser or client app) to supply some kind of secret to decode the feed? Maybe, but this is only strong in proportion to the strength of the key material, and having lots of high-entropy key material per feed makes this cumbersome and hard to use.

Something about confidential and syndication don't really mix. But we don't need another walled garden email/messaging system, and we don't need more web pages to visit (like Tumblr's groups).

I'm working on it. Meantime, what do you think?

When Corner Cases Attack

Notwithstanding the credit given to MTV's Real World, I think reality TV took off with that Fox special that featured a deer knocking a hunter on his rear. That guy just never saw it coming.

Same thing seems to happen too often when we start making architectural adjustments around a corner case in a software application.

The real problem is how to tell when an exception to the initial model -- the corner case or edge case -- is truly a strange bird, versus when it is a big piece of the model that was missed the last time through.

Sounds like something that should be easy to resolve with proper analysis, whether of the thick-stack-of-paper variety, OO variety, or an agile story-based style.

But it's not.

I've recently been doing some work on a project where this mistake was made. At some point in the past, this team looked at an odd duck, thought hard, and realized it was a big piece of their world. Big enough that the underlying infrastructure -- database, network services, etc. -- should be designed with sufficient generality that this corner case would become a mainstream case.

I understand how the analysis went, and why it made sense.

Only problem is, with 20/20 hindsight we can now see it was wrong. Bringing this edge case into the tent made the system an order of magnitude more complex, and it should have been left as an odd hack in a dozen places ... but as development happened over time, on several different module teams, the problem of the added complexity wasn't clear until it was too late.

To be a little more specific, I'll offer an analogy: you're writing a driving game or simulation. A big part of the fun is the spots where the players catch air, jumping their vehicles over obstacles to get ahead. But how to model the behavior of the cars as they fly through the air? A quick hack? or ... integrating elements of flight simulation or 3-D physics simulation?

Well, this team went the equivalent of the flight simulator route, and their hard-enough-to-code 2-D driving game has become a brutal 3-D simulation. All for a few jumps that in retrospect should have been hacked.

My point is not to complain about a historical decision that probably didn't seem bad given the info available at the time. It's to ask about how to avoid these problems in the future.

If you grant my assertion that, in some cases, no straightforward analysis reveals the problem at the moment the "how do we handle this case?" question is asked, then it seems like we need a set of milestones for a posteriori analysis.

We make a design call when we need to make a call, and then we need some specific questions -- and times or situations to ask them -- that will tell us if we're on the right track.

Specifically: What is the best and earliest way to ascertain if we're

1. monkeyhacking in an edge case that really demands a proper refactor
   OR
2. doing a refactor that way overengineers a solution that really just needs a monkeyhack

What do you think?