Software Discipline Tribalism

Unfortunately, people seem rarely able to stop at a reasoned preference -- e.g., "I like X, since X may offer better outcomes than Y" ... and too often end up, at least whenever group persuasion is involved, somewhere more dramatic, personal, extreme, and narrow -- e.g., "I'm the X kind of person who rebels against Y, since Y can offer worse outcomes."

This is as true in cultures of software development as anywhere else.

While many people have been guilty of this unproductive shift in attitudes, it seems many of the Agile development, dynamic languages, small-tools/small-process/small-companies crowd has long since fallen prey to it.

To be sure, there was much temptation to rebel.

At the start of the decade, proprietary Unix was strong; many processes came with expensive consultants, training, books, and tools ... and overwhelmed the projects they were meant to guide; many tools were expensive and proprietary. Web services were coming onto the scene and large players, with licenses and consulting hours to sell, created specs that were unwieldy for the small, agile, and less-deep-pocketed.

When the post-dot-com nuclear winter set in, small companies had no money to pay for any of that, and we got LAMP, Agile, TDD, REST, etc. Opposition to OO, which had been strong in many quarters, suddenly faded as OO was no longer identified (rightly or not) with certain problematic processes. Ironically, many new OO language fans had been ignoring the lightweight, free (speech and beer) processes that some OO advocates had been producing for years.

These have all proven to be useful tools and techniques and have created whole companies and enormous value ... but somewhere along the line, instead of being in favor of these tools and techniques because in some cases they produced better outcomes, either the leaders or the converts started thinking they were the rebels against anything enterprise, strongly typed, thoroughly analyzed, designed and well tooled.

This shift in attitudes does not help the industry ... nor even the clever consultants who lead the charge, deprecating last year's trend for a new one which they just happen to have written a book about.

We desperately need a broader perspective that integrates all of these pieces. There are things manually-written tests just won't do -- tools like Pex can help immensely, even if (or because... )they are from a big company.

Analysis and design are not bad words, while Agile can get dangerously close to simply surrendering to the pounding waves of change (and laughing at goals all the way to the bank) rather than building against the tide, and trying manage to a real outcome on a real budget.

Static languages can get hideously verbose for cases with functor-like behavior (Java and C# [pre-3.0], I'm looking at you). At the same time, go talk to some ActionScript developers -- who have had dynamic and functional for years -- and you'll see an amazing appreciation for the optional strict typing and interfaces in AS3.

REST is great, but in playing at dynamic, it turns out to be rather like C -- it's as dynamic as the strings you pipe into the compiler, and no more. Absent proper metadata, it cannot reflect and self-bind, so it sacrifices features that dynamic language developers love in their day-to-day coding.

Ironically, most of the critical elements of this "movement" -- along with open source -- are being subsumed into the big enterprise software companies at a prodigious pace. Sun owns MySQL and halfway owns JRuby; Java servers may serve more Rails apps than Mongrel/Ebb/Thin/etc. soon, Microsoft is all over TDD, IronRuby, IronPython...

I suppose the sort of tribalizing we see here is at least partly inevitable in any field. But it would serve the entire industry if that "part" could be made as small as reasonably possible. As a young industry with a poor track record and few rules, we ought to be more interested in better software outcomes than in being rebellious.

Non-compete Ruling May Indirectly Help Broken Organizations

Last week's California court ruling against non-competes is interesting, not least because I've had one client who was more interested in a getting me to sign a specially crafted non-compete that he hoped to use in a lawsuit than in having me actually deliver work.

Actually I'm not sure this ruling would even affect independent contractors like me; the view of employment law towards contractors (where it applies at all) is very different from that toward employees.

This ruling may play a small role spurring innovation by making it easier for employees to quit and then compete with their former employer.

I'm also hopeful that the newly prominent threat of such competition may encourage dysfunctional tech organizations to renew their efforts at achieving better group dynamics. (Is there such a thing as therapy for whole organizations? No, I don't mean those motivational or consulting clown-schools that involve offsites and create a distraction ... at least until the check is cashed.)

In addition to the ominpresent threat that one's best employees might leave, there is now renewed emphasis on the fact that a group of them might well leave together and "do it right" if they can.

While the ruling does not of course give anyone license to loot protected IP from the former company, history shows that's rarely the issue in Silicon Valley anyway.

Every development manager should view this as a wake-up call to think about keeping the team on board.

Refactoring: Bittersweet if it Could Easily Have Been Avoided

I've been doing some refactoring on a decent sized app lately. Since there are lots of other people doing stuff and the app is close to a supported 1.0 release, it's like working on the engine of a car while it's cruising down the freeway. Not just tightening the cap on the coolant reservoir, more like converting the engine to run on hemp seed oil and corn cobs instead of gasoline.

On one hand, it's fun: mundane tasks that might feel academic become a lot more interesting when the task is part of the Rubik's cube of after-the-fact architectural change.

On the other hand, a lot of the refactor is an attempt to deal with unnecessary complexity in less complex way. The underlying complexity -- not mediocre code or an initial pass at red-green development -- is really the enemy.

And here's the rub: this app has some network protocol requirements that are (1) outrageously baroque; (2) opaque and undocumented; and (3) completely unnecessary for the app to function for its users.

As a result, it's already eaten over 120 man-months (and a staggering amount of money, let's just say over 8x what you're probably thinking the labor cost was).

At least 2/3 of that time and money (maybe 75%) went to nothing the user will ever see or care about, and nothing that offers any inherent business value or competitive advantage for the owner. That 2/3 is all expended in dealing with the bizarro protocol requirements ... and the issues and complexity it spawns: the design compromises, the testing, the bug analysis and remediation, the refactoring. I'm not even counting the actual delays and feature cuts in this cost.

Where did I get the 2/3 number? Well, the project was initially scoped without this network protocol issue. The estimates were written down. And, in the initial iteration -- about 30 man-months of work over 5 calendar months -- we tracked resource usage and found that core functionality and QA was reliably 1/3 and closely matched the estimates, while accommodating these "other" issues was 2/3. Week in, week out. No surprises. Things haven't changed since.

It's ironic because this pattern is something that often comes up when developing against a true legacy system -- the formats, protocols, and systems in place simply cannot be changed right away for the convenience (or productivity) of a new team. And yet this initiative was green-field, not legacy. It could have used any protocol in the world. There was 0 (zero) install base that needed compatibility with this particular system.

I almost wish I didn't know this information ... It's the one thing that puts a real damper on the refactoring fun: knowing that with even a tiny bit of management planning and the willingness to deal with team issues, it would be unnecessary.

If Developers Love Speed, Why Is a Slow Laptop More Popular than a Fast Desktop?

When I was in college, there was an anthropology meme about how childbirth became riskier when the human pelvis adjusted for walking upright. So the ability to walk and run -- or the brain developments that caused humans to deal with problems by walking and running instead of some other way -- must have offered some massive evolutionary advantage that outweighed increased risk to mother and offspring in childbirth.

I'm not sure where this belief stands now -- whether it's established dogma or the anthro equivalent of an urban legend -- but there seems to be a funky analogy among developers and their dev machines.

I'm amazed by how many devs want to be mobile so bad that they use a laptop as a principal (or only!) development machine. I'm more amazed when these same people then get into a silly debate about "the best tools for the job," whether that's an OS debate, or IDEs, or something else.

Because, like walking upright in the story, a laptop offers mobility at a very heavy price.

I work machines pretty hard -- running server software, virtual machines, development environments / debuggers, lots of browsers, random other tools, some of which even use CPU cycles and not just memory. I appreciate the productivity and uninterrupted "flow" that a really fast machine offers.

Laptop performance is awful compared to desktop machines, and with every passing month a laptop (due to its limited ability for upgrade) falls farther and farther behind its well-maintained desktop counterpart. And a plain ol' $100 motherboard and $250 processor in a desktop will do things that make most laptops implode into a singularity, whether it's the 1333 MHz FSB, the 3+ GHz quad-core CPU, or a graphics card whose cooling pipe alone can't fit inside a laptop.

Even the top end, like the fastest Alienware gear, has some fundamental limitations that sound like desktops from a few years back: 2.8 GHz CPU / 800 FSB / 667 Memory ... and a price tag (with the best options) near $5,000!

It's not an OS thing either: the hyper-popular MacBook Pro, while one of the fastest "conventional/mass-availability/non-gaming" laptops, is a complete lightweight compared to the base 8-core Mac Pro tower (that runs the same $2,800 as a the top-end MacBook Pro).

Don't even get me started on the garbage laptops that most companies give their employees, machines which are optimized for durability, enterprise management, and running Office. Sporting things like 4200 rpm hard drives.

And for the occasional but real necessity of mobility, a $250 cheeseball laptop does a fine job for giving presentations, working with Office, and even a quick hack here or there, so it's not as though there's a huge sunk cost just in being able to bring a slide deck to a client.

Yet ... the ability to move around instead of sitting in one place offers -- or at least appears to offer -- some kind of power that compensates for all of these issues.

Can anyone clue me in on exactly what it is?

TechCrunch vs. The Maker Faire

Mike Arrington wrote one of those shot-heard-round-the-world posts today. It wasn't news, but Mike Arrington writing it on TechCrunch was the news. The Valley in a troubling state? Indeed.

It would bum me out a lot more if I hadn't spent this weekend at the Maker Faire. The fair was a beautiful thing. The people and projects looked great; the companies mostly silly. The bigger they tried to look (Yahoo!) the sillier they looked. The more they focused on doing cool stuff (Microsoft... sorta kinda) the better they looked. But really it was DIY anything and everything. The essence of the geek thought process was there, the thought process that makes Silicon Valley work decade after decade: one part science, one part "I bet if I monkeyed with this a little more, it would be really damn cool," and one part "that is really damn cool -- you need a hand with that?"

Robert Scoble already made this connection. But I want to hammer on it a little more. Spend 15 minutes browsing this flickr stream and you'll feel right as rain. It's meatspace stuff, mostly, some fire and robots and yarn along with the software. But the idea is the same, and the membrane between online and offline has never been thinner.

On Saturday, we spent 20 minutes looking for parking and ended up in the far reaches of a dirt lot where some cargo trailers were parked. The event was well attended.

The peninsula has its own cash-driven strain of lycanthropy, never more than a full moon away, but a lot of people here always want to sit down with a soldering iron or scissors or a blank text editor window and put something new and cool into the world.