Atalasoft: Another Example of Gnarly DRM == Lost Sale

I’m working on a project that involves semi-automated document imaging. Scan, deskew, crop, re-arrange …

It’s on Windows, where every modern scanner hooks into both TWAIN and WIA out of the box, often without even needing a vendor driver, so I just needed a library/toolkit to do the lifting on the app logic side.

Enter Atalasoft DotImage imaging libraries. Does everything you need, works fairly well. Established presence in the market. We start heading in that direction. The Atalasoft bits we needed turn out to be pricey as components go, and we would need a runtime license as well as the development license – but this is a commercial project the success of which would not be diminished by the software costs. So we didn’t blink at the price.

We downloaded the dev SDK, implemented a few features … and we needed to show them to customers. In other cities on other machines. Well, the dev SDK is crippled and doesn’t allow that.

Atalasoft’s sales department generated a 30-day license for me, and sent me the instructions to install and deploy it. And … it half worked. Some machines could run the deployed app. Other machines, the app would crash when the relevant DLLs tried to load, despite deployment of the magic binaries, license files, and other DRM voodoo.

For a brief moment, I thought maybe my app is just broken … but, upon attaching a debugger, I saw that all of these crashes threw the same error. And, since it was .Net, the error was in plain English: Atalasoft’s licensing module was barfing and taking the whole app down.

At that point I could have spend more critical hours trying to navigate around these problems (I’m guessing their pre-sales tech support would have tried) … but … wouldn’t you know it, here is another company offering a similar library, much more agreeable terms, 30-day trial and a seemingly foolproof license key mechanism.

Download, type type build deploy. Success. Haven’t looked back.

Now it’s also convenient that this other product seems to work a little better, has more agreeable legal terms and costs less. But those were not dealbreaker criteria at this stage.

I would never have even gone down the list to this other vendor if Atalasoft’s DRM hadn’t broken my tight-deadline customer demos.

Using AppEngine -- Or Similar Datastore -- To Integrate Complex Legacy Data Formats

I gave a lightning talk last night at the SF Bay Area App Engine Developers, showing some work I've been doing to represent gnarly legacy records in AppEngine so as to maintain source fidelity, minimize upfront analysis, and make them easy to integrate with other systems.

I had started with an XML record that I wanted to parse and represent in the datastore -- without knowing which tags and structures would be present, since this format had, ahem, evolved to obscurity over time, as often happens with real-world legacy records.

Before I talk about my approach, here's why I thought this effort might be interesting to the group: a lot of data structures have a tree structure in common with XML. From C structs and file blocks that include a header, telling which types to cast the next n bytes to (and so on inside of those) ... to mainframe "structured data" records I've encountered which consist of nested records, parsed recursively, with their meanings occasionally opaque, lost to history, or belonging to some partner company.

My approach -- which is simply to create a mapping of how to assemble and disassemble the records -- enables a record to be stored in a single App Engine record. But not as a block (or blob) -- rather with fine-grained addressable fields that are easy to talk to using the GAE Datastore API.

In my case, since my original was XML, I created a mechanism similar to a tiny subset of XPath describing the sequence of tags where a data element lived -- but with the characters changed so that it would be Python and GAE-friendly. That is, instead of "/foo/bar[2]/baz" I used _Foo_Bar__2_Baz.

This let me "flatten" the XML into a set of key-value pairs, while allowing that the XML might contain arbitrary structures injected by others ... and that I might want to inject my own extra structures. This arrangement is perfect for the Expando models in App Engine Datastore, or any similar store (e.g. Hypertable, which is modeled after BigTable, or Microsoft SQL Data Services which uses SQL 2008's sparse tables to similar effect).

So now I can store and retrieve my records. Any fields/subrecords which I understand and care about, I can easily work with from other systems, by mapping to the appropriate "key" in the stored record.

For example, if I'm storing a bunch of catalog data, and another system just cares about enumerating each "Product" with "Name" and "Price," then I can create a facade or wrapper in GAE that maps, say, Price to _Strange_Old_Way_To_Represent_Current_Price, and we're all set.

To be sure, there could be performance issues if you tried to use this to create arbitrary queries and reports against the data. That's not really the purpose and, in my experience, if there are no "shortcuts" to processing these legacy records, then the business folks are not used to being able to make an OLAP cube out of them either. (They probably have a batch or offline extraction process.)

Nonetheless, it's another tool in our chest when we need to work with systems and data that have been out in enough real-world battles to come home scarred with lots of cruft.