Suck Us All Into the Machine: Build FluidDB on Twitter and #Hashtags

I was reading today about a fairly amorphous, tag-based, public "database" concept called FluidDB.

To avoid the usual distractions, I will assume my audience knows all about relational and non-relational datastores, implications of tag-oriented metadata, etc. In fact, since my friends know about this stuff and they are most of my audience, I think I'm not too far afield making this assumption.

One reason I found the FluidDB concept interesting is vanity: about four weeks ago, I spent some time considering building just this kind of database ... on top of Twitter.

Why would one want to do that?

Precisely because Twitter already has a lot of meaningful data curated by humans and tagged with a well-known metadata scheme (hashtags).

In exchange for having very small or hypernormalized data records (since each atomic entry is limited to 140 characters minus the tags and any indexed keys), we get a strange merging of human- and machine-readable data.

Humans could read (and follow, search, etc.) data entities of interest.

And clearly the "goal" of automated (machine) participants (clients) would be to understand as much of the human content as possible, treating it as objects, tuples, logical inferences, or knowledge base "facts."

Moreover, the originator of a tweet, as well as any @-referenced recipients, are critical metadata. They are, actually, tags themselves in way which is linearly independent of the hashtags. That is, a from-@adbreind tweet (entity) marked #database is different from a to/ref-@adbreind tweet marked #database, while #database must be considered to (possibly) have a different sense than it does in, say, a from-@headius tweet. However, the same tools and semantic analyzers can be applied, essentially treating the writer and target of a tweet as special tags.

In this way, our twitter discourse, short enough to make machine understanding tempting even when the packed cultural references make such understanding impractical, merges us into the database and makes us "just another part of the machine."