back to the ivory tower

Disclaimer : This is an opinion piece, not a technical piece. If you want a technical piece, read prior posts or wait a couple of days more, I'm working on some interesting stuff. In this piece I'm voicing a personal opinion only. If you don't like it (the opinion), kudos to you.

I have done my time in the Ivory Towers. Did I like it ? Of course I did ! Developers and architects would come to me - and needed an appointment - and I'd design and create their database or systems the way I liked it, often causing an overhaul of what they had in mind. I could speed up projects or kill them. My word was law and my code reviews were legendary. The little Latin proverbs (in a time before online access was granted) I had in the Friday entry of my company agenda (accessible to all), were coffee-corner talk minutes after they went up.

It was a time of people who knew and understood and people who didn't.

Around the turn of the century this changed. The Internet changed it. Everything became available for everybody and everybody could tinker with it. In the database world SQL-databases ruled supreme. Everybody could understand the mathematics behind those and machines became powerful enough so they became sufficiently performant. Virtualization opened up the system administration world. While dual booting always remained a tricky thing only a geek would do, booting a virtual machine is a no-brainer and the Internet is full of how-to guides.

Everybody could now do everything and this opened the Information Technology gates for a lot of people and small companies with great ideas. The towers toppled.

Recently I notice a pushback, did you notice it too ? SQL databases are no longer hot, no, we've got big data, semantics and map reduce, tripple stores and key-value-stores, ... and most importantly, you will go out of business if you don't have them too, yes, really, look Google and Facebook have them ... you must have them too !

So, following the hype a lot of people buy into all of it. And only afterwards find out they are biting of a lot more than they can chew. For :

• Did somebody explain to you what Big Data is ? Not to mention Cloud Computing, Tripple Stores, semantics, ... And did you find two people that gave you the same explanation on all of those ? Really ? Can I have their email-addresses please ?
• Did somebody explain to you what map reduce is ? You are going to use it to query your data ! Did you look it up ? Do you understand the mathematics ? Really ? Oh, you're a PhD Mathematics ... figures.

It would seem too me that we're moving back to the Ivory Towers, because what all these have in common is that the added value for the companies that create and hype them lies in the fact that they require consultancy to get right and in the end it will be the consultant that says what goes and what does not ... regardless of what the business question was !

I also want to add that a lot of this is far from new. Everybody that ever used LDAP knows that a key-value-store has been around forever. And while a graph database (the thing Facebook is build on) is super flexible and a hell to query in a performant way - can you hear Kerching! too ? - it's roots lie in the CODASYL findings of 1969 (remember IDMS anyone)... yes, the year a man put foot on the moon. Server- and database clusters with quorums ... been there and done that with HP systems and Oracle as early as 2001.

Personally I play with it all (watch this space for upcoming technical posts) and I see the benefits and the drawbacks of many a hype. For me they are all tools to get the job done, the job defines the tool, not the other way around. Remember that before you buy into one ! And the day the new Edgar Codd stands up and starts bringing down the towers of complexity once more ... lets give her/him a Noble Prize !