short and sweet

Sometimes you want to do some horrible things to another human being. It is true, it happens to the best of us. While I love the series on tic-tac-toe in the 1060 newsletter there was literally smoke escaping through my nostrils (and I don't smoke) when I read this was possible :

<rootspace> 
    <!-- provides res:/etc/ConfigRDBMS.xml -->
    <literal type="xml" uri="res:/etc/ConfigRDBMS.xml">
      <config>
        <rdbms>
          <jdbcDriver>org.h2.Driver</jdbcDriver>
          <jdbcConnection>jdbc:h2:/db;CACHE_SIZE=1024</jdbcConnection>
          <poolSize>8</poolSize>
        </rdbms>
      </config>
    </literal>
</rootspace>

It was probably somewhere in a newsletter and I overlooked it. A new feature in one or another module that got mentioned in passing. It is so powerful it deserves another mention right here.

Until now, when I wanted to scaffold, I used filesets. They do the job, allow for some flexibility, blah blah and some more blah ...

Literals ROCk ! Especially in unittests for modules that require configuration to work they offer huge value for little work.

That is it for today, I'm quite sure the god-of-thunder will be having some interesting stuff to talk about in the 1060 newsletter so I'll leave the stage to him.

Before I forget ... happy anniversary 1060 Research !

back to the ivory tower

Disclaimer : This is an opinion piece, not a technical piece. If you want a technical piece, read prior posts or wait a couple of days more, I'm working on some interesting stuff. In this piece I'm voicing a personal opinion only. If you don't like it (the opinion), kudos to you.

I have done my time in the Ivory Towers. Did I like it ? Of course I did ! Developers and architects would come to me - and needed an appointment - and I'd design and create their database or systems the way I liked it, often causing an overhaul of what they had in mind. I could speed up projects or kill them. My word was law and my code reviews were legendary. The little Latin proverbs (in a time before online access was granted) I had in the Friday entry of my company agenda (accessible to all), were coffee-corner talk minutes after they went up.

It was a time of people who knew and understood and people who didn't.

Around the turn of the century this changed. The Internet changed it. Everything became available for everybody and everybody could tinker with it. In the database world SQL-databases ruled supreme. Everybody could understand the mathematics behind those and machines became powerful enough so they became sufficiently performant. Virtualization opened up the system administration world. While dual booting always remained a tricky thing only a geek would do, booting a virtual machine is a no-brainer and the Internet is full of how-to guides.

Everybody could now do everything and this opened the Information Technology gates for a lot of people and small companies with great ideas. The towers toppled.

Recently I notice a pushback, did you notice it too ? SQL databases are no longer hot, no, we've got big data, semantics and map reduce, tripple stores and key-value-stores, ... and most importantly, you will go out of business if you don't have them too, yes, really, look Google and Facebook have them ... you must have them too !

So, following the hype a lot of people buy into all of it. And only afterwards find out they are biting of a lot more than they can chew. For :

• Did somebody explain to you what Big Data is ? Not to mention Cloud Computing, Tripple Stores, semantics, ... And did you find two people that gave you the same explanation on all of those ? Really ? Can I have their email-addresses please ?
• Did somebody explain to you what map reduce is ? You are going to use it to query your data ! Did you look it up ? Do you understand the mathematics ? Really ? Oh, you're a PhD Mathematics ... figures.

It would seem too me that we're moving back to the Ivory Towers, because what all these have in common is that the added value for the companies that create and hype them lies in the fact that they require consultancy to get right and in the end it will be the consultant that says what goes and what does not ... regardless of what the business question was !

I also want to add that a lot of this is far from new. Everybody that ever used LDAP knows that a key-value-store has been around forever. And while a graph database (the thing Facebook is build on) is super flexible and a hell to query in a performant way - can you hear Kerching! too ? - it's roots lie in the CODASYL findings of 1969 (remember IDMS anyone)... yes, the year a man put foot on the moon. Server- and database clusters with quorums ... been there and done that with HP systems and Oracle as early as 2001.

Personally I play with it all (watch this space for upcoming technical posts) and I see the benefits and the drawbacks of many a hype. For me they are all tools to get the job done, the job defines the tool, not the other way around. Remember that before you buy into one ! And the day the new Edgar Codd stands up and starts bringing down the towers of complexity once more ... lets give her/him a Noble Prize !

minimalistic OXO

When you're up against a master it is very noble to let him/her pick both the battleground and the battle. You're going to get beaten though. And while you might learn some important lessons along the way (watch Karate Kid for full list) ... you're still going to get beaten.

Of course I could not refuse to pick up the gauntlet on the Tic-Tac-Toe (or noughts and crosses or oxo as we call it) challenge. But it was with a bit of despair. Sure, I might point out that it looks more like a matrix with elements than a set with cells and that there are in fact 5 diagonals and 5 antidiagonals (only two of those counting for a "Winning Set", that is true), but I might then get slapped around the head with my own weapons. I'm not a mathematician and I'm challenging a PhD in Quantum Physics.

And then I remembered something. I used to play a text MUD (try them, I haven't seen anything graphical that beats them for depth of involvement and gameplay ... once you get used to the interface). Inside the game you could play chess against other players. Yes, really. It was minimalistic. I mean, you had to know the shorthand codes for moving a piece and only newbies would occasionally request a visual (ascii art) view of the board (the rest of us did it all in the head). The only thing the game itself knew was what the valid moves were at a given state for the board.

Did the chess game know about the victory conditions ? Of course not. Only a computer needs to be told what the victory conditions are !

A smile crossed my face when I remembered that. For I can not beat Peter at his own game and I'll be devouring his upcoming newsletter(s) that explain his solution (and no doubt learn lots from them). But a minimalistic OXO in under two hours ? Yes, I can do that.

You find my solution here : 

• It has no concept of rows, columns or diagonals other than is needed for the interface (which is REST, using most of the verbs to get the job done).
• There is a little bit of code, the biggest part going to showing the board (like the ascii art for the chess game).
• The game has no concept of a victory condition, the players can decide what that is.
• It comes with documentation, search for "oxo" after you deploy the module. 
• There is no computer player or any other AI (you can add those as a layer on top if you want them, it's easy to find the rules for a perfect game on wikipedia).

Let the games begin !

second oreilly webcast

Yesterday I gave my second O'Reilly webcast this month. I used a much quieter laptop and it went a lot smoother (for me at least), but you'll understand that I'm quite happy it is now a thing of the past.

You can find the nCode applications I demo'd here. I also noticed that the previous talk is now up on Youtube, so you should be able to find that one there. This one will follow in about a week. The slideshow itself can be viewed on http://netkernelbook.org/wink/wiki/webcast_20120828.

This officially ends my holiday. I'll now be moving on to some serious stuff such as ... tic-tac-toe.

fine grained control

September and the return of the normal flow of things are just around the corner. For this holiday post I'm going to share another snippet of code that I've recently been working on. It gives you fine grained control over NetKernel's kernel properties :

// Author: Tom Geudens
// Date  : 2012/08/15

// The usual suspects for an accessor.
import org.netkernel.layer0.nkf.*;
import org.netkernel.layer0.meta.impl.SourcedArgumentMetaImpl;
import org.netkernel.module.standard.endpoint.StandardAccessorImpl;

// Processing.
import java.util.Properties;
import org.netkernel.container.IKernel;
import org.netkernel.layer0.boot.*;
import java.io.ByteArrayOutputStream;
import org.netkernel.layer0.representation.IBinaryStreamRepresentation;

// The accessor itself.
public class KernelPropertyAccessor extends StandardAccessorImpl {

    public KernelPropertyAccessor() {
        this.declareThreadSafe();
        this.declareArgument(new SourcedArgumentMetaImpl("key",null,null,new Class[] {String.class}));
        this.declareArgument(new SourcedArgumentMetaImpl("value",null,null,new Class[] {String.class}));
        this.declareSourceRepresentation(String.class);
        this.declareInhibitCheckForBadExpirationOnMutableResource(); // golden thread takes care of it
        this.declareSupportedVerbs(
                INKFRequestReadOnly.VERB_NEW|
                INKFRequestReadOnly.VERB_DELETE|
                INKFRequestReadOnly.VERB_EXISTS|
                INKFRequestReadOnly.VERB_SOURCE|
                INKFRequestReadOnly.VERB_SINK);
    }
   
    public void onSource(INKFRequestContext aContext) throws Exception {
        // SOURCE requires one argument, key, which is mandatory for all
        // verbs and checked by the grammar.
        String aKey = aContext.getThisRequest().getArgumentValue("key");
        
        // Check validity of arguments.
        if ( aKey.equals("") ) {
            throw new NKFException("request does not contain a valid key argument");
        }

        // getting the properties data
        Properties vKernelProperties = new Properties();
        IKernel aKernel = aContext.getKernelContext().getKernel();
        vKernelProperties.load(BootPersistence.sourceBootResource("kernel.properties",aKernel).getInputStream());

        if (! vKernelProperties.containsKey(aKey)) {
            throw new NKFException("property - " + aKey + " - does not exist in kernel.properties");
        }
        
        INKFRequest vGTrequest = aContext.createRequest("active:attachGoldenThread");
        vGTrequest.addArgument("id", "gt:kp:" + aKey);
        aContext.issueRequest(vGTrequest);

        // response (string)
        aContext.createResponseFrom(vKernelProperties.getProperty(aKey));
    }

    public void onExists(INKFRequestContext aContext) throws Exception {
        // EXISTS requires one argument, key, which is mandatory for all
        // verbs and checked by the grammar.
        String aKey = aContext.getThisRequest().getArgumentValue("key");

        // Check validity of arguments.
        if ( aKey.equals("") ) {
            throw new NKFException("request does not contain a valid key argument");
        }
        
        // getting the properties data
        Properties vKernelProperties = new Properties();
        IKernel aKernel = aContext.getKernelContext().getKernel();
        vKernelProperties.load(BootPersistence.sourceBootResource("kernel.properties",aKernel).getInputStream());

        INKFRequest vGTrequest = aContext.createRequest("active:attachGoldenThread");
        vGTrequest.addArgument("id", "gt:kp:" + aKey);
        aContext.issueRequest(vGTrequest);

        // response (boolean)
        aContext.createResponseFrom(vKernelProperties.containsKey(aKey));
    }
   
    public void onNew(INKFRequestContext aContext) throws Exception {
        // NEW requires two arguments, key, which is mandatory for all
        // verbs and value, which is only mandatory for NEW and SINK.
        String aKey = aContext.getThisRequest().getArgumentValue("key");
        String aValue = null;
        if (aContext.getThisRequest().argumentExists("value")) {
            aValue = aContext.getThisRequest().getArgumentValue("value");
        }

        // Check validity of arguments.
        if ( aKey.equals("") ) {
            throw new NKFException("request does not contain a valid - key - argument");
        }
        if ( aValue == null) {
            throw new NKFException("request does not contain a valid - value - argument");
        }

        // getting the properties data
        Properties vKernelProperties = new Properties();
        IKernel aKernel = aContext.getKernelContext().getKernel();
        vKernelProperties.load(BootPersistence.sourceBootResource("kernel.properties",aKernel).getInputStream());

        if (vKernelProperties.containsKey(aKey)) {
            throw new NKFException("property - " + aKey + " - already exists in kernel.properties");
        }
        
        // adding new key/value
        ByteArrayOutputStream vBAOS = new ByteArrayOutputStream();
        vKernelProperties.setProperty(aKey,aValue);
        vKernelProperties.store(vBAOS, null);
        
        IBinaryStreamRepresentation vBSR =
                aContext.transrept(vBAOS.toString().toString(), IBinaryStreamRepresentation.class);
        
        BootPersistence.sinkBootResource("kernel.properties",vBSR,aKernel);
        
        // response, the newly created property
        aContext.createResponseFrom(aKey + "=" + aValue);
    }

    public void onSink(INKFRequestContext aContext) throws Exception {
        // SINK requires two arguments, key, which is mandatory for all
        // verbs and value, which is only mandatory for NEW and SINK.
        String aKey = aContext.getThisRequest().getArgumentValue("key");
        String aValue = null;
        if (aContext.getThisRequest().argumentExists("value")) {
            aValue = aContext.getThisRequest().getArgumentValue("value");
        }

        // Check validity of arguments.
        if ( aKey.equals("") ) {
            throw new NKFException("request does not contain a valid - key - argument");
        }
        if ( aValue == null) {
            throw new NKFException("request does not contain a valid - value - argument");
        }

        // getting the properties data
        Properties vKernelProperties = new Properties();
        IKernel aKernel = aContext.getKernelContext().getKernel();
        vKernelProperties.load(BootPersistence.sourceBootResource("kernel.properties",aKernel).getInputStream());

        if (! vKernelProperties.containsKey(aKey)) {
            throw new NKFException("property - " + aKey + " - does not exist in kernel.properties");
        }
        
        // replacing the value of a key
        ByteArrayOutputStream vBAOS = new ByteArrayOutputStream();
        vKernelProperties.setProperty(aKey,aValue);
        vKernelProperties.store(vBAOS, null);
        
        IBinaryStreamRepresentation vBSR =
                aContext.transrept(vBAOS.toString().toString(), IBinaryStreamRepresentation.class);
        
        BootPersistence.sinkBootResource("kernel.properties",vBSR,aKernel);

        // SINK has no response but we do need to cut the golden thread
        INKFRequest vGTrequest = aContext.createRequest("active:cutGoldenThread");
        vGTrequest.addArgument("id", "gt:kp:" + aKey);
        aContext.issueRequest(vGTrequest);
    }

    public void onDelete(INKFRequestContext aContext) throws Exception {
        // DELETE requires one argument, key, which is mandatory for all
        // verbs and checked by the grammar.
        String aKey = aContext.getThisRequest().getArgumentValue("key");

        // Check validity of arguments.
        if ( aKey.equals("") ) {
            throw new NKFException("request does not contain a valid - key - argument");
        }

        // getting the properties data
        Properties vKernelProperties = new Properties();
        IKernel aKernel = aContext.getKernelContext().getKernel();
        vKernelProperties.load(BootPersistence.sourceBootResource("kernel.properties",aKernel).getInputStream());

        if (! vKernelProperties.containsKey(aKey)) {
            throw new NKFException("property - " + aKey + " - does not exist in kernel.properties");
        }
        
        // removing the key
        ByteArrayOutputStream vBAOS = new ByteArrayOutputStream();
        vKernelProperties.remove(aKey);
        vKernelProperties.store(vBAOS, null);
        
        IBinaryStreamRepresentation vBSR =
                aContext.transrept(vBAOS.toString(), IBinaryStreamRepresentation.class);
        
        BootPersistence.sinkBootResource("kernel.properties",vBSR,aKernel);

        INKFRequest vGTrequest = aContext.createRequest("active:cutGoldenThread");
        vGTrequest.addArgument("id", "gt:kp:" + aKey);
        aContext.issueRequest(vGTrequest);

        // response (boolean)
        INKFRequest vKPrequest = aContext.createRequest("active:kernelproperty");
        vKPrequest.addArgument("key", aKey);
        vKPrequest.setVerb(INKFRequestReadOnly.VERB_EXISTS);
        aContext.createResponseFrom(!(Boolean)(aContext.issueRequest(vKPrequest)));
    }
}
 
My code doesn't often come with warnings, but this time I have to state it is delivered "as is" and that changing kernel properties can potentially mess up your system (for which I take no responsibility).

I can also see some possible improvements on the above code, so by all means try it out and change it as you like.

Enjoy !

oreilly webcast

Who was it that said holidays are for the children, not the parents ? Whoever it was, he/she is right ! And it is neither the parents, nor the children that are to blame for this, it is our system of outsourcing children (and ourselves) during the whole year and then being put together (both sides looking upon the other as alien) for several weeks, often in a strange situation.

That's quite enough psychology for one day ... last Tuesday I was honoured (honored too) to be able to give a Webcast for O'Reilly. I must say I was very nervous, my laptop made a horrible sound and I couldn't hear or see the audience (the latter two are normal), so I have no idea how it went, you can get the nCode example module I used on https://dl.dropbox.com/u/65770556/urn.org.elbeesee.demo.ncode-1.0.0.zip.


You can see the slides that accompanied the webcast on http://netkernelbook.org/wink/wiki/webcast_20120807.

A recording was made and as soon as that is available I'll add the link to this post.

Enjoy your holidays !

P.S. There is a slightly more advanced webcast on August 28th. Read about it and register on http://www.oreillynet.com/pub/e/2261. Since the nCode stuff went down well, I will only use nCode on that webcast too (the NetKernel equivalent of : Look ma, no hands !).

P.P.S. No, I haven't gone soft. Word reached me that the white on black hurt the eyes of some of my potential readers.

she sells sea shells on the shore

At the moment I'm spending some time at the Belgian coast, catching my breath after my most recent adventures in Minneapolis (I even - image that - joined an "Happy Hour" where I - for the record - drank Pepsi Cola ... some things will never change).

However there is only so much watching the waves and potty training his son a man can take before he is drawn back to slightly less chaotic things ... NetKernel for example.

Below is a snippet of code that I first wrote for a specific implementation. Then I realised it could be made more general (that's when I added the argument) and right now I'm deciding in which general library it should go.

The snippet is provided as is, without any guarantee. I'm also not getting into any fights over whether or not flags have a place in coding. A long time ago I was taught that using gotos and/or flags was not done by good developers. I guess I'm not a good developer.

// Author: Tom Geudens
// Date  : 2012/07/20


// The usual suspects for an accessor.
import org.netkernel.layer0.meta.impl.SourcedArgumentMetaImpl;
import org.netkernel.layer0.nkf.*;
import org.netkernel.module.standard.endpoint.StandardAccessorImpl;


// Processing
import java.io.File;


public class FlagAccessor extends StandardAccessorImpl {
public FlagAccessor() {
this.declareThreadSafe();
this.declareArgument(new SourcedArgumentMetaImpl("flagname",null,null,new Class[] {String.class}));
}

private String getHome() {
String vHome=System.getProperty("user.home");

// Windows requires a reversal of the slash for a proper file:/ url
// as well as an extra slash in front.
if(vHome.contains("\\")) {
vHome = "/" + vHome.replace(File.separator,"/");
}
return "file:" + vHome;
}


public void onNew(INKFRequestContext aContext) throws Exception {
// One mandatory argument for the onNew function.
String aFlagname = aContext.getThisRequest().getArgumentValue("flagname");


String vFlagDirectory = getHome() + "/.flags/";
String vFlagFile = vFlagDirectory + aFlagname;
Boolean vCreateResponse = false;

if (! (aContext.exists(vFlagDirectory)) ) {
INKFRequest vCreateRequest = aContext.createRequest(vFlagDirectory);
vCreateRequest.setVerb(INKFRequestReadOnly.VERB_NEW);
vCreateResponse = (Boolean)aContext.issueRequest(vCreateRequest);

if (! vCreateResponse) {
throw new NKFException("can not create directoy " + vFlagDirectory);
}
}

if (! (aContext.exists(vFlagFile)) ) {
INKFRequest vCreateRequest = aContext.createRequest(vFlagFile);
vCreateRequest.setVerb(INKFRequestReadOnly.VERB_NEW);
vCreateResponse = (Boolean)aContext.issueRequest(vCreateRequest);

if (! vCreateResponse) {
throw new NKFException("can not create file " + vFlagFile);
}
}
aContext.createResponseFrom(vFlagFile);
}

public void onDelete(INKFRequestContext aContext) throws Exception {
// One mandatory argument for the onDelete function.
String aFlagname = aContext.getThisRequest().getArgumentValue("flagname");


String vFlagDirectory = getHome() + "/.flags/";
String vFlagFile = vFlagDirectory + aFlagname;
Boolean vDeleteResponse = true;


if (aContext.exists(vFlagFile) ) {
INKFRequest vDeleteRequest = aContext.createRequest(vFlagFile);
vDeleteRequest.setVerb(INKFRequestReadOnly.VERB_DELETE);
vDeleteResponse = (Boolean)aContext.issueRequest(vDeleteRequest);
}
aContext.createResponseFrom(vDeleteResponse);
}


public void onExists(INKFRequestContext aContext) throws Exception {
// One mandatory argument for the onExists function.
String aFlagname = aContext.getThisRequest().getArgumentValue("flagname");


String vFlagDirectory = getHome() + "/.flags/";
String vFlagFile = vFlagDirectory + aFlagname;

aContext.createResponseFrom(aContext.exists(vFlagFile));
}
}

Have fun !