different fulcrum ports

I'm very busy preparing the sessions I'll be giving next week @BestBuy HQ in Minneapolis. And you might expect that by now this is like a cut cake (or whatever the English equivalent for that idiom is) for me, no surprises inside ...

Not so, NetKernel surprises me at least once every single day. So when I reviewed the Can I run multiple instances of NetKernel on the same host but on different ports question with Peter Rodgers (aka Thor, don't be misguided by his new twitter account @transrept where he supposedly shows his softer side), his solution (the answer is Yes) was different from mine.

I would - obviously I thought - go into the fulcrum modules and change the port in the [moduleroot]/etc/HTTPServerConfig.xml file :

<SystemProperty name="netkernel.http.frontend.port" default="8081" />
<SystemProperty name="netkernel.http.backend.port" default="1061" />

How could I have known (read the manuals Tom, read the manuals ...) that a SystemProperty tag in a Jetty configuration actually means Java SystemProperty ?

So much more conveniently (and way less intrusive) you can change the way the NetKernel instance is started by changing [install]/bin/jvmsettings.cnf :

-Xmx512m -Xms512m -XX:SoftRefLRUPolicyMSPerMB=100 -Dfile.encoding=UTF-8 -Dnetkernel.http.frontend.port=8081 -Dnetkernel.http.backend.port=1061

The fulcrums will pick this up and ignore the default (another hint that should have told me there was another way to do it) attribute.

Enjoy !


P.S. Of course full credits for this fine tip go to Thor (I wouldn't dare otherwise, have you seen the hammer he carries ?) !
P.P.S. Excellent post by Tony Butterfield (no known aka's) on his Durable Scope blog.

outside the comfort zone

One of the first things that I liked in NetKernel was that I could easily create applications and make them accessible on the intranet/internet without knowing a whole lot about webserver deployment, jar and war, web.xml, ...

I still don't. I come from a system administrator background. While I can drop files in the directories I'm told to drop them in, I haven't really got a clue what the average (non NetKernel) Java webapplication project looks like, where the classes are, where the static content lives, ... In my everyday explorations, NetKernel abstracts all that for me.

That is of course no excuse ! So when I recently heard that Vaadin was in the comfortzone of quite a lot of developers that would potentially benefit from using NetKernel, I challenged myself to either :

• Run a Vaadin application inside NetKernel
• Run NetKernel inside a Vaadin application

Now, using NetKernel from any Java application is trivial, you can point your datasources (or whatever) to a NetKernel exposed endpoint. But that was not the challenge.

I quickly figured out that running a Vaadin application inside NetKernel was beyond me. I could make it so that it ran alongside the other stuff in the fulcrums (= Jetty Webserver), but inside ... no. Maybe it can be done and if anybody knows how I'd be obliged if you tell me, but I could not make that work.

Running NetKernel inside Vaadin however had to work. After all this usage mode is documented in the embedded-demo (courtesy of 1060 Research, just so the thunder god doesn't accuse me of stealing his thunder ... I don't want to end up like Prometeus). And it did ! But - for me - it was not trivial to do it, so here's the low down :

1. Since you are not serving in NetKernel, you need another webserver. I chose Apache Tomcat, which you can find here.
2. The Vaadin website has an explicit HelloVaadin example, created with the Vaadin Eclipse Plugin. So I started up Eclipse and installed that plugin.
3. Back up a step or I guarantee some trouble down the line. I was running the standard Eclipse installation. That got me all kinds of strange errors when creating Vaadin projects, so I installed a fresh Java EE Eclipse and installed the Vaadin Eclipse Plugin on that.
4. With all that done I followed the HelloVaadin example on the Vaadin site to create a new Vaadin project in Eclipse and pretty soon I had that running inside Eclipse using my installed Tomcat.
5. So far so good and I decided that the "Hello Vaadin" message had to be replaced with a NetKernel resource calling a groovy program.
6. You can download my NetKernel module for that here. You'll notice that the module contains an etc directory unused by the module itself that contains a kernel.properties file and a modules.conf file. The former will be used by the embedded NetKernel, the latter contains the locations of the modules that will be known inside the embedded NetKernel. Adapt it to your locations.
7. Time to modify our HelloVaadin example in Eclipse. Under Java Resources -> Libraries you need to add the NetKernel library. And you also need to copy the jars from that to WebContent -> WEB-INF -> lib where you'll also find the vaadin jar itself.
8. Under the Java Resources -> src -> <yourpackagename> you'll find the java program itself. I turned that into this :
   
   package com.example.hellovaadin;
   
   import com.vaadin.Application;
   import com.vaadin.ui.*;
   
   import com.ten60.netkernel.cache.se.representation2.ConcurrentCache;
   import com.ten60.netkernel.cache.se.resolution.ResolutionCache;
   import org.netkernel.container.ILogger;
   import org.netkernel.container.impl.Kernel;
   import org.netkernel.layer0.boot.IModuleFactory;
   import org.netkernel.layer0.boot.ModuleManager;
   import org.netkernel.layer0.logging.LogManager;
   import org.netkernel.layer0.nkf.INKFRequestContext;
   import org.netkernel.layer0.nkf.NKFException;
   import org.netkernel.layer0.nkf.impl.NKFTransportContextImpl;
   import org.netkernel.layer0.urii.SimpleIdentifierImpl;
   import org.netkernel.layer0.util.PropertyConfiguration;
   import org.netkernel.layer0.tools.ExtraMimeTypes;
   import org.netkernel.module.standard.StandardModuleFactory;
   import org.netkernel.urii.IIdentifier;
   import org.netkernel.urii.ISpace;
   
   import java.io.BufferedReader;
   import java.io.InputStreamReader;
   import java.net.URI;
   import java.net.URL;
   
   public class HellovaadinApplication extends Application {
       private static int SYSTEM_RUN_LEVEL = 1;
       private ModuleManager mModuleManager;
       private ConcurrentCache mRepresentationCache;
       private INKFRequestContext mContext;
   
       private void startNetKernel() throws Exception {
           // Create a new micro-kernel.
           Kernel kernel = new Kernel();
   
           // Get system classloader so we can load local physical resources
           ClassLoader classLoader = HellovaadinApplication.class.getClassLoader();
   
           // NetKernel adds support for missing MIME types to the JDK.
           ExtraMimeTypes.getInstance();
   
           // NetKernel requires a logger
           ILogger logger = new LogManager(null).getKernelLogger();
   
           // Set kernel properties.
           // Here read the kernel properties from the property file in the
           // classpath.
           URL kernelProperties = new URL("file:/X:/NK5/project-modules/urn.org.elbeesee.tutorial.embedded.module-1.0.0/etc/kernel.properties");
           PropertyConfiguration config = new PropertyConfiguration(kernelProperties, logger);
           config.setProperty("netkernel.boot.time", Long.toString(System.currentTimeMillis()));
           kernel.setConfiguration(config);
           kernel.setLogger(logger);
   
           // >>>>> Start of optional cache configuration
           // These two caches are optional, if used then add both as a pair
   
           // Representation Cache stores computed resource representations
           mRepresentationCache = new ConcurrentCache(kernel);
           kernel.setRepresentationCache(mRepresentationCache);
           kernel.addConfigurationListener(mRepresentationCache);
   
           // Resolution Cache stores resolved endpoints
           ResolutionCache rcache = new ResolutionCache(kernel);
           kernel.setResolutionCache(rcache);
           kernel.addConfigurationListener(rcache);
           // <<<<< End of optional cache loading code
   
           // Instantiate the modules factories - only use StandardModule here.
           IModuleFactory[] factories = new IModuleFactory[] { new StandardModuleFactory() };
   
           // Create a Module Manager
           mModuleManager = new ModuleManager(kernel, factories);
           InputStreamReader is = new InputStreamReader((new URL("file:/X:/NK5/project-modules/urn.org.elbeesee.tutorial.embedded.module-1.0.0/etc/modules.conf")).openStream());
            BufferedReader r = new BufferedReader(is);
           String line;
   
           // Process the module configuration file
           while ((line = r.readLine()) != null)
           {
               line = line.trim();
               if (!line.startsWith("#") && line.length() > 0)
               {
                   URI source = new URI(line);
                   // Tell the module manager about the module and set its runlevel
                   mModuleManager.addModule(source, SYSTEM_RUN_LEVEL);
               }
           }
   
           // Set run level - this commissions all modules <= RUNLEVEL and moves
           // the kernel to active state.
           mModuleManager.setRunLevel(SYSTEM_RUN_LEVEL);
   
           // Kernel is now live and modules are commissioned!
   
           // Now create a transport context so that we can make requests.
           // Create a reference to the rootspace of our demo module
           IIdentifier spaceIdentifier = new SimpleIdentifierImpl("urn:org:elbeesee:tutorial:embedded:module:public");
           ISpace applicationRootSpace = kernel.getSpace(spaceIdentifier, null, null);
   
           // Create the INKFRequestContext
           mContext = new NKFTransportContextImpl(kernel, applicationRootSpace, this);
           // We're now ready to make requests!
       }
   
       private String getHelloWorld() throws NKFException {
           return mContext.source("res:/hellovaadin", String.class);
       }
       private void stopNetKernel()
       {
           if (mModuleManager != null)
           {    // This blocks further requests and then shuts down the kernel
               mModuleManager.stop();
           }
           if (mRepresentationCache != null)
           {    // This stops the monitor thread in the cache
               mRepresentationCache.stop();
           }
       }
   
       public void close() {
           try {
               stopNetKernel();
           }
           catch (Exception e) {
               e.printStackTrace();
           };
           super.close();
       }
   
       public void init() {
           try {
             startNetKernel();
           }
           catch (Exception e) {
               e.printStackTrace();
           };
           Window mainWindow = new Window("Hellovaadin Application");
           Label label = null;
           try {
               label = new Label(getHelloWorld());
           } catch (NKFException e) {
               e.printStackTrace();
           }
           mainWindow.addComponent(label);
           setMainWindow(mainWindow);
       }
   }
   
   
   
9. If this compiles you've done the setup correctly. All that remains is to run it and lo and behold ... the message is served through a NetKernel groovy endpoint (the console will tell you if you did something wrong, you should see a small NetKernel instance being booted up in there). 

That is it. Well, that was a whole lot and if somebody tells me the learning curve of NetKernel is steep again, try doing the above without a lot of knowledge about Java webapplications (like I did) !

One more quid pro quo. The above is not perfect. Files modules.conf and kernel.properties should be loaded from the application's classpath rather then from fixed files. Since this was my first Vaadin experience I didn't know (nor could I quickly find it) how to do that. Etc.

Enjoy !

great minds do not think alike

A London (where I taught a NetKernel class @BSI  - British Standards Institute - last week) university has flyers with that slogan. And it - as you can see - took hold in my mind. I perceive it to be more true than it's often used counterpart, for if everybody threads the same paths of thought, no progress can be made.

So I had a bit of an uneasy moment when I received links to http://ifttt.com/wtf and http://durablescope.blogspot.be/2012/06/scratching-itch.html from two great minds within a timerange of 5 minutes.

For one is the more general case of the other. The first one was forwarded with the this would be easy to do in NetKernel, would it not intention, the second one is an answer to that ... yes it is.

It may be obvious from my surprise, but there was no communication on this topic between the two. Great minds may not think alike, but they surely think on the same topic.

You probably saw the following map already, it was rather hot when GM decided to pull it's FaceBook budget not too long ago :

Social media are hot and (in some form or another) there to stay. However, they are complex beyond belief (which was one of the reasons given to explain GM's move). We need social mashups. And NetKernel is indeed an excellent platform to build those. So what are you waiting for ?


That's it for today, next week we dive back into the more technical areas.

Cache makes NetKernel go round

It was a long climb up from the earth's core. Today I have a small technical tip. Here's the code snippet we start with :

public void onSource(INKFRequestContext aContext) throws Exception {
   String lFilename = aContext.getThisRequest().getArgumentValue("filename");
   String lResult;

   if (!aContext.exists("pds:/rsk/" + lFilename) {
     String lUUID = UUID.randomUUID().toString();
     aContext.sink("pds:/rsk/" + lFilename, lUUID);
   }

   lResult = aContext.source("pds:/rsk/" + lFilename, String.class);

   INKFResponse response;
   response = aContext.createResponseFrom(lResult);
}

Not too complex is it ? We want to read a persisted file. If it doesn't exist yet we create it first. Is that then my technical tip ? No, there's a small caching problem with the above code. When the file does not exist yet, the result is not cached.

And small though that problem may be, it can add up over time if you do lots of creating. The problem is of course the sink statement. Since that changes the very resource we want to cache the result of the onSource request can not be cached. So here is the solution :

public void onSource(INKFRequestContext aContext) throws Exception {
    String lFilename = aContext.getThisRequest().getArgumentValue("filename");
    String lResult;

    if (!aContext.exists("pds:/rsk/" + lFilename) {
      String lUUID = UUID.randomUUID().toString();
      subrequest = aContext.createRequest("pds:/rsk/" + lFilename);
      subrequest.addPrimaryArgument(lUUID);
      subrequest.setVerb(INKFRequestReadOnly.VERB_SINK);
      subrequest.setHeader("exclude-dependencies",true);
      aContext.issueRequest(subrequest);
   }

   lResult = aContext.source("pds:/rsk/" + lFilename, String.class);

   INKFResponse response;
   response = aContext.createResponseFrom(lResult);
}


By setting the exclude-dependencies on the sink request, that request is ignored when determining the cachebility of the onSource.

Did you notice my smugness ?  The problem is of course the sink statement. Very nice Tom. Only ... the above still doesn't cache. Oops.With both feet back on the ground, I called in the help of the caching ninja himself ... *drumroll* ... Tony Butterfield. I gave him no more than a Visualizer trace, he didn't need (nor want) the code. Obviously the age old use the force, read the source is no longer in effect.

He probably had a good laugh at my expense ... and then said ... good work on the sink, but the remaining problem is of course the exists statement.

The what ? The exists statement ? Here's the final code :

public void onSource(INKFRequestContext aContext) throws Exception {
   String lFilename = aContext.getThisRequest().getArgumentValue("filename");
   String lResult;

   INKFRequest subrequest = aContext.createRequest("pds:/rsk/" + lFilename);
   subrequest.setVerb(INKFRequestReadOnly.VERB_EXISTS);
   subrequest.setHeader("exclude-dependencies",true);
   Boolean lPDSExists = (Boolean)aContext.issueRequest(subrequest);

   if (! lPDSExists) {
     String lUUID = UUID.randomUUID().toString();
     subrequest = aContext.createRequest("pds:/rsk/" + lFilename);
     subrequest.addPrimaryArgument(lUUID);
     subrequest.setVerb(INKFRequestReadOnly.VERB_SINK);
     subrequest.setHeader("exclude-dependencies",true);
     aContext.issueRequest(subrequest);
   }

   lResult = aContext.source("pds:/rsk/" + lFilename, String.class);

   INKFResponse response;
   response = aContext.createResponseFrom(lResult);
}


Yes, the exists was the remaining problem. If the resource does not exist, it obviously starts out as false. Next we sink the resource that is linked to the exists ... and that result (the false) is no longer valid. Which also makes the result of the onSource uncacheable.

Once more  exclude-dependencies comes to the rescue.

Now, before you start using this as your big trick, remember that it is important you understand why and when you want to circumvent the default caching behaviour ! In the above case it may have brought a benefit, but now the code needs more documentation too.

Have a great week !

P.S. In case you think I enjoy punning and envision myself quite the funny guy ... here's a couple of the obvious puns I did not use :
- Code for cache.
- It's all about cache Baby, it's all about you and NetKernel.
- To cache or not to cache, that is the question !
- In Tony Butterfield we trust, all others must use cache.