Practical NetKernel: April 2012

It has been brought to my attention that I can write posts here on practically anything that is closely or remotely linked to NetKernel and even IT in general. Today I want to discuss something with you that has been a thorn in my side for years. Logging.

Some 19 years ago, when I - fresh out of school - started in IT, I also started getting daily listings (on paper) with loggings. I would get an overview of the program-identifiers I requested, another list would be almost identical but contain file-identifiers and so on.

My listings were peanuts compared to the ones the old hands got. In fact, you could easily determine the status/position of someone by looking at the amount of listings with loggings he/she got. The higher the position, the higher the pile.

Many a lament of many a tree went unheard as it was cut down to support this habit. And of course, most of those listings got no more than a single glance before being thrown away.

IT grew up, companies became environmentally aware, I started doing database and system administration, paper listings had to go, mails and chats and log consoles ... replaced them. Until this very day I come in in the morning, take one look at the loggings and run to the toilet or the coffee machine (or both, order may depend upon the quality of the coffee).

Please, please, log as sparsely as you can :
Why ? Overhead is overhead. Even if it is small.
Why ? Storage costs money. Even if it is not a lot. Do you really want to have more loggings about your data (often repeating the data) than you have data ?
Why ? Well, at the end of the day somebody has to look at all that data. And I've spend my share of nights sitting next to the console jockey (I was there for a rollout of something usually and ... I admit, they had better coffee than we had) wading through all kinds of messages popping up. I also sat next to the audit person going through one nights worth of database changes/logins/logouts to find the culprit of a certain rather clever (not so clever that I didn't notice) change on a top secret database.

NetKernel has a nice logging system. By all means check it out and use it. Logging in NetKernel is asynchronously and used in moderation will have a negligible impact on your application. Do spare a thought though for the poor human that will have to check them !

P.S. If you did come here to learn about swamp logging, check here.

My schedule has been a little hectic the last couple of weeks, but this afternoon I finally found time to revisit the Space Listing. So, without further ado, here's the rootspace that contains my version of it :

<rootspace

name="System Accessors"

public="true"

uri="urn:org:elbeesee:ext:system:accessors">

<id>system.SpaceStaticResourceList.accessor</id>

<class>org.elbeesee.ext.system.SpaceStaticResourceListAccessor</class>

<identifier>active:ssrls</identifier>

</active>

</grammar>

</accessor>

<regex>res:/resources/stylesheets/.*</regex>

</fileset>

<uri>urn:org:netkernel:xml:core</uri>

</import>

<uri>urn:org:netkernel:ext:layer1</uri>

</import>

</rootspace>

Nothing special there, same definition as the original active:sls. It's in the code that things are different, so lets have a look at that :

package org.elbeesee.ext.system;

// Author: Tom Geudens

// Date : 2012/04/08

// The usual suspects for an accessor.

import org.netkernel.layer0.nkf.*;

import org.netkernel.layer0.meta.impl.SourcedArgumentMetaImpl;

import org.netkernel.module.standard.endpoint.StandardAccessorImpl;

// Processing.

import java.io.File;

import java.net.URI;

import java.util.Iterator;

import org.netkernel.container.IKernel;

import org.netkernel.module.standard.StandardSpace;

import org.netkernel.layer0.boot.BootUtils;

import org.netkernel.layer0.representation.IHDSNode;

import org.netkernel.layer0.representation.impl.HDSBuilder;

import org.netkernel.layer0.urii.SimpleIdentifierImpl;

import org.netkernel.urii.ISpaceWithIdentity;

import org.netkernel.urii.impl.Version;

public class SpaceStaticResourceListAccessor extends StandardAccessorImpl {

public SpaceStaticResourceListAccessor() {

this.declareThreadSafe();

this.declareArgument(new SourcedArgumentMetaImpl("space",null,null,new Class[] {String.class}));

this.declareArgument(new SourcedArgumentMetaImpl("version",null,null,new Class[] {String.class}));

}

private IHDSNode listDirectory(INKFRequestContext aContext, File aDirectoryFile) {

return listDirectory(aContext, aDirectoryFile, "res:/");

}

private IHDSNode listDirectory(INKFRequestContext aContext, File aDirectoryFile, String aResourcePath) {

HDSBuilder lDirectoryBuilder = new HDSBuilder();

File[] lDirectoryFiles = aDirectoryFile.listFiles();

lDirectoryBuilder.pushNode("resources");

for (int i=0; i < lDirectoryFiles.length; i++) {

if (lDirectoryFiles[i].isDirectory()) {

String lResourcePath = aResourcePath + lDirectoryFiles[i].getName() + "/";

lDirectoryBuilder.importChildren(listDirectory(aContext,lDirectoryFiles[i],lResourcePath).getFirstNode("/resources"));

}

else {

lDirectoryBuilder.addNode("resource", aResourcePath + lDirectoryFiles[i].getName());

}

lDirectoryBuilder.popNode();

return lDirectoryBuilder.getRoot();

}

public void onSource(INKFRequestContext aContext) throws Exception {

// One mandatory argument

String aSpaceIdentifier = aContext.getThisRequest().getArgumentValue("space");

// One optional argument

Version aVersion = null;

if (aContext.getThisRequest().argumentExists("version")) {

aVersion = new Version(aContext.getThisRequest().getArgumentValue("version"));

}

// Processing

IKernel lKernel = null;

ISpaceWithIdentity lSpace = null;

StandardSpace lSS = null;

String lSource;

lKernel = aContext.getKernelContext().getKernel();

lSpace = lKernel.getSpace(new SimpleIdentifierImpl(aSpaceIdentifier), aVersion, aVersion);

if (lSpace == null) {

throw new NKFException("Space is not found");

}

if (!(lSpace instanceof StandardSpace)) {

throw new NKFException ("Space is not a standard module space");

}

lSS = (StandardSpace)lSpace;

lSource = lSS.getOwningModule().getSource().toString();

lSource = BootUtils.fixURIString(lSource);

IHDSNode lModuleXML = null;

IHDSNode lRootSpace = null;

if (lSource.startsWith("file:")) {

lModuleXML = aContext.source(lSource + "module.xml",IHDSNode.class);

}

INKFRequest subrequest = aContext.createRequest("active:xslt");

subrequest.addArgumentByValue("operand", lModuleXML);

subrequest.addArgument("operator", "res:/resources/stylesheets/module.xsl");

subrequest.addArgumentByValue("spaceid", aSpaceIdentifier);

subrequest.setRepresentationClass(IHDSNode.class);

lRootSpace = (IHDSNode)aContext.issueRequest(subrequest);

File lSourceFile = new File(URI.create(lSource));

IHDSNode lSourceContent = listDirectory(aContext,lSourceFile);

HDSBuilder lValidResources = new HDSBuilder();

lValidResources.pushNode("resources");

Iterator<IHDSNode> lIterator = lRootSpace.getNodes("/rootspace/fileset").iterator();

while(lIterator.hasNext()) {

IHDSNode lFileset = lIterator.next();

String lGlob = null;

String lRegex = null;

lGlob = (String)lFileset.getFirstValue("glob");

lRegex = (String)lFileset.getFirstValue("regex");

if (lGlob != null) {

System.out.println("fileset glob = " + lGlob);

}

if (lRegex != null) {

Iterator<IHDSNode> lIterResource = lSourceContent.getNodes("/resources/resource").iterator();

while (lIterResource.hasNext()) {

IHDSNode lResource = lIterResource.next();

String lResourceString = (String)lResource.getValue();

if (lResourceString.matches(lRegex)) {

lValidResources.addNode("resource", lResourceString);

}

lValidResources.popNode();

// One response

INKFResponse response = null;

// response = aContext.createResponseFrom(lRootSpace.getRoot());

// response = aContext.createResponseFrom(lSourceContent.getRoot());

response = aContext.createResponseFrom(lValidResources.getRoot());

}

What happens here ?
1) Determine the module of the rootspace that is passed as an argument.
2) Extract the rootspace from the module.xml of that module.
3) Determine all possible file resources of the module.
4) Loop through the filesets defined in the rootspace and use those to filter the file resources so only the valid ones remain.

This is the stylesheet that is used to extract the rootspace from module.xml :

<xsl:stylesheet xmlns:xsl="http://www.w3.org/1999/XSL/Transform"

xmlns:nk="http://netkernel.org"

version="1.0">

<xsl:output method="xml"

indent="yes"

encoding="UTF-8"

omit-xml-declaration="yes"/>

<xsl:param name="spaceid" nk:class="java.lang.String" />

<xsl:template match="/module">

<xsl:copy-of select="rootspace[@uri=$spaceid]"/>

</xsl:template>

</xsl:stylesheet>

And ... done.

Or so it seems, you might have noticed I labeled this entry "part 1". There's some work to be done yet :
* Next to regex, a fileset can also be glob or grammar defined. All three should be handled (at the moment only regex is).
* A fileset definition can contain a rewrite. I want to get the real valid resources, so we need to provide a rewrite too.
* A module can be expanded or jarred. At the moment the accessor only works for expanded modules. I want jars too.

As you can see there's room for a "part 2" (and beyond ?). Watch this space ! There might be other requirements you are interested in. Let me know and I'll add them.

Since this was a lot of code real fast, my next blog entry will once again be a philosophical one. You've been warned !

Practical NetKernel

2012/04/20

logging

2012/04/08

space listing - the future - part 1