2016/12/02

Into the wasteland

However strange this may sound, Resource Oriented Computing is not used everywhere yet. While this is only a matter of time of course, it may cause some inconvenience during that period.

Hence the need for transports. Transports bring external events into the ROC abstraction and take responses back out.


http://www.deviantart.com/art/Wasteland-Truck-521012152

The obvious choice when it comes to transports is http. And sure, out-of-the-box NetKernel comes with two http-transports enabled, one running on port 8080, also known as the Frontend Fulcrum and one running on port 1060, also known as the Backend Fulcrum. All other transports are pretty much forgotten/overlooked.

To change that we are going to take a look at the SMTPTransport today. In human words, I am going to send mails to NetKernel and I want NetKernel to do something with them. To make things even more interesting, the body of the mail is going to be a fully specified declarative request (no worries, I'll show an example further on) that I want NetKernel to execute.

You will need to install the email-core module for this. This is not installed by default but it is readily available in Apposite and has been there for quite a long time. After installing it you can take a look at the documentation here.

In short, the SMTPTransport listens on port 25000 (you can configure this) and turns incoming mails into smtp:message requests. We can very easily handle those with a mapper.

<mapper>
 <config>
  <endpoint>
   <grammar>
    <active>
     <identifier>smtp:message</identifier>
     <argument name="from" desc="from address" min="1" max="1" />
     <argument name="to" desc="to address" min="1" max="1" />
    </active>
   </grammar>

   <request>
    <identifier>active:groovy</identifier>
    <argument name="operator">res:/resources/groovy/message.groovy</argument>
    <argument name="from" method="as-string">[[arg:from]]</argument>
    <argument name="to" method="as-string">[[arg:to]]</argument>
   </request>
  </endpoint>
 </config>
 <space>
  <endpoint>
   <id>com:ebc:smtp:requestprocessor:transport</id>
   <prototype>SMTPTransport</prototype>
   <private/>
  </endpoint>

  <import>
   <uri>urn:com:ebc:smtp:requestprocessor:import</uri>
   <private/>
  </import>
 </space>
</mapper>

Like I said, what we want to do is treat the incoming message-body as a declarative request. All that takes in the way of code is this :

// arguments
Document aBodyDoc = (Document)aContext.source("emailMessage:/part/0/body",Document.class);
//

// processing
ILogger vLogger=aContext.getKernelContext().getKernel().getLogger();
RequestBuilder vBuilder = new RequestBuilder(aBodyDoc.getDocumentElement(), vLogger);
INKFRequest vRequest = vBuilder.buildRequest(aContext,null,null);
aContext.issueAsyncRequest(vRequest);
//

// response
INKFResponse vResponse = aContext.createResponseFrom("smtp request processed");
vResponse.setMimeType("text/plain");
vResponse.setExpiry(INKFResponse.EXPIRY_ALWAYS);
//

I admit, this is not production-ready code (no error handling for one), but it does the job quite nicely. What remains to be shown is my import rootspace :

<rootspace
name="ebc smtp requestprocessor import"
public="false"
uri="urn:com:ebc:smtp:requestprocessor:import">

<fileset>
<!-- contains groovy resources -->
<regex>res:/resources/groovy/.*</regex>
</fileset>

<import>
<!-- contains SMTPTransport -->
<uri>urn:org:netkernel:email:core</uri>
</import>

<import>
<!-- contains DOMXDAParser -->
<uri>urn:org:netkernel:xml:core</uri>
</import>

<import>
<!-- contains active:groovy -->
<uri>urn:org:netkernel:lang:groovy</uri>
</import>

<import>
<!-- contains SimpleImportDiscovery, DynamicImport -->
<uri>urn:org:netkernel:ext:layer1</uri>
</import>

<endpoint>
<prototype>SimpleImportDiscovery</prototype>
<grammar>active:SimpleImportDiscovery</grammar>
<type>smtprequestprocessor</type>
</endpoint>

<endpoint>
<prototype>DynamicImport</prototype>
<config>active:SimpleImportDiscovery</config>
</endpoint>

</rootspace>

Lets quickly run through that. Groovy module and sources ... check. The email-core module ... check. So what do we need the xml-core for ? Well, actually the email message body is a binary stream. And as you can see in our code we need an Element (that would be a org.w3c.dom.Element). Transreption comes to our aid and the xml-core module contains the transreptor we need.

All the above speaks for itself but you may wonder why I have a dynamic import set up in there. Well, remember that we want to execute requests ? Here's an example :

<request>
    <identifier>active:csvfreemarkerasync</identifier>
    <argument method="as-string" name="in">file:/C:/nkwork/UKCOMPANY/input/BasicCompanyData-2016-11-01-part1_5.csv</argument>
    <argument method="as-string" name="out">file:/C:/nkwork/UKCOMPANY/output/company_1_5.ttl</argument>
    <argument method="as-string" name="template">file:/C:/nkwork/UKCOMPANY/template/company.freemarker</argument>
    <argument method="as-string" name="separator">,</argument>
    <header name="forget-dependencies">
        <literal type="boolean">true</literal>
    </header>
    <representation>java.lang.String</representation>
</request>

There is however no way that my message handler is going to be able to launch that request, active:csvfreemarkerasync in this case, without having access to it. Now, there's several ways we can make this happen ... I rather like the dynamic import way.

So how does it work ? Launch your module and it will go and listen at port 25000. To send it a mail we'll go commandline. For Windows, we'll use CMail.

cmail -from:practical.netkernel@gmail.com -to:practical.netkernel@gmail.com -host:localhost:25000 -body-file:request.txt

The file request.txt obviously contains ... the request. There is no more to it. By the way, with powershell you can even do it without an extra tool :

$EmailFrom = "practical.netkernel@gmail.com"
$EmailTo = "practical.netkernel@gmail.com"
$Subject = "sending a request to netkernel"
$Body = [IO.File]::ReadAllText(".\request.txt")
$SMTPServer = "localhost"
$SMTPClient = New-Object Net.Mail.SmtpClient($SmtpServer, 25000)
$SMTPClient.EnableSsl = $false
$SMTPClient.Credentials = New-Object System.Net.NetworkCredential("none", "needed");
$SMTPClient.Send($EmailFrom, $EmailTo, $Subject, $Body)

For Linux too there are many many many ways to do this (the following requires heirloom-mailx) :

cat /var/tmp/request.txt | mailx -v -S smtp="localhost:25000" practical.netkernel@gmail.com

To close this post, if you are wondering why I'm using commandline-tools ... well the above makes adding NetKernel requests to your automation-chain (or DevOpsBot, or whatever) very simple.