interesting times

If you're looking for the episode on logging, soon my friend, soon. 
If you've noticed my last post of 2012 was also titled interesting times, you are correct.

A year ago I promised Tony Butterfield that I would force a breakthrough for Resource Oriented Computing / NetKernel in 2013. With about eight hours left in 2013 here in Brussels, Belgium, I must admit defeat as far as that claim goes.

People that know me personally know that I do not take defeat - in any area - lightly or graciously. 

It has been a good year but not the exceptional year that I envisioned when I made that claim.

So, here are some of my ROC related targets for 2014 :

• At least one ROC backed game playable in one of the mainstream social networks.
• Positioning ROC as the best solution for Linked (Open) Data projects.
• Positioning ROC as the green IT solution. With the point of no return for climate change set in 2015, computing must start contributing too.
• A new book.
• <any target you, my constant readers, care to set ?>
• ...

Best wishes and see you in 2014 !
 

back to the beginning ... unit test

In IT a lot of fuss is made these days around testing and logging. The former is used to avoid introducing bugs into software and the latter is used to be able to find the inevitable bugs that are introduced into software.

And no, the above is not said in a light or satirical tone. I wouldn't dare, the industry has gotten quite anal on these subjects, as - to give one example - Rick Hickey found out when he compared Test Driven Development (TDD) to learning to drive by bumping into the guardrails. Testing and logging are to be taken very serious and they have become big moneymakers in their own right.

This does say a lot about the IT industry as a whole, but I'm not going there either ...

Today I'm going to discuss testing within the Resource Oriented Computing worldview, logging will be in the spotlight next time.

Why test ?

• unit test : to check if a piece of code delivers on its promise 
• integration / system test : to check if a piece of code works in combination/interaction with other pieces of code in a complex system
• regression test : to check if a piece of code still works after a change has been introduced elsewhere in a complex system 

You probably have other definitions in your textbook, mine go back to my initial IT training in the early nineties. Yes, I am that old.

When you're working with ROC/NetKernel, you'll soon notice that there's less fuss about testing. Reasons :

• less code
• smaller components
• less complexity

Another - maybe less obvious - reason is that constraints are typically applied at the end of the Resource Driven Development (RDD) cycle - more on Construct Compose Constrain in a later post - not at the beginning. RDD and TDD approach the same problems from opposite sides.

Time to discuss the testing framework in NetKernel. Yes there is one and yes, it works with resources. Hey, what did you expect ? Last time I created a service for the Amazon Web Service tools. You can find the source for that module, urn.com.amazonaws.sdk on Github.

You'll find three spaces in the module.xml. The public urn:com:amazonaws:sdk space that contains the service, the private urn:com:amazonaws:sdk:import space that contains the internally used resources and tools and the public urn:com:amazonaws:sdk:unittest space. Can you guess what that last one is for ?

There is an ongoing debate whether or not the unit tests should be included in a module or live in a separate module. As with the naming of spaces (I've gotten some comments on that) ... whatever suits you best. If you are adamant about not deploying tests to production, you might want to split them off. If you want to have tests for your private spaces - as I do - you must include them.

The testing framework works - like the fulcrums - with a dynamic import. It searches for public spaces with a res:/etc/system/Tests.xml resource and pulls those spaces inside.

It is a bit confusing given the tag-names, but the resource describes groups (!) of tests. Some people prefer one group for the whole module, I usually have one group per space that I want to test :
            <tests>
                <test>
                    <id>test:amazonaws:sdk:import</id>
                    <name>amazonaws sdk import unittest</name>
                    <desc>amazonaws sdk import unittest</desc>
                    <uri>res:/resources/unittest/sdk-import.xml</uri>
                </test>
                <test>
                    <id>test:amazonaws:sdk</id>
                    <name>amazonaws sdk unittest</name>
                    <desc>amazonaws sdk unittest</desc>
                    <uri>res:/resources/unittest/sdk.xml</uri>
                </test>
            </tests>
 
Each group contains the uri of the resource with the actual list of tests for that group, in this case res:/resources/unittest/sdk-import.xml and res:/resources/unittest/sdk.xml. Lets look at the former :
            <testlist>
                <test name="SOURCE regions.xml resource">
                    <request>
                        <identifier>res:/resources/xml/regions.xml</identifier>
                        <verb>SOURCE</verb>   
                    </request>
                    <assert>
                        <notNull/>
                    </assert>
                </test>
            </testlist>
 
And the latter :
            <testlist>
                <test name="SOURCE s3 eu-west-1 resource">
                    <request>
                        <identifier>res:/amazonaws/baseurl/s3-eu-west-1</identifier>
                        <verb>SOURCE</verb>   
                    </request>
                    <assert>
                        <stringEquals>s3-eu-west-1.amazonaws.com</stringEquals>
                    </assert>
                </test>
            </testlist>

These test-lists can contain as many tests as you deem necessary. In this case I have one in each. I check if regions.xml is there and I check one case of the service.

If you've been properly indoctrinated you'll probably wince in agony at the above. Only two tests ? What about some more cases ? Bear with me, if you want to add more, you can.

Let us have a look at the XUnit Tests tool first.

I filtered the results so you only see the two relevant test-lists. If you want to run a lot of tests at once, select the test-lists you want to run :

and select execute :

Or you can select one test-list with view :

and execute that :

Note that this is not only a GUI, it is quite easy to add this to an automation and get the test-results as XML documents. Everything is a resource !

I'm not going to run through all the XUnit Testing documentation, but lets add a couple more tests so you all feel a bit more at ease. First in res:/resources/unittest/sdk-import.xml :
            <testlist>
                <test name="SOURCE regions.xml resource">
                    <request>
                        <identifier>res:/resources/xml/regions.xml</identifier>
                        <verb>SOURCE</verb>   
                    </request>
                    <assert>
                        <notNull/>
                    </assert>
                </test> 
                <test name="SOURCE regions.dat resource">
                    <request>
                        <identifier>res:/resources/xml/regions.dat</identifier>
                        <verb>SOURCE</verb>   
                    </request>
                    <assert>
                        <exception/>
                    </assert>
                </test>
            </testlist>
 
If you then run the tests for that list this is what you get :

and next in res:/resources/unittest/sdk.xml :
            <testlist>
                <test name="SOURCE s3 eu-west-1 resource">
                    <request>
                        <identifier>res:/amazonaws/baseurl/s3-eu-west-1</identifier>
                        <verb>SOURCE</verb>   
                    </request>
                    <assert>
                        <stringEquals>s3-eu-west-1.amazonaws.com</stringEquals>
                    </assert>
                </test> 
                <test name="SOURCE s4 eu-west-1 resource">
                    <request>
                        <identifier>res:/amazonaws/baseurl/s4-eu-west-1</identifier>
                        <verb>SOURCE</verb>   
                    </request>
                    <assert>
                        <exception/>
                    </assert>
                </test>
             </testlist>

If you then run the tests for that list this is what you get :

Oops ... that wasn't quite what I intended. What went wrong here ? Select exec to see the actual result of the request :

The grammar of the res:/amazonaws/baseurl/s4-eu-west-1 request is valid, so the regions.xml resource is passed through the xslt transformation and comes up with unknown as the result. A valid result. Not an exception. So lets change our test-list again :
            <testlist>
                <test name="SOURCE s3 eu-west-1 resource">
                    <request>
                        <identifier>res:/amazonaws/baseurl/s3-eu-west-1</identifier>
                        <verb>SOURCE</verb>   
                    </request>
                    <assert>
                        <stringEquals>s3-eu-west-1.amazonaws.com</stringEquals>
                    </assert>
                </test> 
                <test name="SOURCE s4 eu-west-1 resource">
                    <request>
                        <identifier>res:/amazonaws/baseurl/s4-eu-west-1</identifier>
                        <verb>SOURCE</verb>   
                    </request>
                    <assert>
                        <stringEquals>unknown</stringEquals>
                    </assert>
                </test>
                <test name="SOURCE blabla resource">
                    <request>
                        <identifier>res:/amazonaws/baseurl/blabla</identifier>
                        <verb>SOURCE</verb>   
                    </request>
                    <assert>
                        <exception/>
                    </assert>
                </test>
            </testlist>

And the result :

Much better. And here's my opinion : the extra test I added to res:/resources/unittest/sdk-import.xml is useless as it doesn't test functionality, the two extra tests I added to res:/resources/unittest/sdk.xml are useful as they do test functionality. It may not always be so clear and if you feel you've got to err on the side of caution, by all means do. Just remember that your purpose is solving problems, not writing tests.

Right, that's a lot of information again. Don't forget to look through the XUnit Test documentation, you'll find that you can adapt and extend it to your own preferences.

As I promised, next time I'll discuss logging and as a bonus I'll also explain what that Limiter endpoint (did you notice it in the unit test space ?) does. Keep on ROCing in the mean time !