And no, the above is not said in a light or satirical tone. I wouldn't dare, the industry has gotten quite anal on these subjects, as - to give one example - Rick Hickey found out when he compared Test Driven Development (TDD) to learning to drive by bumping into the guardrails. Testing and logging are to be taken very serious and they have become big moneymakers in their own right.
This does say a lot about the IT industry as a whole, but I'm not going there either ...
Today I'm going to discuss testing within the Resource Oriented Computing worldview, logging will be in the spotlight next time.
Why test ?
- unit test : to check if a piece of code delivers on its promise
- integration / system test : to check if a piece of code works in combination/interaction with other pieces of code in a complex system
- regression test : to check if a piece of code still works after a change has been introduced elsewhere in a complex system
When you're working with ROC/NetKernel, you'll soon notice that there's less fuss about testing. Reasons :
- less code
- smaller components
- less complexity
You'll find three spaces in the module.xml. The public urn:com:amazonaws:sdk space that contains the service, the private urn:com:amazonaws:sdk:import space that contains the internally used resources and tools and the public urn:com:amazonaws:sdk:unittest space. Can you guess what that last one is for ?
There is an ongoing debate whether or not the unit tests should be included in a module or live in a separate module. As with the naming of spaces (I've gotten some comments on that) ... whatever suits you best. If you are adamant about not deploying tests to production, you might want to split them off. If you want to have tests for your private spaces - as I do - you must include them.
The testing framework works - like the fulcrums - with a dynamic import. It searches for public spaces with a res:/etc/system/Tests.xml resource and pulls those spaces inside.
It is a bit confusing given the tag-names, but the resource describes groups (!) of tests. Some people prefer one group for the whole module, I usually have one group per space that I want to test :
<tests>
<test>
<id>test:amazonaws:sdk:import</id>
<name>amazonaws sdk import unittest</name>
<desc>amazonaws sdk import unittest</desc>
<uri>res:/resources/unittest/sdk-import.xml</uri>
</test>
<test>
<id>test:amazonaws:sdk</id>
<name>amazonaws sdk unittest</name>
<desc>amazonaws sdk unittest</desc>
<uri>res:/resources/unittest/sdk.xml</uri>
</test>
</tests>
Each group contains the uri of the resource with the actual list of tests for that group, in this case res:/resources/unittest/sdk-import.xml and res:/resources/unittest/sdk.xml. Lets look at the former :
<testlist>
<test name="SOURCE regions.xml resource">
<request>
<identifier>res:/resources/xml/regions.xml</identifier>
<verb>SOURCE</verb>
</request>
<assert>
<notNull/>
</assert>
</test>
</testlist>
And the latter :
<testlist>
<test name="SOURCE s3 eu-west-1 resource">
<request>
<identifier>res:/amazonaws/baseurl/s3-eu-west-1</identifier>
<verb>SOURCE</verb>
</request>
<assert>
<stringEquals>s3-eu-west-1.amazonaws.com</stringEquals>
</assert>
</test>
</testlist>
These test-lists can contain as many tests as you deem necessary. In this case I have one in each. I check if regions.xml is there and I check one case of the service.
If you've been properly indoctrinated you'll probably wince in agony at the above. Only two tests ? What about some more cases ? Bear with me, if you want to add more, you can.
Let us have a look at the XUnit Tests tool first.
I filtered the results so you only see the two relevant test-lists. If you want to run a lot of tests at once, select the test-lists you want to run :
and select execute :
Or you can select one test-list with view :
and execute that :
Note that this is not only a GUI, it is quite easy to add this to an automation and get the test-results as XML documents. Everything is a resource !
I'm not going to run through all the XUnit Testing documentation, but lets add a couple more tests so you all feel a bit more at ease. First in res:/resources/unittest/sdk-import.xml :
<testlist>
<test name="SOURCE regions.xml resource">
<request>
<identifier>res:/resources/xml/regions.xml</identifier>
<verb>SOURCE</verb>
</request>
<assert>
<notNull/>
</assert>
</test>
<test name="SOURCE regions.dat resource">
<request>
<identifier>res:/resources/xml/regions.dat</identifier>
<verb>SOURCE</verb>
</request>
<assert>
<exception/>
</assert>
</test>
</testlist>
If you then run the tests for that list this is what you get :
and next in res:/resources/unittest/sdk.xml :
<testlist>
<test name="SOURCE s3 eu-west-1 resource">
<request>
<identifier>res:/amazonaws/baseurl/s3-eu-west-1</identifier>
<verb>SOURCE</verb>
</request>
<assert>
<stringEquals>s3-eu-west-1.amazonaws.com</stringEquals>
</assert>
</test>
<test name="SOURCE s4 eu-west-1 resource">
<request>
<identifier>res:/amazonaws/baseurl/s4-eu-west-1</identifier>
<verb>SOURCE</verb>
</request>
<assert>
<exception/>
</assert>
</test>
</testlist>
If you then run the tests for that list this is what you get :
Oops ... that wasn't quite what I intended. What went wrong here ? Select exec to see the actual result of the request :
<testlist>
<test name="SOURCE s3 eu-west-1 resource">
<request>
<identifier>res:/amazonaws/baseurl/s3-eu-west-1</identifier>
<verb>SOURCE</verb>
</request>
<assert>
<stringEquals>s3-eu-west-1.amazonaws.com</stringEquals>
</assert>
</test>
<test name="SOURCE s4 eu-west-1 resource">
<request>
<identifier>res:/amazonaws/baseurl/s4-eu-west-1</identifier>
<verb>SOURCE</verb>
</request>
<assert>
<stringEquals>unknown</stringEquals>
</assert>
</test>
<test name="SOURCE blabla resource">
<request>
<identifier>res:/amazonaws/baseurl/blabla</identifier>
<verb>SOURCE</verb>
</request>
<assert>
<exception/>
</assert>
</test>
</testlist>
And the result :
Much better. And here's my opinion : the extra test I added to res:/resources/unittest/sdk-import.xml is useless as it doesn't test functionality, the two extra tests I added to res:/resources/unittest/sdk.xml are useful as they do test functionality. It may not always be so clear and if you feel you've got to err on the side of caution, by all means do. Just remember that your purpose is solving problems, not writing tests.
Right, that's a lot of information again. Don't forget to look through the XUnit Test documentation, you'll find that you can adapt and extend it to your own preferences.
As I promised, next time I'll discuss logging and as a bonus I'll also explain what that Limiter endpoint (did you notice it in the unit test space ?) does. Keep on ROCing in the mean time !