Hurry hurry here's your chance,
See the mystery and romance
Ok, no romance in this episode but quite a bit of mystery. These are exciting times. I recently got pinged by the new CEO of 1060 Research. Yes I know, I did a double-take too. More than one in fact. Peter Rodgers is now VP of Leisure. One Charles Radclyffe (there's a proper Brit name if I ever saw one) has now taken it upon him to try and bring Resource Oriented Computing to the masses.
 |
| https://www.deviantart.com/art/At-Library-707812195 |
I'm also still working for Neo4j. A year has gone by and what a ride it has been so far. Literally too, I've travelled more distance in the last year than I have in the 44 years before it. Combined.
Today I want to revisit the Neo4j client for NetKernel that I had already created last time. The 1.0.0 version works but has a serious flaw. It creates (and closes) a connection pool for every query that you do. That's not efficient.
This is however not a trivial problem :
- You've got to keep track and manage a connection pool for each database configuration that you have.
- When a database request comes in you've got to identify the correct connection pool and get a connection/session from that one (and not another).
- When a configuration changes you've got to close and reopen the connection pool with the new parameters.
- When a connection pool isn't used you might want to close it to conserve resources.
- ...
The first is transreption. Remember the definition for that ? An isomorphic (lossless) transformation of one representation to another. We all understand (if not that definition) the fact that an XML document can be represented as a JSON document without any data loss. Here we take that one step further and state that the configuration for a specific database can be transrepted to the connection pool for that specific database. This solves a lot of the above problems.
The second is the caching mechanism. And there is a reason folks that this is the patented part of NetKernel. In this case it solves all of the other problems. Lets go through a typical scenario :
- I do a first database request and pass the configuration of the database
- A transreption happens (connection pool can not be found in cache) and the connection pool is created
- I get a connection/session from the connection pool and my request is executed
- I close the connection/session
- I do a second database request and pass the same configuration
- The transreption does not happen, the connection pool for that configuration comes from cache
- I get a connection/session from the connection pool and my request is executed
- I close the connection/session
- I do a third database request and pass a different configuration
- A transreption happens (connection pool can not be found in cache) and the connection pool is created
- Because memory is - maybe - low the caching mechanism tries to find unused resources in cache and finds the first connection pool. It is closed.
- I get a connection/session from the connection pool and my request is executed
- I close the connection/session
- If unused for a while the caching mechanism may find the second connection pool to be unused and closes it.
Lets just have a look at what the ConnectionPoolRepresentation looks like :
package com.ebc.neo4j.client;
import org.neo4j.driver.v1.*;
public class ConnectionPoolRepresentation implements AutoCloseable {
Driver mDriver;
public ConnectionPoolRepresentation(String aURL, String aUsername, String aPassword) {
mDriver = GraphDatabase.driver(aURL,
AuthTokens.basic(
aUsername,
aPassword
)
);
}
public Session getSession() {
return mDriver.session();
}
@Override
public void close() throws Exception {
mDriver.close();
}
}
Yes I know, there's no error handling yet. But look at the - lack of - size on that thing ! And here's the relevant pieces of code in the actual database-request-handling accessor. The first piece sources the configuration and does the transreption (if necessary) :
ConnectionPoolRepresentation vCPR = null;
if (aContext.getThisRequest().argumentExists("databaseconfiguration")) {// databaseconfiguration is an optional argument
try {
vCPR = aContext.source("arg:databaseconfiguration", ConnectionPoolRepresentation.class);
}
catch (Exception e) {
throw new Exception("RowsAccessor: no valid - databaseconfiguration - argument");
}
}
else {// neo4j:databaseconfiguration is the default
try {
vCPR = aContext.source("neo4j:databaseconfiguration", ConnectionPoolRepresentation.class);
}
catch (Exception e) {
throw new Exception("RowsAccessor: no valid - neo4j:databaseconfiguration - resource");
}
}
The second piece is the actual usage of the connection pool :
// processing
IHDSMutator vMutator = HDSFactory.newDocument();
vMutator.pushNode("results");
Session vSession = null;
try {
vSession = vCPR.getSession();
StatementResult vResult = vSession.run(aCypher);
while (vResult.hasNext()) {
Record vRecord = vResult.next();
vMutator.pushNode("row");
for (Pair <String, Value> vListEntry : vRecord.fields()) {
process_listentry(vSession.typeSystem(), vMutator, vListEntry.key(), vListEntry.value());
}
vMutator.popNode(); // row
}
}
catch (Exception e) {
}
finally {
if (vSession != null) {
vSession.close();
}
}
vMutator.popNode(); // results
IHDSDocument vRepresentation = vMutator.toDocument(false);
//
Now, the urn.com.ebc.neo4j.client-2.0.0 module is obviously an utility module so there's no fancy things to show, but these are some of the requests you can do in the request trace tool :
<request>
<identifier>active:rows</identifier>
<argument name="databaseconfiguration">
<literal type="hds">
<config>
<url>bolt://127.0.0.1</url>
<username>neo4j</username>
<password>xxxYYYzzz</password>
</config>
</literal>
</argument>
<argument name="expiry">
<literal type="long">30000</literal>
</argument>
<argument name="cypher">
<literal type="string">MATCH (n) RETURN n LIMIT 5</literal>
</argument>
</request>
<request>
<identifier>active:rows</identifier>
<argument name="expiry">
<literal type="long">30000</literal>
</argument>
<argument name="cypher">
<literal type="string">MATCH (n) RETURN n LIMIT 5</literal>
</argument>
</request>
Obviously these requests expect you to have a Neo4j instance running. The second request will create the configuration from the environment parameters of the user running NetKernel. For example on Windows :
set neo4j_databaseurl=bolt://127.0.0.1
set neo4j_databaseuser=neo4j
set neo4j_databasepassword=xxxYYYzzz
An that concludes this post. Next time we're going to take a look at the NeoSemantics work of my colleague and friend Jesus Barrasa and how we can take that to the next level with NetKernel.
