Implementing Entity Services using NoSQL – Part 3: CouchDB

Following on from Part 2 of this series where I created and deployed the Product Entity Service using the SOA ‘contract-first’ technique, I’m now going to work on the NoSQL database aspects of the service implementation.

As I already mentioned in Part 1, I’ve already selected CouchDB as my NoSQL database and the Ektorp library as the database driver.

CouchDB – The Relaxed Database.

CouchDB is essentially going to act as my document store. Natively it stores its documents in JSON format (actually a binary version called BSON I think) and it requires no prior knowledge of the document’s structure. You can pretty much store anything, and even mix documents of different types in the same database if you want to.

Because there are no prior setup steps such as DDL scripts or XSD schemas, getting started can be very quick. In fact if you can use CURL you don’t need anything, just issue HTTP commands to CouchDB and your stuff gets stored. As it says on the tin – CouchDB is fairly relaxed. For a fuller explanation on getting started with the basics of CouchDB see CouchDB: The Definitive Guide.

Getting from Java to JSON format.

Of course when we’re in Java, JSON as a String based representation isn’t that convenient for us to use. This is where Ektorp steps in, using the Jackson Java-to-JSON library to smooth things over. Jackson facilitates an out of the box POJO to JSON conversion process behind the scenes within Ektorp.

Jackson is an important feature for this project because I want to acheive a clean and hassle free development flow from XML to Java Objects to Database Documents and back again. Jackson is a key component in making this work as we’ll see later.

CouchDB’s Document Storage Pre-Requisites.

Although CouchDB doesn’t need a schema, it does need two basic pieces of data for each document: a unique id and a document revision number. These data items help with managing the documents and implementing the idempotency rules that help maintain document integrity in multi user environments. CouchDB expects these fields to be named ‘_id’ and ‘_revision’. _id can be assigned by the user or by the database during create operations. _revision is assigned by the database and increments upwards each time a document’s record is updated.

Now obviously I didn’t want database specific fields to go into my XML documents, so my definition of a product has a field called ‘Id’ and a field called ‘Revision’. Unless I do something, this document would not meet the necessary criteria for storage in CouchDB, and strange things would start to happen like extra _id and _revision fields being added to the database records at runtime that didn’t match the Id and Revision of the XML document I asked CouchDB to store. I don’t want to change my XML schema for a Product in order to add these database specific fields, so what do I do?

Cleverly, Jackson can be configured to rectify this problem without touching the Java/JaxB definition of the ‘Product’ object that is derived from the Product XML schema. It can be told to remap the Product’s ‘Id’ and ‘Revision’ fields to the CouchDb ‘_id’ and ‘_revision’ fields at runtime. This maintains a degree of loose coupling but allows me to use the same JaxB generated Java objects throughout my code saving a lot of time and effort.

Accessing the Database.

CouchDB is not accessed via JDBC and it doesn’t have a traditional JDBC driver. Instead it uses a REST interface (http based GET, PUT, POST, DELETE, etc.) and communicates using JSON formatted content.

Ektorp provides some helper classes to help you work with the CouchDB database. There is a Connector class that can be instantiated to establish a workable connection to the database, and a customisable RepositorySupport class that offers type-safe convenience methods for interacting with the database and its records.

Creating a DAO.

Once correctly customised by extension and class-typing, the RepositorySupport class can be used for all your basic Data Access Object requirements such as Get, Create, Update and Remove operations. It can also generate CouchDB views automatically based purely on the name of the methods you add to it (as long as they follow certain rules). This makes it easy to add ‘find’ methods to your DAO such as ‘findByManufacturerName’ or ‘findByCategoryId’. Finally, if you need more sophisticated views or map/reduce queries, it can help with those too.

Pulling it all together.

By configuring Jackson and by using Ektorp to create a DAO, it’s now just a case of writing some integration tests to make sure it all hangs together. The tests I used initially are quite simple, I asked my DAO to…

1. create a fresh JaxB Product object and assign it an ID
2. save it to my CouchDB ‘Product’ database
3. read the Product object from the ‘Product’ database using it’s ID
4. modify the Product object & update it
5. retrieve it once more, checking the revision was increased
6. finally, delete the Product object & check that attempts to read it now fail

If the DAO code can do all these things, then I have the basic behaviours that I need for my Product Entity Service implementation. However, because it’s an integration test, I need a working CouchDB service to be available during the testing cycle. Maven can help with integration testing by using the Maven Failsafe plugin to bind these kinds of tests to the integration-testing specific parts of the Maven build lifecycle. This prevents mixing integration tests in with normal unit tests which usually have fewer dependencies and runtime requirements.

Getting CouchDB working locally is pretty simple, but it’s also possible to use a free cloud hosted CouchDB development instance if you can’t be bothered with the install and set-up process. I’ve tried both, and they work equally well.

What’s next?

Now my CouchDB DAO is complete it’s time to move into the final stages of the project where I’ll link up the DAO behaviours to the Web Service capabilities I created earlier. To do this I’ll be using Java Enterprise Edition 6.

If you’d like an email notification when the next instalment is published then follow the link on the right to subscribe.

Continue to Part 4.

Reference: Implementing Entity Services using NoSQL – Part 3: CouchDB from our JCG partner Ben Wilcock at the SOA, BPM, Agile & Java blog.