Implementing Entity Services using NoSQL – Part 5: Improving autonomy using the Cloud

Ben WilcockSeptember 10th, 2012Last Updated: October 22nd, 2012

0 167 6 minutes read

In the previous posts I discussed how I went about building my SOA ‘Entity’ service for Products by using a combination of Java Web Services, Java EE and the CouchDB NoSQL database. In this final post in the series I’m going to leverage some of the technical assets that I’ve created and implement some new user stories using some popular SOA patterns. Take a look at Part 1 and Part 2 as well.

My current Product Entity Service implementation is very business process agnostic, and therefore highly re-usable in any scenario where consumers want to discover or store Product information. However, as it stands the Product Entity Service is designed to be used within a trusted environment. This means that there are no restrictions on access to operations like Create, Update or Delete. This is fine within a strictly controlled corporate sandbox but what if I want to share some of my service operations or Product information with non trusted users?

Lets imagine that in addition to our in-house use of the Product Entity Service we also wanted to cater for the following agile ‘user story’…

So that: My published product information is accurate and constantly available to off-site customers.
As a: Sales Director & IT Manager.
We want to: Offer a highly-available ‘read-only’ copy of my product information to off-site users and systems.

This scenario is not uncommon in business and I’ve implemented user stories like it in the past with some of my clients. But what’s fantastic about the technologies that we’re using to implement our Entity service (Java/JAX-WS and CouchDB NoSQL) is that they make developing this scenario very easy.

First of all, the architectural design.

In order to implement this user story I’m going to call upon two more SOA design patterns for the service architecture – Service Data Replication and Concurrent Contracts. In post two, we already talked about the ‘contract-first’ approach so I won’t go into any more detail other than to say that it still applies here. The contract is still a Standardised and Decoupled Contract.

Service Data Replication is a pattern that helps you to achieve high levels of service autonomy and availability by creating whole copies of the data required by the service elsewhere on the infrastructure. This copy can then be used instead of the original in order to balance the load on the infrastructure.

The Concurrent Contracts pattern is used when ‘[an existing] service’s contract may not be suitable for or applicable to all potential service consumers’. For example, when security, protocol or accessibility concerns require that something similar (but not the same) be made available to a specific subset of users (like off-site consumers or mobile devices with restricted processing power or bandwidth).

In order to implement our new user story, we’ll use both of these patterns together to provide a ‘read-only’ version of the Product Entity Service. However, by providing a second ‘read-only’ version of the Product Entity service, you could also say that we are partially implementing the Redundant Implementation SOA pattern, because we’re offering a second redundant option for some of the service’s key operations.

The resulting architecture looks something like this (click to enlarge)…

The service contract.

The original Product Entity Service offered five operations – Get, Find, Create, Update and Delete but offering all these capabilities to outsiders is not necessary and would probably be quite problematic (in terms of security, integrity, etc.). We certainly don’t want any external users creating or updating our product information without our permission.

Therefore, in our web service contract for the new ‘Read-only Product Entity Service‘ I’ve removed the Create, Update and Delete operations completely and only provided Get and Find. All the datatypes remain the same (the same Product.xsd describes the product entity etc.). Keeping the datatypes the same is important because I’m deliberately applying the Canonical Schema and Schema Centralisation patterns and utililising the Standardised Service Contract principal in order to avoid transformation.

The Java code.

With this new read-only service I’m still working contract first, so I’ve created a new Maven project who’s first task during a build is to import the Read-only Product Entity Service’s WSDL contact and create from it a set of JAX-WS service interfaces and data objects.

From this point on, I can reuse some of the code that I’ve developed already for the ‘full’ Product Entity Service. By refactoring my previous codebase into modules, I can even get Maven to treat the original code as a dependency for this new service. In essence, the bits that I’m interested in are the EJB’s and DAO’s I created for the business and persistence logic of the Get and Find operations.

By reusing the existing code and by creating a Glassfish server on the Amazon cloud, I can stand-up the new service in record quick time and be halfway to completing the user story. All I need now is some replicated Product data to work with…

Starting the data replication.

Couch DB offers a fantastic enterprise class replication system out of the box and for free. This makes implementing the Service Data Replication SOA pattern very easy.

Couch DB’s built in data replicator can replicate whole databases of documents between any two CouchDb instances that are available on the network or over the web. In this case I’ve created a CouchDb database at a hosted provider called IrisCouch. They can provide me with secured CouchDB instances or various sizes at the drop of a hat and will look after all the necessary infrastructure and backup requirements. For small users they even do this free.

In order to setup a replication I just need to use the CURL command-line tool to issue specific instructions via HTTP to my local CouchDB instance. These instructions tells my local CouchDB service to replicate my Product data with my remote CouchDB database on IrisCouch over the web.

The database replication instruction is a JSON document and looks something like this…

{'source':'products',
 'target':'https://username:password@instance.iriscouch.com:6984/products',
 'create_target':true,
 'continuous':true}

In essence this JSON instruction says “replicate the local Products database documents to the remote iriscouch instance, forever“. Immediately on issuing the command, CouchDB sets to work and starts replicating my local data to my remote database instance. This includes updates and deletes and any ‘design documents’ stored in the Products database. This replica of Products is then immediately available to my Read-only Product Entity Service which is hosted in the Amazon EC2 cloud.

From this point onwards, service consumers using the Get or Find operations on this service will see a replica of the data that is used in-house. The information will be kept up to date by the replication.

Finally, the user acceptance test.

So how well did we do against our new user story’s requirements?

By hosting the read-only version of the Product Entity Service on the Amazon cloud, we’ve created an off site web-service that that is highly available. The data it provides is an exact replica of the data we use on-site with only the smallest amount of latency between the two copies under normal conditions.

If my on-site network goes down, the read-only cloud based version of the Product Entity Service will carry on regardless, and because we’re using the Amazon cloud infrastructure it can benefit from almost unlimited runtime resources if necessary. Availability shouldn’t ever be a problem. We can add more machines at any time, and offer load balancing and potentially spread machines over multiple continents if necessary.

My guess is that the Sales Director will be pleased that our customers can always see the information in our product catalogue and customers themselves should be be pretty satisfied with the comprehensive and reliable service that they now receive. Finally, the IT Manager will be pleased that network security remains intact and that the new off-site hosted service will cost next to nothing to run and be very reliable.

All that remains is to publicise the Read-only Product Entity Service endpoint to our customers and support them in their use of it. All in all a pretty successful day at the office.

Would you like to try the Read-only Product Service for yourself?

The endpoint details can be found in the SOA Growers Simple Service Repository. Follow the ‘Service Repository’ link and look for the ‘R20121231? release. In there you fill find links to the service’s WSDL, WS-I certificate and link to a live demo web-service endpoint hosted on an AWS micro-instance.

The best way to experience the live demo is to download the SOAP-UI test suite. This test suite requires SOAP-UI v4 (which can be downloaded for free). The test suite contains a simple test of all the operations available on the service.

Catch-up with the entire blog series online…

That’s probably the last in this series on building entity services with Java and CouchDB. If you missed any of the earlier blog posts in this series and you would like to catch up, the other entries are listed below…

Don’t forget to share!

Reference: Implementing Entity Services using NoSQL – Part 5: Improving autonomy using the Cloud from our JCG partner Ben Wilcock at the SOA, BPM, Agile & Java blog.