Sunday, December 7, 2014

Basic integration with Spring Boot

Introduction

The less I hack into a third-party piece of software, the better. Even if it’s open source software I know that any modification to the core system is likely to make support and upgrades more difficult. Integration through Application Programming Interfaces (APIs) is a really handy approach to take that can reduce the impact of coercing systems to work together. Essentially, you have data coming in and out of the system from/to different sources. Unfortunately, these sources don’t often work in a manner that just lets you “wire up” the communications.

Enterprise Integration Patterns provides a highly useful set of patterns that can help you solve a variety of integration problems. Spring Integration provides a number of mechanisms for integrating systems. In this article I’ll outline and build a very basic integration example.

For incoming data (from our source system) an “inbound channel” is used. The inbound data may then be handled in some way (maybe enhanced or transformed). The result is then sent to an “outbound channel”(to our client/receiving system). Spring Integration provides us with a very useful set of ready-to-go inbound/outbound channels:

  • Message Queues
  • Filesystem
  • FTP
  • Databases (JDBC, JPA)
  • Mail
  • Syslog
  • Twitter
  • Web sockets
  • Web services

Sitting between the inbound and outbound channel is some logic that might convert, extract, enhance etc the data.

The example I’ve prepared is based on a situation I’ve seen over numerous projects. Essentially, a source system will create a data export (usually a comma-separated file) on a periodic basis and put the file into a directory. The client system grabs the file from the directory and then processes it in some way. Often, the client system can’t accept the data as-is and needs to manipulate it in some way to meet its needs. A SysAdmin will turn to bash and cron to create a file poller and the developer will maybe reach for perl. I’d recommend that both could easily start to reach for Groovy and Spring Boot + Integration.

Implementation

Design

The example system is a very basic implementation of the File Transfer pattern that will:

  • Monitor a directory for new files that match a name pattern (Inbound file channel)
  • Copy new files to another directory (Outbound file channel)
  • Log that this is being done (Basic service activator)

Drawn using the enterprise integration icons the process appears thus:

Integration diagram for inbound and outbound file channels

It’s a very basic model and, of itself, isn’t something with a great utility. However, it provides an basic implementation for building on. The key features in the example are:

  • Spring Boot takes almost all of the heavy lifting around preparing the application
    • Including allowance for application properties to use defaults, be passed in via the command line, or be provided in a configuration file
  • Spring Integration’s built-in file channel adapter handles the inbound and outbound file
  • The file is passed (by the service activator) through to a POGO (Plain Old Groovy Object) handler
    • At present this handler just logs the fact it’s been called and returns the file without any processing.

Components

The following components are utilised in the solution:

Code

The code for this article is located in the Workbench Bitbucket repository.

Discussion

The build.gradle file defines the build and dependencies. You’ll see it makes use of the spring-boot plugin and the spring-boot-starter-integration dependency. As I’ve used the Gradle wrapper you should be able to run the code without installing Gradle - just run ./gradlew run.

When the system is running you can look in /tmp and see the input and output directories. Add a file into input and you’ll see that the running Gradle project log will show you a message from demo.Handler that the file was copied. Check out the input directory and the file is gone - it’s now in output.

Let’s take a look at what’s happening:

  • The src/main/resources/application.properties file sets the default inputDir, inputFilePattern and outputDir values
  • The src/main/resources/integration.xml file defines the Spring Integration:
    • An inboud-channel-adapter monitors files in inputDir that match the inputFilePattern (which is an Apache Ant-style pattern)
      • It also performs auto-create-directory - to make sure the inputDir is there
      • You’ll see that a poller is defined to run periodically
    • An outbound-file-channel uses the outputDir configuration for the output directory
      • Again, auto-create-directory is used
      • The mode uses REPLACE so that old copies of files in outputDir are written over
      • You’ll also see delete-source-files is used - this removes the inputDir copy
    • A service-channel is configured for the outbound channel.
    • The service-activator really brings it all together by putting the demo.Handler class’s handleFile method between the inbound and the outbound channels.
      • This class is defined in src/main/groovy/demo/Handler.groovy and it’s pretty basic
  • To get the whole thing running, src/main/groovy/demo/Application.groovy loads the configuration (including integration.xml) and starts up the system.

Whilst you can run this all in Gradle, take a look in the build/libs directory and you’ll find boot-integration-demo-0.1.0.jar - it’s quite large because it’s self-contained. You can run it using java -jar build/libs/boot-integration-demo-0.1.0.jar and this makes it easily distributable.

Spring Boot also gives us the ability to override the settings in application.properties via a few options. Each property can be changed using double dash (--) parameters on the command line - allowing the command line below to reconfigure the essentials:

java -jar build/libs/boot-integration-demo-0.1.0.jar --inputDir=/tmp/input2 --outputDir=/tmp/output2 --inputFilePattern=*.txt

This is really a very basic example to build off. There’s a wide range of directions you could take, including:

  • Push the file at a database, web service or message queue
  • Create a more sophisticated Handler.java that manipulates the file or extracts specific information

Anyway, Spring Integration combined with Spring Boot and Groovy provides a really strong toolset for tying together your software ecosystem.

Further reading

If you’re new to Spring Boot and Spring Integration then it’s a good idea to head to spring.io and look at their documentation and tutorials - they’re really useful.

Check out Enterprise Integration Patterns and the seminal book on the topic:

G. Hohpe and B. Woolf, Enterprise Integration Patterns: Designing, Building, and Deploying Messaging Solutions. Addison-Wesley Professional, 2004. 

The Apache Camel website also provides a great index of the patterns as well as links to drawing tool stencils.

No comments :

Post a Comment