The less I hack into a third-party piece of software, the better. Even if it’s open source software I know that any modification to the core system is likely to make support and upgrades more difficult. Integration through Application Programming Interfaces (APIs) is a really handy approach to take that can reduce the impact of coercing systems to work together. Essentially, you have data coming in and out of the system from/to different sources. Unfortunately, these sources don’t often work in a manner that just lets you “wire up” the communications.
Enterprise Integration Patterns provides a highly useful set of patterns that can help you solve a variety of integration problems. Spring Integration provides a number of mechanisms for integrating systems. In this article I’ll outline and build a very basic integration example.
For incoming data (from our source system) an “inbound channel” is used. The inbound data may then be handled in some way (maybe enhanced or transformed). The result is then sent to an “outbound channel”(to our client/receiving system). Spring Integration provides us with a very useful set of ready-to-go inbound/outbound channels:
- Message Queues
- Databases (JDBC, JPA)
- Web sockets
- Web services
Sitting between the inbound and outbound channel is some logic that might convert, extract, enhance etc the data.
The example I’ve prepared is based on a situation I’ve seen over numerous projects. Essentially, a source system will create a data export (usually a comma-separated file) on a periodic basis and put the file into a directory. The client system grabs the file from the directory and then processes it in some way. Often, the client system can’t accept the data as-is and needs to manipulate it in some way to meet its needs. A SysAdmin will turn to
cron to create a file poller and the developer will maybe reach for
perl. I’d recommend that both could easily start to reach for
Groovy and Spring Boot + Integration.
The example system is a very basic implementation of the File Transfer pattern that will:
- Monitor a directory for new files that match a name pattern (Inbound file channel)
- Copy new files to another directory (Outbound file channel)
- Log that this is being done (Basic service activator)
Drawn using the enterprise integration icons the process appears thus:
It’s a very basic model and, of itself, isn’t something with a great utility. However, it provides an basic implementation for building on. The key features in the example are:
- Spring Boot takes almost all of the heavy lifting around preparing the application
- Including allowance for application properties to use defaults, be passed in via the command line, or be provided in a configuration file
- Spring Integration’s built-in file channel adapter handles the inbound and outbound file
- The file is passed (by the service activator) through to a POGO (Plain Old Groovy Object) handler
- At present this handler just logs the fact it’s been called and returns the file without any processing.
The following components are utilised in the solution:
- Spring Boot
- Framework for building Spring-based applications
- Spring Integration
- Library of integration patterns
- Programming language
- Build tool
The code for this article is located in the Workbench Bitbucket repository.
build.gradle file defines the build and dependencies. You’ll see it makes use of the
spring-boot plugin and the
spring-boot-starter-integration dependency. As I’ve used the Gradle wrapper you should be able to run the code without installing Gradle - just run
When the system is running you can look in
/tmp and see the
output directories. Add a file into
input and you’ll see that the running Gradle project log will show you a message from
demo.Handler that the file was copied. Check out the
input directory and the file is gone - it’s now in
Let’s take a look at what’s happening:
src/main/resources/application.propertiesfile sets the default
src/main/resources/integration.xmlfile defines the Spring Integration:
inboud-channel-adaptermonitors files in
inputDirthat match the
inputFilePattern(which is an Apache Ant-style pattern)
- It also performs
auto-create-directory- to make sure the
- You’ll see that a
polleris defined to run periodically
- It also performs
outputDirconfiguration for the output directory
REPLACEso that old copies of files in
outputDirare written over
- You’ll also see
delete-source-filesis used - this removes the
service-channelis configured for the outbound channel.
service-activatorreally brings it all together by putting the
handleFilemethod between the inbound and the outbound channels.
- This class is defined in
src/main/groovy/demo/Handler.groovyand it’s pretty basic
- This class is defined in
- To get the whole thing running,
src/main/groovy/demo/Application.groovyloads the configuration (including
integration.xml) and starts up the system.
Whilst you can run this all in Gradle, take a look in the
build/libs directory and you’ll find
boot-integration-demo-0.1.0.jar - it’s quite large because it’s self-contained. You can run it using
java -jar build/libs/boot-integration-demo-0.1.0.jar and this makes it easily distributable.
Spring Boot also gives us the ability to override the settings in
application.properties via a few options. Each property can be changed using double dash (
--) parameters on the command line - allowing the command line below to reconfigure the essentials:
java -jar build/libs/boot-integration-demo-0.1.0.jar --inputDir=/tmp/input2 --outputDir=/tmp/output2 --inputFilePattern=*.txt
This is really a very basic example to build off. There’s a wide range of directions you could take, including:
- Push the file at a database, web service or message queue
- Create a more sophisticated
Handler.javathat manipulates the file or extracts specific information
Anyway, Spring Integration combined with Spring Boot and Groovy provides a really strong toolset for tying together your software ecosystem.
If you’re new to Spring Boot and Spring Integration then it’s a good idea to head to spring.io and look at their documentation and tutorials - they’re really useful.
Check out Enterprise Integration Patterns and the seminal book on the topic:
G. Hohpe and B. Woolf, Enterprise Integration Patterns: Designing, Building, and Deploying Messaging Solutions. Addison-Wesley Professional, 2004.
The Apache Camel website also provides a great index of the patterns as well as links to drawing tool stencils.