11.2 Messaging and Integration

Back around the dawn of computing, sending a message between systems was easy: you wrote the data onto a magnetic tape, took the tape out of the computer, and carried it over to the other computer. Assuming the two computers shared a common data format, you were all set. When networks were invented, programmers devised integration techniques based on direct network connections. An application opened a connection to another application, and if the network was running and the target application was responding, the two systems could interact with each other, as long as they shared a common format for data. The problems with this approach should be familiar: the system wouldn't work if one application was down, or if there were intermittent network failures, or if the systems had different data formats. Building direct linkages and custom protocols were also a time-intensive and nonstandard approach, which had to be revisited with each new application or new version. And did we mention the data formats?

There are other approaches; many systems communicate with each other by reading and writing shared files, or by accessing shared tables within a database. The former method works well when two applications are running on the same physical machine or connected by a reliable network that supports file sharing. The latter works well for multiple applications and applications spread across a variety of systems. Both approaches have the benefit of being asynchronous: you don't need to have all the applications running in order for any single system to read or write data. But programmers still need to agree on formats and create the infrastructure to share the data resources. Programmers also have to be particularly careful of concurrency issues (see Chapter 10 for more on concurrency).

The original Unix mail system used the shared-files approach, allowing users on the same system to run the mail program at different times to leave messages for others and to read messages left for them. But with the rise of the Internet, and particularly with the development of always-on TCP/IP networks, users started to send messages to people using different computers, and even at different locations. The result was first UUCP and then SMTP, both of which provide a mechanism for routing messages between systems without having to actually create a direct connection between the sender and the recipient.

SMTP, in particular, did two things. First, it provided a standard, allowing different vendors and different platforms to send email to each other. MS Exchange, Lotus Notes, and Unix mail all use vastly different internal formats, but the common SMTP message and address formats made it largely a non-issue. Second, SMTP allowed administrators to set up an SMTP server for their domains. SMTP servers received messages from users and routed them to other systems for distribution. If the destination system was unavailable, the SMTP server could hold on to the outgoing message for a few hours and try again, rather than ignominiously failing.

SMTP servers were the first widely deployed messaging middleware. They introduced most users to the idea of asynchronous messaging, in which a message is sent and received at different times. In the last few years, asynchronous messaging has become a big deal in the enterprise development area because it provides the same advantages SMTP provides for email: it develops a standard method of packaging information and an infrastructure that allows messages to be sent reliably.

Figure 11-1 shows a simple use case diagram, where every single interaction could be implemented via a messaging approach. The customer emails a form to the software implementing the Purchase Product use case, which in turn sends a message to the software implementing the order processing use case, which, in turn, interacts with the Purchasing Server system actor via another set of messages.

Figure 11-1. Use cases that can use messaging

11.2.1 Types of Messaging

Deciding to use asynchronous messaging in your application is a big step, but it's by no means the last. There are a number of ways to implement asynchronous messaging, each involving a different Quality of Service guarantee. The U.S. Postal Service, for instance, promises to deliver a first class letter within four days, and makes their best effort to deliver the letter intact. Most letters duly arrive in the expected timeframe and in reasonable condition. However, some letters get lost, and others are damaged. Because this isn't good enough for every snail-mail enabled application, the USPS provides a set of additional delivery services: return receipts, certified mail, Priority Mail, and Express Mail. An Express Mail letter is all but certain to arrive at its destination the following day, in good condition.

Messaging middleware is the same. Depending on the sophistication and configuration of your messaging platform, you can achieve a variety of different service levels. The higher levels involve more maintenance and are generally more expensive to configure. In this chapter, we're going to talk about two different kinds of messaging. The first is guaranteed messaging, which is implemented via middleware and provides high reliability and control. We use JMS to handle guaranteed messaging in Java. The second is nonguaranteed messaging, which lacks the reliability and security of guaranteed messaging, but can be implemented with much less programmer time and less cost for external middleware.

11.2.1.1 Guaranteed messaging

Guaranteed messaging uses middleware to ensure that a message is properly delivered to its destination, both ungarbled and in a reasonably timely fashion. The middleware implementing the messaging framework guarantees a message sent to a valid destination eventually arrives, even if the destination is temporarily unreachable.

Guaranteed delivery doesn't imply complete certainty. Specifying a nonexistent destination won't get your message delivered, regardless of how expensive your middleware is. And if a destination system is unavailable for a long period of time, the middleware will probably return the message to its sender rather than wait indefinitely. What is guaranteed, however, is the contract between the messaging client and the server governing how a message will be handled. Undeliverable messages won't vanish into the night but will be handled according to a specific and well-known set of rules.

Message Oriented Middleware systems implement guaranteed messaging. Senders and recipients work through a MOM server, which is responsible for tracking all elements of the message lifecycle. MOM servers often provide a range of options for the message content itself: XML, Java objects, raw binary data, and so on. J2EE clients interact with MOM servers using JMS, the Java Message Service. JMS, which is part of J2EE 1.3, provides a standard programmatic interface to the different MOM implementations, much as JDBC does for database servers. Sonic Software's SonicMQ and IBM's MQSeries products are two of the leading MOM products.^[2]

^[2] For more on JMS, see Java Message Service by Richard Monson-Haefel and David Chappell (O'Reilly).

11.2.1.2 Nonguaranteed messaging

Nonguaranteed messaging is like standard postal service. Your message will probably get there, but you don't know exactly when or in what condition. You also don't know if something goes wrong: there's no notification when your letter slips out of the bag and gets wedged in a corner of the Duluth post office.

Internet email, as defined by RFC 822, is the archetypical nonguaranteed messaging system. In fact, it's very consistent: messages sent to a valid address almost never get lost, and are delivered in a timely manner. In a controlled environment, receipt rates in the five-nines (99.999%) range aren't out of the question. Of course, delivery guarantees are only part of the issue. RFC 822 messaging provides no guarantee that a message will only be delivered and processed once, or that the message won't be changed in transport (corporate firewalls and virus detection packages frequently modify the format of a message). The application is responsible for verifying integrity and making sure that the order of processing is handled correctly. There are also no security guarantees; with SMTP, for instance, it is trivially easy to create a "spoof" message that appears to be from a particular source: receipt of a message provides no assurance that the message is genuine.

In return for a little extra work, you get the basic global infrastructure for free. Nonguaranteed messaging is often the only option when dealing with widely distributed sites, getting integrations up and running quickly with external partners, or when one actor in the messaging interaction is a human.

In J2EE, email integration is handled via the JavaMail API, which, out of the box, supports sending messages via SMTP and receiving them via POP3 or IMAP.^[3] JavaMail is actually a generic messaging API, and additional plug-ins are available to support other messaging schemes, such as NNTP, the protocol behind Usenet.

^[3] For more on JavaMail, see Chapter 12 of Java Enterprise in a Nutshell, Second Edition, by William Crawford, Jim Farley, and David Flanagan, and Chapter 10 of Java Enterprise Best Practices by the O'Reilly Java Authors (both published by O'Reilly).

11.2.1.3 Persistent and nonpersistent messages in JMS

There's another wrinkle worth mentioning before we get to the patterns. While most MOM systems support guaranteed messaging, you can use MOM in guaranteed or nonguaranteed mode. The tradeoff, as usual, is performance: the server can handle more messages in nonguaranteed mode than in guaranteed mode, but there is a chance that a message won't be delivered.

JMS allows outgoing messages to be tagged as either persistent or nonpersistent. A nonpersistent message will be delivered to recipients at most once, but possibly never. Marking a message as persistent tells the middleware to use a store-and-forward approach to message delivery, ensuring that a message is delivered once and only once. The two delivery mechanisms have different implications depending on the type of message distribution pattern you're using, so we'll touch on this subject a bit more in the next section.