(I’m going to reply here and link to here in the comment)
I particularly like your observations on when durable messages work against you. I work in the finance industry and as you note we often use a mix of durable and non-durable messaging solutions. For applications like price streams you may need to ship thousands of messages a second but don’t care if you lose a few, but for the submission of orders you must be certain that you don’t lose any.
There are several messaging middleware providers that target the finance industry specifically — e.g., TIBCO — that try to address the low-latency requirement. We have used non-transactional MSMQ and have got up to a few thousand messages per second; TIBCO claims to support up to 50,000 messages per second! However, they don’t specify if you need a z/OS mainframe to do that…
Your comments on very large messages were also interesting. The decision to allow multi-message orders seems like it caused really far-reaching changes to the design. I wonder if you considered solving the problem “internally” by inserting a message processor in your inbound message stream that broke up large messages – a splitter – then you have smaller messages to deal with but you are in control of the message order and flow. If necessary you could have diverted them into another queue and maintained order that way and allowed other messages to “overtake” in the regular queues. Of course, this is still something that is painful as it changes the way messages flow and you still have to parse the 50MB XML. However, in the case I have seen with similar problems the counterparty was such a lumbering behemoth there was no chance of them being able to refactor their solution to make any changes to the message format or the message choreography (which I think is a lovely way to say the “request-response pattern for the message conversation”).
That is the power and the pain of messaging: it provides a clean interface – the message format – to work with counterparties, but if the message conversation starts to change then the asynchronous nature of messaging can make the changes pop up everywhere in the message chain.
But, great article to introduce a really interesting subject.