We have a number of teams that are considering – or already moving towards – message-based architectures. We made some mistakes coupling applications and databases, will we make mistakes with messaging that we don’t know about yet?
What is message-based?
I’m sure you can find some great definitions online.
Systems (driven by people or other systems) interact by putting messages into a store that offers some guarantees about storing messages exactly once. The messages can have metadata that allow them to be labelled in such a way that systems know something about the content of the message. We might have a queue of messages that preserves order and labels messages with a “topic”, or we might have multiple queues that is for a different use.
For me the big questions are:
- Who are the actors?
- When do actors act?
Who are the actors? Many-to-many interactions explodes the client-server picture
From the first time-sharing mainframes, we have had central “servers” that control a resource, with many clients calling in. With messaging that model might still be true, but isn’t always. Now the message queue is the central resource, but the things sending and pulling messages don’t have to be one to one. It also means that integration between systems that you built and systems that you bought changes, sometimes for the better.
Also: you now have no idea who’s doing your work, but on the plus side you don’t need to care who’s doing the work, so when it changes no harm done.
When actors act? Timing is everything
The timing of a synchronous call to the “server” is a simple thing (well not really, it’s actually hella complex but networking sockets fix most of it). You call, it responds. With messaging, that is no longer true. Threading is implicit now and is everywhere. If you previously had got away with not considering timing and ever used thread.sleep to fix something, then those days are gone.
There are 2 things about messaging systems that make a difference to the timing: the message transport and the message distribution. You might be able to have a guaranteed delivery and a guaranteed order to your messages, but if you do then you probably are giving up some other things. Also, messaging systems only get interesting when you have a message distributor that lets actors “subscribe” and or get push notifications or distribute to multiple clients.
Where’s my schema?
You used to understand changes to database schema, and how to make those changes. When we have a new database schema we have to migrate data. We know what changes in business requirements are easy to absorb, and what is hard. We know how to make a database schema that can absorb some changes. We’ve learnt how to source control our database schemas, or even test the database.
But with messaging a lot of that is gone, but we still have a structure. The different ways of listing messages by topic and in different queues offers a lot of ways of structuring data on the queues. That means that you have a lot of options and you can change things easily. But! That’s scary as well. Things that change can break. How do we source control all that? And who does the work? Developers or infrastructure/ops staff who are admins on the machine?
What’s interesting is that you don’t need to pick one model; when messages are used you can replicate into multiple structures independently. Of course, more formats is more work.
Read the book
This book is great. If you are working with messages you should read it. It shows what you can do when you stop thinking about messaging as a way of doing RPC and really put the message queues at the heart of your architecture.
Enterprise integration patterns
Here’s the site of the book slightly less easy to read. For me, the best part is this overview of application integration options.