By Charles Chen．Feb 4, 2022
When building a microservices architecture, one of the most important questions to answer is how the different services will communicate.
The two most fundamental patterns are (are there others?):
- RPC/RPI — Remote Procedure Call/Remote Procedure Invocation; it’s exactly what it sounds like and is a synchronous request/reply style of interaction that we are all familiar with.
- Messaging — passing messages asynchronously between the different services; sometimes, broadcasting or publishing those messages to multiple services simultaneously.
Both patterns have been with us long before “microservices” became a thing. MSMQ, for example, goes back to the late 90’s and I’m sure simply extends other ideas and concepts that existed before.
When building modern cloud-scale microservices, I think there is a distinct advantage to leveraging a message-oriented architecture in place of a RPC-oriented architecture.
- Scalability — it allows the various services to scale independently and according to their own performance profiles rather than synchronously incurring cost for parts of the application that have different throughput.
- Resiliency — by using messaging, it allows systems to buffer in the event that some part of the system is misbehaving. Messaging — unlike synchronous RPC — typically has a built-in retry mechanism (by redelivering the message).
- Decoupling — using messages allows the different services to be entirely decoupled. Each service only need know that it is responsible for pushing a message and any number of services can then consume that message. This allows services to avoid tight coupling and allows adding services in the future without additional coupling.
However, there are also challenges to overcome as well:
- Complexity — namely, the messaging system becomes another layer to operate, manage, and maintain. That said, it helps overcome a different problem of complexity with RPC: service discovery.
- Latency — message-oriented architectures will generally have more latency as messages move through a network and need to be serialized and deserialized, queued and dequeued.
- Cohesiveness — (for lack of a better term) there’s something to be said for the high visibility, traceability, and “cohesiveness” of direct RPC calls that makes it easy to follow the path from start to finish. In contrast, a message-oriented architecture may require more documentation to better understand who the producers are and whom all of the consumers are and in which way they depend on the producers.
- Client signalling — in many cases, the action initiated by a user may require a response; not everything can be fire-and-forget. In a message-oriented architecture, this has to be overcome with some mechanism for signalling to the client that an action completed. If the system is interactive, then more thought has to go into the design of the user experience (I have a future article and video in mind on addressing this).
With the introduction of cloud-scale, managed messaging services from Google, Amazon, and Microsoft, I think that the biggest challenge of complexity has been addressed by offering fully managed, highly scalable, high throughput messaging services.
Let’s take a look at how to work with Google Cloud’s Pub/Sub service and how we can set up local development and testing.
Google Pub/Sub provides a command line emulator using the gcloud command, but we can also grab a container with it all set up for us already: https://github.com/marcelcorso/gcloud-pubsub-emulator
To use this image, we can simply execute:
docker run --rm -ti -p 8681:8681 -e PUBSUB_PROJECT1=gps-demo,test-pub:test-sub messagebird/gcloud-pubsub-emulator:latest
And we’ll have a fully configured emulator up and running with a topic test-pub and subscriber test-sub set up for us.
This is a great tool and mechanism for sandboxing development of a message-oriented architecture so that you can test the behavior and prototype error handling, traceability, and other aspects of the system design with no risk (I really wish Microsoft would offer a similar capability with Azure Service Bus, but I’ve made do with using namespaced topics in Service Bus directly).
To publish a message, we can create a simple publisher:
And a subscriber:
In both cases, the code doesn’t need to create the topic nor the subscription because the container startup already provisions these for us.
I’ve created a sandbox repo with a fully configured setup that allows you to quickly play around with different messaging scenarios with Google Cloud Pub/Sub.
With this setup, it’s really easy to test various scenarios for interacting with a message-oriented approach to microservices. In a future article, I want to build a more complex example using Google Cloud Run and a local mechanism for simulating how Google Cloud Run triggers integrate with Pub/Sub.
Cheers!
For development on Apple M1 silicon, you’ll also need this native gRPC library (already packaged into the dependencies in the repo): https://github.com/einari/Grpc.Core.M1
The original article published on Medium.
