Distributed Systems: an introduction to Publish-Subscribe (pub/sub)

More and more of the web is moving to microservice architecture, which allows for loosely-coupled services to work together to provide functionality to users. While these services will often communicate with each other via network requests, the publish-subscribe model is another common way for this collaboration to occur.

There are many different technologies that can be used to implement pub-sub, including but not limited to Apache Kafka, Amazon Kinesis, Google Pub/Sub, and Microsoft Event Hubs. The purpose of this post is not to go into the details of implementation or discuss the pros and cons of these specific technologies, but to provide definitions of the words and an overview of how it works.

I talk about two different types of events that can be passed through a pub-sub model, Change Data Capture events and “business events” here, though these messages could of course be anything!

What is decoupling and why is it good?

Before we dive into pub-sub, let’s take a quick detour to decoupling. If we’re not convinced of the value of decoupling, pub-sub won’t do us any good!

Decoupling means reducing the dependency between pieces of a system. This can mean two classes in the same codebase or two services in a system built on microservices, or anything in between.

Ok, but what do we mean by “reducing the dependency”? Obviously the different parts…