I’ve been pondering a common misconception: Microservices are synchronous. Or, Microservices are asynchronous. Both are actually missing the point!

The use of common libraries when implementing microservices using HTTP as a transport has many characteristics of synchronous coding. A request is made, and then one blocks until the result is availble, and then processes the result of the request. That sounds like synchronous code.

However, it’s only the way the code is written that makes it synchronous. For instance, if instead of using a library that takes the request details and returns a set of results, one could use a library that issues the request and then returns without waiting for the eventual results. That would shift the model to asynchronous programming.

So, are microservices synchronous or asynchronous?

What’s the real difference? Actually, it’s easier to show than tell …

Parallel vs Sequential

Before worrying about the synchronous vs. asynchronous, the more abstract concepts of parallel vs. sequential need to be examined. Here is a parallel process diagram:

There is a layout that looks like A then B then C might be fired in that order. This is nothing more than accidental sequence. The meaning of that UML diagram is that Tasks A, B, and C may run in any order, and until each has completed, the computation won’t be considered done. In particular, the tasks can run in parallel (literally, if there’s three cores, or three distributed machines). Said another way, tasks A, B, and C have no inter-dependencies. They are independent tasks.

It is legal to implement this as a simple sequential set of steps:

1. Task B
2. Task A
3. Task C

Why B, A, C, vs A, B C? Why not? ANY order is legal, so any assumption of what a developer might do to put them sequentially is irrelevant. I gave a legal sequence that would accomplish the above computation.

How could it actually be implemented in parallel, though?

That would require some form of concurrency support. The launching is trivial, but the completion requires a barrier–completion can’t be claimed until all three tasks have completed. Of course, it would also need to “complete” in some way if one (or more) of the tasks fail but that’s outside the scope of this article.

What might a true parallel launch look like? Here’s pseudo code for such:

begin
  let cb = new CompletionBarrier()
  launch new TaskA(), cb
  launch new TaskB(), cb
  launch new TaskC(), cb
  cb.blockUntilDone()
end

I intentionally used nothing like an existing language just to avoid language war issues. How this runtime will launch the tasks, how it will update the barrier, etc., are implementation details. But, notice that launching in this does not wait for the newly created object to start, do its work, and exit.

The system (this thread, process, whatever the implementation model) is blocked until all the tasks launched complete. They might, of course, complete (and launch) in any order, including fully concurrently.

Fundamentally, this is what differentiates parallel vs. sequential: in parallel, the order is undefined and irrelevant and in sequential it is not.

It’s common for most people to reason about sequential processing. So much so that when systems are heavily concurrent the teams make questions on non-sequential processing a strong part of their interview process…and they’re right.

Easy vs. Hard

However, it’s not true that sequential is inherently easier than parallel. The reason it’s easier for most is because most have never been taught about parallel. Even the way computer programming is described, where the computer is told what to do “step by step,” implies sequential. The reality is more nuanced.

It’s harder to do what hasn’t been learned. Learn parallel, concurrent, distributed, etc., and it’s no longer hard. It’s hard to drive a stick shift car, until you learn. Then it’s hard to endure an automatic (really hard — I still drive a stick).

Because I’ve been doing non-sequential since the 90s, I actually tend to think in parallel terms, even when the language or environment doesn’t support it. Because doing so frees me from accidental dependencies.

Async vs. Sync

So, then, what about asynchronous vs. synchronous, and why the diversion through parallel vs. sequential?

Async does not mean parallel, and sync does not mean sequential! The terms refer in particular to the mechanism of function call, not the concurrency model.

In a synchronous world, the request blocks until the response arrives. In an asynchronous world, the request is made and then eventually the response “becomes available.” There is no blocking. The less formal terms, then, for async vs. sync is “non-blocking vs. blocking.”

Here’s an example of sequential synchronous:

  taskA()
  taskB()
  taskC()

Basic default code.

Here’s an example of sequential asynchronous (JS for instance works this way — this is still pseudo-code!):

  await taskA()
  await taskB()
  await taskC()

The await allows the engine (the JS runtime, such as V8 in Node) to perform each of the tasks to completion before doing the next without blocking other “awaiting” jobs. It’s really syntactic sugar for something (again, pseudo-code) like this:

  runUntilDone(new TaskA(), 
    {runUntilDone(new TaskB(), 
      {runUntilDone(new TaskC())})
    })

Of course, the real JS await also captures the return value and allows the handling of exceptions, but it always expands into a set of continuations “behind the scenes”.

Notice that Python also supports an async/await model now. The point of the model is to give the illusion that even though non-blocking calls and continuations are being used, it’s “just” sequential code. Nothing special. Believe that at your peril, the reality is that there are many futures and continuations in play and even though the syntactic sugar is nice be sure to understand what’s really happening or debugging becomes … let’s call it interesting.

So await and async allow non-blocking code to act sequential. They are not tools for parallel code. What is in JS?

JS on its own doesn’t do real user-space concurrency. It does something a bit more subtle. When a developer’s code is running, that code has 100% of the “thread.” The parts that are threaded (for JS engines are multithreaded engines) are the IO handlers, which work with the buffers and queues and such. That’s what allows JS to have multiple concurrent networking connections at once, even though it only uses a single thread for handling all the developer’s code.

The engine (and the JS language supported in the engine) is really designed to be an event-driven system, not a concurrent system. That’s why without using await/async syntax it’s a continual passage of call back functions.

JS developers use multiprocessing for concurrency … each process is “heavier” than a thread but has its own instance of the world. Whether it’s another V8 instance or whether a browser does threads with marshalling the net effect is that there’s “another” JS context that shares nothing more than a communication mechanism (pipe, socket, etc.).

Embracing Parallelism and Asynchronous Calls

Even if other tools are used, queues are a very easy way to think about a fully parallel and async world.

This is what a genuinely async parallel style often looks like:

workers.enqueue(new TaskA())
workers.enqueue(new TaskB())
workers.enqueue(new TaskC())

There’s no attempt to setup a barrier. Whatever effects A, B, and C have will be enqueued “somewhere” and something ELSE worries about what to do next.

Of course, this does imply that TaskA will be pulled before TaskB (queues DO force ordering) but there’s no intent or implication that B and C won’t be pulled (and perhaps even started and finished!) before TaskA even starts. The order in which they start and finish is nondeterministic.

What might the workers object be? Could be a thread pool. Could launch the new objects to AWS Lambda. Could push it out into a Kafka queue or SQS. Could just do the work sequentially (makes it REALLY easy to write test-cases when you have a workers that is sequential…). It could do almost anything. It doesn’t matter.

The enqueue call is technically sequential — but the consequence of the call does not block the return from the enqueue.

When Parallel and Async is Miserable

Then why don’t more people use async and parallel routinely?

Because an enormous amount of real code works like this:

let from_a = taskA()
let from_b = taskB(from_a)
let result= taskC(from_b)

That makes explicit the dependency on the results of a prior computation. There are some tricks and tools to parallelize that, but they are certainly not easily understood.

Most people would in fact have written the above this way:

let result = taskC(taskB(taskA()))

Now, what if taskB(…) happens to be an async call, but the others aren’t? Then the entire computation has to become await/async compatible. It propagates.

This is one of the main reasons I actually avoid the async/await syntax and use message buses and queues for internal routing. It allows me to easily invoke (synchronously) even when there are asynchronous effects because of message bus delivery and enqueuing through the queues. It makes it much easier to test.

But, a side-effect of my style is that I’m never hiding the “complexity” of a message and queue driven model. I find that the dramatically lower coupling, to the point of not “baking in” sources and destinations, makes for cleaner logic and simpler testing.

Other people find that they can’t “sit down and read the code” and know what it does. My logic uses state machines and guards instead of nested layers of calls and callbacks and instead of nested ifs and case statements.

The coupling, once removed, also removes the ordering.

Modeling the Real

Most of the real world is in fact parallel and async. Here’s a “program” that I suspect most people are familiar with:

let pizzaria = phonebook.fetchPizzaPlace()
pizzaria.orderPizza(new PizzaRequest(["meat lovers", "extra cheese"]))

Now, would you stand motionless until the pizza arrives? Probably not …

let tvRemote = room.getRemote("tv")
tvRemote.play(["movie", "Fifth Element"])

And you’ll be interrupted so you do this:

let door = room.getDoorbell()

door.onRing(new VisitorHandler())

And that’s it. You’re now watching a movie until the pizza arrives, and you’ll even handle it if an Amazon delivery comes first, because the VisitorHandler() is smart.

That’s what reality looks like. We might also take a phone call, while the movie plays, while we’re waiting for a pizza. Those are all different concurrent things. They are all in parallel. And they are all async.

If you model a genuinely hard problem, it’s almost always going to be easier to model it as people would do it which will be parallel and async. Translating that into sequential/synchronous can be much harder than it appears.

Learn to program such that your problem and your solution work in the same vocabulary and your assumptions remain consistent between problem and solution.

Conclusion

Be very certain that you understand what’s really happening whenever you use libraries, tools, languages, etc., that “help you” by simplifying the syntax and “making” asynchronous calls seem synchronous or offering concurrency mechanisms.

Forcing the world into sequential steps well is actually very hard and implies dependencies even if those dependencies are purely happenstance. This in turn causes artificially forced sequential implementations to diverge from the problem space model. This is the normal case for most software and it’s part of why teams are so eager to re-write instead of fix.

The real world is a great model and we know it works. Let’s allow the computers to do what we do.

Keep the Light!