I’ve been skimming Dice to look at the market lately. Of course, most job postings are written with a list of libraries, but I realized when the Wall Street Journal and I spoke about it in my posting on The Staffing Crisis I never mentioned the reason why this approach doesn’t actually help the companies seeking people.

I can appreciate that teams want someone that knows Spring Boot and RESTful API design. They might also want to know how to consume Kafka messages, and they might want to know how to use Redis for pubsub, streams, or state.

What does any of this have to do with dependencies?

Dependencies

What are dependencies in software?

There are actually a huge number of kinds of dependencies. But in this case, I’m going to focus on just a small aspect of dependency management: the dependencies within a public API of a library or framework.

An example of such a dependency is a web controller for Spring Boot:

@RestController
public class OtterExampleController {
   @GetMapping("/")
   public String index() {
      return "Otter Example Invoked via Spring Boot";
   }
}

Now, in order to use those annotations (which are the ONLY parts of the code that actually refer to Spring Boot) the following import lines needed to come first:

import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.RestController;

So, aside from @RestController and @GetMapping, everything else is “plain old Java.”

That’s a testament to the quality of the Spring Boot library. Good tools should have minimal dependencies (there’s that word) to do their job.

Now, obviously using Spring Boot in a production system requires more of the tool than just the controller mapping mechanism. But what’s funny, it doesn’t require a lot more. The reason for using the tools is precisely because each of the tools, the dependencies, solves a specific set of problems.

This is why having such powerful libraries is a boon to people building modern enterprise systems. If you wonder how much code that replaced, feel free to start in Java with parsing HTTP request headers from a stream …

The Ugly

Now, as you start doing real systems, not toy examples, the libraries and frameworks start to permeate the system. This permeation is both natural and terrible.

Well written code has to be testable.

In order to be testable, well written code must have minimal dependencies. Any code that uses Spring Boot (such as my OtterExampleController) can’t be tested as easily as code that doesn’t use any external library at all. This is why the SDET ends up with mocks and layers of complex configuration. Bad programming makes for complex testing.

So, we have two competing goals:

1. The code must be easily and reliably tested
2. The code must take advantage of the frameworks and libraries

These are not conflicting goals. Not if your team members are skilled and your architects do their jobs properly.

However, naive coding practices, such as inserting dependencies freely into the code that represents your problem domain, are common in the industry. And we suffer terribly from it. The need for each person to understand all the dependencies (which is reflected in those job listings) is symptomatic of this.

Direct Dependency

One obvious solution is that there should be an interface between the code that represents the business work (such as processing accounting entries) and the code that represents the solution space (such as routing RESTful web requests).

This is not just my view. “Program to the interface” is a common guideline in Java, for instance. The reason for such guidance is expressly to reduce dependencies and to make it easier to test. I don’t need to mock (or instantiate!) a web client/web server in order to test a transaction handler if I can instantiate the objects that do the work and do not depend on the path that reached them.

For instance, here’s a dumb domain class:

public class Lamp {
   private bool lit;
   private bool energized;
   private Light light;
   public Lamp(Light aLight) { this.light = aLight; }
   public void energize() {...}
   public void deEnergize() {...}
   public void switch() {...}
}

That class (yes, the lamp again) requires a state machine internally in the three functions and it also requires knowledge of how the Light API works. It has a dependency on the Light.

If Light is an interface, then it’s easy to pass in a mock object. That makes testing easier. If Light is a class, then … that entire implementation dependency is drawn in too.

Of course Lamp could be wholly independent of the Light …

public class Lamp {
   public Enum Control { DoNothing, LightOn, LightOff }
   private bool lit;
   private bool energized;
   public Lamp() {...}
   public Control energize() {...}
   public Control deEnergize() {...}
   public Control switch() {...}
}
   

This makes the Lamp have no external dependencies at all. It returns the domain intention of the call instead of doing the work directly. That leaves the burden of using the Light (which this class is unaware of) with the caller.

It’s still stateful, but it’s trivial to test it without any mocks. The tester even knows exactly what actions the lamp expects to control.

Indirect Dependency

It’s very common to leak dependencies up the chain. Meaning some low-level code needs to “do something” (perhaps, set some header in an HTTP response) and so the HTTP response has to be passed through layers and layers of things that merely pass it but don’t use it.

This is where architecture comes in, and why good lead developers and architects are so important.

Junior Developer says, “Oh, we need the HTTPResponse …” to which the lead developer says, “No, you don’t. You need to allow the caller, which has an HTTPResponse, to understand why IT needs to do something in the HTTPResponse…”

How? Many ways.

1. Return a domain object that has rich intentions so the caller can understand the results in the domain (this separates the recognition of the internal happening to the processing of the happening)
2. Use exceptions! (I hate to be obvious, but if something fails, an exception that carries the details is more useful than trying to handle the failure with HTTPResponses deep in domain code)
3. Use an async model and send an event or message (more advanced)

There are other solutions, of course. The key is to decide, “We’re not going to leak our communication boundary details into our domain logic.” Once that decision is made, amazingly, solutions manifest rapidly and easily.

Then, surprise, code improves.

The Consequences of Dependency Mismanagement in Staffing

Imagine a larger team, 20+ people. On a team of four, where everyone wears all hats and there’s only a few hundred thousand lines of code, these issues don’t manifest as painfully.

In the large team, using RESTful web services, there are actually many concurrent areas of interest.

First, there’s the design of the actual RESTful services. The decision for what data, what links, etc., that allows the power of RESTful to be done at all. Regardless of whether it’s distributed behind a load balancer to a K8s driven set of containers or running on a laptop as a monolithic set of jars, the decision of the RESTful API itself is pure architecture and crucially important.

Implementing the endpoints to the API using Spring Boot is where the boundary hits. The server code needs to receive the HTTP request, route it, and pull out all the needed parts (and another entire set of libraries and tools such as JWT) and decide what the request’s intention is. Once it knows the intention, it calls domain code which is easily testable as defined above!

The people who implement the endpoints need to know Spring Boot, Spring HATEOAS, JWT, etc. That’s actually a small group of people. They’re the only ones that actually need to understand the boundary between receiving a request and bundling results.

Then, should it be decided to drive a batch process through Kafka queues and use the same domain objects, it’s easy to do this! A Kafka expert could be hired for just those new boundary cases…and then they could be done and leave.

When the code is not well isolated (or well abstracted) then every person, including people writing accounting (or lamp) logic must know it as well. THAT is when you need experts in all the libraries for each seat.

That has a few huge negatives for staffing:

1. It increases the skills of all team members needed (making it harder to bring in and train juniors, for instance)
2. It decreases the pool of available hires
3. It puts all the projects with poor architecture in competition for that limited pool

Projects with better code not only need fewer people, but they require fewer lists of dependencies.

And isn’t the reason for using all these dependencies to reduce our time to market and our costs?

Code matters. Architecture matters. A quick review of Dice and … it isn’t looking good for the software quality out there.

Dependencies are the enemy. They must be encircled and isolated — they are expensive, dangerous, and should be respected accordingly.