I’m sure everyone’s dying to hear how the mockless unit tests are going. It’s been almost two months since we started this service, and we’re smack bang in the middle of brownfield iterative development: adding new features to existing ones, fixing bugs, etc. So it seems like now is a good time to reflect on whether this approach is working or not.

And so far, it’s been going quite well. The amount of code we have to modify when refactoring or changing existing behaviour is dramatically smaller than before. Previously, when a service introduces a new method call, every single test for that service needed to be changed to handle the new mock assertions. Now, in most circumstances, it’s only one or maybe two tests that need to change. This has made maintenance so much easier, and although I’m not sure it mades us any faster, it just feels faster. Probably because there’s less faffing around unrelated tests that broke due to the updated mocks.

I didn’t think of it at the time, but it also made code reviews easer too. The old way meant longer, noisier PRs which — and I know this is a quality of mine that I need to work at — I usually ignore (I know, I know, I really shouldn’t). With the reviews being smaller, I’m much more likely to keep atop of them, and I attribute this to the way in which the tests are being written.

Code hygiene plays a role here. I got into the habit of adding test helpers to each package I work on. Much like the package is responsible for fulfilling the contract it has with its dependants, so too is it responsible for providing testing facilities for those dependants. I found this to be a great approach. It simplified the level of setup each dependant package needed to do in their tests, reducing the amount of copy and pasted code and, thus, containing the “blast radius” of logic changes. It’s not perfect — there are a few tests where setup and teardown were simply copied-and-pasted from other tests — but it is better.

We didn’t quite get rid of all the mocking though. Tests that exercise the database and message bus do so by calling out to these servers running in Docker, but this service also had to make calls to another worker. Since we didn’t have a test service available to us, we just implemented this using old-school test mocks. The use of the package test helpers did help here: instead of having each test declare the expected calls on this mock, the helper just made “maybe” calls to each of the service methods, and provided a way to assert what calls were recorded.

Of course, much like everything, there are trade-offs. The tests run much slower now, thanks to the database setup and teardown, and we had to lock them down to run on a single thread. It’s not much of an issue now, but we could probably mitigate this with using random database names, rather than have all the test run against the same one. Something that would be less easy to solve are the tests around the message bus, which do need to wait for messages to be received and handled. There might be a way to simplify this too, but since the tests are verifying the entire message exchange, it’d probably be a little more involved.

Another trade-off is that it does feel like you’re repeating yourself. We have tests that check that items are written to the database correctly for the database provider, the service layer, and the handler layer. Since we’re writing tests that operate over multiple layers at a time, this was somewhat expected, but I didn’t expect it to be as annoying as I found it to be. Might be that a compromise is to write the handler tests to use mocks rather than call down to the service layer. Those tests really only validate whether the hander converts the request and response to the models correctly, so best to isolate it there, and leave the tests asserting whether the business rules are correct to the server layer.

So if I were to do this again, that’ll be the only thing I change. But, on the whole, it’s been really refreshing writing unit tests like this. And this is not just my own opinion; I asked my colleague, who’s has told me how difficult it’s been maintaining tests with mocks, and he agrees that this new way has been an improvement. I’d like to see if it’s possible doing this for all other services going forward.