Had a great venting session with someone at work about the practices of micro-services, the principals of component driven development, mocking in unit tests, and interfaces in Go. Maybe one day I’ll write all this up, but it was so cathartic to express how we can do better on all these fronts.

If anyone is to ask what I think, here it is in brief:

1. Micro-services might be suitable for what you’re building if you’re Amazon or Google, where you have teams of 20 developers working on a single micro-service. But if you’ve got a team of 20 developers working on the entire thing, you may want to consider a monolith instead. Easier to deploy, easier to operate, and you get to rely on the type system telling you when there’s an integration problem rather than finding it out during runtime.
2. The idea of component driven design — which is modelled on electrical engineering principals whereby a usable system is composed of a bunch of ICs and other discrete components — is nice in theory, but I think it’s outlived it’s usefulness for most online services. It probably still makes sense if you’re stuck in the world of Java and J2EE, where your “system” is just a bunch of components operating within a container, or if you actually are vending components to be reused. But most of the time what you’re working on is an integrated system. So you should be able to leverage that fact, rather than think that you’re building and testing ICs that you expect others to use. You’re not likely going to completely replace one component for another when you need to change a database (which you’re unlikely to do anyway). You’re more likely going to modify the database component instead. So don’t make that assumption in your design.
3. This also extends to the idea of unit testing, with the assumption that you must test the component in isolation. Again, you’re not building ICs that you’re expecting to transplant into other projects (if you are, then keep testing in isolation). So it makes more sense to build tests that leverage the other components of the system. This means building tests that actually call the components directly: the service layer calling the actual database driver, for example. This produces a suite of tests that looks like a staircase, each one relying on the layers below it: the database driver working with a mock database, the service layer using the actual database driver, and the handlers using the actual services. Your unit test coverage should only be that of the thing you’re testing: don’t write database driver tests in your handler package. But I don’t see a reason why you shouldn’t be able to rely on existing areas of the system in your tests.
4. The end result of doing this is that you’re tests are actually running on a mini-version of the application itself. This naturally means that there’s less need to mock things out. I know I’ve said this before, but the idea of mocking out other services in unit tests instead of just using them really defeats the idea of writing tests in the first place: gaining confidence in the correct operation of the system. How can you know whether a refactor was successful if you need to change the mocks in the unit test just to get them green again? Really, you should only need to use mocks to stub out external dependencies that you cannot run in a local Docker container. Otherwise, run your tests against a local database running in Docker, and use the actual services you’ve built as your dependencies. And please: make it easy for devs to run the unit tests in their IDE or in the command line with a single “make” command. If I need to set environment variables to run a test, then I’ll never run them.
5. Finally, actually using dependent services directly means there’s less need to defined interfaces up front. This, after starting my career as a Java dev, is something I’m trying to unlearn myself. The whole idea of Go interfaces is that they should come about organically as the need arise, not pre-defined from above before the implementation is made. That is the same level of thinking that comes from component design (you’re not building ICs here, remember?). Just call the service directly and when you need the interface, you can add one. But not before. And definitely not because you finding yourself needing to mock something out (because, again, you shouldn’t need to mock other components of the system).

Anyway, that’s my rant.

Continue reading →

I’m waiting for a data migration to finish, so I’ve naturally got migration scripts on my mind.

There’s an art to writing a good migration script. It may seem that simply throwing together a small Python script would be enough; and for the simpler cases, it very well might be. But it’s been my experience that running the script in prod is likely to be very different than doing test runs in dev.

Continue reading →

Manuel Moreale wrote an interesting post today about sharing stuff online:

Life can be joyful and wonderful and marvellous. But it can also be a fucking nightmare. And yes, it’s important to celebrate the victories and to immortalise the glorious moment. But it’s also important to document the failures, the shitty moments, the dark places our minds find themselves stuck in. It’s all part of what makes us unique after all.

Continue reading →

While poking through some old files this morning I came across probably the first bit of HTML I’ve ever written, way back on the 10th April 1996¹.

I think I vaguely remember making these. We were in Castlemaine staying over at my grandparents house and Dad bought along his laptop for us kids to play with (complete with a passive-matrix LCD and 2 hours of battery life). It was the evening and I was mucking around with Netscape Navigator. This was before we got the internet at home so I’m not quite sure why we even had Navigator. It may have been that Dad was using it for work, or maybe it came preinstalled on the laptop. But whatever the reason, it was there, and I was playing around with it.

Continue reading →

I’ve been thinking a little about how to support indexing in UCL, as in getting elements from a list or keyed values from a map.  There already exists an index builtin that does this, but I’m wondering if this can be, or even should be, supported in the language itself.

I’ve reserved . for this, and it’ll be relatively easy to make use of it to get map fields. But I do have some concerns with supporting list element dereferencing using square brackets. The big one being that if I were to use square brackets the same way that many other languages do, I suspect (although I haven’t confirmed) that it could lead to the parser treating them as two separate list literals. This is because the scanner ignores whitespace, and there’s no other syntactic indicators to separate arguments to proc calls, like commas:

Continue reading →