pseudoramble

This seems like the kind of thing that could change year-by-year, but it came up in a Hacker News thread recently. So I thought I’d jot down my current thoughts, which are a sum of my previous experiences and changes over time.

Good unit tests are extremely valuable

They tend to be executed the fastest, so I can run them with a watcher and execute them each time I change some files. They’re also the shortest in length when implemented well, which makes understanding the specifics of the test better.

They also help me do things like organize my thoughts, and explain the intent of my code much better. It’s great that they can run and at least verify the cases that were covered are handled.

My definition of a unit is changing

I used to think of a unit as mostly a code-unit. So, an individual function, module, or class. Often in code bases I stick with that because that’s what the standard was at at the time. I think in newer code bases I would move away into a concept that focuses more deeply on the behavior and expected outcomes more specifically, but within the confines of the code base itself.

Example: if I’m building a standard CRUD app with a layered architecture like:

[Controller] -> [Service] -> [Repository] -> DB

Originally I would define a unit here as the controller, the service, the repository, and ignored the DB. It’s sort of a silly definition when you think about it for unit tests. What I care about for unit tests is that when somebody says “Give me all of the users where the age is > 30” that they eventually asked the repository for the correct thing and did appropriate processing, not that the Controller invokes the Service for a particular method.

I think a future code base would combine at least the Controller + Service into a single set of tests. Consolidating this would make ratio of test code to lines tested significantly improved, with fewer tests having better value. I think I would simply mock the Repository interface here since that feels like a fairly clean break that I could define independently from the logic itself, though a fake DB seems like a perfectly fine alternative too.

Unit tests should not be used everywhere

There are places where it just doesn’t make sense to unit test. In particular, if your code is defined in terms of another thing (an external API, database, whatever) then I don’t believe you’re actually looking at a single unit. You end up mocking the dependency out, and at that point you’re trying to cover something outside your span of control implementation-wise, which I think leads to inaccurate assumptions about how that dependency works.

Using the example above, I think it’s reasonable to say that the Repository class is strictly defined in terms of your backing database, a think which again you don’t control implementation-wise. You can use ORM’s or other database abstractions to maybe reduce that fact, but no matter what your code at some point have to generate a query and give it to the database. You could implement the test to validate the query is written correctly, but that requires fairly weak tests that could break when minor query changes are made. Maybe you implement a fake database then instead, which improves things but doesn’t make it clear that your queries work against the actual database.

In this case, rolling up a small actual instance of the DB and running integration tests between the two increases the value substantially. It helps your tests reduce the knowledge of implementation details of how things are stored and retrieved, and you can focus on the behavior of the code itself. I’ve used this technique too to actually migrate Repository code to a different database provider. It’s a pretty powerful tool, one we would never have had if we only used unit tests.

You may be thinking “Yeah, duh” but I don’t think it’s so obvious. I’ve had this argument multiple times with others who disagree with me on it. It has been a while, but I think the gist of their arguments are usually that if you mock the dependencies carefully enough you can get the desired outcomes, and that the process of setting up test databases is too much. The latter point is legitimate, although I think given how powerful computers are and the availability of cloud resources nowadays reduces this burden.