Easy, single-statement unit tests

Our testing framework lets us test our code more extensively, and with less effort. This makes the code simpler, easier to refactor, and results in fewer bugs.

Audience requirements: Java; unit testing.

Some background: our architecture

We divide all our java classes into 2 categories:

  • data/noun classes store data only, and have almost no logic, except for construction preconditions.
  • verb classes are stateless and implement the business logic.

Opinions may differ on this style, but it works well with dependency injection (e.g. Google Guice), and we have seen it used successfully elsewhere, so we like it.

We make heavy use of data classes instead of ‘more raw’ types. For example, we have a Partition class for representing “a collection of items with weights that sum to 1, i.e. 100%”. It is better than a simple Map<T, Double> because:

  1. Single check: We only need to check once that sum(values) ~= 1, at the Partition constructor. Otherwise, if 7 methods took a raw Map<T, Double> argument with implicit partition semantics, we would need to have 7 preconditions in the code. That would clutter the production code, and also require 7 more tests. Secondarily, it’s better performance to check the preconditions only once.
  2. Clearer semantics than relying on the name of a parameter, which may not be descriptive – or even correct.
  3. Type safety avoids bugs. Example: say OptimizationResult stores the solution from the optimizer in a Map<AssetClass, Double>, but there’s no restriction that the values sum to 1 (e.g. perhaps it is in dollars, not %)1. Passing a Partition instead of an OptimizationResult will not compile, which is good. However, with a raw Map<AssetClass, Double>, it is possible to make such a mistake.

With 600+ data classes, and 185,000+ lines of test code (as of this writing), it pays to have systematic ways to test data classes, and to simplify the testing of verb classes.

Matchers

We make extensive use of the Hamcrest matcher framework. You can think of a matcher as a test-only, parametrizable equals().

We only implement hashCode & equals in our data classes in very few cases, such as when its instances will be used as keys to some map. Otherwise, we prefer using a matcher:

  • Since matcher code lives in test files, we avoid cluttering up prod code with test-oriented logic.
  • Matchers are like a parametrizable equals() in a way. For instance, we can compare two objects subject to an epsilon passed in, and use a different epsilon in different cases.
  • Matchers make it easier to read and write unit tests of ‘verb classes’.

Our data classes each have corresponding test classes that extend RBTestMatcher, a class that we built (simplified below):

public abstract class RBTestMatcher {
  public abstract T makeTrivialObject();
  public abstract T makeNontrivialObject();
  public abstract T makeMatchingNontrivialObject();
  protected abstract boolean willMatch(T expected, T actual);
}

Here is a concrete example for a TradeAmounts class, which stores the $ amount bought and sold, broken down by stock.

public class TradeAmountsTest extends RBTestMatcher {

  public static TradeAmounts emptyTradeAmounts() {
    return tradeAmounts(emptyBoughtTradeAmounts(), emptySoldTradeAmounts());
  }

  @Override
  public TradeAmounts makeTrivialObject() {
    return emptyTradeAmounts();
  }

  @Override
  public TradeAmounts makeNontrivialObject() {
    return tradeAmounts(
        new BoughtTradeAmountsTest().makeNontrivialObject(),
        new SoldTradeAmountsTest().makeNontrivialObject());
  }

  @Override
  public TradeAmounts makeMatchingNontrivialObject() {
    return tradeAmounts(
        new BoughtTradeAmountsTest().makeMatchingNontrivialObject(),
        new SoldTradeAmountsTest().makeMatchingNontrivialObject());
  }

  @Override
  protected boolean willMatch(
        TradeAmounts expected,
        TradeAmounts actual) {
    return tradeAmountsMatcher(expected).matches(actual);
  }

  public static TypeSafeMatcher tradeAmountsMatcher(
        TradeAmounts expected) {
    return makeMatcher(expected,
        match(v -> v.getBoughtTradeAmounts(), f -> boughtTradeAmountsMatcher(f)),
        match(v -> v.getSoldTradeAmounts(),   f -> soldTradeAmountsMatcher(f)));
  }

}

Subclasses of RBTestMatcher just need to implement:

  • makeTrivialObject to return the most trivial object possible (in this case, nothing traded).
  • makeNontrivialObject to return most general object possible (e.g. we bought and sold different amounts, and in multiple stocks – not just one).
  • makeMatchingNontrivialObject that should ‘match’ makeNontrivialObject, but not be equal in the traditional sense of equality: e.g. the two could be off by some epsilon.
  • willMatch: a simple predicate to return true if two objects are similar enough to consider to be matching.

By convention, we also add a static method that returns a matcher (here, tradeAmountsMatcher). Hidden behind the syntactic sugar of makeMatcher() and match() is the fact that it in turn relies on boughtAmountsMatcher and soldAmountsMatcher. Since all of our data test classes adhere to the RBTestMatcher convention, this is not a restriction.

Of course, additional tests may be necessary, most commonly for preconditions. For example, TradeAmountsTest also tests (not shown here) that a stock cannot both be bought and sold.

There are several advantages to having every data test class extend RBTestMatcher:

  • We get a free unit test for the matcher logic. TradeAmountsTest inherits a unit test called RBTestMatcher#matcherMetaTest() (not shown), which checks that:
    • makeTrivialObject does not match the two non-trivial objects
    • all 3 objects match themselves
    • makeNontrivialObject matches makeMatchingNontrivialObject
  • We get a free unit test for toString(), also inherited from RBTestMatcher. It’s a simple test that just calls toString() and ignores the return value, but at least it ensures no exceptions are thrown.
  • We some free rudimentary testing of the constructors, insofar as no exceptions are thrown from within the 3 make*Object methods.
  • It standardizes the format of the data class test code, making it easier to read.
  • It lets other tests construct a realistic test object without needing to know how to construct one (e.g. knowing what would constitute valid constructor parameters). For example, TradeAmountsTest#makeNontrivialObject builds an object by utilizing in turn BoughtTradeAmounts#makeNontrivialObject.

Single-statement unit tests

Another big benefit of all data classes having matchers is that verb class unit tests can be much more concise – in some cases, a single Java statement.

Take a look at this extensive test2 of summarize() in TradesToTradeAmountsSummarizer. The logic is fairly straightforward: it adds up the corresponding amounts bought and sold in the Trades object in the input, and summarizes to a TradeAmounts object, which only records amounts bought and sold.

assertThat(
    makeTestObject().summarize(
        trades(
            boughtTrades(iidMapOf(
                STOCK_B1, boughtTradesWithOrder(
                    buyOrder(STOCK_B1, buyQuantity(11), price(100.10), DUMMY_TIME),
                    rbSetOf(
                        boughtTrade(STOCK_B1, buyQuantity(2), price(100.02), DUMMY_TIME),
                        boughtTrade(STOCK_B1, buyQuantity(3), price(100.03), DUMMY_TIME))),
                STOCK_B2, boughtTradesWithOrder(
                    buyOrder(STOCK_B2, buyQuantity(44), price(100.04), DUMMY_TIME),
                    rbSetOf(
                        boughtTrade(STOCK_B2, buyQuantity(5), price(100.05), DUMMY_TIME),
                        boughtTrade(STOCK_B2, buyQuantity(6), price(100.06), DUMMY_TIME))))),
            soldTrades(iidMapOf(
                STOCK_S1, soldTradesWithOrder(
                    sellUnspecifiedLots(STOCK_S1, sellQuantity(11), price(200.10), DUMMY_TIME),
                    rbSetOf(
                        soldTrade(STOCK_S1, sellQuantity(2), price(200.02), DUMMY_TIME),
                        soldTrade(STOCK_S1, sellQuantity(3), price(200.03), DUMMY_TIME))),
                STOCK_S2, soldTradesWithOrder(
                    sellUnspecifiedLots(STOCK_S2, sellQuantity(44), price(200.04), DUMMY_TIME),
                    rbSetOf(
                        soldTrade(STOCK_S2, sellQuantity(5), price(200.05), DUMMY_TIME),
                        soldTrade(STOCK_S2, sellQuantity(6), price(200.06), DUMMY_TIME))))))),
    tradeAmountsMatcher(
        tradeAmounts(
            boughtTradeAmounts(iidMapOf(
                STOCK_B1, money(doubleExplained(  500.13, 2 * 100.02 + 3 * 100.03)),
                STOCK_B2, money(doubleExplained(1_100.61, 5 * 100.05 + 6 * 100.06)))),
            soldTradeAmounts(iidMapOf(
                STOCK_S1, money(doubleExplained(1_000.13, 2 * 200.02 + 3 * 200.03)),
                STOCK_S2, money(doubleExplained(2_200.61, 5 * 200.05 + 6 * 200.06)))))));

This was a clean implementation as a single Java statement.

Let’s imagine we didn’t have tradeAmountsMatcher, and instead implemented hashCode/equals in TradeAmounts, BoughtAmounts, and SoldAmounts.

A simple assertEquals might not work, because double arithmetic may result in tiny deviations. Of course, we could relax the various equals() methods to check equality subject to some epsilon. However, that gets complicated quickly: the epsilon may not always be a fixed amount, yet equals() cannot take epsilon as a parameter.

Therefore, the test logic would have to be something like this (shown in pseudocode):

  • Get output of summarize()
  • For BoughtAmounts:
    • Check that the map has entries for B1 and B2 (no more or fewer stocks)
    • For B1, confirm the $ amount
    • For B2, confirm the $ amount
  • Same for SoldAmounts

The deeper the structure of an object (e.g. a tree of collections of trees), the faster this will get unwieldy very quickly. Instead, our test matcher allows us to write such a test as a single Java statement.

Summary

We are big proponents of automated testing, instead of manual QA. To that end, we have built infrastructure to make it much easier to write and read automated tests. This results in test code of higher quality and wider coverage, making it easier to improve the code via refactors, and to catch bugs.


Notes

1. This is not a good example in the context of our own code base, because we rarely use plain doubles anyway. Even if we used raw maps in our code, the Partition would be a Map<AssetClass, UnitFraction>, where UnitFraction is forced to be within 0 and 1, and the OptimizationResult (if it were in $ terms, which it is not) would be a Map<AssetClass, Money>. Those two are not interchangeable, even by accident. A better example would be a TargetPartition and HeldPartition, where each can instead be represented as a raw Partition, with the former representing the target/goal, and the latter representing the current holdings before any trading. The two are conceptually different, so we should never want to use one instead of the other, yet both contain a Partition inside them.

2. Inside this test code, doubleExplained() is a handy in-line way to show an actual value that gets used in a test alongside the calculation that gave rise to it. Otherwise, the actual value used may look arbitrary. Also, DUMMY_TIME is a predefined constant which we use to imply that time does not matter in a calculation; here, that makes sense, because we are just summing dollar amounts.

Author: Rowboat Advisors

Rowboat Advisors builds software for sophisticated and fully automated portfolio management for the financial advisor industry.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s