Test Flow and Method Contracts

facebooktwitterdiggdzonestumbleuponredditdelicious


 
Today’s (long overdue) blog entry is inspired by a recent twitter discussion I’ve been following. Uncle Bob (aka Robert Martin) made the bold statement that 100% test coverage should simply be a matter of conscience.
 
Now, I’m not going to delve into my thoughts on the discussion. However, I find it distressing that one of the biggest arguments I see against high test coverage appears to be “but tests don’t guarantee the code works…”.
 
As Bob said, “Tests cannot prove the absence of bugs. But tests can prove that code behaves as expected.”
 
 

What are automated tests?

Proponents of automated testing list a great many reasons why they believe in it. To name a few:

  • Prevent code/bug regression
  • Ease of refactoring
  • Provide confidence in code behaviour
  • Reduce (or eliminate?) time spent testing manually

 
All of these points actually come back to one thing:
Tests mean you know what the code does
 
Note that I didn’t claim the code works, just that you know what it does. That differentiation is important. I’ll talk more about the infallibility of tests after.
 
 

Function Contracts

Those familiar with design by contract or Liskov’s Substitution Principle are familiar with the idea of preconditions and postconditions:
 
Precondition – a condition or predicate that must always be true just prior to the execution of some section of code or before an operation
Postcondition – a condition or predicate that must always be true just after the execution of some section of code or after an operation
 
A more formal coding of pre and post conditions takes the form of Hoare Triples. A Hoare Triple is a statement which essentially says “given some precondition, the execution of my code will produce some postcondition”.
 
 

Tie it Together

While most of us don’t think about it, automated tests are Hoare Triples.
 
Demonstration:

@Test
public void get_shouldReturnCachedValue_givenValuePutInCache() {
  //setup
  String key = "key";
  String expectedValue = "value";
  Cache cache = new Cache();
  cache.put(key, expectedValue);
 
  //execute
  String result = cache.get(key);
 
  //assert
  assertThat(result, equalTo(expectedValue));
}
 
@Test
public void get_shouldHaveValue_givenValuePutInCache() {
  //setup
  String key = "key";
  String expectedValue = "value";
  Cache cache = new Cache();
  cache.put(key, expectedValue);
 
  //execute
  boolean result = cache.has(key);
 
  //assert
  assertTrue(result);
}

 
If you put this in terms of a Hoare Triple ({P} C {Q}):
{ cache.put(x, y); } r = cache.get(x) { r == y }
{ cache.put(x, y); } r = cache.has(x) { r == true }
 
You also might note that the method signature I chose is simply the Hoare Triple written as C {Q} {P}. This is the current practice of the team I work on, but ensuring all three clauses of the triple are distinguishable in a test name has been extremely valuable to us.
 
Note: The test could be more terse, but splitting setup/execute/assert allows us to think in terms of {P} C {Q}
 
 

Taking it a Step Further: Mocking

Some of you may be thinking “ok yeah, that test was easy”. It’s true. In the example we didn’t have to interact with external dependencies. But how can we ensure the system works correctly with external dependencies?
 
Note: The concept of “from repository” is somewhat simplified to keep things short. Imagine a more complex world where a couple of DAOs had to be combined to create a Member.

Cache mockCache = mock(Cache.class);
Repo mockRepo = mock(Repo.class);
MemberLookupService service = new MemberLookupService(mockCache, mockRepo);
 
@Test
public void getMember_shouldReturnMemberFromCache_whenCachedValuePresent() {
  //setup
  String memberId = "member id";
  Member member = new Member(memberId);
  when(mockCache.hasKey(memberId)).thenReturn(true);
  when(mockCache.get(memberId)).thenReturn(member);
 
  //execute
  String result = service.getMember(memberId);
 
  //assert
  assertSame(member, result);
}
 
public void getMember_shouldReturnMemberFromRepository_whenCachedValueNotPresent() {
  //setup
  String memberId = "member id";
  Member member = new Member(memberId);
  when(mockCache.hasKey(memberId)).thenReturn(false);
  when(mockRepo.find(memberId)).thenReturn(member);
 
  //execute
  String result = service.getMember(memberId);
 
  //assert
  assertSame(member, result);
}
 
public void getMember_shouldPlaceMemberInCache_whenValueLookedUpFromRepository() {
  //setup
  String memberId = "member id";
  Member member = new Member(memberId);
  when(mockCache.hasKey(memberId)).thenReturn(false);
  when(mockRepo.find(memberId)).thenReturn(member);
 
  //execute
  String result = service.getMember(memberId);
 
  //assert
  verify(mockCache).put(key, result);
}

 
Here we’ve created three Hoare Triples.
{Cache.has == true; Cache.get(x) == y; } r = getMember(x) { r == y }
{Cache.has == false; Repo.getMember(x) == y} r = getMember { r == y}
{Cache.has == false; Repo.getMember(x) == y} r = getMember { Cache.has(x) == true }
 
 

Reasoning About our Code

 
Now that we’ve built up a set of Hoare Triples, let’s attempt to reason about our code. We have established a system with the following rules:
 
{ cache.put(x, y); } r = cache.get(x) { r == y }
{ cache.put(x, y); } r = cache.has(x) { r == true }
{Cache.has == true; Cache.get(x) == y; } r = getMember(x) { r == y }
{Cache.has == false; Repo.getMember(x) == y} r = getMember { r == y}
{Cache.has == false; Repo.getMember(x) == y} r = getMember { Cache.has(x) == true }
 

Based on this, let’s create a scenario and pose a question. Here’s the scenario:

  • Cache.put has not been called with key “Travis”
  • “Travis” exists in the Repository, it is not null

 
The question:
Is it possible for MemberLookupService.getMember(“Travis”) to return null?
 
For the answer, I’ll refer you to Modus Ponens. In specific, when given the rule “P => Q”, if you know “not P” you cannot reason about “Q”. All potential values for “Q” are possible.
 
So can “getMember” return null? Yes. We’ve not established any rules about what “has” does when there’s nothing in the cache.
 
 

Fix the bug

To fix the bug, we need to add a couple more tests, as well as whatever code makes our entire test base pass:
 

@Test
public void get_shouldReturnNull_givenEmptyCache() {
  //setup
  String key = "key";
  Cache cache = new Cache();
 
  //execute
  String result = cache.get(key);
 
  //assert
  assertThat(result, is(nullValue()));
}
 
@Test
public void has_shouldReturnFalse_givenEmptyCache() {
  //setup
  String key = "key";
  Cache cache = new Cache();
 
  //execute
  boolean result = cache.has(key);
 
  //assert
  assertFalse(result);
}


Creating the following rules:
{new Cache} r == Cache.has(x) {r == false}
{new Cache} r == Cache.get(x) {r == null}
 
Based on our earlier scenario, we will no longer receive null:
{new Cache} r == Cache.has(Travis) {r == false}
{Cache.has(Travis) == false; Repo.getMember(Travis) == y} r = getMember { r == y}
 
 

The Code Behaves as Expected

While I don’t spend every day thinking about Hoare Triples and the predicate calculus behind my system, it’s all still there. Whether reasoning formally about our system, or informally, we do it based on what we believe the rules of our system to be.
 
Tests prove that these logical rules exist. Correct tests prove that they are the rules that we think they are. Whether the test is manual or automatic, as long as it is correct it can prove that we are correct about what rules govern our software.
 
Of course, this requires the tests to be correct. Tests are fallible as well. Automating our tests is how we address the fallibility of the tester, but I’ll go into that next time.


facebooktwitterdiggdzonestumbleuponredditdelicious

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s