Damir's Corner.

What code are your unit tests testing?

Mocks can be a helpful tool for replacing external dependencies in unit tests. However, caution is required when you embark on that route or you could end up with tests that don't really test your code under test.

Let's say you want to test the following method (from an all-too-common to-do list example):

public async Task<ToDoItem> CompleteAsync(int id)
{
  var item = await this.repository.GetAsync(id);

  if (item == null)
  {
    throw new ArgumentException($"No item with id {id}.", nameof(id));
  }

  item.Completed = true;
  return await this.repository.UpdateAsync(item);
}

It's responsible for marking a to-do item as complete. It depends an a repository which uses EF Core for persistence.

In a unit test, you would usually mock the repository so that you could test the method in isolation. The following test method appears to be doing just that:

[Test]
public async Task CompleteAsyncWorks()
{
  var repositoryMock = new Mock<IToDoRepository>();
  var service = new ToDoService(repositoryMock.Object);

  var incompleteItem = new ToDoItem
  {
    Id = 1,
    Label = "Write test",
    Completed = false,
  };

  var completeItem = new ToDoItem
  {
    Id = incompleteItem.Id,
    Label = incompleteItem.Label,
    Completed = true,
  };

  repositoryMock
    .Setup(m => m.GetAsync(It.IsAny<int>()))
    .ReturnsAsync(incompleteItem);
  repositoryMock
    .Setup(m => m.UpdateAsync(It.IsAny<ToDoItem>()))
    .ReturnsAsync(completeItem);

  var result = await service.CompleteAsync(incompleteItem.Id);

  Assert.That(result.Completed, Is.True);
}

However, this test would succeed even if the method under test didn't mark the to-do item as complete. Take a closer look at it and try to figure out why.

The only assertion in the test is verifying the status of the item returned by the method under test which is passed directly from the mocked repository UpdateAsync method. Since this value is hardcoded in the test, it doesn't matter how the method under test is modifying its instance of the to-do item. At best, this test is testing that the mocking framework works as expected.

And how can the test be modified so that it will test the method implementation instead? Most importantly, it must verify the status of the to-do item instance that the method under test is modifying. For that to work, the mocked UpdateAsync method should return the same value that is originally passed to it just like its real counterpart would:

[Test]
public async Task CompleteAsyncMarksItemAsComplete()
{
  var repositoryMock = new Mock<IToDoRepository>();
  var service = new ToDoService(repositoryMock.Object);

  var item = new ToDoItem
  {
    Id = 1,
    Label = "Write test",
    Completed = false,
  };

  repositoryMock
    .Setup(m => m.GetAsync(It.IsAny<int>()))
    .ReturnsAsync(item);
  repositoryMock
    .Setup(m => m.UpdateAsync(It.IsAny<ToDoItem>()))
    .Returns<ToDoItem>(i => Task.FromResult(i));

  var result = await service.CompleteAsync(item.Id);

  Assert.That(item.Completed, Is.True);
}

The new test is much better at its job. If we break the implementation of the method under test, the test will fail.

The test code can still be difficult to understand without looking at the implementation of the method under test. Also, if we change the implementation of the method, it's likely that the test will also have to be changed.

In such cases, integration tests are often a better fit than unit tests. Instead of testing a single unit in isolation (the CompleteAsync method), we would test some if its dependencies at the same time.

For our method, we could include the repository and the EF Core database context. To keep everything in process and avoid file I/O we can use the in-memory database provider:

[Test]
public async Task CompleteAsyncMarksItemAsComplete()
{
  var serviceProvider = new ServiceCollection()
    .AddDbContext<ToDoContext>(opts => opts.UseInMemoryDatabase("ToDoDb"))
    .AddTransient<IToDoRepository, ToDoRepository>()
    .AddTransient<ToDoService>()
    .BuildServiceProvider();

  var service = serviceProvider.GetRequiredService<ToDoService>();
  var item = await service.CreateAsync("Write test");

  var result = await service.CompleteAsync(item.Id);

  Assert.That(item.Completed, Is.True);
}

This test also verifies that the method under test changes the to-do item status as expected. And it does it without mocking any of the dependencies. This makes the test less fragile, and also easier to understand. I decided to use dependency injection to simplify my job of composing service dependencies.

You can find code with all three tests in my GitHub repository. Try breaking the CompleteAsync method (e.g. set the Completed flag to true) and check which of the tests fail as they should.

When mocking dependencies in unit tests, you should always make sure that you're still testing the method under test and not the mocking framework. As a quick sanity check, try breaking the method under test and run the test again. It should fail, or you're doing something wrong.

If setting up mocks for your unit tests is getting too complicated, consider writing integration tests instead. As long as you make sure that all the code is still running in process and without I/O, you can keep them as fast and reliable as unit tests.