16/11/2016

3 reasons why I don't use strict mocks

Home


Source: own resources, Authors: Agnieszka and Michał Komorowscy

The majority, if not all, of mocking frameworks provides 2 types of mocks i.e. strict & loose. The difference between them is that the strict mocks will throw an exception if an unexpected (not configured /set up) method was called. I prefer to use loose mocks because with strict ones unit tests are fragile. Even the small change in the code can cause that unit tests will start failing. Secondly, if you need to set up many methods a test becomes less readable. Now, I can see one more reason.

Today, my colleague showed me a bug in my tests that were using strict mocks. He started investigation because we observed the following strange erros occuring during builds on the server:

System.Runtime.Serialization.SerializationException: Unable to find assembly 'Moq, Version=4.5.22.0, Culture=neutral, PublicKeyToken=69f491c39445e920'.

It was not easy to find a root cause because he couldn't reproduce a problem locally. However, finally he found what was wrong. The problem laid in the fact that some of classes being mocked were implementing IDisposable interface. It means that at some point of time the garbage colector was trying to dispose them. However, the strict mocks were not expecting calls to Dispose method so they were throwing exceptions.

If you want to reproduce a problem try the following simplified code.
using System;
using Moq;
using Moq.Protected;

namespace Sandbox
{
    class Program
    {
        static void Main(string[] args)
        {
            var mock = new Mock<BaseClass>(MockBehavior.Strict);

            //To fix a problem uncomment this line
            //mock.Protected().Setup("Dispose", ItExpr.IsAny<bool>());

            //If you call Dispose method directly in your code, then you also must setup the public Dispose method.
            //However, in this case the public Dispose method must be virtual because Moq can work only with virtual methods.
            //mock.Setup(x => x.Dispose());

            var obj = mock.Object;
        }
    }

    /// <summary>
    /// A model implementation of IDisposable interface
    /// https://msdn.microsoft.com/pl-pl/library/system.idisposable(v=vs.110).aspx
    /// </summary>
    public class BaseClass : IDisposable
    {
        private bool disposed = false;

        public void Dispose()
        {
            Dispose(true);
            GC.SuppressFinalize(this);
        }

        protected virtual void Dispose(bool disposing)
        {
            if (disposed) return;

            if (disposing) { /* ... */ }

            disposed = true;
        }

        ~BaseClass() { Dispose(false); }
    }
}
To sum up. The 3rd reason why I prefer loose mocks over strict ones is that sometimes it may be difficult to figure out which methods are actually used.

2 comments:

Unknown said...

I cannot agree with You. When adding new method to class breaks Your tests, it's obviously a design issue. Broken tests may tell You, that You are violating Open-Closed principle. So You want to rethink Your solution.
That exception You provided - main problem is calling Dispose in destructor. If i remember correctly, You cannot tell when destructor will be called, and You cannot be 100% sure that it will be finished(example - You want to free the handle, which was "destroyed" before(Weak reference, any other thingie that allows GC to remove object, even, when it can be reached). So, IMHO, calling "Dispose" method should be determined accurately by programmer.
Thrid reason You provided - when You cannot tell when method is used - something is wrong. If Your code is nondeterministic - how You can assure that it works?

Michał Komorowski said...

"When adding new method to class breaks Your tests, it's obviously a design issue."

Adding a method to a class will not break unit tests based on strict mocks (at least it shouldn't). However, using this new method can break unit tests because strict mocks works in this way. Unit tests can also start failing if you change order of methods calls, change values of parameters... The problem is that unit tests will start failing not because something is wrong in your code, but because a configuration of a test is not proper.

"That exception You provided - main problem is calling Dispose in destructor...."

You are right that it's not guaranteed when a destructor will be called. However, it is how Dispose pattern should be implemented according to my knowledge. Of course, it's better to explicitly call Dispose method or use using clause and not to wait for GC.

"Thrid reason You provided - when You cannot tell when method is used - something is wrong."

You have to remember that unit tests is not the same thing as a fully fledged application. In the real application, for example a DI container may be responsible for disposing objects. Whereas in unit tests we rather creates objects on our own. In that case we may "forget" that a given class is disposable (as in the example).

Besides, my 3rd reason doesn't state that it is difficult to say "when method is used". It says that it is difficult to say "which method are used". It may sound surprising but if you work with big projects, that are developed by many developers... you simply doesn't know all the details and you may overlook that some methods should be configured for a strict mock.

Post a Comment