The tricky cases in testing - Visibility

Separate aspects: Integration vs. operation

 

During implementation, I always observe the IOSP, which Integration Operation Segregation Principle. It states that a method is either Integration or Operation should be. The following excerpt shows this using a simple example:

public class Configuration
{

    public static IDictionary ToDictionary(string configuration) {
        var settings = SplitIntoSettings(configuration);
        var keyValuePairs = SplitIntoKeyValuePairs(settings);
        var dictionary = CreateDictionary(keyValuePairs);
        return dictionary;
    }

    internal static IEnumerable SplitIntoSettings(string configuration) {
        return configuration.Split(';');
    }

    internal static IEnumerable<KeyValuePair> SplitIntoKeyValuePairs(
            IEnumerable settings) {
        foreach (var setting in settings) {
            var keyAndValue = setting.Split('=');
            yield return new KeyValuePair(
                keyAndValue[0], keyAndValue[1]);
        }
    }

    internal static IDictionary CreateDictionary(
            IEnumerable<KeyValuePair> keyValuePairs) {
        var result = new Dictionary();
        foreach (var keyValuePair in keyValuePairs) {
            result.Add(keyValuePair.Key, keyValuePair.Value);
        }
        return result;
    }
}

The method ToDictionary belongs to the category Integration. This method is responsible for calling other methods. It integrates these methods. The methods SplitIntoSettings, SplitIntoKeyValuePairs and CreateDictionary are against it Operations. These methods contain the Domain logic and do not call any other methods, apart from methods from frameworks.

Domain logic - Logic that relates to the topic of the application.

Test strategy

 

With the clear separation of integration and operation, it makes a lot of sense to use the Operations isolated to test. These methods are leaves in the dependency tree, so they are already isolated anyway. The integration methods, on the other hand, have dependencies on the methods that they integrate. Testing them in isolation would mean replacing the real methods in the test with dummies. Although this is technically possible, the benefit is disproportionate to the effort involved. The I test integration methods so only with integration tests. As a result, I have few integration tests via the public API and many unit tests for the operations.

For the example ToDictionary I only need a single integration test. The method consists of a linear sequence of method calls. There is no branching and no loop. Consequently, the test coverage of this method is already achieved with a single integration test. I test my standard example here and check whether the following string is correctly transformed into a dictionary:

"a=1;b=2;c=3" -> {{"a", "1"}, {"b", "2"}, {"c", "3"}}

I then test the many syntactic details of this transformation with unit tests of the operations.

• I test whether the string is correctly split at the semicolons with unit tests on the method SplitIntoSettings.
• I test the decomposition of a single setting into key and value at the equals sign with unit tests on the method SplitIntoKeyValuePairs.
• I test the correct structure of the dictionary with unit tests on the method CreateDictionary.

This approach has the advantage that the individual tests are easier to formulate than with pure integration tests via the public API. This is mainly due to the fact that the structure of the respective test data is simpler and more focused. I can check the SplitIntoSettings method with the following test cases, for example:

"a;b" -> "a", "b"
"a;;b" -> "a", "b"
";a" -> "a"
"a;" -> "a"

These test cases are easier to formulate because it is not relevant which syntax the Settings (this is how I refer to the strings between the semicolons in this example).

The overall advantage is that a single unit test usually fails in the event of a problem. This gives me a very focused indication of exactly where the problem lies. If all tests were run exclusively via the public API, several integration tests typically fail in the event of a problem and it is therefore unclear which detail is causing the problem.

Open visibility with InternalsVisibleTo

 

The question remains as to how the operations can be called in the automated unit tests. Normally, the internals of the class would be set to private is set. This means that the operations would not be easily accessible for automated tests. I could call them via reflection. But that would be too much trouble for me. Quite apart from the fact that I would then have to be careful when refactoring, because the method names would then appear as strings in the tests.

So I set the operations to internal and complete the InternalsVisibleTo attribute on the implementation assembly. Please note that you must use the Assembly names of the assembly to which you want to grant access to the internal symbol. After creating a new project, the assembly name corresponds to the project name. However, if you change the project name, the original assembly name is initially retained. In the InternalsVisibleTo must contain the assembly name, not the project name.

 

Through the InternalsVisibleTo attribute, the internals of the class are visible to the test assembly and can therefore be tested automatically. These are whitebox tests because the tests now have knowledge of the internal structure and functionality of the implementation.

However, the internals are now also visible in the implementation assembly. The with internal marked methods of the class can be called from other classes within the same assembly. This means that the visibility of the internals is not only extended to the tests, but unfortunately also to the implementation assembly. I accept this disadvantage in favor of good testability. Within the team, the procedure must be known to all developers in order to avoid dependencies on internal methods. From a pragmatic point of view, I don't see this as a disadvantage. Even without this test strategy, developers should not be dependent on internal methods without thinking carefully about the possible consequences. For me internal on the same level as we private: internal, private stuff that you keep your hands off. Regular code reviews within the team can ensure that problems with dependencies are identified in good time.

Conclusion

 

Whitebox tests allow me to differentiate between integration tests and unit tests at the method level. Integration tests, related to the methods, test the public API, while unit tests target the internals of the solution. In this way, I achieve a healthy distribution of the number of tests: few integration tests, many unit tests.

The visibility of the internals must be softened somewhat for this test strategy, as C# does not offer any special "open for testing only" visibility, which is between private and internal should be.