The articles in this series are all collected here: Exploring Smart Unit Tests (Preview)
[Update: Smart Unit Tests has been released as IntelliTest with Visual Studio 2015 Enterprise. My experience with IntelliTest is largely the same as with the preview version as shown in this article.]
Existing Code
As I mentioned previously, one of the best uses I can see for Smart Unit Tests is to run it against existing code to either generate a baseline set of tests or to fill in gaps of the existing tests. So, I decided to do some experimentation with some existing code that I had.
This is the sample project that I use in my presentation "Clean Code: Homicidal Maniacs Read Code, Too!". This code has a suite of unit tests already in place, and we use those tests as a safety net when we refactor the code. You can see a video of that refactoring on my YouTube channel: Clean Code: The Refactoring Bits.
This is a view model in a project that uses the MVVM design pattern. This means it contains the presentation logic for a form, and we have a set of 19 tests already in place:
Here's the "Initialize" method for this class:
We won't go into details too deeply here (you can get that from the video). The short version is that we have a couple of helper methods to get objects out of our dependency injection container. In this case, we are using the Service Locator pattern (which is considered an anti-pattern by some). This code is based on a real application, and we'll see a bit of funkiness as we go along.
To see a sample test, let's look at the 2nd line. Here, we get a model from our DI container and assign it to the backing field of our "Model" property. The test for this is pretty straight forward:
In our test, we create an instance of our view model and pass in our DI container as a parameter (we'll see how this gets initialized in a bit). Then we call the "Initialize" method and make sure that the "Model" property is not null.
We initialize our DI container in the test setup:
This creates an instance of our DI container (which is a UnityContainer) and assigns it to our class-level "_container" field. At the bottom of this method, we call into different methods to populate the PersonService (our service) and CurrentOrder (our model) in the container. These are split into separate methods so that we can easily put together different test scenarios -- for example, by registering a service that throws an exception to see how that is handled by the view model.
Here is the where we create a mock of the service:
This is a bit complex. This is because our service (represented by the "IPersonService" interface) uses the Asynchronous Programming Model (or APM). This is a pattern where we have pairs of "xxxBegin" and "xxxEnd" methods. This makes mocking and testing a bit interesting, and this is why we have the "IAsyncResult" types.
At the very bottom of this method, we put the mock service into the DI container with the "RegisterInstance" method.
Running Smart Unit Tests
So, we have a not-very-simple class. But at the same time, we do have reasonable isolation of our dependencies so that we can replace them in our tests.
Let's see how Smart Unit Tests responds when we run against the "Initialize" method. As a reminder, here's the method:
When we right-click on the method and choose "Smart Unit Tests", we get the following results:
What we get is a number of warnings. For this list, I selected the "Object Creation" issues. As we see, Smart Unit Tests is having some problems creating instances of our objects.
Here are the details for the first item:
This tells us that Smart Unit Tests had to take its "best guess" of how to create our class. Notice that we get a hint: "Click 'Fix' to Accept/Edit Factory."
Applying "Fix"
Let's give this a try. But instead of doing this on our CatalogViewModel object, I'll do it on the UnityContainer object (our DI container) since it will be a bit easier to initialize.
Just like when we chose "Allow" in the last article, when we choose "Fix", it will create a new project in our solution and take us to the code we can modify.
Here is the factory method that we get by default:
This simply creates the UnityContainer object, but it doesn't initialize it at all like we are doing in our hand-written tests.
One thing I found a bit interesting (that has absolutely nothing to do with what we're talking about) was the "sad robot" parameter:
But what's more important is that we look at the comments.
This tells us that we need to fill in the blanks to handle the different ways that we can initialize our container. For this, I brought in some of the code that I had in the original unit tests to mock up the service and mock up the model. Then I added some parameters so that we could run tests with either of these objects included or not included (to test our error states).
Rather than creating separate methods, I just in-lined the code (to try to get things working -- we can always refactor it later).
A Bit of a Roadblock
If we re-run the Smart Unit Tests, we'll see that the warning that we "Fix"ed is now gone. We can do the same type of thing for our other methods as well. Unfortunately, I ran into a bit of a roadblock as I was doing this.
For the last item "Could not create an instance of System.Collections.Generics.List`1 <Common.Person>", when I tried to "Fix" this, I got an error:
I wish it had been able to generate a factory method stub for me, because this would have been fairly easy for me to implement. In fact, we created a list to use for testing earlier:
This error is a reminder that we are still dealing with a preview release here. There are still a few bugs to be worked out.
A Bigger Issue
As mentioned earlier, I'm still trying to figure out if Smart Unit Tests has a place in my workflow for building applications. Unfortunately, this particular experience is leading me away from Smart Unit Tests.
Object Initialization
This particular class is already fairly well tested; there are a few scenarios that aren't covered, but it's mostly there. If I do need to spend effort in putting together mocks and test factories, then I might as well write the tests manually. This lets me think in my normal way of testing rather than having me try to put together code that is conducive to the computer generating the tests.
Parameterization
The other concern that I have regarding testing this particular class is that it is a view model. View models are generally state driven (by the properties) and have a number of methods that operate on those properties. This is normal in the object-oriented programming world -- it's really the main reason we might want to use OO.
But Smart Unit Tests works primarily by coming up with parameters that will exercise the various code paths. How does it react when it walks up to a class with very few parameterized methods?
Here are the public methods for the CatalogViewModel class:
Notice that the constructor takes a parameter (our DI container), and most of the other methods don't have parameters. The 2 methods that take "object person" as a parameter are actually a bit of a problem, too. Inside the method, the parameter is immediately cast from "object" to "Person". The reason we have "object" as a parameter is to make it easier to interact with the XAML UI. It's not the best solution, but it generally works. So, in order to test these methods, we really need to be supplying a "Person" as a parameter rather than simply an "object". This could be difficult for Smart Unit Tests to figure out (and I never quite got that far in this case).
Functional Programming
It seems like Smart Unit Tests would really excel in functional-style programming where we are passing in discrete parameters and returning discrete results. This is much different from testing object state. And we saw this in action when we tested our code for Conway's Game of Life which had these functional characteristics.
As a non-functional example, in the hand-written test, we call the "Initialize" method and then check the state of the "Model" property. This is not based on parameters or return values. The changes are based on the state of the object, including both the "container" field and the "Model" property.
Functional-style methods are extremely easy to test (since they don't deal with mutable object state or have weird internal dependencies). Being able to auto-generate tests for functional-style methods is nice to have, but I'm not sure how needed it is.
Wrap Up
I will be the first to admit that I may be missing something really big when it comes to Smart Unit Tests. Feel free to leave comments if you have any ideas or insights; I'd like to get some additional thoughts on the topic.
I really like the idea of having tests that exercise every code path. I like the idea of being able to "fill in the gaps" in my current tests. I like the idea of being able to generate tests against legacy code.
But if it turns out that I need to put a bit of effort into factory methods and mocking in order to get the tests to auto-generate, I'd rather put that effort into just creating the tests myself. As I mentioned, I'd rather build tests in a way that I (a human) understands than spend time to build test scaffolding that an automated-testing system (a computer) understands.
I'll be exploring Smart Unit Tests further (including looking at "Fix" with simpler dependencies). And I'll be sure to share any breakthroughs or "a-ha" moments.
Happy Coding!
No comments:
Post a Comment