Monthly Archives: February 2011

The Right Level of Attention to Detail

I’ve been accused from time to time of not having enough attention to detail, or letting some things slip through the cracks. That is why when I notice something slipping through the cracks, or some detail being out of place I get concerned. The theory is, if I’ve spotted it, what else is wrong.

Of course, there are some things that I am overly obsessive about in my professional career:

  • Structure of version control trees.
  • Naming conventions used for projects within Visual Studio solutions.
  • Templates used for SharePoint sites.
  • Names used for build definitions for Team Foundation Server.

That is just a few of them. Of course, someone could probably write a book about the things I don’t care about when it comes to home life…

What is a *.trx file?

This is another post to follow on from my “What is a *.vsmdi file?” post, this time just a quick note about *.trx files. A *.trx file is generated when you execute tests using MSTest. By default they are placed in a sub-directory called “TestResults” with an associated directory containing input and output files and other associated test result data like Code Coverage and Test Impact Data.

Most unit testing frameworks have a similar capability where they output results into a log file that is then used by the server they use for continuous integration to present the outputs for that build. Team Foundation Server actually ingests the *.trx file into its database and presents the information via the analytics cube. So it is actually quite a useful thing, although most people don’t need to track their *.trx files locally and they are certainly never checked into version control.

Sometimes developers have functional automation tests which they cannot run on their build servers, but they may still want to keep track of tests passing and failing. Here the *.trx file being generated can help because after the local test run it is possible to “publish” those tests results to Team Foundation Server and associate them with a build that ran previously. Obviously it is then the responsibility of the developer that they are associating the test results with a build that best represents the code/system that was under test.

Deployment Items & MSTest Performance

As I mentioned earlier, the test result file also has an associated directory, in which you will find two sub-directories, “In” and “Out”. When a test is executed the test assembly and associated dependencies are copied into this directory and then run. The idea behind this is that the tests run in an isolated part of the file system which makes it possible to pick up issues that might be associated with specific file locations. The [DeploymentItem] attribute can be used to push additional files into this directory as an explicit dependency.

Some people don’t like this behaviour which is understandable. Its one of those differences between testing frameworks that make you love it or hate it – but you have to appreciate the reason why it does it. In Visual Studio 2010, the MSTest framework got some significant performance improvements, and one of the changes they made was automatically disabling “deployment” which is that whole file copy business. As soon as you start using deployment items though it gets turned back on and you pay a performance penalty.

What is a *.vsmdi file?

imageThere was a comment on an internal mailing list yesterday about *.vsmdi files as an example of “why you shouldn’t use MSTest”. Before I jump right in and agree with the author of that statement it might be worth examining what a *.vsmdi file is, and its place in the overall plumbing of MSTest.

The *.vsmdi file (MSTestContrib.vsmdi in my example) contains a list of tests which exist across all projects within your solution. Double-clicking on the *.vsmdi file inside Visual Studio will launch the “Test List Editor” which allows you to organise these tests into a hierarchy which you can then use to selectively execute tests.

When Visual Studio 2005 shipped Microsoft assumed that most people would use this view to arrange which tests they wanted to run and that it would be particularly useful when you had a large number of tests.

Fast forward to Visual Studio 2010 and the reality is somewhat different. Most developers who are using MSTest simply use the Test View window to control which tests they want to run, and in almost all cases they run all of the tests (because that is what a good TDD developer would do). The *.vsmdi file is really just a relic of a poor assumption about how Visual Studio was going to be used.

Originally the use of the *.vsmdi file was a requirement to get MSTest integration with MSBuild working on a TFS build server, however this requirement has since been removed once again removing the utility of the file.

To top it off, Visual Studio 2010 now includes features which make it possible to better target their test execution effort (if they do have a very large number of tests) by using the “Impacted Tests” view. This view relies on code coverage and test impact data to interactively suggest which tests you should run based on the changes you have made since your last test execution. So the *.vsmdi file is now largely redundant.

So what, why should I care about the *.vsmdi file?

VsmdiProblemsUnfortunately the *.vsmdi file, as well as being redundant has always had some issues around version control and read-only flags which is ironic given that they are part of the Team Foundation Server tool-chain. The screenshot on the right shows an all too common scenario where due to either file locks or some other mysterious issue, multiple *.vsmdi files are being generated. It seems to happen most when developers are working together on a project and frequently adding tests.

Assuming that you are just using the Test View to run your tests these files are perfectly safe to delete. If you want to try and tackle the problem, there is an imperfect solution. Simply remove all the files from your disk and in version control (assuming you are using TFS here, or any version control system that is integrated with Visual Studio), then instruct the version control system to perform an exclusive check-out.

Like I said, it’s not a perfect solution, especially for agile development teams, but it will stop those annoying file1.vsmdi, file2.vsmdi … file20.vsmdi scenarios coming up.

Is this a good reason not to use MSTest?

I’d say it is a pretty weak argument by itself. Most people have a preference from one unit testing framework over an other. Some people prefer xUnit over NUnit, others prefer MbUnit. Every unit testing framework has its benefits as well as its quirks but people seem to point frequently to MSTest as a commonly despised testing framework.

Personally I like it – even with its faults, its a personal preference. Am I lesser developer because I use MSTest? I don’t like to think so. I enjoy some of the out-of-the-box integrated capabilities such as Impacted tests, and the data collectors as well as good integration with Team Foundation Server and Lab Management. Everyone is entitled to their own opinion though!

Reflector, why not pay for it?

I’m surprised at the somewhat negative reaction by the .NET community about RedGate’s recent announcement about charging $35.00 dollars for the basic version of .NET Reflector. For those who don’t know the history, Lutz Roeder handed .NET Reflector over to RedGate so that they could maintain and enhance it into the future.

.NET Reflector is an awesome tool and I’ve used it countless times to get a better understanding of “how stuff works”. I wouldn’t be the .NET developer I am today without this tool. So I’m definitely going to cough up the $35.00 dollars, and you know what, now that I’ve sat down and thought about it, I might go and get the Pro version.

My only hope is that Lutz got a good deal when he handed over the tool to RedGate, but that doesn’t diminish the value of the tool, and if Lutz had decided to start selling it I would have happily have paid for it then as well.

Using .NET Reflector is not some special right that the .NET community has. We’ve been lucky to get it for free for so long given how useful it is. Looking forward to version 7.0!