Scenario Coverage Analyser for TFS

Paul Stovell has just publically released a tool that he worked on internally at Readify called the Scenario Coverage Analyser for TFS. The tool integrates the code coverage results with an attribute based mechanism for identifying what regions of code relate to different Scenario work items. After the build this data is then slurped into a modified data warehouse where it can then be reported upon. Paul has a great write up on his blog on how it all works.

Certain kinds of programming challenges cry out for TDD.

For the past three weeks (bit of a break in between) I’ve been practicing Test Driven Development, although I didn’t really walk in during the morning and say “I’m going to do TDD” – the problem simply lended itself to that approach.

The two programming tasks that I’ve had over those two weeks both requirement to convert some kind of input to some kind of output. The code I was writing was replacing an existing system that was now end of life so I had a number of samples of input files and output files.

My approach was to write a test for each matching input/output pair and then drive my API to do the transformation that I would then compare (reference output to actual output). I started off thinking a little bit about how I would like to drive the API and then I started implementing it.

Obviously when I started all the tests failed, and in fact for the one that I just finished it took five days to get the first test to pass. In order to know more about how well I was tracking over that period of time I actually got the unit test to do a byte-by-byte comparison of the input and tell me how far I got up to.

I found that the tests help focus me from an implementation perspective because I knew I had to get the first test passing, and once I had done that the other four tests would need to pass as well, because they didn’t I realised that the rule I had come up with to do the conversion wasn’t quite right. The kinds of problems having multiple match inputs/outputs helped me solve were:

• String trimming.
• Timezone conversions.
• Coping with missing data.

In the end I got all five tests passing and I asked for some more recent samples. In the end I had nine tests passing and one failing, and as it turned out the tenth test (the one that failed) actually was a bug in the existing system which my code didn’t manifest, in that case we changed the test condition

Overall it has been a fairly positive testing experience.