Monthly Archives: December 2010

MSTestContrib 1.0.0 available for download/NuGet.

Late last night I uploaded the RTM release of MSTestContrib 1.0.0 to the MSTestContrib project portal up on CodePlex. This release of MSTestContrib includes the ability to write BDD-style unit tests simply by sub-classing MSTestContrib.Specifications.Specification and decorating the class and methods with a few attributes.

Getting Started

The easiest way to get going with MSTestContrib is to install it using the NuGet package manager. If you have ASP.NET MVC 3.0 RC2 or above installed you already have NuGet installed. On the NuGet console issue the command “install-package MSTestContrib” with the unit test project selected as the default project.

MSTestContribInstallPackage

If you don’t have access to NuGet you can still download a zip file that contains just the MSTestContrib DLL from the project portal releases page. The MSTestContrib.dll file needs to be referenced by the unit test project. Once this is setup you can create a new test class as per the normal process and then sub-class MSTestContrib.Specifications.Specification and add some attributes. The test method implementation just uses a series of methods to build up the BDD-style statement. The following code sample (gist 760628) shows a specification without any implementation detail provided.

Running this test using MSTest will produce an inconclusive result. The result summary view functions as you would normally expect.

MSTestContribTestResults

Drilling in a bit closer to the individual test result you can see that MSTestContrib has reproduced a textual representation of the BDD-style test, spelling out the specification description, scenario description and the various BDD statements. At the end of each statement there is a “NotImplemented” indicator which is the cause of the inconclusive test result.

MSTestContribInconclusiveResult

The following code sample (gist 760641) shows a completed test specification including not only the plain text description of the test case but also a function/expression that implements that description.

If the Add(…) method on the calculator threw an exception then the test result output would provide not only the text-based representation of the specification but also all of the exception detail. The following screenshot provides an example of the output.

MSTestContribFailedResult

It’s not pretty but there are limits to how much formatting you can put into the error message within the MSTest framework.

MSTestContrib Goals and Non-Goals

I wrote MSTestContrib primarily as a BDD-framework that I was happy using. There are dozens of BDD frameworks out-there including some really great ones. I was inspired by StoryQ but I wanted something that was simpler still. By taking dependencies on MSTest (the out of the box unit testing framework that comes with Visual Studio 2010) I hope to reduce barriers to entry for developers who have not done BDD-style development before.

Over time I am hoping that MSTestContrib will be extended with other framework elements which support teams that are investing in MSTest. If Microsoft produces tooling in MSTest which makes elements of MSTestContrib redundant then those features will be depreciated.

Advertisements

JavaScript InfoVis Toolkit published to NuGet package repository.

Recently I’ve been playing around with the JavaScript InfoVis Toolkit developed by Nicolas Garcia Belmonte. The toolkit is a comprehensive library for producing rich info-graphics including styling and animation and is fed off a JSON input. I was looking at using it to visualise an organisational chart but it can do much more than that.

After playing around with it a little bit I decided that it would be handy if the toolkit was actually contained in a NuGet Package. So I created a fork and pushed it into the temporary NuGet Package repository. The package has now been accepted and is available from the package manager console in Visual Studio 2010 (with NuGet installed, or ASP.NET MVC 3.0 or above).

Getting Started

The first thing you need to do is open up your ASP.NET application inside Visual Studio 2010. The package assumes that you are using ASP.NET MVC, but to be honest it’ll work just fine with plain-old ASP.NET as well (except that the samples will be located under /Content/JitExamples/).

JITSolutionExplorer

Once you have your project open and selected, go to the Package Manager Console and issue the command “install-package JavaScriptInfoVisToolkit”. This will install the package into the ASP.NET application.

JITPackageManager

The package contains three script files (jit.js, jit-yc.js, excancas.js) and a number of samples (under /Content/JitExamples). Once you are comfortable with how the toolkit works you can simply delete the examples.

JITSolutionExplorerInstalled

If you want to get a feel for what the package can do, set the various HTML files under JitExamples to your start page and hit F5. Enjoy!

What can Microsoft learn from others?

Here are a few random thoughts about various Microsoft products compared to some of their smaller competitors:

What can Microsoft learn from Git?

  1. Distributed version control is popular amongst developers, and developers create pull for developer tools so you want to keep them happy.
  2. Distributed version control can get complicated, so even though you have to do it to be a cool kid you need good visualisation tools, not to dumb it down, but to empower developers to make good decisions.
  3. Marking files as read-only increases friction, developers want a file based version control system, not a solution-based one.
  4. Developers want a version control system that can tell them about what has changed in a directory, not one that they have to go fishing for each new file to add it into the system.
  5. There is definitely room for improvement with Git, time for Microsoft to become of best of breed in version control.

What can Microsoft learn from GitHub?

  1. The current generation of developers have embraced Internet-facing collaboration portals, and the next generation will expect that is the way that the world works.
  2. Collaboration features like Gist are enticing developers to experiment with the likes of GitHub.
  3. Developer identity is at once independent of organisations, and potentially part of multiple. I shouldn’t have to change accounts to switch to working on a private-pet project to a commercial project for a customer.

What can Microsoft learn from Ruby & Ruby on Rails?

  1. Its all well and good to have an IDE, but if you rely on convention over configuration then your IDE doesn’t need to be anywhere as sophisticated. When I do Rails development I use TextMate. I miss things like intellisense, but all the other designers and stuff aren’t nearly as important.
  2. The .NET Framework is pretty damn fine, the beauty of the .NET Framework when compared with the framework that ships with Ruby and Ruby on Rails is something to behold. Lesson? You don’t need to emulate everything from other platforms.

What can Microsoft learn from AgileZen?

  1. With AgileZen its very easy to get started on a small personal project, and it doesn’t necessarily need to be tightly integrated with other ALM-related features such as version control. Kanban should be a process template that comes out of the box in Microsoft tools that support any kind of task management.

What can Microsoft learn from Heroku?

  1. You need to have enough faith in your platform to support co-locating different customer applications on a single machine. You shouldn’t have to spin up a new VM for each application.
  2. You should be able to get something up and running for free to enable the developer community to experiment. Doing the above will actual enable you to do this quite cheaply.
  3. You need a good deployment model (actually, Windows Azure isn’t too bad here, but the Git approach with Heroku is interesting).

Anyway – that’s enough from me.

Database Migrations vs. Database Differencing

Paul Stovell just wrote a fantastic article on database deployment, if you haven’t read it, go and do it now. It underlines the critical thinking that is being applied to the relationship between database administrators, software developers, their roles and responsibilities and the artefacts that they manage.

This is actually a debate/conversation that has been going on for some time internally about the best way to manage database migrations, earlier this year that debate/conversation actually spawned this post, but I think Paul has made a really good case for why the database migrations approach (as opposed to database differencing) works for him.

Similar Solutions, Same Problem

To be honest, I’ve always been conflicted about this. The migrations approach is seductive and other than being a little bit more cumbersome to get started I think it yields some pretty good results – and most importantly it totally appeals to my inner developer. I think that DB Pro is very quick and easy to get up and running and if used properly you can get the desired approach even if there are a few scenarios where it falls down and would force me to use the migrations approach.

I disagree with Paul fundamentally that the state based approach to database updates is dumb. I consider the database differencing approach, and the database migrations approaches to be two valid solutions to the same problem. I’ve produced the following diagram to help put it into context.

image

There are those people out there that are happy to define the state that they want to get to, and trust the tool to get them to get them from that state by building the migration for them. Then there are those people who define the migration, and trust the tool to keep track of what migrations have been applied. What it comes down to us understanding “what could go wrong”.

Corrupted Starting State

Despite the development teams best efforts managing changes to the database schema in version control, there will be times where the starting state of a database prior to the upgrade process does not match what the developer intended. In an ideal world the developers would test the upgrades against a copy of the database, but I’ve seen situations where this is not feasible because the database might be 1TB in a remote datacentre which means there is a lag between database copies arriving (if at all).

Using the database migrations approach this can sometimes lead to a problem where someone has made a change to the production database directly which blocks the migration script from working. I’m not saying that is a reason not to use database migrations, but its one of the things that “could go wrong”.

Because tools like DB Pro use a database differencing approach they don’t suffer this particular problem because they can compensate and change the database state to the desired configuration. The problem is – what if that original change was made for a reason, and along comes DB Pro and wipes it out (well, not quite, it has protections against data loss), once again, this isn’t a reason not to use DB Pro, its just another thing that “could go wrong”.

The point I am trying to make is that no matter which tool you use, there are situations will will trigger update failure, regardless of whether you are using a differencing approach or a migrations approach. It isn’t sufficient to sit back and say if all my scripts run, the database is in a good state.

Update Failure Recovery

Once you’ve got yourself stuck with an update situation you need to know how to get yourself out. With DB Pro, it will normally abandon changes to the database if it detects a situation where data-loss will occur or the scripts won’t run. But not working does not equal recovered.

What you need to do with DB Pro is get the offending schema changes into your definition of what you want the state to look like so that you can continue to move forward. With the migrations tool you need to hope you have a tool that will detect failures and rollback the changes so you are not left in an unknown state. A decent migration tool will do this for you – so I’m not arguing this as a point of difference.

Where the migration story can suck is that if you have got an uncontrolled change in the production database, you need to figure out how to replicate it – ironically a differencing tool will help you here because you can migrate the database up to the point that the change was made, perform a difference between  your local developer database and the production database and insert a migration in the middle. The truth is you are unlikely to need that migration anywhere other than your development and test systems but it is important to capture that intermediate state.

Inconsistent Approaches

The problem with database migrations is that there are too many cooks in the kitchen. Within the average software development project you have software developers modifying the schema in development, testers mucking around with sample data in the test environment, and database administrators giving the schema some treatment post deployment in production. What makes it worse is that everyone has their own ideas on how they want to accept changes to the environment that they control.

Paul and I could argue all we like about our preferred migration tool, but unless the DBA accepts it as a valid way to push a schema forward then we are both hosed, and we need to present him with some diff scripts to run. In both the differencing scenario, and the migration scenario this is pretty simple, Paul would just hand over his T-SQL scripts, and I would use the differencing tool to pre-generate the T-SQL scripts. But wouldn’t it be nice if we all played nicely?

Paul hit the nail on the head with a few of his suggestions. You really must use version control, you really do need to test your updates on a regular basis, obviously the best way to do this is via a CI build that works against a copy of the production database (as fresh as you can get).

I would go one step further and say that database administrators should also be forced to use the same process that the development team are using (and ideally put them on the development team). This way if they want to change an index on the database, then they should be forced to make that change via either migration scripts or a tool like DB Pro and push it to production. Doing this would have a dramatic effect when trying to reduce the number of failed database deployments.

Multiple Application Databases

All of this is grand in theory, but there is one more thing to consider when it comes to database migrations. The database is not always completely owned by a single application. I’ve seen customers that have 50+ applications talking to the same database, and they don’t even attempt to understand the boundaries of each application – it is truly a shared, stateful component. In situations like this the database needs to be treated like a separate product with its own release schedule, but governed by all the same rules (version controlled, continuously tested). Various dependent projects submit changes to this database, and where possible they get rolled out in advance – which kinda discourages radical changes to the schema.

Final Thoughts

I think that this space is getting interesting at the moment and I’m glad that we are seeing some debate on this topic. What I hope is that we end with the changes to the base platforms and tools required to make this a smoother process. For example I would love to see a BCL change for .NET which introduced a “System.Configuration.Migration” to compliment “System.Configuration.Install”, and then have all of the Microsoft tools target producing “migration libraries” which plug directly into this framework extension.

I also think that migrations isn’t just a database problem. And I am really pleased to see the likes of Fluent Migrator and Machine.Migrations making early moves into solving the more global problem of how to go from state A to state B cleanly, whether you are using a relational database, a document database or any other kind of stateful system.

Finally, I’ve been disappointed by DB Pro. The attempts to get other database vendors to buy into this process has largely failed, Microsoft has relied on Quest to produce TeamFuze which integrates DB Pro with Oracle, and the IBM DB2 plug-in for DB Pro seemed to go no where. If there really is a strong argument for using the migration approach over the differencing approach it is because of lack of tooling. At least with a migration approach you can “roll-your-own” fairly easily.