Put documents into TFS where your intended audience will find them.

Here is a great post on the VertigoSoftware blog on where the best place to store documents in TFS are. As they point out there are three locations where you can store documents, on the associated WSS portal, as a file attachment on a work item, and in the version control store.

The key piece of wisdom? It depends on who the intended audience is.

I must admit that I use a similar technique – although I find that using work items is the easiest (for me – but I am often my own audience). After that – if I need to share them outside the development team I will use the portal site.

Bug 1894: TFS Integrator fails to kick off one or more builds when more than one are triggered.

This is a bug on TFS Integrator that I just logged on our internal TFS installation. I thought that I would post it here for those that have started using TFS. We will get a fix out for this soon – we may take the opportunity to use Windows Workflow Foundation (now that it is stable) so it could take a few weeks before someone has the time available. Bug follows.

I suspect that Team Build is the problem here in that it doesn’t support queuing on its side of the fence. I uncovered this with the CrmOnTime project. The CrmOnTime project consists of three seperate builds CrmOnTimeCommon, CrmOnTime and CrmOnTimeServices. When CrmOnTimeCommon builds dependencies are replicated to the other two projects. If these projects kick off their builds at the same time they seem to fail if they are using the same build server. If they each kick off on a different build server then there is no problem.

Unfortunately we still have to implement the fix in our code because we are a Continuous Integration engine our users should rightfully expect not to have to run multiple build servers just to get multiple dependent builds kicking off – we just need to queue the requests. I’m thinking that moving forward we should replace the internal state management with an event driven workflow or something like that.

Temporary Workarounds:

Structure the sleep periods such that you allow one build to complete before the other gets kicked off – or use multiple build servers.

Getting Started with TFS Integrator

ATTENTION: TFS Integrator is now obsolete. Please use TFS Continuous Integrator and TFS Dependency Replicator. For more information, read the post on why TFS Integrator is now obsolete.

---

This is a post that I’ve been meaning to write for a while. Earlier this year, when Chris Burrows joined our team we had the opportunity to spend a little bit of time building a TFS extension that Darren and I dreamed up whilst we were working on a client project together.

As you are probably no doubt aware, Team Foundation Server does not ship out of the box with a Continuous Integration capability. A lot of people, including myself consider this a glaring omission – but as a developer I have to appreciate the demands of tight delivery schedules.

What is Continuous Integration?

Continuous Integration (CI) is the process of continually scanning the source code repository for changes, and, once they have been detected automatically kicking off a build process to verify that the code compiles successfully. In many instances teams will tag on a round of build verification tests to ensure that not only does the code compile, but that it doesn’t catch on fire when you try to run it (smoke tests).

While not unique to agile methods, continuous integration is certainly one of the tools that agile development teams to keep a handle on their code base as it evolves throughout an iteration. It is no surprise then that when agile teams came to use TFS that there was a collective “what the” when it didn’t include a CI engine.

How did we build our TFS CI engine?

Fortunately for us, the team at Microsoft did build TFS with an eye on extensibility and they included in the system an “event service” which we can plug into to listen for events in the source code repository. The way it works is that whenever you do something significant in Team Foundation Server, such as checking in a file, that TFS sub-system notifies the TFS eventing system which then broadcasts that message to any subscribers. This is how you get an e-mail when another developer checks in some code.

This same eventing mechanism can actually be instructed to send a notification via the SOAP format to a web-service endpoint which might be hosted in something like ASP.NET, or event Windows Communication Foundation.

TFS Integrator uses this facility to kick off a build in the Team Build component of Team Foundation Server. When TFS Integrator initialises it reads a configuration file to identify which parts of the source tree it is interested in listening to then subscribes to the check-in event notifications that the eventing system sends out.

When it receives a notification it sleeps for a specified period of time then kicks off a build with the Team Build facility. Once the build is completed the eventing system sends how a notification to subscribers as to the outcome. The completed interaction between the system components looks like this:

 

All of this automation makes use of the tools available from the Visual Studio 2005 SDK which you can download from Microsoft.

How to configure TFS Integrator to support Continuous Integration?

Before we start you need to understand that the current build of TFS Integrator relies on the .NET 3.0 runtime (RC1) being installed. For this reason we don’t currently recommend installing it on the same computer as either your application or data tiers. Once you have downloaded and installed the .NET 3.0 runtime (RC1), the next thing that you are going to need to do is download the latest build of TFS Integrator.

The setup package itself does nothing more than drop a few files onto the file system, the rest of the configuration requires some manual intervention. The installation files are dropped into the “TFS Integrator” directory under program files.

The “TFS Integrator” directory contains three seperate configuration files:

• TfsIntegrator.exe.config
• ContinuousIntegration.xml
• DependencyReplication.xml

The first file, TfsIntegrator.exe.config contains the bootstrap information required to get TFS Integrator talking to your Team Foundation Server installation. In this configuration file there are three main settings that you should be interested in. These are BaseAddress, RegistrationUserName and TeamFoundationServerUrl.

BaseAddress is the address of the server itself. We would have liked to have allowed the program to determine this itself, but it can get tricky in multi-homed systems. The value that you need to provide here is the address that Team Foundation Server will be able to communicate to TFS Integrator on via the event notification service. Provided this address maps to the local host, the TFS Integrator service will self register the port as it starts up.

The next setting, RegistrationUserName is the account that you want TFS Integrator to register subscriptions under. Typically this is the same account that is used to run the service (especially in a domain configuration). Finally, the TeamFoundationServerUrl is the address of the Team Foundation Server that TFS Integrator will need to talk to.

Once the initial configuration is done the service can then be installed. So that it can run headless. You do this by running the “TfsIntegrator.exe” program with the “-i” switch.

 

After the installer registers the service you just need to go into the Services MMC snap-in (services.msc) and specify the logon credentials that the TFS Integrator service is going to use. This account should have access to TFS, be the same account as specified in the RegistrationUserName and have sufficient rights to write files to the “TFS Integrator” directory in program files.

Now that the basic configuration for TFS connectivity is out of the way its time to actually modify the ContinuousIntegration.xml file. The ContinuousIntegration.xml configuration file contains all the information that TFS Integrator needs to trigger the build of a “team build type” in Team Foundation Server.

The root element in the document is ContinuousIntegration, and under that you can have one or more TeamProject elements. The TeamProject element defines the name of the team project to which TFS Integrator will listen to events for. This element in turn contains multiple Build elements. The build element defines what path in the version control store to listen for changes in, what build type to kick off, and on what build machine. The sleep attribute defines the settle period that TFS Integrator will wait for further check-in notifications before spinning off the build.

Once you have made the modifications to the above configuration file you should be able to start up TFS Integrator. Congratulations! But what about DependencyReplication.xml?

What the heck is Dependency Replication?

Dependency Replication is a process that most development teams have to do as the scope of what they are trying to achieve grows. The idea is that within your organisation you might have a set of common frameworks that you use and maintain. Rather than linking in the source code into each new project that you undertake you treat the framework like an internal product which you build seperately.

The problem you then have is integrating the updated binaries into the other projects that depend on them. This can be quite a time consuming process, so much so that teams will often give up and end up compiling hundreds of different versions of the same code into their code base – which creates a maintenance nightmare.

TFS Integrator includes a dependency replication engine which extends the continuous integration feature. It does this by listening for the build completion even from TFS and using that to trigger the execution of some code which picks up the compiled build output and checking it back into Team Foundation Server in a specified location.

The effect is that with dependency replication the feedback loop is complete and dependant components can be linked together.

In the case of the framework example we could build up a scenario where TFS Integrator gradually rebuilds your entire code base.

1. Developer checks in some code in the framework code base.
2. Check-in notification sent to TFS Integrator.
3. TFS Integrator kicks off a build in Team Build.
4. Build completes.
5. Notification of successful build completion sent to TFS Integrator.
6. TFS interator checks in some of the drop files into version control.
7. Check-in notification sent to TFS Integrator.
8. TFS Integrator kicks off a build in Team Build.
9. Build completes.
10. Notification of successful build completion sent to TFS Integrator.
11. Rinse, repeat.

In the past I’ve spent significant amounts of time getting similar systems going on top of version control systems like Visual SourceSafe where sharing was used as the replication mechanism. I actually think that this approach is much cleaner and more obvious.

How to configure TFS Integrator to support Dependency Replication?

Like continuous integration, the configuration of dependency replication is controlled via a configuration file – in this case, DependencyReplication.xml.

The root element of the configuration file is DependencyReplication. Within it there are multiple TeamProject elements which define the TeamProject and what the root folder is (I’m not sure what the root folder configuration item is off the top of my head, but I suspect its the relative path to which check-ins will be made). Within each TeamProject element, one or more Dependency elements are defined which specify which build type the team  project depends on. Inside each Dependency element is a File element which specifies the drop location relative source file any dependencies that need to get checked back into source control. The destination is the fully specified path to put those files into version control.

In the configuration file above we are saying that SystemB depends on some files from SystemA (Output1.dll and Output2.dll) and that those files should be checked back into the $/SystemB/SystemB/Dependencies location in version control. The interesting thing to note is that the file name is also specified in the Destination path. This allows configuration managers to rename files as part of the replication process.

Once both continuous integration and dependency replication configuration is complete, you need to restart TFS Integrator to get it to pick up its latest settings.

Next Steps

If you are interested in trying out TFS Integrator in your environment, just go ahead and download it. We are very keen to see people use this tool because it is genuinely useful to us, and we think it will be useful for you too. We probaly won’t be giving away the source code at this stage but unless a future version of Team Foundation Server makes it redundant we will continue to make revisions to the code base and support it (as best we can).

If you want to get support for TFS Integrator then I recommend that you join the OzTFS mailing list and community which was recently setup by Grant Holliday, you can subscribe via e-mail. Once you are subscribed you can send an e-mail with your question. If you can’t post to that mailing list for any reason, just send an e-mail to me.

Thanks

Finally – before I sign off on this post, I want to thank a few people. TFS Integrator was not developed by me – in fact I can’t even claim sole responsibility for its design. Both Darren Neimke and Chris Burrows both helped get this software out the door. Chris in particular put in hours above and beyond the call of duty to get this out the door.

DEV316: Extending Team Foundation Server

David Lemphers said it best when he said that Team Foundation Server is more a platform than a product. In the lead-up to Tech.Ed 2006 in the US I presented a web-cast on how to Extend Team Foundation Server.

I actually had to get up at about 4am in the morning to double check all my configurations before presenting that session live at 6am – to say I was tired would be an understatement. Still – I covered a fair bit of material in that session.

The session that I am presenting will not be the same as the web-cast! I will cover much of the same material but I will demonstrate it by showing off some really useful extensions to Team Foundation Server. The two that I’ll definately be covering include:

• Grant Holliday’s TFS Bug Snapper
• Readify’s TFS Integrator

Bug Snapper is a tool that Grant Holliday wrote which allows testers to take a screenshot, annotate it and submit it to Team Foundation Server. I know that Grant got quite a bit of traffic to his blog when it was released and I think it is something that people would be interested to see.

TFS Integrator is a continuous integration engine for Team Foundation Server which also takes care of copying build outputs from the drop location back into the source tree where other projects can use them as dependencies for their build. We have actually sent this out for BETA testing already but we are looking to make our first public release soon – hopefully before Tech.Ed.

DEV309: Visual Studio 2005 Team Foundation Server – Applying Version Control, Work Item Tracking and Team Build to Your Software Development Project

On the Thursday during Tech.Ed I will be co-presenting a session with local VSTS MVP Joe Sango. During the session we’ll be having a practical look at how to use Team Foundation Server’s Version Control, Work Item Tracking and Build Automation features in a software development project.

To make it interesting, Joe and I will each choose a different process template (MSF Agile or MSF CMMI) and walk through the typical workflow. Because the session only goes for an hour and fifteen minutes we’ll have to do it fairly quickly, but it will be a good opportunity to see the two processes executed side by side.

I suspect what you will learn is that while the governance processes are quite different, the day to day developer tasks are much the same in that they continue to use the tools to check-in code against work items and create automated builds – how you get to a work item which you can code against is where the action happens.

DEV303: Concurrent Development with Branching in Team Foundation Server

In my previous post I outlined the list of sessions that I would be presenting at Tech.Ed this year. The first cab off the rank is DEV303, Concurrent Development with Branching in Team Foundation Server (CommNet VCal). The talk itself is actually derived from a talk that I gave at Code Camp Oz earlier this year.

The talk didn’t go that well at Code Camp Oz because the machine I was using didn’t really have the grunt to run TFS quickly enough for demonstrations. At Tech.Ed this shouldn’t be a problem because I’ll have access to the Dev’Garten to perform my branching operations on, as well as a couple of other backup options.

Most of the theorhetical content for the talk actually comes from the following blog posts as well as a good number of years being responsible for the build master/configuration management role within development teams.

1. TFVC: Why use branching?
2. TFVC: The Terminology of Branching
3. TFVC: Concurrent Development with Branching
4. TFVC: Reducing Source File Contention with Branching

In addition to covering the theorhetical content I’ll look at the practical/mechanical aspects of how to do branching and merging in Team Foundation Version Control.

Who would like to test our Continuous Integration server for Team Foundation Server?

At Readify we (well, one guy mostly) have been building a continuous integration extension for Team Foundation Server. Obviously building a continuous integration extension for Team Foundation Server is pretty straight-forward but what we have done is try to simplify the process by removing the dependency on IIS (but we do rely on .NET 3.0 BETA 2 currently – for WCF).

The other significant feature is that you don’t need to use BisSubscribe.exe, just install the service onto a seperate build server (we don’t recommend putting it onto the same box running the application or data tiers just yet because of the BETA dependency). When the extension starts up (runs as a service) then it will self register with the TFS event notifications service.

We are already using this build internally and it is working great! We still have a few issues we want to address before putting it out there for wider distribution, but it is certainly a functional piece of software. If you want to try it out just leave a comment on this post or shoot me an e-mail directly at mitch.denny@readify.net.

Teamprise turns 1.1

Martin Woodward sent me an e-mail this morning telling me that the Teamprise Java/Eclipse-based front end for Team Foundation Server is now at revision 1.1. This is great news because this release fixes an issue that stopped the plug-in working in a great number of Windows networking environments. Team Foundation Server users that also have some Java developers will find this especially good news.