I recently completed an assessment and subsequent construction and deployment of a development management system for our software development team at Matrix

While Microsoft has brought out Team system for doing source control, automated build/test, its a fairly expensive product. It has the advantage of being an 'install and go' kind of thing, but the cost of it makes it prohibitive for smaller development teams.

There are also lots of great open source tools out there for performing these tasks that work very well. At Matrix where I work, I have assembled a set of tools that are doing a great job of letting us manage a very extensive set of code across a diverse team of people in a quick, easy and efficient manner. This blog post will review the tools we are using for this, and give examples of how to set it up.


Source Control:

At the heart of every development system is a good source control system. I have tried out a number of these including Visual Source Safe, Microsoft Team System, CVS and SVN.

Visual Source Safe:

Microsoft Visual Source Safe is easy to setup, but painful to use and it has a lot of issues. Its ok for small groups of people, but does not scale well. I found it works best for teams under 5 people who work on a small number of projects, or as a personal source control system.

Microsoft Team System:

We did research Team System - but ruled it out mainly due to its cost, and the fact it is ecosystem tied. The pricing structure, as with anything Microsoft makes, is very complicated, so its hard to get an idea of what it actually costs to set this up. Team System covers a lot more than just source control - its a full development management system from source control through to build/test and software project health tracking.

I think for teams that do strictly Microsoft centric development who are fully immersed in the Microsoft world, and can leverage the value from the Team System Visual Studio clients it would be a very good tool. A full detail product comparison of the visual studio offerings can be downloaded HERE.

CVS and SVN:

As anyone in the open source community knows, CVS and SVN are the best systems out there. CVS as I understand it was the original source control system. I have used it - and its the most comprehensive and some say the most powerful source control system out there. However, SVN is the one we ended up going with, as we needed something that was a bit easier to work with, was simple to setup and had a good variety of clients to choose from.

After initially starting with a standard linux based SVN system, which we got working without a hitch, we ended up switching over to a great tool I found called VisualSVN. We were just using Tortise SVN to the linux system, but with more junior developers ran into a few problems with the lack of direct IDE support for SVN in visual Studio. Through doing some research I found VisualSVN which offers a $49.00 client that wires beautifully into Visual Studio. Makes using SVN as easy (if not easier) than the Microsoft equivalents.

The same company that makes VisualSVN also makes a VisualSVN server - which packages all the SVN bits for use on Windows Server. We were originally using VisualSVN against the linux based repository, but ended up switching over to the VisualSVN version simply because VisualSVN server has a nice GUI On it for managinge the repository using MMC snap in, and it integrates directly with Active Directory for security, thus eliminating the need for managing a second set of passwords etc. The best part is the VisualSVN server is a free download.

I would recommend VisualSVN for anyone looking for a powerful, easy to use alternative to Microsoft's tools.

Unit Testing:

On the Unit Testing front, there are several alternatives. The Microsoft Team Systems Test framework, Nunit and the more recent derivative Xunit. There is a good comparison of these on the Xunit site. At Matrix, we never got far enough into Microsoft Team System to try out the Visual Studio Test and Test system framework, so I cannot comment on it.

Nunit/Xunit:

I was using Nunit up until last year, when I went to the Patterns and Practices confrence and saw Jim Newkirk (who developed both Nunit and Xunit framework and is a long time proponent of unit testing) lecture on the subject. There are a number of things he did when building Xunit that make it a better testing framework - including having setup/teardown in constructor/destructor, and changing the way the tests are run by randomizing them. This forces each test to succeed or fail on its own, and ensures that there are no unforseen interdependancies of the tests.

We have been using Xunit for a while now, and it works very well.

TestDriven.Net

If you are looking for a tool that can be used to run the Nunit/Xunit tests directly from inside Visual Studio, check out TestDriven.Net - I have tried it out, and it works very well. It allows either Nunit or Xunit to be run directly inside visual studio. You can run one or many tests, and you can run tests under debug. This is very useful for framework level debugging as well as simply checking test results before committing code. Tests can be used as an entry point for code that would normally require a UI to trigger them for debugging purposes.

It therefore adds more value to building unit tests especially for lower system levels that don't have a presentation layer, since it gives a very simple quick way to step through code. They offer a personal, professional and enterprise license which are all pretty well priced.

Automated Build Tools:

The two main tools out there seem to be MSBuild, which is also part of Team System, and Nant.

Nant:

Most people will likely be familiar with Nant which is a derivative of Ant. At its simplest, Nant is a .net focused build tool that uses an XML file to generate .net compiler output. It can be used for a lot more than just that though, and has a very extensive set of commands associated with it. So extensive, I found it took a while to get a handle on how to best set it up.

I found a great tool though to help with the code markup though called NantBuilder. Its a simple IDE for building and debugging Nant scripts. It has auto complete, and a list of Nant codes and what they do, so is a great way to learn Nant very quickly. Its also very inexpensive, and they ave a 30 day trial as well.

Documentation:

I used to use NDoc - but it seems to have died out. There are a few other options out there such as Doxygen and a number of commercial products, but fortunately someone from Microsoft appears to have stepped up with the Sandcastle project. (project blog is here)

Sandcastle works very well, and generates docs for managed code using a very similar approach to Ndoc. It however supports all the new language elements such as generics. In addition, there is a GUI editor available for it also on codeplex here. This makes the process of setting up documentation generation for class libraries very simple.

Code Analysis:

From a .net perspective there are two main free tools out there that are available for doing static analysis that I am aware of. One is Style Cop (blog here), and the other is FX Cop.

Style Cop:

Style Cop looks at the source code from a code layout/design perspective using a set of style and consistency rules. These rules are derived from Microsoft's .net Framework design guidelines. Style cop produces a set of reports that show where source code is out of compliance with these guidelines. The tool can be tuned to suit individual development environments by changing the sensitivity and excluding rules that don't apply. Other rules can be added as well. Its useful to get an idea of how well developers are following standards guidelines.

FX Cop:

FX Cop anlayzes .net assemblies and generates reports on design, possible performance issues, potential security problems etc. It is also based on the .net Framework Design Guidelines produced by Microsoft. As with Style Cop, it can be tuned to suite individual development environment needs.

NCover:

Another great code analysis tool that I have found very useful is NCover. It looks at the amount of code that is being excercised by automated unit tests, and generates a report.

This is useful information as it clearly shows areas where code is not being executed, which either means unit tests are missing, or the code is in fact 'dead'. It can also show branches of code that are not being tested - for example in the case where there may be conditional or object strategies that branch but whose branches are not covered by current unit tests.

Conclusion:

There are a fair number of tools outlined here. In our environment, we use:

  • SVN - Source Control through VisualSVN server and Visual SVN client
  • Xunit - for unit testing. We have a copy of TestDriven.net for test debuggging.
  • Nant - Build automation Scripts. Nantbuilder I found useful to help get up and running with this.
  • Sandcastle - Documentation generation, and the Sandcastle UI for building the doc generation script
  • FX Cop and Style Cop - for static code analysis
  • NCover - for code coverage analysis

Because we have a small development team, automation is critical to being able to take advantage of these. So while it takes a bit of work to get the automated build/analyze/test scripts setup initially, it really pays off in terms of getting consistent reports, tests and documentation generated.

This requires though a tool which can act as the driver for all of these - and the best tool out there is ThoughtWorks Cruise Control, which comes in both an open source and a commercial version. We are now using the commercial version for our build management system, but the great news there is ThoughtWorks has a great licensing system that is tiered, so even with a small team such as ours, we can take advantage of a very powerful build automation system for initially no cost other than our time.

In the next post, I will delve in detail into how we set Cruise Control up to drive all these other tools. I will also include the scripts required to generate output from each tool as an example.