menome

Silverlight Managed Extensability Framework, Prism and MVVM Light

On further research, we found that the Prism framework is not quite ready for version 4 silverlight. The Prism team are currently revamping Prism to use MEF (Managed Extendability Framework) and while they have begun producing code drops (they are on Drop 3), these are still in alpha, so we can't use them yet.

So to bridge the gap while the Patterns team works towards full release, we decided to use the MVVM Light framework from Laurent Bugnion of Galasoft. It provides a simplified version of Prism's feature set, and has proven to be a great framework to work with.

MVVM light provides:

Commanding - RelayCommand object that can be used with Sivleright 4 button commands, and has an extension that bridges other controls that don't yet have commanding
Messaging - A great messenger framework for decoupling viewmodels from each other by sending messages instead of binding
Blend support - MVVM light has been built with Blend support in mind, and works very well with Blend. This gets around another deficiency in the current Prism implementation, which doesn't play well with Blend.
ViewModelBase - base class for viewmodels
Templates - for Blend and Visual studio. It also include some code snippets to help with code writing.

I would recommend using MVVM light if you are looking for a good, simple way of making MVVM easier and more clear to work with. It works extremely well with MEF - we have now refactored a complex Silverlight application to use MVVM light and MEF in less than a week.

Sean Wildermuth has a great 4 part example series on using MVVM light with MEF to build an MVVM based Silverlight application. The source included with the examples also has a fantastic error handling, status message and centralized IsBusy indicator setup that leverages the MVVM messaging framework.

Silverlight Managed Extensability Framework and Prism

We have been doing extensive development using Silverlight, and have run into some barriers with the design of the application we are working on. Some of the issues we were able to mitigate with MVVM pattern. However, as the app has become more complex, dependencies have crept in between the viewmodels and the parent application.We are also running into trouble managing commands and events associated with the application - how do you have several components subscribe to the same event set without having to connect viewmodels together?

The application's Xap file has also grown to be fairly large, and it will be necessary to split the application up into separate chunks that are loaded only when required. Our current structure will not allow this.

Upon doing research into how to address these issues, I discovered the Prism framework from the Microsoft Patterns and Practices group. This appeared to be a great way to address a number of the issues.

Prism is a framework for composing applications from separate modules. It contains an approach for injecting components into others using the Unity framework (also a patterns and practices framework) to eliminate dependencies. It also has a very nice looking commanding approach implemented through an event aggregator publish/subscribe mechanism.

However, during reading through commentary on Prism, I also found the Managed Extendability Framework, from a different part of Microsoft which looked initially like it was a competing approach to solve similar issues.

Microsoft often does things like this (take the whole Linq/ Linq-SQL/Ado.net entity framework/Windows Communication foundation maze as an example...) in which they come out with several almost the same but not completely technology bits that overlap. Inevitably one of the approaches will be adopted at the expense of the other, which forces developers who adopted the wrong one to refactor to the winner.

This meant more research into the details of the frameworks. There has been a large amount of discussion about the two frameworks and what roles they play. It looks like that while there is overlap, they do address slightly different issues.

MEF appears to be the 'lower level' framework and allows each component to 'import' things they need. Each component is marked with 'export' to mark things they provide- mef goes through the apps and matches them up. In this way, you can create components that are loosely coupled that assemble themselves at run time.

So it is essentially the 'glue' (or duct tape in Canadian terms) that ties components together instead of using tightly coupled approach.

Prism has a similar concept for eliminating component coupling, but instead uses Unity (a Patterns and Practices dependency injection framework I think) for this purpose.

It then goes on to add guidance and tools for building modules, commands, and binding modules and commands to make an overall application. So Prism is more focused on the user interface concern side of the equation, where MEF operates at the component level.

In fact, MEF was included in .net 4.0 and sivlerlight 4.0, which means that its approach 'won out' in terms of what it does over the the Unity approach used in Prism. It looks like for Prism release 4, the Prism team are refactoring the framework to use MEF.

So - it looks like what we can do is use MEF to glue the components together, then use Prism to handle the user interface composition aspect via the modules and event aggregator.

Its a bit frustrating having to figure out two separate (for now) frameworks to accomplish our goals, and it would be nice if the different bits of Microsoft communicated better on these sorts of things, but at least I found out about the path of this before committing to fully using one approach.

Here are some additional resources:

Interview by Craig Bator Project Manager Quest For Canadas Smartest IT

We were interviewed by Craig Bator PM of the Quest for Canada's Smartest IT.

We talk about the philosophy we use at Matrix for building an agile IT and software development team that works directly with leadership to meet the ever changing needs of the business.

See it here: http://smartest-it.ca/craigs_chronicles.aspx

WCF Debugging 404 error

WCF is a great technology, but it is difficult to configure, and often the error messages are cryptic - if in fact you can get it to return exceptions at all.

Fiddler is a great way to debug WCF based services, but even that sometimes doesn't work if you are using the visual studio/cassini based web host.

Fortunately, adding the following code into your web.config file will put any exception information into a log file:

 <system.diagnostics>
    <sources>
      <source name="System.ServiceModel"
              switchValue="Information, ActivityTracing"
              propagateActivity="true">
        <listeners>
          <add name="traceListener"
              type="System.Diagnostics.XmlWriterTraceListener"
              initializeData= "c:tempwcfErrors.svclog" />  
        </listeners>
      </source>
    </sources>
  </system.diagnostics>

WCF

Clustering points for Silverlight Bing maps control

We are now finally underway building the first version of what will become the Menome system.

One of the things we needed was a way of clustering points on a map. The idea is to reduce the number of points as the user zooms out by combining points together to keep performance up and to reduce map clutter.

The algorithm operates by creating a 'screen grid' and passing this to the server. As the user zooms and moves around the map, the boundary window they are looking at in lat/long, the height and width of the current window in pixels and the zoom level are passed to the server.

The first level of filtering happens on the SQL server by doing an intersection of the boundary window with the point geography stored in the SQL server.

Points inside the boundary are returned to the web server, where above a given zoom threshold (10 for now), they are passed through the clustering class. The clustering class works by creating a grid out of the current window. Two nested loops are created, which loop through the grid cells. Each points coordinates converted from lat/long to pixel, and checked against the current grid cell. Points found in the same grid cell are grouped together into a single point, and stored into the grid cell list indexed by the X and Y cell index multiplied together.

Fortunately, Johannes Kebeck has posted an algorithm for doing this on his fantastic blog The algorithm he presents is setup to return javascript, and is written in VB, so we changed it slightly to return a SiteLocationPoint object and to operate in C#, since that is the language we are using for the application we are building.

We have turned this into a class, which takes a list of points, and returns a list of clustered points. The main change we had to make to the algorithm itself was modifying the it to use a Dictionary to store the clustered points in the gridCells object instead of a jagged array, which cannot be declared dynamically in C#.

We used a dictionary with the X*Y grid cell as a key for storing/extracting points in a given grid cell. The other modification we need to make, which I will post once we figure it out, is a way of changing the clustering density.

Click more to get the source code.

using System;
using System.Collections.Generic;
using System.Linq;
using System.Web;
using System.Collections;
 
namespace Matrix.Web.SiteManager
{
    public class ClusterSitePoints
    {
        private const int GridSize = 40;
 
        //Constants for the Clustering
        //(addressable area in Bing Maps)
        private const double MinLatitude  = -85.05112878D;
        private const double MaxLatitude  = 85.05112878D;
        private const double MinLongitude  = -180D;
        private const double MaxLongitude = 180D;
 
 
        public ClusterSitePoints(double north, double west, double south, double east, long mapWidth, long mapHeight, long mapZoomLevel, List<MatrixSiteLocationPoint> pointsToCluster)
        {
            this.east = east;
            this.west = west;
            this.north = north;
            this.south = south;
            this.mapHeight = mapHeight;
            this.mapWidth = mapWidth;
            this.mapZoomLevel = mapZoomLevel;
            this.pointsToCluster = pointsToCluster;
        }
 
        public List<MatrixSiteLocationPoint> ClusterPoints()
        {
            List<MatrixSiteLocationPoint> clusteredPoints = new List<MatrixSiteLocationPoint>();
 
            long numberOfCellsX = Convert.ToInt64(Math.Ceiling(Convert.ToDouble(this.MapWidth) / GridSize));
            long numberOfCellsY = Convert.ToInt64(Math.Ceiling(Convert.ToDouble(this.MapHeight) / GridSize));
            long numberOfCells = numberOfCellsX * numberOfCellsY -1;
 
            //MatrixSiteLocationPoint[numberOfCells][] gridCells = new MatrixSiteLocationPoint[][] { };
            //ArrayList gridCells = new ArrayList();
            Dictionary<int, MatrixSiteLocationPoint> gridCells = new Dictionary<int, MatrixSiteLocationPoint>();
            //Array MatrixSiteLocationPoint = Array.CreateInstance(typeof(MatrixSiteLocationPoint),
 
            long ulTotalX = 0;
            long ulTotalY = 0;
 
            LatLongToPixel(this.North, this.West, this.MapZoomLevel, ref ulTotalX, ref ulTotalY);
 
            long poiTotalX = 0;
            long poiTotalY = 0;
            long poiMapX = 0;
            long poiMapY = 0;
 
            foreach (MatrixSiteLocationPoint point in this.PointsToCluster)
            {
                LatLongToPixel(Convert.ToDouble(point.Latitude), Convert.ToDouble(point.Longitude), this.MapZoomLevel, ref poiTotalX, ref poiTotalY);
                poiMapX = poiTotalX - ulTotalX;
                poiMapY = poiTotalY - ulTotalY;
 
                // populate array with clustered points
                for (int x = 0; x < numberOfCellsX - 1 ; x++)
                {
                    if(( x * GridSize <= poiMapX) && (poiMapX < (x + 1) * GridSize))
                    {
                        for (int y = 0; y < numberOfCellsY - 1 ; y++)
                        {
                            if ((y * GridSize <= poiMapY) && (poiMapY < (y + 1) * GridSize))
                            {
                                MatrixSiteLocationPoint clusterPoint = null;
 
                                if (gridCells.ContainsKey(x * y))
                                {
                                    clusterPoint = gridCells[x * y];
                                    clusterPoint.NumberOfPointsInCluster += 1;
                                    clusterPoint.Location = clusterPoint.Location + " : " + point.Location;
                                    clusterPoint.LocationDescription = "Site Point Cluster";
 
 
                                    if (point.MatrixSiteKey.MatrixSiteCountOfProjects != null)
                                    {
                                        //clusterPoint.MatrixSiteKey.MatrixSiteCountOfProjects.NumberOfProjectsInSite += point.MatrixSiteKey.MatrixSiteCountOfProjects.NumberOfProjectsInSite;
                                        clusterPoint.NumberOfProjectsInCluster += point.MatrixSiteKey.MatrixSiteCountOfProjects.NumberOfProjectsInSite;
 
                                    }
 
                                }
                                else
                                {
                                    clusterPoint = point;
                                    clusterPoint.NumberOfPointsInCluster = 1;
 
                                    if (point.MatrixSiteKey.MatrixSiteCountOfProjects != null)
                                    {
                                        //clusterPoint.MatrixSiteKey.MatrixSiteCountOfProjects.NumberOfProjectsInSite += point.MatrixSiteKey.MatrixSiteCountOfProjects.NumberOfProjectsInSite;
                                        clusterPoint.NumberOfProjectsInCluster += point.MatrixSiteKey.MatrixSiteCountOfProjects.NumberOfProjectsInSite;
 
                                    }
 
                                    clusterPoint.IsClustered = true;
                                    clusterPoint.Location = clusterPoint.Location + " : " + point.Location;
                                    clusterPoint.LocationDescription = "Site Point Cluster";
 
                                }
                                gridCells[x * y] = clusterPoint;
                            }
                        }
                    }
                }
            }
 
            // extract point clusters
            foreach (MatrixSiteLocationPoint point in gridCells.Values)
            {
                clusteredPoints.Add(point);
            }
 
            return clusteredPoints;
        }
 
        private void LatLongToPixel(double latitude, double longitude, long lvl, ref long pixelx, ref long pixely)
        {
            latitude = Clip(latitude, MinLatitude, MaxLatitude);
            longitude = Clip(longitude, MinLongitude, MaxLongitude);
 
            double x  = (longitude + 180) / 360;
            double sinLatitude  = Math.Sin(latitude * Math.PI / 180);
            double y = 0.5 - Math.Log((1 + sinLatitude) / (1 - sinLatitude)) / (4 * Math.PI);
 
            UInt32 mapSize = Offset(Convert.ToInt32(lvl));
            pixelx = Convert.ToInt64(Clip(x * mapSize + 0.5, 0, mapSize - 1));
            pixely = Convert.ToInt64(Clip(y * mapSize + 0.5, 0, mapSize - 1));
 
        }
 
        private double Clip(double n, double minValue, double maxValue)
        {
            return Math.Min(Math.Max(n, minValue), maxValue);
        }
 
        private UInt32 Offset(int level)
        {
            UInt32 bit = new UInt32();
            bit = 256;
            return bit << level;
        }
 
        #region (Property) North
        private double north;
        /// <summary>
        /// Gets/Sets Northern coordinate of map window
        /// </summary>
        public double North
        {
            get
            {
                return north;
            }
            set
            {
                north = value;
            }
        }
        #endregion
 
        #region (Property) South
        private double south;
        /// <summary>
        /// Gets/sets Southern coordinate of map window
        /// </summary>
        public double South
        {
            get
            {
                return south;
            }
            set
            {
                south = value;
            }
        }
        #endregion
 
        #region (Property) East
        private double east;
        /// <summary>
        /// Gets/Set eastern coordinate of map window
        /// </summary>
        public double East
        {
            get
            {
                return east;
            }
            set
            {
                east = value;
            }
        }
        #endregion
 
        #region (Property) West
        private double west;
        /// <summary>
        /// Gets/Sets Western coordinate of map window
        /// </summary>
        public double West
        {
            get
            {
                return west;
            }
            set
            {
                west = value;
            }
        }
        #endregion
 
        #region (Property) MapWidth
        private long mapWidth;
        /// <summary>
        /// Gets width of map
        /// </summary>
        public long MapWidth
        {
            get
            {
                return mapWidth;
            }
            set
            {
                mapWidth = value;
            }
        }
        #endregion
 
        #region (Property) MapHeight
        private long mapHeight;
        /// <summary>
        /// Gets/Sets height of map
        /// </summary>
        public long MapHeight
        {
            get
            {
                return mapHeight;
            }
            set
            {
                mapHeight = value;
            }
        }
        #endregion
 
        #region (Property) MapZoomLevel
        private long mapZoomLevel;
        /// <summary>
        /// Gets/Sets zoom level of map
        /// </summary>
        public long MapZoomLevel
        {
            get
            {
                return mapZoomLevel;
            }
            set
            {
                mapZoomLevel = value;
            }
        }
        #endregion
 
        #region (Property) PointsToCluster
        private List<MatrixSiteLocationPoint> pointsToCluster;
        /// <summary>
        /// List of points to cluster
        /// </summary>
        public List<MatrixSiteLocationPoint> PointsToCluster
        {
            get
            {
                return pointsToCluster;
            }
            set
            {
                pointsToCluster = value;
            }
        }
        #endregion
    }
}

GIS Technology Stacks

In order to produce a single view of GIS data derived from multiple sources, it is necessary to abstract the rendering of the data from the actual datasources. This is similar to the concept of a Services Oriented Architecture in which business data are rendered from their system domain to a domain specific language that suits the needs of the enterprise.

The term that is given to the systems required to perform the rendering function are called ?stacks? in the GIS world.
The Stack typically is composed of: Spatial Database, Internet Mapping Server (map services), Mapping Client, and Web Mapping Client.
There are a few different technology streams available for provisioning stacks. I have been doing research into the different stacks available for service enabling GIS data.

Database:

Used for storing, indexing and querying spatial data. The most common of these is the ShapeFile or File Geodatabase. This is simply a set of specially configured files that combined together make a -database-. In the world of -real- databases, there are several common options

- Oracle Spatial - supposed to be the best in terms of accuracy and speed. It is expensive, and requires specially trained resources to keep it running

- PostGres/PostGIS - best open source spatial database. It is fast, reliable and widely supported. There are good toolsets for it on most platforms

- Sql Server Spatial - New spatial database on the block already has decent driver support, work is still required on optimization. The Geometry (planar) data type has support now on Mapserver and Geoserver, but not the Geography datatype. ESRI requires the use of the ESRI server for full connectivity.

- ESRI SDE - ESRI spatial extension for Oracle and SQL server - expensive, accurate, requires ESRI specific expertise. This product was moved into the ESRI Server offering as of version 9.3

Currently at Matrix (where I work) we are using:
- SQL server for base and field data
- SHP (ESRI Shape files) for project data used for spatial analysis and other core GIS functions
- PostGres/PostGIS for hydrogeology data, and possibly base data.

Internet Mapping Server (IMS):
Internet Mapping Servers provides mapping services. These appear to be the key piece that we are missing to make this whole thing work on an enterprise scale.

IMS typically are subdivided into WMS (Web Mapping) and WFS (Web Feature Mapping) servers (Article HERE describing the differences) . The Web Mapping Services are typically image based tiles that are rendered by the server through a map template from the datasources defined for that template, and output to image tiles. This is the most commonly used approach, and is what Google Maps, Virtual Earth etc are doing in the backend on a massive scale. The issue here is that while tiles are the fastest way of returning properly formatted spatial data, the objects (or Features in GIS speak) represented on the images cannot be directly manipulated. Additional query to database directly type techniques can be used to get around this if necessary. A better approach though is to use Tiled Map Services for base data or displaying dense datasets that do not change often, and use Features (vector/object data) for dynamic datasets or in places where the object density is reduced (higher zoom levels)
The Web Feature Service returns actual object or in GIS speak Feature definitions. A typical IMS server will support returning data in both WMS and WFS type services. The WFS data return is typically an XML stream formatted in KML, GML, or GEORSS format. There are probably some proprietary ones as well for ESRI specific data.

The players out there on this front are:

- ESRI Server - full featured, very expensive, but does it all. It also has the ability to do spatial analysis through a web browser.

- Mapserver - older open source IMS system we have tried it at Matrix. It works well, but has a more 'command line' approach to layer provisioning and requires a higher understanding of specific mapping techniques to make it work.

- FME Server - new this year. It is very useful for doing spatial transforms, analysis and providing data from a variety of sources, but is not an IMS server in production. It has a complicated licensing scheme based on engines that makes it difficult to get sense of what the cost of the system is. However, FME tries to mitigate this by giving users a long evaluation period to asses what they might need.

- Manifold IMS - very inexpensive, fast, but I was not crazy about the implementation. It is built around the idea of using the Manifold user interface to generate code for a website. The downside is the code that is generated is an older style postback server approach instead of a more modern Ajax style user interface. It is likely possible to customize this to give a better user experience, but it seems like a lot of work. However, for the cost its an excellent alternative.

- MapGuide - open source and commercial versions available. The open source version is the prior year's commercial offering. Seems to be good, but has spotty support. Similar to Manifold, it Is built more around a software factory approach, so is quick to get up and running without programming.

- GeoServer - open source, widely supported, has a very nice administration interface, and decent driver support. Supported by a very active community. It also has an associated commercial support style model called OpenGeo, which packages the various open source technologies together into one offering.
It has a wide support base, uses XML for its layer definitions, has a very nice looking administration console, and excellent documentation.

Geoserver also has several add-ons that look really good as well. One of immediate interest is GeoWebCache, which is a WMS tile caching and optimization server in effect a spatial proxy server.
Another of interest, although very new on the scene is a WFS based versioning module, of which one branch appears to be looking into using SVN as the backing for spatial feature version control (http://geoserver.org/display/GEOS/Versioning+WFS)

Map Editing Clients:
These are clients that allow GIS experts to create, manage and manipulate the spatial data directly. They can derive spatial data from a variety of sources to produce an overall map.

Typical clients are:

- ArcMap - which is what ESRI collectively calls its mapping clients, and it is broken up into three levels in terms of increasing cost/features: ArcView,ArcEditor and ArcInfo
- ArcView - Basic ESRI spatial data editing tool - designed for single user single datasource
- ArcEditor - Encompasses ArcView plus adds multiple user editing
- ArcInfo - All the features of the other Arcs plus spatial analysis

- Manifold - Low cost equivalent to ArcMap. Not as powerful as the ESRI offerings, but does some things better/faster than ArcMap. It is very useful as a general non-GIS client for editing spatial data.

- Quantum GIS - open source GIS editor - pretty good equivalent to the ArcMap in terms of features etc.

Udig - open source GIS client by refractions research who build and maintain PostGis. This is an excellent well thought out easy to use client, and I would recommend it

At Matrix, we will likely continue to use the core ESRI ArcMap series for the main GIS team. Manifold will likely be used for people who need to do desktop editing, but are doing so on a more limited basis. Quantum GIS has not been experimented with enough to know if it will be useful at Matrix.

Web Mapping Clients

There are a number of web mapping clients available. They are all roughly feature equivalent, and offer slightly different advantages. If the underlying data provisioning is done correctly, supporting more than one client is not a huge outlay of time

Typical Web Mapping Clients are:

Google Maps - the most common client. Base data provided by Google must be paid for on a $10,000 per year license for commercial use. Javascript based, relatively easy to work with. The APIs however are designed more towards support consumer mashup style development, and can be confusing to work with for developers used to more enterprise GIS style APIs

Virtual Earth (now bing maps) - comes in two versions now: javascript and Silverlight. We have successfully used the javascript version to build basic demonstration map tools. However, the Silverlight one offers a lot more promise for corporate mapping applications. It is much faster, supports c# based development and can leverage WCF style services. For Matrix Rapid development, we think this will be the best client to focus on building a framework around. Virtual Earth also provisions base maps at a cost roughly equivalent to the Google Maps cost ~10,000 US per year.

Open Layers - open source equivalent to Google Earth and Virtual Earth javascript mapping clients. It offers similar functionality, but does not come with a base maps subscription. OpenStreetMaps provides basic street level mapping data, but will not be of much use for Matrix given that most of the work focuses in areas where the OpenStreetMap approach will not work. There are a number of sources Matrix GIS team has studied on this front for base map provisioning

Time Machine fails with Error mounting volume

I use a Macbook pro for doing all my software development. Since switching over to Mac, I can't imagine using any other computer to develop with. The combination of the OSX operating system with the power of the Macbook Pro makes for an unbeatable development platform.

I have also eliminated all windows machines from our house now as well. I have a linux server for my websites, an Imac for our 'TV', which also acts as our primary media server for music, photos and family media, and an old 17" Powerbook G4 that my wife uses. I have the three Mac machines backing up to a 1 terabyte Timecapsule.

The Timecapsule is a great innovation on Apple's part, and has already saved my life on a few occasions. Not only does it handle all my media etc. but with the new version of VMWare fusion, I am able to have automatic backups of my local development and document files that I use on the Windows side.

Time Capsule was easy to setup, and had run without an issue until a few weeks ago, when I started getting errors during backups on my Macbook Pro.

These were 'Backup failed - Error mounting volume', or 'Network error during backup'. After digging around a bit through various log files etc. I found that the likely culprit had nothing to do with the network, but was the fact that my hard drive had bad files and directories on it.

Resolving this issue involved booting up from the OSX CD and running the Disc Utiltiy Repair function on the Macbook Pro hard drive. Since doing this, Timecapsule is up and running again without a problem.

It would be nice if Time Capsule/Time Machine could produce a more informative message, or offer to take some kind of corrective action in the event of a failure like this instead of the misleading cryptic message it had generated. Thus this blog post so hopefully the next person who hits the problem will be able to fix it without having to dig through log files...

Team based development systems - Article 1 - Tools

I recently completed an assessment and subsequent construction and deployment of a development management system for our software development team at Matrix

While Microsoft has brought out Team system for doing source control, automated build/test, its a fairly expensive product. It has the advantage of being an 'install and go' kind of thing, but the cost of it makes it prohibitive for smaller development teams.

There are also lots of great open source tools out there for performing these tasks that work very well. At Matrix where I work, I have assembled a set of tools that are doing a great job of letting us manage a very extensive set of code across a diverse team of people in a quick, easy and efficient manner. This blog post will review the tools we are using for this, and give examples of how to set it up.

Source Control:

At the heart of every development system is a good source control system. I have tried out a number of these including Visual Source Safe, Microsoft Team System, CVS and SVN.

Visual Source Safe:

Microsoft Visual Source Safe is easy to setup, but painful to use and it has a lot of issues. Its ok for small groups of people, but does not scale well. I found it works best for teams under 5 people who work on a small number of projects, or as a personal source control system.

Microsoft Team System:

We did research Team System - but ruled it out mainly due to its cost, and the fact it is ecosystem tied. The pricing structure, as with anything Microsoft makes, is very complicated, so its hard to get an idea of what it actually costs to set this up. Team System covers a lot more than just source control - its a full development management system from source control through to build/test and software project health tracking.

I think for teams that do strictly Microsoft centric development who are fully immersed in the Microsoft world, and can leverage the value from the Team System Visual Studio clients it would be a very good tool. A full detail product comparison of the visual studio offerings can be downloaded HERE.

CVS and SVN:

As anyone in the open source community knows, CVS and SVN are the best systems out there. CVS as I understand it was the original source control system. I have used it - and its the most comprehensive and some say the most powerful source control system out there. However, SVN is the one we ended up going with, as we needed something that was a bit easier to work with, was simple to setup and had a good variety of clients to choose from.

After initially starting with a standard linux based SVN system, which we got working without a hitch, we ended up switching over to a great tool I found called VisualSVN. We were just using Tortise SVN to the linux system, but with more junior developers ran into a few problems with the lack of direct IDE support for SVN in visual Studio. Through doing some research I found VisualSVN which offers a $49.00 client that wires beautifully into Visual Studio. Makes using SVN as easy (if not easier) than the Microsoft equivalents.

The same company that makes VisualSVN also makes a VisualSVN server - which packages all the SVN bits for use on Windows Server. We were originally using VisualSVN against the linux based repository, but ended up switching over to the VisualSVN version simply because VisualSVN server has a nice GUI On it for managinge the repository using MMC snap in, and it integrates directly with Active Directory for security, thus eliminating the need for managing a second set of passwords etc. The best part is the VisualSVN server is a free download.

I would recommend VisualSVN for anyone looking for a powerful, easy to use alternative to Microsoft's tools.

Unit Testing:

On the Unit Testing front, there are several alternatives. The Microsoft Team Systems Test framework, Nunit and the more recent derivative Xunit. There is a good comparison of these on the Xunit site. At Matrix, we never got far enough into Microsoft Team System to try out the Visual Studio Test and Test system framework, so I cannot comment on it.

Nunit/Xunit:

I was using Nunit up until last year, when I went to the Patterns and Practices confrence and saw Jim Newkirk (who developed both Nunit and Xunit framework and is a long time proponent of unit testing) lecture on the subject. There are a number of things he did when building Xunit that make it a better testing framework - including having setup/teardown in constructor/destructor, and changing the way the tests are run by randomizing them. This forces each test to succeed or fail on its own, and ensures that there are no unforseen interdependancies of the tests.

We have been using Xunit for a while now, and it works very well.

TestDriven.Net

If you are looking for a tool that can be used to run the Nunit/Xunit tests directly from inside Visual Studio, check out TestDriven.Net - I have tried it out, and it works very well. It allows either Nunit or Xunit to be run directly inside visual studio. You can run one or many tests, and you can run tests under debug. This is very useful for framework level debugging as well as simply checking test results before committing code. Tests can be used as an entry point for code that would normally require a UI to trigger them for debugging purposes.

It therefore adds more value to building unit tests especially for lower system levels that don't have a presentation layer, since it gives a very simple quick way to step through code. They offer a personal, professional and enterprise license which are all pretty well priced.

Automated Build Tools:

The two main tools out there seem to be MSBuild, which is also part of Team System, and Nant.

Nant:

Most people will likely be familiar with Nant which is a derivative of Ant. At its simplest, Nant is a .net focused build tool that uses an XML file to generate .net compiler output. It can be used for a lot more than just that though, and has a very extensive set of commands associated with it. So extensive, I found it took a while to get a handle on how to best set it up.

I found a great tool though to help with the code markup though called NantBuilder. Its a simple IDE for building and debugging Nant scripts. It has auto complete, and a list of Nant codes and what they do, so is a great way to learn Nant very quickly. Its also very inexpensive, and they ave a 30 day trial as well.

Documentation:

I used to use NDoc - but it seems to have died out. There are a few other options out there such as Doxygen and a number of commercial products, but fortunately someone from Microsoft appears to have stepped up with the Sandcastle project. (project blog is here)

Sandcastle works very well, and generates docs for managed code using a very similar approach to Ndoc. It however supports all the new language elements such as generics. In addition, there is a GUI editor available for it also on codeplex here. This makes the process of setting up documentation generation for class libraries very simple.

Code Analysis:

From a .net perspective there are two main free tools out there that are available for doing static analysis that I am aware of. One is Style Cop (blog here), and the other is FX Cop.

Style Cop:

Style Cop looks at the source code from a code layout/design perspective using a set of style and consistency rules. These rules are derived from Microsoft's .net Framework design guidelines. Style cop produces a set of reports that show where source code is out of compliance with these guidelines. The tool can be tuned to suit individual development environments by changing the sensitivity and excluding rules that don't apply. Other rules can be added as well. Its useful to get an idea of how well developers are following standards guidelines.

FX Cop:

FX Cop anlayzes .net assemblies and generates reports on design, possible performance issues, potential security problems etc. It is also based on the .net Framework Design Guidelines produced by Microsoft. As with Style Cop, it can be tuned to suite individual development environment needs.

NCover:

Another great code analysis tool that I have found very useful is NCover. It looks at the amount of code that is being excercised by automated unit tests, and generates a report.

This is useful information as it clearly shows areas where code is not being executed, which either means unit tests are missing, or the code is in fact 'dead'. It can also show branches of code that are not being tested - for example in the case where there may be conditional or object strategies that branch but whose branches are not covered by current unit tests.

Conclusion:

There are a fair number of tools outlined here. In our environment, we use:

SVN - Source Control through VisualSVN server and Visual SVN client
Xunit - for unit testing. We have a copy of TestDriven.net for test debuggging.
Nant - Build automation Scripts. Nantbuilder I found useful to help get up and running with this.
Sandcastle - Documentation generation, and the Sandcastle UI for building the doc generation script
FX Cop and Style Cop - for static code analysis
NCover - for code coverage analysis

Because we have a small development team, automation is critical to being able to take advantage of these. So while it takes a bit of work to get the automated build/analyze/test scripts setup initially, it really pays off in terms of getting consistent reports, tests and documentation generated.

This requires though a tool which can act as the driver for all of these - and the best tool out there is ThoughtWorks Cruise Control, which comes in both an open source and a commercial version. We are now using the commercial version for our build management system, but the great news there is ThoughtWorks has a great licensing system that is tiered, so even with a small team such as ours, we can take advantage of a very powerful build automation system for initially no cost other than our time.

In the next post, I will delve in detail into how we set Cruise Control up to drive all these other tools. I will also include the scripts required to generate output from each tool as an example.

Configuring Silverlight to use a secured WCF Service

Configuring Silverlight to access a NT Authenticated WCF Services :

add .xap -> application/x-silverlight-app to the mime type in IIS to get IIS to recognize and transmit the mime type
add .xaml -> application/xaml+xml to the mime types in IIS to get IIS to recognize and transmit the mime type
add the clientaccesspolicy.xml to the root folder in IIS to get the cross domain working for wcf debugging
Ensure that you have the IIS virtual directory security set to use NT authentication only

If we modify or create new services inside the service class, it is necessary to make sure the URL is set correctly in the ServiceReferences.ClientConfig file in the Silverlight project when you update the service reference. It defaults to a value that is different than the value visual studio uses when the debugger is run, which causes an exception to be thrown when the silverlight call goes through to the service.

Clientaccesspolicy.xml file:

The clientAccessPolicy.xml file is a file silverlight uses to determine rules around accessing domains. Without this file, Silverlight throws a ‘404’ error. Docs indicate it has to be in the root of the website – not in the virtual directory

<?xml version="1.0" encoding="utf-8"?>
<access-policy>
<cross-domain-access>
<policy>
<allow-from http-request-headers="*">
<domain uri="*"/>
</allow-from>
<grant-to>
<resource path="/" include-subpaths="true"/>
</grant-to>
</policy>
</cross-domain-access>
</access-policy>

Web.config WCF Settings:

Here’s what worked

security binding has to be setup to use transportCredentialsOnly, with Windows for the type.
This binding has to be bound to the basicHttpSerivce endpoint behaviour
the MEX binding that is created by the service generation tool has to be deleted or it seems to confuse things once security is enabled
The service must be setup to use aspNetCompatibilityEnabled = true – and associated code and includes added into the service class. This is done automatically by the add new SilverlightWCF service

<system.serviceModel>
<bindings>
<basicHttpBinding>
<binding name="basicHttpSecuredBinding">
<security mode="TransportCredentialOnly">
<transport clientCredentialType="Windows" />
</security>
</binding>
</basicHttpBinding>

</bindings>
<behaviors>
<serviceBehaviors>
<behavior name="EmployeeDashboardServiceBehavior">
<serviceMetadata httpGetEnabled="true" />
<serviceDebug includeExceptionDetailInFaults="false" />
</behavior>
</serviceBehaviors>
</behaviors>
<serviceHostingEnvironment aspNetCompatibilityEnabled="true" />
<services>
<service behaviorConfiguration="EmployeeDashboardServiceBehavior"
name="EmployeeDashboardService">
<endpoint address="" binding="basicHttpBinding" bindingConfiguration="basicHttpSecuredBinding"
name="basicHttpEndpoint" contract="EmployeeDashboardService" />
</service>
</services>
</system.serviceModel>

Silverlight over SSL With Authentication:

Client Access policy file has to have some additional attributes added to it as below:

        1: <?xml version="1.0" encoding="utf-8" ?>

       2: <access-policy>

       3:   <cross-domain-access>

       4:     <policy>

       5:       <allow-from http-request-headers="*">

       6:         <domain uri="http://*"/>

       7:         <domain uri="https://*" />

       8:       </allow-from>

       9:       <grant-to>

      10:         <resource include-subpaths="false" path="/"/>

      11:       </grant-to>

      12:     </policy>

      13:   </cross-domain-access>

      14: </access-policy>

Client Side:

<configuration>
   <system.serviceModel>
       <bindings>
           <basicHttpBinding>
               <binding name="basicHttpEndpoint" maxBufferSize="2147483647"
                   maxReceivedMessageSize="2147483647">
                   <security mode="Transport" />
               </binding>
           </basicHttpBinding>
       </bindings>
       <client>
           <endpoint address="https://name.svc"
               binding="basicHttpBinding" bindingConfiguration="basicHttpEndpoint"
               contract="ServiceReferenceEmployeeDashboard.EmployeeDashboardService"
               name="basicHttpEndpoint" />
       </client>
   </system.serviceModel>
</configuration>

Silverlight

While I am critical of Microsoft, its because I know they can do better than they have in recent years. If you put the dismal performance of Vista and the horrible marketing campaigns of late, Microsoft outside the consumer and platform sphere has created some incredibly innovative products for developers over the last few years. Linq, and WCF are just two recent examples - not to mention the new SQL server 2008 which is amazing as well - more on that in a later post...

Unfortunately, it is Microsoft business model that needs innovation - not their tools. It is mired in the world of PC software - often to the detriment of new and innovative ideas.

Fortunately, Silverlight looks like a breakthrough technology for Microsoft: mainly because it shows a change of attitude. Microsoft is finally getting serious about cross platform support, and the web as the application platform. I have often wondered why Microsoft has not made a .net VM host for other platforms - Mono aside. The obvious answer being fear of cannibalizing its windows franchise. I personally think Microsoft, if it were serious about this battle for the web, would make windows free for consumers, and instead shift its focus to a subscription service based model: say $50 a year for support for Windows or something to that effect. It would cost it a fortune initially, and would affect their stock price (could it get worse??) but would force the company to formally acknowledge what has already happened: Operating systems are a commodity.

Anyway - fantasy aside - It seems that Silverlight is at least a start at Microsoft opening its platform up to other operating systems.

Silverlight allows for .net managed code to run on any platform using Windows Presentation Framework. I am very excited about this, as many of the apps we are looking at building can then be made to run on any platform. It is of course possible to do this with Ajax, which I have done, but it will be nice to have the advantage of a full fledged debugger and a nice set of tools that finally is graphic designer friendly on the Microsoft side (Google of course has had this for a while with its GWT - I have never liked Microsoft's native implementation of asp.net Ajax - its complicated and messy).

I already have a version of a managed code graph written in C# using the open source Visifire control working in IE, Firefox, and Firefox on the mac (have to try linux next)... using the same code base. This graph gets its data from a WCF service that pulls data from our accounting system.

Here it is running in firefox on my MacBookPro:

Getting this working though was a bit difficult from a configuraion standpoint.