24 July, 2008 21:53
The Big Switch?
Posted by mimo under [ Ideas , Systems Architechure , Opinions , Rants ][ (0) Comment ] | [ (0) Trackbacks ]
Just finished reading The Big Switch – excellent book. The key to
making the firm I work for (Matrix Solutions) successful is to minimize overhead and
infrastructure costs as much as possible. While Matrix’ only product is
information, as your book points out, we are in the business of
collecting, interpreting and managing knowledge for our clients – not
building and managing IT infrastructure.
Therefore - I am always looking for ways to reduce costs of IT by leveraging wherever possible the kinds of ‘utility’ services discussed in the book, and have for some time been a believer in SAS.
Matrix right now is still very much in the ‘older’ model of IT services, in which we still own and manage an IT infrastructure based on Microsoft servers and applications which the company owns.
The first shift away from this came when we moved all of the servers offsite to a data center provider (Q9), which eliminated the need for our firm to build and manage the power, cooling, and security plumbing many organizations require. There was a great deal of skepticism at the senior management level regarding this move – mostly centered around connectivity issues: it was an uncomfortable concept for them to have the servers hosted offsite. Since the move, stability, security and uptime have all increased.
Based on this success, the second step to moving to a utility model of computing, was to recommend using a hosted solution for one of our key business systems – the HR/Payroll system.
We chose to externally host the HR/Payroll system for the following reasons:
· HR/Payroll systems is maintained by a fairly small internal community of users
· contains highly confidential data which should be more secure stored offsite by a company specializing in handling these types of sensitive data than they would be internally with our more limited resources
· These systems and the companies providing them are well established
· The reporting requirements are fairly light compared to those found other core business financial systems (reports are fairly linear and simple relatively speaking)
· The data handled by HR/Payroll systems should be well suited to being operated on through a browser style hosted application
Unfortunately, the experience turned out to be an unmitigated disaster – which has made painfully clear to me some major business related weaknesses in the Utility model for computing described in the Big Switch, and current hype around 'Cloud Services'.
The first issue came from the fact that the provider we chose, while apparently well established and reputable mislead us regarding the capabilities of their hosted application and its readiness for commercial use.
A typical feature of hosted systems are their ability to be rapidly deployed and changed. This seems to have lead to a situation even worse than that found in the traditional software world: going from of rushing product cycles to get software out there to the live deployment of the endless beta. This was the case with the HR system in question. While the provider demonstrated the system to us, and assured us it was feature complete for use it was nowhere near ready for production use.
The core data entry system, while crude, worked reasonably well. So our HR staff worked away at getting information put into the system. However, what the provider did not tell us was that the reporting engine was not fully functional. Once our data was in the system, we discovered that in fact many of the reports were not working correctly.
Further investigation revealed that the reporting engine hardly functioned at all – outside the few core simple reports they had shown during the demonstration, and we had tested during the setup phase.
The provider promised us that they were addressing the issues, and that they would be fixed in a few weeks. This was in August 2007 – by December, an ‘upgrade’ to the reporting engine rendered the entire engine non-functional, leaving us in a situation where we could not generate ANY reports from the system. Queries to the provider went unanswered – turned out they were going through some kind of re-organization/merger.
The situation degraded further. A subsequent upgrade did not improve the system, and the re-organization ended up in the termination of the VP we were dealing with, along with 70 other members of the staff, including many of the technical resources.
We have also found out that we could not move back to their non-hosted system, were not in fact even be able to get the data out of the system in order to move to another platform. If we had had the database in house, at least we could have extracted the information. They do not have an API capable of allowing us to download the data either: so we are faced with the unpleasant choice of continuing to be held hostage to a system that does not work well and meet our needs, or starting all over again from scratch with a new system.
I believe that the technical issues associated with having fully hosted applications such as this one (and Amazon’s recent outages) will gradually be worked out over time.
But is one fundamental difference with applications hosted within a corporate firewall: my boss can come down the hall and fire me if I screw up, replace me with someone else who will then be able to take the application/data and take it in a new direction.
Certainly there is still a level of lock-in even with internally hosted enterprise application. Despite that, we can fire a vendor of an internally hosted application, and replace their product with another one at our leisure since the data can be migrated from one system to another because we control the application, and more importantly, the database and the data it contains (we just finished doing this successfully with the accounting system).
Another factor that is becoming apparent is that the value proposition of fully hosted solutions may not be as compelling as it seemed a few years ago. The reason for this is due to services such as Q9 and virtualization. Q9 gives us world class infrastructure, but doesn’t (yet) do much to address the servers and management issue.
Q9 charges on a power basis – so the more servers we get, the higher the cost. Traditionally, the more servers you have, the more staff required to support them. We just finished virtualizing nearly all of our servers – so we have gone from about 30 physical servers down to 3. This has cut the power consumption, and thus Q9 costs down significantly. In addition –the rollback/replication/management capabilities of VMware allow us to much more easily managed the ‘servers’ than ever before – so even though our server count has gone up, we are not finding an increase in workload (or associated staff increase).
The other thing we are finding is that providers of traditional enterprise software are offering what I would call a ‘hybrid hosted’ service. The provider of our financial/accounting system is an excellent example of this. They basically handle the maintenance of the servers that run their application as part of our maintenance agreement through remote access through me. For example, we are just about to do an upgrade to their system. We schedule a time where they patch in to my computer, then install whatever things they need to update their system. It’s not as seamless as having them simply upgrade their servers and us getting the updates through the browser, but it does have the advantage of allowing us to manage the changes to the user community in that we can test and schedule upgrades, thus avoiding unpleasant situation such as what happened with the HR system where it suddenly stopped working for the majority of our reports (and no rollback or backout capability).
Anyway – just some comments from recent experiences in this area. I still think there will be a switch – but it may not prove to be a ‘big’ one..
Therefore - I am always looking for ways to reduce costs of IT by leveraging wherever possible the kinds of ‘utility’ services discussed in the book, and have for some time been a believer in SAS.
Matrix right now is still very much in the ‘older’ model of IT services, in which we still own and manage an IT infrastructure based on Microsoft servers and applications which the company owns.
The first shift away from this came when we moved all of the servers offsite to a data center provider (Q9), which eliminated the need for our firm to build and manage the power, cooling, and security plumbing many organizations require. There was a great deal of skepticism at the senior management level regarding this move – mostly centered around connectivity issues: it was an uncomfortable concept for them to have the servers hosted offsite. Since the move, stability, security and uptime have all increased.
Based on this success, the second step to moving to a utility model of computing, was to recommend using a hosted solution for one of our key business systems – the HR/Payroll system.
We chose to externally host the HR/Payroll system for the following reasons:
· HR/Payroll systems is maintained by a fairly small internal community of users
· contains highly confidential data which should be more secure stored offsite by a company specializing in handling these types of sensitive data than they would be internally with our more limited resources
· These systems and the companies providing them are well established
· The reporting requirements are fairly light compared to those found other core business financial systems (reports are fairly linear and simple relatively speaking)
· The data handled by HR/Payroll systems should be well suited to being operated on through a browser style hosted application
Unfortunately, the experience turned out to be an unmitigated disaster – which has made painfully clear to me some major business related weaknesses in the Utility model for computing described in the Big Switch, and current hype around 'Cloud Services'.
The first issue came from the fact that the provider we chose, while apparently well established and reputable mislead us regarding the capabilities of their hosted application and its readiness for commercial use.
A typical feature of hosted systems are their ability to be rapidly deployed and changed. This seems to have lead to a situation even worse than that found in the traditional software world: going from of rushing product cycles to get software out there to the live deployment of the endless beta. This was the case with the HR system in question. While the provider demonstrated the system to us, and assured us it was feature complete for use it was nowhere near ready for production use.
The core data entry system, while crude, worked reasonably well. So our HR staff worked away at getting information put into the system. However, what the provider did not tell us was that the reporting engine was not fully functional. Once our data was in the system, we discovered that in fact many of the reports were not working correctly.
Further investigation revealed that the reporting engine hardly functioned at all – outside the few core simple reports they had shown during the demonstration, and we had tested during the setup phase.
The provider promised us that they were addressing the issues, and that they would be fixed in a few weeks. This was in August 2007 – by December, an ‘upgrade’ to the reporting engine rendered the entire engine non-functional, leaving us in a situation where we could not generate ANY reports from the system. Queries to the provider went unanswered – turned out they were going through some kind of re-organization/merger.
The situation degraded further. A subsequent upgrade did not improve the system, and the re-organization ended up in the termination of the VP we were dealing with, along with 70 other members of the staff, including many of the technical resources.
We have also found out that we could not move back to their non-hosted system, were not in fact even be able to get the data out of the system in order to move to another platform. If we had had the database in house, at least we could have extracted the information. They do not have an API capable of allowing us to download the data either: so we are faced with the unpleasant choice of continuing to be held hostage to a system that does not work well and meet our needs, or starting all over again from scratch with a new system.
I believe that the technical issues associated with having fully hosted applications such as this one (and Amazon’s recent outages) will gradually be worked out over time.
But is one fundamental difference with applications hosted within a corporate firewall: my boss can come down the hall and fire me if I screw up, replace me with someone else who will then be able to take the application/data and take it in a new direction.
Certainly there is still a level of lock-in even with internally hosted enterprise application. Despite that, we can fire a vendor of an internally hosted application, and replace their product with another one at our leisure since the data can be migrated from one system to another because we control the application, and more importantly, the database and the data it contains (we just finished doing this successfully with the accounting system).
Another factor that is becoming apparent is that the value proposition of fully hosted solutions may not be as compelling as it seemed a few years ago. The reason for this is due to services such as Q9 and virtualization. Q9 gives us world class infrastructure, but doesn’t (yet) do much to address the servers and management issue.
Q9 charges on a power basis – so the more servers we get, the higher the cost. Traditionally, the more servers you have, the more staff required to support them. We just finished virtualizing nearly all of our servers – so we have gone from about 30 physical servers down to 3. This has cut the power consumption, and thus Q9 costs down significantly. In addition –the rollback/replication/management capabilities of VMware allow us to much more easily managed the ‘servers’ than ever before – so even though our server count has gone up, we are not finding an increase in workload (or associated staff increase).
The other thing we are finding is that providers of traditional enterprise software are offering what I would call a ‘hybrid hosted’ service. The provider of our financial/accounting system is an excellent example of this. They basically handle the maintenance of the servers that run their application as part of our maintenance agreement through remote access through me. For example, we are just about to do an upgrade to their system. We schedule a time where they patch in to my computer, then install whatever things they need to update their system. It’s not as seamless as having them simply upgrade their servers and us getting the updates through the browser, but it does have the advantage of allowing us to manage the changes to the user community in that we can test and schedule upgrades, thus avoiding unpleasant situation such as what happened with the HR system where it suddenly stopped working for the majority of our reports (and no rollback or backout capability).
Anyway – just some comments from recent experiences in this area. I still think there will be a switch – but it may not prove to be a ‘big’ one..
