This is due to several things:
- Often times, the authors have a clear industry bias. Two of the papers I reviewed had clear conflicts of interest. This doesn't automatically invalidate findings, but it is difficult to believe a company would be dumb enough to trash their own products and services.
- Completely different methodologies for arriving at total-cost figures. As always, there are numerous ways to Fudge The Numbers™
- Completely different use cases. Depending upon the required task, Windows and Linux may have different TCO numbers. No two companies are identical, and different companies have totally different needs. Companies that develop software, for example, are going to have totally different requirements than a company that sells insurance.
- Dubious sources, claims, or arguments. Many papers had sources that were improperly cited, or that I could no longer locate.
- For certain comparisons, objective figures are not easily expressed. For example, evaluating "usability" is a difficult proposition. Evaluating "stability" is a comparable dilemma. Some studies attempt to factor these details in, but do so in a very superficial manner.
Here are some of the articles I read:
- CyberSource (2004) Linux vs. Windows. Total Cost of Ownership Comparison
- Didio, L. (2005) 2005 North American Linux & Windows TCO Comparison Report, Part 1
- Yamamoto Krammer, M. (2005) Costs and benefits Still Favor Windows Over Linux Among Midsize Businesses
- Windows 2000 Verses Linux in Enterprise Computing
...that said, I'd like to demonstrate the above uncertainty by brutally harassing one of the above articles. I didn't find this article much better or worse than the others (in general, the quality is poor); this is for illustrative purposes only.
The CyberSource paper concluded that open source software saved businesses 19 to 36%, depending upon various factors and conditions. The most important factoid I drew out of this study was the fact that CyberSource is "the second longest running open source solutions company in the world." Immediately, it's hard to be objective about their findings, because there's a clear conflict of interest. For the same reason I wouldn't trust MSFT to release honest, accurate information about TCO figures, it is hard to take this evaluation seriously.Still, there are some compelling arguments in the paper that should be discussed. CyberSource does clearly state they are vendors (and thus proponents) of Open Source Software solutions, so they let their biases be known early on. Furthermore, they make the claim that the scales have been tipped in MSFT's favor by:
- Not applying survey information indicating it takes less resources to support Linux
- Not factoring in viruses, malware, spyware, etc.
- Not counting system downtime from "reboots" and "crashes"
- Tripling the budget for Linux support to counter MSFT's claim that external experts would be required to support OSS solutions.
At first these would seem to be gigantic "favors," but let's take a moment to look each gift horse in the mouth.
The first scale-tipper isn't particularly compelling, given that other TCO reports had "survey information" stating the exact opposite: Windows requires less support resources than Linux. I was also unable to verify the source of this claim because it was a dead hyperlink; since the veracity cannot be established, it shall be struck.
Second, viruses....what can be said of viruses on Windows? As a "source," the article links a listing of viruses that have been seen in the last month. The source does not state how many times a virus was found, nor what platform the virus was targeting, nor if the underlying security vulnerability has been patched. It is an interesting list; it is not, however, a useful source to illustrate the security of Windows XP or Vista. In the context of TCO, it's almost worthless as a resource.
(also worth noting is that the listing of viruses is misleading because the list enummerates viruses rather than the specific exploit being targeted, so it is possible the list gives a false sense of insecurity)
Furthermore, other studies have little qualm with factoring in security costs for Linux--so the claim that these sorts of costs are not attributable to Linux is immediately suspect. I believe the reason for this is security in a mid to large business has more to do with protecting internal resources from employees, rather than from outside threats. At my last job, which was a fortune-50 company, our biggest threat vectors were clearly from within.
Regarding system downtime and "crashes," the article bases its claim that Linux is more stable and less prone to downtime with a single source. CyberSource claims that "our research indicates that open source systems rarely if ever suffers such problems," and further claims that "none of the most robust systems tracked by research firm Netcraft UK are Windows." Looks like they haven't viewed their own source in a while: the top three positions are occupied by Windows Server/IIS configurations. Also, this listing is almost exclusively based on web servers, and the dominant web server is Apache (60% market share, verses 30% market share for IIS, as of December 07); assuming equal weighting in terms of stability, Apache should dominate the list.
Personally, I don't think the netcraft numbers are actually relevant to stability/downtime. However, they are the only figures CyberSource provides as evidence of Windows being more prone to downtime and crashes. I personally find it to be unconvincing.
The final advantage is much more tangible. CyberSource triples the budget for Linux support in response to the claim that Linux may require more external consultants than Windows. This is a honest scale-tipper, in my opinion; they factor the costs of outside consultants in the Windows system at $45k, and the numbers rise to $135k for OSS solutions.
Cybersource then dives into the actual numbers, and there's definitely some funny math going on. Here is a compilation of costs for the Windows system:
...the first item is in direct conflict with CyberSource's claim that they weren't going to factor in the cost of viruses, spyware, malware, etc. For a 250-computer company, that comes to $9,475. To be honest, this number should be deducted. Either that, or they need to estimate the whole cost (including support). There are also free anti-virus solutions available; CyberSource glosses over the fact that free software isn't just for Linux. (more on this later)
The second item--Window Server 2003, is assigned a dollar figure of $3,999. This is not entirely clear, because Cybersource is specifically referring to the cost of Windows Server 2003 Enterprise Edition. Other versions of Windows Server 2003 have much lower costs. The only sizable difference between Enterprise and Standard is the number of physical processors and the amount of RAM supported; Standard costs $999. It is not necessarily true that a business of 250 employees would need Enterprise.
The third item is Microsoft Commerce Server. Again, omitted is the fact that the $20k figure is for the Enterprise Edition. Standard Edition is $7k.
Regarding Microsoft ISA 2004, for some inexplicable reason CyberSource goes with the cheaper version. The pricey Enterprise Edition is $6k per processor, or a cool $75k for 25 processors.
SQL Server has more editions with more complicated pricing schemes than I care to ennumerate in this post; prices range anywhere from $4k up to $25k; at any rate, the Cybersource figures are likely inflated.
The quoted price for Visual Studio .NET is for some sort of Enterprise Edition; MSFT has moved on from 2003 and 2005 is the latest in sliced bread. Prices for VS.NET 2005 range anywhere from $300 for the lowly Standard edition, up to $10k+ for Team Suite with MSDN Premium Subscription. Considering the CyberSource TCO figures only a single copy of VS.NET into the figures, there is literally no point in buying anything beyond the standard edition. Hell, the absolutely free Express Editions might suffice.
Next, we have the largest costs in software: operating systems and office software. First off, the CyberSource article specs out full retail pricing for Windows XP Pro; this is stupid. If a business is buying 250 copies, that's well into the world of bulk licensing. Ditto for office. The office software, however, is even more troublesome. One thing CyberSource did was create an analogous software package for their Linux-based solution. Included as the office suite is the venerable Open Office application (which I am actually a big fan of). What CyberSource doesn't make clear is that you can run Open Office on Windows, and it works great. So charging the Windows based configuration $300 times 250 computers is weird, and possibly unfair. Either MSFT Office has something that Open Office lacks (which would be the only good reason to cough up that much money), or these TCO figures reek of dead fish.
Finally, the icing on the cake: the Microsoft Software Insurance Program, which CyberSource computes to be a recurring cost of 25% a year for server OSs, and 29% a year for desktop OSs. I know of no company that purchases this program. Even a complete idiot can see this is a bad investment: CyberSource estimates it to be $211k per year, which is more than the cost of operating systems or office software. Given that MSFT releases a new operating system or office platform every two to five years, it doesn't take a rocket scientist to see this is a bad deal.
The total cost for the MSFT Software package ended up being $504,712, verses $90 for the OSS solution. $211,000 was the Software Insurance Program. $173,000 was the office software. Removing those two items gets us to $120k in software costs, which completely obliterates the lead for the OSS solution. This is also not taking into account the fact that volume licensing can further reduce that figure, and that Apache, PHP, MySql, and other industrial grade OSS products are available on a Windows platform.
This is a consistent trend in all the studies I reviewed: why no hybrid environment? Why can't we run Windows Operating systems, Firefox browsers, Open Office, VS.NET 2005, and Unix servers running Apache, MySql, and PHP? Why must consumers continually be presented with this "us verses them" reality? To see that sort of attitude from MSFT is not totally surprising; to see it from the open source community, on the other hand, is disturbing.
Even administrative jobs get broken into this "us" verses "them" concept; is it outside the realm of reason that a single individual could be good at administering both Windows and Linux computers? Most nerds I know are proficient with several operating systems; on my desk, at this very moment, are Windows XP, Vista, and Ubuntu OS installs. At school, I typically use machines running Red Hat, unless I'm interested in using VS.NET, in which case I use XP Pro. I like all of them, more or less. Were I an employer, I would want to hire IT administrators who knew all of it--not just half of the picture.
I should reiterate that this study was pretty bad, but so were the rest of the studies I read. Some of them weren't even studies; they were surveys pretending to be legitimate TCO comparisons, which is equally as debatable as Cybersource's number fudging.
