Where *nix and security meet the general public
Monthly Security Scorecard - another view
In response to Jeff Jones’ Monthly Security Scorecard I did some research on Secunia and made some statistics to answer his. Jeff’s Scorecard is quite minimal, in my opinion and, as pointed out by some of the comments, is missing some interesting facts. These facts include the outstanding advisories for example and of course the amount of Software installed. Since Linux installs a lot more software the numbers are a bit slanted, however even if I only take the numbers from Secunia in regards to advisories, vulnerabilities fixed etc., things still look quite different then on Jeff’s charts.
I took the liberty and counted only the advisories and patched vulnerabilities from November 2006 onwards. Furthermore I added a few more Operating Systems to put things a bit more into perspective and to see if there are any differences by vendors. The OS’s listed are:
- Windows XP Professional (according to Secunia, this is Service Pack 2)
- RedHat Enterprise Linux 4
- OpenBSD 4.0
- FreeBSD 6.x
- Windows 2003 Server Standard Edition
- Ubuntu Linux 6.06
- Mac OS X 10.4
First off, we have the amount of advisories issued per month (click to enlarge):
As you can see Ubuntu Linux is the clear leader followed by RedHat and Mac OS X. This leads me to believe that the advisories are increased because of the sheer amount of software that these Systems install. Notice the BSD’s are quite low.
Next off we have the patching of these vendors during the couple of months in 2006.
Again, something interesting. Notice FreeBSD, the Windows and Mac OS X had outstanding advisories, whereas the Linuxes have 100% patch track. Something that might be a benefit of this “community” thing?
So how are we doing in 2007 with patches then?
Again the Windows, Mac OS X and FreeBSD have outstanding advisories whereas the Linuxes have 100% patch rate.
NOTE: The outstanding advisories for FreeBSD are and were LOW criticality whereas Windows and Mac had a mixture of medium and semi-low.
So, how about criticality in those months of 2006?
Wow, if you look at that image you might get a shock at first glance. But then you notice that the spikes are moderately critical advisories. Ubuntu seems to have a lot of them, by the way. Now what gets me here is that Both Windows versions are the only 2 Operating systems that have a Critical advisory here…
But is 2007 any better?
Actually, in regards to the Windows versions, it isn’t. But the spike that Ubuntu keeps carrying is still there. This time though its in less critical even though there are already 8 highly critical advisories. Considering that Ubuntu has 100% Patch rate though, it doesn’t make this as bad as it looks.
Conclusion:
Do we have an ultra-secure Operating System? No way, not even OpenBSD, though it comes darn close to it. What we do see throughout these graphs though is that the open source community seems to be doing a better job in at least providing fixes to the Linux vendors, who then ship it out.
Something negative has to be said about this though. If you have a mission critical server, you do NOT want to read that there are already 31 patches available for your server, this year alone! When you have your maintenance window, it will be quite a drastic update. Then again, do you want to read that your mission critical Domain Controller has a critical vulnerability outstanding and there is no patch yet?
All in all, what does this really mean for us in real life? Nothing much really since the maintenance windows won’t change and generally seen this is not THAT big of a deal. I will keep this updated at the end of every month though, to match Jeff’s scorecards.
DISCLAIMER: I do not work for any Linux vendor or Microsoft. As a matter of fact I work in a non-tech industry. I do use Mac OS X as my OS of choice though. Am I biased? Probably, but at least I TRIED to use only the number available and pretty much all relevant numbers I could find (yes on ONE source only but still…).
Technorati Tags: enterprise, freebsd, interesting facts, linux, mac os x, openbsd, redhat, scorecard, secunia, ubuntu, vulnerabilities, windows 2003 server, windows xp professionalPopularity: 12% [?]
This is a very good assessment. However, you missed didn’t follow Jeff’s methodology. He threw out security issues for Linux applications that did not have an equivalent application installed by default on windows. I think if you followed his methodology, you would come up with a similar comparison.
Also, someone really needs to do a days-of-vulnerability analysis (number of days any vulnerability affects the system).
What are these “outstanding advisories” in FreeBSD? I am not sure I even understand what you mean by that term. Can you be more specific and if you know of a vulnerability that the FreeBSD Security Officer and the Security Officer Team does not know about, would you please tell them about it? I am sure they would like to know.
http://www.freebsd.org/security/
RHEL4 might have had 30+ updates, but.. That domain controller you mention probably won’t be running 90% of the stuff those patches are for. IM clients, email clients, etc… So what is the relevance?
Good morning, sorry for the late replies.
z: The point I am more trying to bring across is that Jeff’s numbers do not make much sense since his are the amount of fixes released per month. He totally ignores the amount of outstanding vulnerabilities. Now the question is, would you rather have a flaw open or patched?
Charlie, Advisories are potential problems. Since I took the advisories from the secunia pages here is a link to the FreeBSD 6.X one:
http://secunia.com/advisories/23721/
But as I mentioned, its a low criticality.
Wendell, you are absolutely spot on. Yet the issue I tried to focus on was that there are 30+ updates yet 0 outstanding. In my book that speaks very well for RedHat (and Ubuntu). There are vulnerabilities in 3rd party programs and yet they are quite fast in releasing those patches.
I think you missed one of the points made in one of Jeffs previous posts. The reason Linux never has any outstanding vulnerabilities on Secunia is NOT because they have been patched. It is that Secunia has no efficient way of confirming that vulnerability X affects vendor Ys distribution. Therefore the vulnerabilities only appear against Ubuntu / Redhat / etc. WHEN A PATCH IS RELEASED, thus giving a 100% 0 hour fix time. Conversely if a flaw is found in Windows, it affects Windows and thus can be added as outstanding immediately.
While I do take Jeff’s analysis with a pinch of salt due to his working for MS and thus having a potential conflict of interests I do think that much of his analysis IS correct.
Mog, you make a good point, but wouldn’t the same apply for the BSD’s? Why do they then have and outstanding advisory? I think for a lot of applications what you say is true, but there are also some that are widely known or affect Linux or a software package for Linux, and still they issue patches fast.
To me, this just proves that the open source communities work a whole lot faster at fixing vulnerabilities then the proprietary vendors.
One thing I didn’t see, or maybe I missed it, was something along the lines of Net Vulnerabilities in Core software. I would take what OpenBSD installs by default when you first install it, get a list of the packages it has, look for those packages (and their dependencies) on the other operating systems, and then just see how many vulnerabilities where there for them at the beginning of the month, how many at the end. This I think would indicate who is fixing things faster and being more active.
On a completely different note, what did you use to generate the graphs ?
Well, I am ashamed to say, Microsoft Office for Mac 2004
I would love to use OpenOffice but I don’t like the X interface and NeoOffice just isn’t quite as polished IMHO. That suggestion about the packages sounds pretty interesting but a LOT of work…
I shall have to see, unless you are volunteering some of your time?
Darn it ! I was hoping to get you to do the work !
I do not mind helping out with the effort. Two can probably get it done faster then one. You have the email address that you can contact me on. Let me know.
I thought you might have used some CLI based tool for doing
the graphs, like gnuplot or such.
[…] here is the roundup for March 2007 following the same principles and Operating Systems as in the original article EXCEPT that we added Windows Vista […]
[…] here is the roundup for March 2007 following the same principles and Operating Systems as in the original article EXCEPT that we added Windows Vista […]
[…] here is the roundup for March 2007 following the same principles and Operating Systems as in the original article EXCEPT that we added Windows Vista […]
[…] a security point of view, matter of fact, OpenBSD is the best choice for a box leaved alone, that is, without updates done from a system […]
[…] punto di vista della sicurezza, dati alla mano, OpenBSD risulta essere la scelta migliore nel caso di una installazione lasciata a se stessa, […]