Information security concerns of open source software: case heartbleed Janne Määttä, Samuli Heimonen Abstract 1. Introduction 2. Security in Open Source Software in general 3. Case: Heartbleed 4. A look at issues in other significant OSS projects 5. Conclusions References Abstract In this paper we look at the security concerns in open source software development and attempt to promote healthy discussion over credibility of such software.The goal of this paper is not to come up with any definitive answers, but rather raise awareness and postulate on some of the concerning issues in terms of security. The way we go about it is by looking at some common practices in OSS as well as practical examples of cases where security has failed ­ most notably the ‘heartbleed’ case. The drive behind this paper is to challenge the common impression that open source can, in general, be considered safe just because of the sheer number of eyes on the code and because anyone at any point can evaluate it to point out flaws. 1. Introduction April 2014 a critical bug in OpenSSL left potentially millions of server vulnerable to security breach. The bug had in fact gone on unnoticed for two years since it was committed in new year eve of 2012 by a guy called Robin Seggelmann. This bug has then been named Heartbleed by the engineer working at Codenomicon and the name came from the heartbeat system in OpenSSL where the bug was discovered. Sometimes described as one of the worst vulnerabilities on internet, it leaves us with the question: how and why did it happen. Was the underlying cause in open source nature of the OpenSSL or was this simply a human error. OpenSSL team had their code review process set up yet it failed to spot the issue and it remained undetected until 2014. Many Open Source people claim that Open Source is more secure because they have more eyes looking the code and by that it achieves greater security level than proprietary softwares which in perfect word it probably would. Since this world is far from perfect this argument isn’t so clearly so and it may actually quite the contrary. In this paper we will attempt to postulate on whether or not the fact that open source code can be viewed by anyone is significant contributor on security of the said software, or if in fact the open nature predisposes the code to flaws resulting from carelessness or even malicious intentions. While at it, we also contemplate on some of the common practices in OSS and possible improvements in terms of security. We selected heartbleed for your case study since it’s one of the biggest security vulnerabilities that has been detected for a while. Even though there are not many documented cases of anyone using it for targeted attacks, it's surely one that has gotten most of attention in the media. While heartbleed was a major incident, it most certainly was not the only case of overlooked design flaws. Just in year 2012 ­ 875 million people had their personal information compromised in various ways, and quite likely many of these could have been prevented if people were more concerned of security and would pay more attention to it. There is not much documentation about how much of that number was caused by open source software and how many by proprietary ones, but it’s safe to say both kind of softwares were responsible for that quite astonishingly high number. (Lewis, 2014) Since the heartbleed only affected OpenSSL and not the proprietary SSL implementations it’s only fair to investigate if having source code released openly is more safe or does it open more doors for those who want to use it for malicious purposes. One could easily postulate multiple actors who could benefit of such widely spread exploit out in the wild and use it gain intelligence or monetary benefits. If someone had the will and capabilities of orchestrating something like this, they could easily get away with it. Unlike in corporate world, in the world of open source the contributors are rarely liable for their actions. In fact there are virtually no consequences for committing poor code. Also people who actively seek these kind of vulnerabilities and sell them for highest bidder, which is more than one could get by simply committing fix even to project that has reward system for bugs, on the black market are also not punished. It comes no surprise since some of the clients are in fact law enforcement or intelligence agencies. (Lewis, 2014.) Core idea behind the Open Source Software is that when you release your software openly to everyone, it means anyone can find bugs in it, fix them or contribute new features into software. Idea for the open source began after many manufacturers companies realized that software too was competitive advantage for them and started to guard their software more carefully from prying eyes. This caused lot of oppositions from people who were accustomed to modifying their software to their needs and as most of them were academic researchers and engineers they had the know how to start a projects of their own. They started to re­create various software as free versions of them and even started to create completely free operating system from popular Unix operating system. This has opened up the software markets and allowed people to use free of charge software rather than buying licenses to proprietary softwares. Originally named as Free Software was later changed to open source software which was more describing of its true nature which is not simply free of charge, but opens up its source code for anyone who wishes to see it and contribute back to the original project or create own project based on the it. To support this ideology several organizations have been created like Free Software Foundation and Open Source Initiative. They help to define the values and terminology that is related to open source in this ever changing and fast evolving era of technology.
2. Security in Open Source Software in general When it comes to security of open source softwares, great number of Open Source people argue that more eyes on code means that bugs and vulnerabilities are more likely to be found and fixed prior to anyone being able to use them to do harm. On the other hand not everyone are playing by the same rules, which means that they can discover a vulnerability and use it to their own advantage. Not only can they use existing vulnerabilities, but they can also introduce new ones by committing code to the set project and hide malicious code into their commit deliberately. It may be nearly impossible to distinguish between deliberately created, carefully crafted exploit and just careless mistake. It certainly isn’t making it easier to spot these flaws by looking at code when there’s potentially hundreds of thousands of lines of code spread across hundreds, if not thousands of files. This may well be the case with popular open source projects and certainly is so with OpenSSL(Greer & al., 2014). In large projects like that you are unlikely to find single developer who has closely familiarized themselves with the entire code base. More likely each developer are specialized in very specific parts of the code and hence may not even have their eyes on rest of the code. This already reduces the chance of eye pairs alone spotting mistakes. In complex projects it may in fact be so that it’s not even viable to entirely rely on humans for quality assurance of the code. Or at least in large scale open source projects such degree of Q&A would require immense efforts from people who don’t even get paid to do it which can affect the quality for good or bad of their work since the reward is non existent and there are little to no consequences for slacking off or missing errors. (Carvalho & al, 2014) A change in the way open source projects test their software and how the developers think of security is required. Overtly complex software calls for powerful testing tools to ensure its robustness. For greater coverage one ought not only scan the code itself for potential issues, poor practices, bad formatting and compiler warnings, but also to perform tests on the resulting shippable package. Some flaws may be challenging to find just by looking at code ­ be it a person or software looking at it. Sometimes the resulting binaries are combination of multiple components ­ some of which may not have been covered in tests. In fact it may not even be possible to cover everything if the project has external dependencies. Imported libraries can also contain flaws in them, and by importing them, you introduce the same flaws in the software you ship. You might not have the source code for these libraries, or testing them is out of the scope, so there is little that can be done in that sense just by looking at your code. Some projects use reward based system for rewarding people who found these kind of vulnerabilities, but they are not only ones that are interested in them, because law enforcement and intelligence agencies are also paying to acquire such information in order to gain advantages against criminal and other spying endeavours. Most open source projects have some kind of review process for the commits, especially for new committers, though this can eliminate some of the threats it’s nearly impossible to check every commit in such a detail that would guarantee that no vulnerability is introduced in it ­ be it intentional or just a mistake by the author. There can of course be set of automated test that are run on code, but no matter how sophisticated the tests are, they never completely rule out bugs be them intended or not. Going through all these process can be hard since bigger the project grows, the more commits come in and more people are required to check out commits. This means that at some point those developers that have made solid commits in the past, are trusted to use their own judgement when it comes to checking code for vulnerabilities. This of course opens up possibility for developer to infiltrate the project or possibility of identity thief can still allow malicious code to be inserted into code base if no other security checks or policies are put in place. (Lewis, 2014) Even if you were able to reach wide code coverage in your unit tests, that still doesn’t mean your software is safe against extreme corner cases, or malice in general. Unit tests and most other forms of white­box testing methods are mainly testing whether the code satisfies requirements and not so much how secure it is. Security testing is a whole different field of testing and requires different kind of expertise than you would normally find on your average programmer. Much like the more “traditional” testing, security testing can be automated as well. A commonly used term for automated security testing is ‘fuzzing’. Fuzzing is the process of automating semi­random testing of input channels on software. It can be either black­ or white­box method mainly measuring input and output of target software, and monitoring it for unexpected behaviour.Even though fuzzing does help to create random test samples it’s still requires people to create the testing platform, if one does not already exists, and creating the boundaries for the property that is being tested. As the project grows, changes and evolves so must the software used to test it or otherwise some bugs may not be detected or some features may have changed so much that original software is no longer able to test it. This can also increase the required time and effort to maintain the fuzzer for the project unless there is a third party software that is suited for neds of the project and it can keep up with it. (Carvalho & al, 2014) Raising security awareness amongst developers is as crucial, if not even more so as using powerful tools and enforcing policies. When it comes to open source projects, the developers who are participating and contributing are doing so on their own free time. They rarely get paid for working on the project and hence might easily have the attitude that if they make a mistake, then who cares ­ it’s not like they’re going to get fired. And even more so when something does go horribly wrong, who can be held responsible for it? Sure you can blame the guy who made the mistake, but then again he was probably just trying to be kind by contributing to the open source community and using such software often includes terms that you’re doing so on your own risk. To take legal action might be uncalled for, but perhaps there should be more discussion on what to do with the ones that are responsible for introducing major flaws in widely used free software. They should be held accountable in some form so as not to give the false impression that it’s no big deal if one accidently made a bug that exposed numerous services to exploits. However if open source community starts persecuting people who commits bugs can have negative effect on amount of people who are willing to commit to them out of fear of the consequences if they make a mistake in their commit. There needs to be change in attitude and awareness over such issues. Taking a legal action against people who commit bugs into open source projects is difficult since it’s quite hard to proof one way or the other if it was intended or simply human error, unless same person can be proven to used it exploit services. 3. Case: Heartbleed Heartbleed is a bug in the OpenSSL encryption software, which is used to establish secure communications between computers by acting as Transport Layer Security (TLS) implementation. TLS acts as secure transport layer by encapsulating higher, application level protocols. It's widely used all over internet to securely transfer information on online stores, online banking, email, social media and whole array of other services. (Carvalho & al, 2014) The heartbleed bug enables hostile intruders to get data from servers memory by abusing the faulty keep­alive message system known as heartbeat. In the system client sends keep alive message, which includes message length field. The server takes naive approach and implicitly trusts that client sends correct message length. If client says message is longer than it actually is the server then adds data from its memory to the reply message, which may contain sensitive information and even the servers own private key which enables the attacker then to decrypt all traffic and impersonate the original site. (Carvalho & al, 2014) The bug was originally discovered by Google research team on april 1st 2014 and by Finnish cyber security company Codenomicon a few days later. Both companies discovered the issue independently but Google's team was first to report the issue to the OpenSSL developer team. The bug got named by one of the engineers working at Codenomicon, and they launched an advertising campaign with website and logo to inform people about this severe security issue. Patch for the issue was released about week after discovery and despite the potential to be biggest vulnerability of the time, there were no known major breaches caused by heartbleed. The bug was introduced in 2012 by Germany based programmer called Robin Seggelmann. In an interview Seggelmann has stated that he was only trying to improve OpenSSL by implementing multiple bug fixes. According to him he had missed validating one of the variables containing length by mistake. This mistake also went unnoticed by the one who reviewed his code. An understandable human error for sure, but still the timing of it does leave a seed of doubt. The bug was introduced in new year’s eve while most people were out celebrating. This would make for a fine opportunity to slip in some malicious code as there would be less watchful eyes on the code. Of course Seggelmann would deny such allegations pointing out that the bug was simple programming mistake ­ a human error. To be fair it does appear to be rather typical mistake and one should avoid making claims to the contrary until there is some concrete evidence to suggest that it was indeed done on purpose. (​
O'Neill​
, 2014) Given how crucial OpenSSL is to countless of organizations trusting their transport layer security on it, there is the question how on earth did the bug slip through all the watchful eyes. Given that OpenSSL team only have 13 people running the most used and important internet technology it’s understandable that sooner or later slip up like the heartbleed occurred even though there are academic people who are familiar with security issues like Seggelmann who has Phd and his thesis subject was about secure end­to­end communications (​
O'Neill​
, 2014). There were some speculations circulating that the bug was covertly introduced on purpose to give some party so called zero day exploit which only they would have known. Potentially spreading to millions of servers, this would have given malicious actor significant capacities to acquire otherwise restricted data. With all the government surveillance and cyber warfare going on, it wouldn’t exactly be surprising if it turned out that the bug was indeed targeted attack by some governmental entity. Since the usage of the flaw does not leave any trace of it, it's impossible to say if any agency or other actor may have abused the bug or what information they might have acquired. We may never know this for sure, but it does teach us an important lesson: open source and free software in general may be widely spread, which makes security all the more crucial, and none of the security or quality properties are inherently guaranteed even if there is a large, active community working on the project. This was made painfully obvious by heartbleed case. Because of this it would be wrong to claim that openness alone would lead people to spot flaws more easily. It takes a bit more than just pair of eyes to find the non­trivial bugs in code. Firstly it would require the project to have strict guidelines in quality assurance, testing procedures, coding practices and approval procedures. Secondly it would require good awareness in security concerns, which is no easy task. An average programmer has little to none background in information security, and might in fact lack even basic awareness on the subject. This means the change would have to happen all the way from academic level. An individual open source project could tackle the problem by enforcing policies to used specific kinds of tools and procedures aimed at security testing. 4. A look at issues in other significant OSS projects While not open source, it might be worth pointing out that interestingly enough just few months prior to heartbleed being discovered ­ there was another somewhat similar case with apples TLS implementation. Known as ‘goto fail’ bug, it had existed in apple’s operating system for over a year prior to being found (Boyes & al, 2014). Clearly not even dedicated Q&A in a big corporation can catch all the flaws, but to get a better understanding of whether the fact that there are more eye pairs looking at the code is by itself enough of a guarantee at code quality, we ought to look at other cases too. After all one incident might just be a rare exception, and as such would not suffice as evidence to draw any conclusions. Whereas heartbleed may have got more publicity than any other exploit in recent times, it isn’t quite the only serious case. In fact exploits are being reported and abused constantly. There are several databases available on internet archiving and publishing known vulnerabilities in both open and proprietary software alike. One such database is CVE ­ Common Vulnerabilities and Exposures, which is funded by the National Cyber Security Division of the United States Department of Homeland Security. A quick search on their database for commonly used open source software yields perhaps even surprising amount of results. As an example a search for popular blogging software known as ‘wordpress’ gives over thousand results listing anything from XSS to SQL injection vulnerabilities. A bug known as ‘shellshock’ can also be found there under the identifier ​
CVE­2014­6271​
. This particular bug also gained fair amount of attention when it was first discovered. It is a vulnerability in unix bash shell allowing attacker to run arbitrary shell commands. Having brought it up, let’s take a look at perhaps the most significant open source project of all time ­ the Linux operating system. Linux is in many ways the symbol of open source software, and it has extensive user base as well as contributors. While linux is in general trusted platform by countless of enterprises, government institutions and home users alike, it still isn’t free of vulnerabilities. Entering linux as keyword in CVE database results in thousands of records. A code quality and security auditing tool known as Coverity has been scanning through linux kernel code base since 2006 and over the years has found numerous medium to high impact issues all over the code. In year 2013 the tool was able to find 5092 defects, out of which 3346 were fixed the same year (Coverity inc, 2014). These issues were in the kernel itself, device drivers and networking stack. If even one of the most significant open source projects is subject to code defects, it would be only fair to say that there is nothing inherent in OSS which would guarantee security. 5. Conclusions With bugs like heartbleed and shellshock, open source software is gradually under closer scrutiny than ever, and for a good reason too. Open source is not likely to go away any time soon. In fact current trend tends to be moving more and more towards open source. So what can be said about security in free and open software. After a careful look into Heartbleed bug and other security issues found in open source softwares it safe to say that just by making a project to open source does not inherently mean that security of the project would increase in comparison to proprietary ones. That being said going Open Source can increase level of security if a proper steps are taken and policies and safeguards are placed so the project remains manageable despite the increased effort put into security. If therese processes and policies are made good enough and are also followed then it can indeed increase the security. Quite many, if not most of the biggest projects currently running have done these kind of things to increase their security level, but since their code base is quite vast it’s impossible to rule out potential human errors from the equation. Average open source software project is likely to have some security issues that could be addressed but simply do not have sufficient resources to do so. More eyes in the code means more secure software was the claim we investigated and in our research we found out that simply having more eyes to look up the code base of project does not as such increase the security, but there are other steps that also need to be taken. Fact that source code can be seen by all does not also as such decrease security as most of proprietary software people have us believe. Though it's obvious that allowing previously unknown people to commit code to project can have compromising effect there are means that can be used to mitigate the potential for it. So in the final analysis it can be said that more eyes in the code does not make software more secure, but it can be good starting point towards long journey into more secure project by using various means that we have discussed about in this article. References Boyes, H.A.; Norris, P.; Bryant, I.; Watson, T., "Trustworthy software: lessons from ‘goto fail’ & heartbleed bugs," in ​
System Safety and Cyber Security (2014), 9th IET International Conference on​
, vol., no., pp.1­7, 15­16 Oct. 2014 Carvalho, M.; DeMott, J.; Ford, R.; Wheeler, D.A., "Heartbleed 101," in ​
Security & Privacy, IEEE , vol.12, no.4, pp.63­67, July­Aug. 2014 Coverity inc, “Coverity Scan: 2013 Open Source Report”, 2014 Geer, D.E.; Kamp, P.­H., "Inviting More Heartbleed," in ​
Security & Privacy, IEEE​
, vol.12, no.4, pp.46­50, July­Aug. 2014 Lewis, James A. Heartbleed and the State of Cybersecurity, ​
American Foreign Policy Interests Sep/Oct2014, Vol. 36 Issue 5, p. 294­299​
, 2014 Patrick Howell O'Neill​
(2014), ​
Meet Robin Seggelmann, the man who accidentally created Heartbleed,​
The Daily Dot, http://www.dailydot.com/technology/heartbleed­bug­robin­seggelmann/​
, ​
Referred to 14.11.2015