Security Blog

Unfortunately the old analogy does not work as well as it used to, as the Forth Bridge is not being painted constantly (and in fact was never the case!). However there is a permanent maintenance crew working on the bridge, so it does work at one level.

Servers require maintenance; all sorts of maintenance, although here we are concerned only with security patches. Applying security patches is not a one off process, or something to be done only when someone reminds you of the necessity.

If you patch a server every three months, then the maximum amount of time the server may be vulnerable to a new vulnerability is three months. A vulnerability may well come out minutes or seconds after you have finished applying patches.

This is of course a rapidly changing situation, so the advice may well change.

The main Heartbleed blog entry has a bit of information about what to do about passwords, but to make it more plain …

University Accounts

In the case of your University account password, we are not currently recommending that you change your password. However we will be advising that those people who have used our vulnerable servers should change their passwords, but it currently appears to be a tiny minority of the entire population.

Google

Unfortunately the information regarding Google is somewhat contradictory. Some of Google’s services have had this vulnerability, but would have been fixed very early – one of the researchers investigating this issue was part of Google’s Security Team. Some sites are indicating that Google users should change their account passwords.

However the unofficial advice from a Google engineer is that there is no need to change passwords at this time. They go on to suggest that regularly changing passwords is always recommended – as we do.

Other Web Sites

In the case of web site passwords (Yahoo, etc.), we recommend checking with the web site operators for advice. Most of the top 1000 web sites were not found to be vulnerable (for example Facebook was not), so changing passwords for these sites is unnecessary.

If a web site announces that it is advisable to change passwords, then we advise that you do so. Unfortunately the announcements may not be immediately obvious, and some sites collating information may contain misleading information.

Given the serious nature of this vulnerability, it is worth taking the unusual step of explaining what it is and what makes it so serious. The vulnerability can be found within certain versions of a software component called OpenSSL. This is very widely used within the industry by server vendors (typically Linux vendors), by software vendors (such as Apache), and by vendors of appliances.

(with thanks to xkcd.com)

If in doubt it is safest to assume that OpenSSL is being used because it is likely to be used in unexpected places.

How Does This Affect Me?

If you are not a server administrator, you may be wondering whether you are at risk. The short answer is that nobody knows how much of a problem this will be for you. For example, out of the top 1000 web sites around the world, 47 were known to be vulnerable as of the 8th April.

Which means that any account passwords for those 47 web sites may be compromised, or all encrypted communication may have been disclosed. The answer here is to read any announcements on the web sites about this vulnerability and take any actions they suggest – which may include changing your passwords.

It is also necessary to keep any client software updated; most news concentrates on the server side but clients are also vulnerable. However whilst they are vulnerable, nobody has demonstrated any exploits for the client side, so you aren’t actually actively being attacked.

But it may well not remain that way. So aim to apply updates to your machines – laptops, phones, tablets, etc.

Of course if you are a server administrator, there’s quite a bit of work to be done …

But What Is It?

When connecting to an SSL-enabled service such as a secure web site, an SMTP mail service, an encrypted mail service, etc., a negotiation takes place to determine what encryption should be used. This negotiation includes lots of functionality that is optional including a “heartbeat” function.

The code implementing this functionality is vulnerable, and allows an attacker to read information from the server they are attacking. Including the encryption private key.

What Are The Consequences?

If you are running a vulnerable service, and an attacker steals your SSL private key then the attacker can :-

1. Decrypt any communication with your service that they manage to intercept.
2. Implement a man-in-the-middle attack with perfect credibility, as they can impersonate your server with as much authenticity as your own server.

This lasts whilst you continue to use the compromised private key and is not fixed by simply fixing the vulnerability; the compromised certificate needs to be revoked and a new one used.

It is not possible to determine if a private key has been compromised or not.

How Is This Fixed?

There are several stages to dealing with this vulnerability :-

1. Identifying the vulnerable servers and services. This is not a simple issue as the vulnerability will show up in unexpected areas. It’s not even as simple as identifying servers running the vulnerable software release; whilst enabled by default, the “heartbeat” functionality can be turned off.
2. Update all of the vulnerable servers and services.
3. Generate and distribute new certificates for all of the identified services.
4. Revoke potentially compromised certificates.

It is important that new certificates are not distributed to services that still need to be updated.

Has It Been Exploited?

The short answer to that is that nobody knows. The vulnerability has been present within the OpenSSL code for more than a year, so it is possible that unknown hackers have known about this.

Indeed it is the sort of vulnerability that people like the NSA and GCHQ would be keen to make use of.

When the exploit is used, there is no way of detecting from server logs what has happened so any historical exploitation would have gone undetected. There are signs that people are actively hunting for vulnerable services – either for checking purposes (how many sites are vulnerable?) or to try exploitation.

Further Information

This blog entry will be updated as further information becomes available. In the mean time :-

1. The CERT announcement of vulnerability #720951. CVE vulnerability announcement.
2. The Heartbleed website.
3. Very technical SAN webcast (requires log in).
4. Simplified technical explanation of Heartbleed. But is perhaps simplified to the point of inaccuracy.
5. Detailed, but probably the best technical explanation from a mainstream media organisation (The Register).
6. JANET Certificates and advice.
7. Heartbleed scanner (which may give false errors). And another one (which looks to be more reliable). And a third which gives a lot more information. However it may be illegal to scan servers you don’t own/operate.
8. Tenable blog entry.
9. It may be a lot harder to get the private key than initially suspected.

This is most certainly worth a listen :

http://www.bbc.co.uk/programmes/b03xzyy5

There are many ways to dupe people out of their login credentials…

In the examples below,  what looks like an authentic email contains a link to an ‘important’ GoogleDoc:

http://www.symantec.com/connect/blogs/google-docs-users-targeted-sophisticated-phishing-scam

http://nakedsecurity.sophos.com/2012/05/30/phishing-with-help-from-google-docs/

Remember – don’t click on links in email! (if you don’t feel safe about clicking on the above, then use them in a Google search)

If you’ve ever wondered why we insist that users must log on to the University wireless network, then this article from the BBC Click news site should help to explain things…….

Free wi-fi hotspots pose data risk, Europol warns

By Dan Simmons  Click presenter

http://www.bbc.co.uk/news/technology-26469598

Or…. if you don’t like to click on links on webpages – Google search for BBC News Technology + Free wi-fi data risk

As you may be aware, the University has been subjected to an NTP DDoS attack which caused multiple short intervals when the Internet was unavailable. Given the intensity of the attack, JANET (our “ISP”) have implemented a firewall rule that only permits NTP access from their servers.

In addition, we have ourselves implemented a firewall rule that only allows NTP access to our official NTP servers. This has implications to anyone who has done their own thing in the past – not that there was anything wrong with that, but in the light of the DDoS attack it has been necessary to tighten things up.

If you do run your own NTP server – which you will know because you will have configured it in the past – then you should reconfigure the server with the details below :-

tinker panic 0
# Added according to Vmware recommendation
server ntp0.port.ac.uk
server ntp1.port.ac.uk
server ntp2.port.ac.uk
# Three servers to use (currently)
driftfile /var/lib/ntp/drift/ntp.drift 
# Keep track of how useful the local clock is
logfile   /var/log/ntp
# A log file. Do we need this?
keys /etc/ntp.keys
trustedkey 1
requestkey 1
# Details of security keys - which we don’t use
disable monitor
# Disable the dangerous stuff

The key elements to this configuration are :-

1. The server list of ntp0.port.ac.uk, ntp1.port.ac.uk, and ntp2.port.ac.uk.
2. The “disable monitor” which prevents the NTP server from taking part in a denial of service attack.

In addition, it is highly desirable to restart the NTP service once a day – /etc/init.d/ntp restart – which can be set up as a cronjob. This has practically no effect on the availability of the NTP service (it happens very quickly) and means that changes to the list of NTP servers take effect quickly.

And we’re planning on changing one of the NTP servers imminently.

One of the questions that arises from time to time is just how often do we get scanned by some kind of attacker looking for vulnerabilities?

This is a bit of a tough question to answer as it involves looking into why scans are performed, and our firewall doesn’t yet have the ability to ask an “attacker” why they are doing what they are doing. There are many different reasons why our network may be scanned :-

1. Scans looking for vulnerabilities.
2. Scans from monitoring sites looking for specialised vulnerabilities such as NTP, DNS, chargen, etc. These scans are looking at the size of the problem rather than trying to use the vulnerability.
3. Scans from network monitoring sites and/or researchers looking at how big the Internet is.
4. Random “noise” from typos, etc.

Of course there is no easy way to determine the proportions, but let us assume that a 1/4 of all scans are malicious.

But back to the original question: How often do we get scanned?

By taking an unused network – one that has never been used or allocated – we can use the firewall logs to look at “attackers” trying to connect to hosts on these networks. For one day (Sunday 26th January, 2014), this amounted to some 41,154 connection attempts.

But a single “attacker” may connect to multiple addresses within this network, so how many unique “attackers” probed something on the unused network? For the same date, it was 3,218.

Using our guess of 1/4 of those were malicious scans looking for vulnerabilities, we had 805 unique attackers on the 26th January. But is this some kind of exceptional day?

To determine that we need to look at the number of scans over the whole of January :-

So, no it wasn’t exceptional – we really do see thousands of scans per day.

This post is a bit more technical than the usual, and covers a kind of attack that is not only the kind of attack we may fall victim to, but an attack that we could participate in! Defences against this sort of attack are difficult, but avoiding the risk of participating in the attack is far easier.

You could argue that there is no real benefit to the organisation in avoiding participating in attacks as we are not the real victims in such circumstances; however it is (and has always been) considered to be ethical behaviour to avoid causing problems for others on the Internet. And of course the University wishes to be an ethical organisation!

When an attacker wishes to perform a denial of service attack against a victim, there are several possibilities, but one of the common varieties is to simply overwhelm the victim with as much traffic as possible. This is achievable with a simple denial of service attack if the attacker has access to more resources than the victim.

However for an attacker to overwhelm a victim with more resources, they will have to use a distributed denial of service attack where they use the resources of many computers and networks to flood the victim. This is commonly performed using a collection of compromised machines (a botnet), but there is another option …

If the attacker can :-

1. Forge network traffic so that it appears to be from the intended victim.
2. Make use of a service that will respond to such traffic with an answer to a question that is much larger in size than the question.

Then the attacker has themselves a viable attack. A simple illustration of this most simple form of this attack appears below :-

As you can see, I have labelled the attacker’s ISP as “Bad ISP”. This is because unless the ISP allows for forged network traffic, then the attack cannot get past the first hurdle. It has long been best practice to block forged traffic (see BCP38), and yes we do.

Unfortunately there are vast numbers of ISPs out there who do not block forged traffic.

Once an attacker has access to a poorly configured network that allows them to forge the source address in network packets, then they have to find the next step – a service that can be used to amplify the attack. The essentials of such a service are :-

1. That it responds to a small question with a large answer. This gives the amplification factor that multiplies the amount of traffic that the attacker can send.
2. That it is based around UDP rather than TCP. Although TCP can be used in such attacks, it is very much more complicated and hasn’t been used as yet. UDP is simpler as there is no negotiation involved.
3. It is also helpful if the service is “light-weight” in that it can easily handle very large numbers of questions and answers.

As it happens two very widely used services – DNS and NTP – are ideal for this kind of attack. If they are configured in a naive manner, the attacker can use them to amplify attacks considerably.

To simplify things I have left out some additional details. The attacker is likely to use a large collection of compromised machines to send the questions to the naively configured services. This increases the amount of denial of service traffic enormously … if an attacker has 1,000 compromised machines and the amplification factor is 10, then 1,000 streams of 1Mbps turns into an attack of 10Gbps.

Most initiatives to combat such distributed denial of service attacks concentrate on fixing the naive configuration of applications, although fixing the “bad” ISPs would solve a whole class of different attacks. As an example, the DNS amplification attack has been known about for ages, but the NTP amplification attack has only recently been used.