The Second Life Security Breach revisited

On September 8th Linden Lab reported on their blog that an intruder was able to access the Second Life database through the web servers via a 0-day exploit on September 6th. After two days of investigation Linden Lab came to the conclusion that the likelihood of compromised data is high enough to reset all passwords and tell the public about the incident. This happened on September 8th. According the Linden Lab “some of the unencrypted customer information stored in the database was compromised, potentially including Second Life account names, real life names and contact information, along with encrypted account passwords and encrypted payment information”.

So I will try to summarize a bit what has been happening and what it might mean.

The Timeline

According to an email sent out by Linden Lab on September 9 and blog posts from the Second Life blog, here’s a little rundown on what has happened:

September 3 – intrusion attempts might have started

September 5 – the first successful attempt might have happened

September 6 – unusual activity in the database led to an investigation by Linden Lab and led to closing the intrusion. A forensic investigation began to check whether data has been compromised or not (maybe this was related to this and this blog post?)

September 8 – Linden Lab decided that there was “a substantial likelihood that customer account information could have been accessed” and thus invalidated all password for this reason as well as announced this fact to the public.

Investigations are ongoing and law enforcement has been informed and a letter was sent to all residents in compliance with SB 1386 (see this forum post by Ginsu Linden for more details on what this means).

What is an exploit anyway?

An exploit is some bug in a software which allows unauthorized access to either the whole system or parts of it like data. The latter (as told by LL) has happened in this case. We don’t have much information about what has happened in detail but I assume that one of their used software packages had an exploitable bug which might have allowed to pass SQL statement directly to the underlying MySQL database (SQL is a standardized query language for retrieving and modifying data in databases such as MySQL). For instance in former exploits of such tools it might has been possible to inject SQL statements into web form data in some tricky ways and getting those executed on the database.
This of course means that only data can be accessed this way which is actually in the database to which the webserver is connected to.
So this seems likely to have happened here (not necessarily through a buggy form but there are eventually different ways to accomplish this depending on the bug found.
But all this is merely speculation by myself and thus do not take it too serious.

What information has been revealed?

According to the initial blog post it potentially has been

• Second Life account names
• real life names and contact information
• encrypted passwords
• encrypted payment information

The password issue should have been solved by resetting the passwords. So left is basically real life and encrypted payment information which probably means credit card numbers. Both of these things are problematic to a wide range of the Second Life population. The first because many people tend to hold their second and first lives separate and they are thus very uneasy that this information might get revealed (see also the New World Notes post here). Regarding the credit card information it is obvious that you don’t want to share your credit card number with anybody except of course all those companies you buy stuff from ;-).

People do also worry whether chat logs have been revealed as this might be as problematic as the former points. A post to the Second Life Answers forum still has to be answered by Linden Lab. It also proves the point that it’s in general a bad idea to store chat logs as it’s sort of compromising my privacy (especially when it might get hacked. But even in the case when only LL employees have access to it it’s problematic IMHO. More so when no information is revealed who and in which circumstances has access and how long they are stored).

While there’s not much you can do about the potential release of your real life data you might think about whether to get a new credit card number or not. Let’s look into that a bit.

Is my credit card number still secure?

According to Linden Lab the credit card number was stored encrypted in the database. According to the Security FAQ sent via email they use MD5 with a salt as encryption for both the passwords and the payment information. As not everybody is familiar with encryption methods (honestly I am also an expert at least when it comes to probabilities of breaking it) here’s a little explanation of what MD5 is about.

MD5 is basically an algorithm which takes a file or string and encodes it into a hash (checksum). So when I encode “Second Life” with this algorithm then I get the MD5 hash “c9a3f646ec6a1f70c23edfe072f0b3b6”. This hash is 128 bits long (or 32 characters here with each having a range of 0…9abcef = 16 digits = 2^4 = 4 bits). In fact it is one of the characteristics of MD5 that it is always 128 bit long regardless of how big the input file is.

Another feature of MD5 is that it’s not reversible. So you cannot take this string and use some reverse algorithm and get “Second Life” out of it. It’s only one-way.
MD5 hashs are used for several things. For instance they are used for password. In the database the password is not usually stored in clear text but as an MD5 hash. So when you enter your password then the MD5 hash gets computed and compared to the MD5 hash stored in the database. If they are the same then you get access.
They are also used for big files in order to verify their integrity. If somebody changes that file and you compute the MD5 hash afterwards then it is not the same anymore as it was before and when you compare this hash to the one you might have stored in a more secure location (like on CD-R) then you can decide whether it’s the original file or not.

But there are problems with it, too, of course. As the resulting hash is always 128 bit you can imagine that two files might eventually create the same MD5 hash. This is called a collision. This can also happen for short strings such as passwords. This means that you can eventually find a string which creates the same hash and an intruder can get logged in with that string although it’s not the same one as original password.
It now depends on how difficult it is to find such strings. One method is definitely using brute force. Brute force means that you just try all sorts of possible passwords and try to login. Usually login will be disabled in many system when you try wrong passwords for too long but when you have a database with all the pre-computed MD5 hashs then you can of course check against that first and only try to login with the passwords you might have discovered that way.

As this website suggests breaking clear md5 hashes is not that difficult but what Linden Lab uses are actually salted md5 hashes. Salted hashes are created not simply from the string itself but from two strings. What the second string is is up to the programmer, it could be the registration date or something computed from various fixed parameters. Without knowing what this string actually is it will get very hard to crack this password (or credit card number in the other case). Linden Lab also just said they were using salts but still we don’t know how they look (and this is probably good as the attacker might have use for this information. Still it security-through-obscurity which is no real security nevertheless). We just should hope it’s not a too easy salt and hopefully no static one.

Regarding the passwords there should be no threat anymore anyway as they all have been reset. What stays is the disclosure of the encrypted credit card information. According to Linden Lab the credit card information is encrypted in the same way, with a salted MD5 hash. Without the salt I’d think of breaking this number as relatively easy as you know at least that it’s only numbers.
With salts things get more complicated and time consuming depending on the salt used.

I am a full-blown MD5 expert but I’d say that for now the credit card numbers are probably safe. There are at least more probable ways of getting it stolen like at any shop or restaurant. And additionally credit card companies usually have a fraud detection system in place although I haven’t made any experiences with those yet.
In the end it’s also the question of how much criminal energy that attacker has. It’s still a difference (I hope) for somebody to eventually stumble or search for an exploit and use it than to break credit card numbers and make big purchases.

So for me it probably means that I won’t get a new card but when anybody out there is a better expert or I might have misunderstood something (LL, please tell!!) then tell so in the comments.

How the situation was handled by Linden Lab

Well, one might argue and some people of course do why Linden Lab waited 2 days before informing the public. They explain that in their security FAQ and it’s also visible from the timeline above. Before September 8 they haven’t really been sure whether actual data was compromised. Another reason might also be that you want to make sure that all the bugs are fixed and no further security breach can happen as an announcement like this might attract even more attackers. So for me it sounds ok and invalidating all the passwords was definitely the way to go to be on the safe side.

A different thing is support. It was apparent that many users will have problems when their password got reset as they might have forgotten their security question answer or their email address is not valid anymore (or never was). In this case they need support hotlines but those were closed on the following weekend. Linden Lab also seemed a bit uneasy with opening them as apparently they will get overrun with not so happy customers (here’s a recent blog post by them on how it was handled). Nevertheless it is crucial for you in order to portrait yourself as professional that you have support lines open all day and night and weekends especially in such cases. With every user joining it will get even more important.

Of course they added more mechanisms to reset your password and thus kudos to the devs in place. Missing phone support nevertheless stays a problem.

What could be done better (or what might have been different already)

In the case of Second Life it is apparent that real life information and credit card information are very precious things (the first one seems even more crucial to residents it seems to me). Thus those need to be protected very well. For this reason I don’t understand why this information is actually stored in a database which is directly connected to the web server. There is the possibility of storing all such data on separate machines in separate databases which are not directly connect to either the web or the grid. And when you need the data you still can issue some web service which fetches that data for the script. And further you can also enhance the security of this link by using SSL client and server certificates. It is of course still hackable but it affords much more knowledge about the system (as it’s custom made) and it has more barriers in front of the actual data. You might also need to get direct access to the host machine for such a hack to work which is still harder than just searching for zero-day exploits of popular software packages.
Of course such things seem to be in place as the unencrypted payment information was not revealed. But then I wonder why real life data and especially the payment info hash was in the web server accessible database.

Other things which could be improved would be the use of another hash algorithm such as SHA-1 or at least applying the MD5 algorithm more than once.

Anyway, I hope Linden Lab is looking into getting maybe some external security experts to visit them (not because they might not be aware of that themselves but external eyes sometimes come with a different perspective and ideas).

What you can do

There’s several things you can do such as:

• never use the same password at different web sites.If you did so with your Second Life password then you should quickly change it.
• don’t use simple password. make it as long as possible and as random as possible. Mix all sorts of letters, numbers and special characters.
• regarding credit card numbers you need to judge for yourself. The probability to get them cracked might not be that high but with enough time and energy they could possibly be cracked.
• you might also want to use a one-time credit card number if it’s available from your credit card company. Such numbers might add a usage fee though (although it’s not really understandable to me why they don’t offer security for free).
• you might additionally want to check http://www.privacy.ca.gov/sheets/cis1english.htm and similar sites.

As this is a complicated topic especially when you don’t have that much information please comment when you think I have described something incorrectly.

PS: On a somewhat related note, Tateru Nino reports via NWN a big increase of new signups since the security breach got public. So it’s a bad thing on the one side but it also seems to be great publicity on the other hand. Either everybody now wants to get their data stolen or they assume it’s safe now 😉 (ok they don’t need to give any credit card information for a basic account if they don’t want to so they should be safe then anyway).

secondlife virtualworld security creditcard lindenlab taotakashi hacker exploit