Jon Oberheide is the CTO of Duo Security, an Ann Arbor-based startup developing kick-ass two-factor authentication. In his free time, Jon dabbles in kernel exploitation, mobile security, and beer brewing. Dan Rosenberg is a security consultant at Virtual Security Research. In his free time, Dan mercilessly crushes both userspace and kernel targets and does his best to make Linux just a little bit more secure.
Your talk at Immunity was titled “Stackjacking your way to GRSEC/PAX Bypass.”
Before we get started, there’s been some dispute over the term “bypass” that I’d like to avoid. Something more along the lines of “Exploitation of grsecurity/PaX Kernels” would be more accurate A bit of clarification on terminology would also be helpful. Typically, a “vulnerability” is considered to be a mistake in the design or implementation of a piece of software that may be leveraged by an attacker to cross some privilege boundary. In contrast, an “exploit” is a piece of code or technique used to take advantage of a specific vulnerability (or multiple vulnerabilities) in order to cross that boundary. By these definitions, the data in the charts we presented is for vulnerabilities, not exploits — each issue was identified as a flaw that was relevant from a security perspective, but certainly not every one of those flaws had an exploit written for it. In other words, vulnerabilities are discovered, and exploits are written.
What is stackjacking?
“Stackjacking” is the name we came up with for our complete exploit technique. Its origin is in the fact that we leverage certain properties of the kernel stack to execute our attack, and that we enjoy poking fun at buzzwords.
You open your talk discussing interesting Linux exploits over the past few years. It looks like there was a big jump in the number of exploits discovered between 2003 and 2004. Why is that?
I could only wager a guess: the 2.6 branch of the Linux kernel was released in 2003, which was coupled with a significant increase in usage and the addition of a lot of new code. One thing worth noting is that while 2003-2005 saw sharp spikes in discovered vulnerabilities, a majority of the issues reported were considered low severity, as rated by CVSS criteria. In contrast, more recent years have had higher numbers of higher severity issues.
You also review the most interesting exploits for Linux in 2010, what are they? Do they share any common traits?
The exploits we discussed were all published by Jon and me. The notable trend was that the majority of the vulnerabilities behind these exploits, including issues in Econet, Reliable Datagram Sockets (RDS), and Controller Area Network (CAN), were in exotic networking protocols that the average user wouldn’t use. Unfortunately, Linux has a facility that allows unprivileged (non-root) users to load these packet families at runtime if they’ve been compiled as modules, which most distributions do by default. This significantly increases the attack surface of the kernel, since now attackers can find vulnerabilities in rarely used, less rigorously tested networking code and exploit them on a typical Linux distribution. One of the other takeaways here was that grsecurity includes a configuration option called MODHARDEN that disables this auto-loading support, preventing unprivileged users from loading these modules. Ubuntu and Debian have caught on to this as well, and more recent versions have started to disable support for these exotic protocols by default.
What does it tell you when almost half the exploits for Linux were found by just six people?
It tells me that there is very little community effort focused on auditing the Linux kernel for security vulnerabilities, which is a shame.
What is grsecurity/PaX? What does it do? How does it go about doing that?
grsecurity and PaX are a set of third-party patches, maintained by Brad Spengler and the PaX team, that can be applied to the Linux kernel to provide a complete set of security enhancements. This includes hardening both userland applications as well as the kernel itself. Although the list of features that affect user applications is extensive, for the purpose of our talk we focused on the features that were specifically designed to make kernel exploitation more difficult. The full feature list and the patches themselves are available at http://grsecurity.net. The features we specifically discussed were:
GRKERNSEC_HIDESYM, which reduces known sources of information leakage that attackers can use to target exploits. This feature aims to turn the kernel into a black box, where the attacker has no knowledge of where any internal data or code resides in memory. GRKERNSEC_MODHARDEN, which significantly reduces the attack surface of the kernel by preventing the automatic loading of kernel modules by unprivileged users. PAX_UDEREF, which leverages x86 segmentation to create a strict separation of user and kernel addresses. Many vulnerabilities and exploits take advantage of the fact that Linux’s memory model allows the kernel to seamlessly access data, and even execute code, residing in the userland portion of a process’ address space. As a result, many exploits, having leveraged some vulnerability to gain control of the kernel’s execution flow, simply map a suitable payload into the memory of a userland process and divert execution to that payload. The kernel will then execute this code without complaint. UDEREF prevents exploit techniques such as these and all other cases where the kernel may incorrectly access data or code residing in userspace. PAX_KERNEXEC, which implements stronger page protections in the kernel. Many popular targets for vulnerabilities that give an attacker the ability to write into kernel memory, such as global data structures (IDT, GDT, system call table, etc.) and function pointer tables, are explicitly marked as read-only.
If this is a tool to harden a Linux installation, how are people getting around it? How did the two of you come to find this exploit at the same time? Were you working together?
There has been almost no public work on exploitation of grsecurity/PaX kernels. Brad Spengler has repeatedly issued public invitations for researchers to do work on this topic, and this was part of the reason Jon and I started looking at this. We had both been doing work on Linux kernel exploitation, and after a few conversations about collaborating on this talk it became clear that we had independently come up with several of the same ideas.
From your talk it seems you both found different ways to achieve the same steps. Is this accurate? You both seem to have your own way of groping the stack. Is this two ways of going about the same exploit or could these actually be considered separate exploits?
Before we get too far in, some clarification needs to be made. Jon and I presented two techniques that could be used in conjunction with certain types of vulnerabilities to successfully exploit a grsecurity/PaX kernel. We did not disclose real vulnerabilities as part of our talk. As a result, our talk was essentially a theoretical study in exploitation, with our implementation code using a synthetic vulnerability for demonstration purposes. Yes, we developed two separate ways of achieving one step of our exploit technique. The end result of both of these techniques is the same, but the way to get there is somewhat different.
If a KStack Leak is so common or considered useless, what has changed to make it so dangerous?
In the vanilla (non-hardened) Linux kernel, kernel stack leaks continue to be mostly harmless. In that environment, there are so many other easily accessible sources of information leakage that jumping through all the hoops we did is completely unnecessary. However, one of the main contributions of our talk was that in hardened kernel environments, where it’s assumed that the attacker knows nothing about the memory layout of the kernel, one of these leaks may provide the pinpoint of visibility an attacker needs to target an attack.
What are these three steps to get at the root in a hardened Linux install?
Our exploit technique requires several distinct stages. We began our discussion by stating our assumptions. Namely, we assume that we are dealing with a hardened kernel with all the appropriate kernel protections mentioned above. Additionally, we assume the attacker has an arbitrary kernel write vulnerability and a kernel stack information leak. Kernel write vulnerabilities are admittedly somewhat rare, but they are probably more common than most classes of kernel vulnerabilities, which tend to be somewhat esoteric. Given such a vulnerability, it’s almost trivial to gain root privileges on a vanilla kernel, but it’s much more difficult on a grsec kernel.
- The first stage of our attack involves leveraging the kernel stack leak vulnerability by using a library I wrote, called libkstack, to infer the location of the base address of the current process’ kernel stack.
- Next, Jon and I presented two techniques for combining this knowledge with an arbitrary write vulnerability to gain the ability to read arbitrary kernel memory. Jon’s technique involves writing into a process’ kernel stack while it’s in a “sleepy” system call, such as sleep(), wait(), etc. Jon uses a separate process to write to a pointer stored on the sleeping process’ kernel stack that is then used as the source address for a copy_to_user() call. By overwriting this source address, Jon was able to cause the kernel to copy back the contents of kernel memory of his choice back to the user. My technique leveraged a piece of metadata residing in a structure (the thread_info struct) located at the base of every process’ kernel stack (whose location we now know). Ordinarily, when the kernel copies data back and forth between userspace and kernel space, it checks that the source or destination pointer (as appropriate) for the copy resides in the range for userspace addresses. On 32-bit Linux, user addresses are less than 3gb (0xc0000000). If this checking weren’t in place, the kernel might accidentally take a kernel address provided by a user via a system call and copy data to it, allowing a trivial compromise. In particular, the “addr_limit” member of the thread_info struct is used for this checking, and as such it normally contains the value USER_DS, which corresponds to this 3gb split. On a vanilla kernel, if an attacker could somehow change this value in kernel memory to ULONG_MAX (0xffffffff), it’s game over, because then all checks on copying data back and forth between userspace and the kernel would succeed, and the attacker could read from and write to arbitrary kernel memory. However, with PAX_UDEREF enforcing segmentation, I needed to go through a few extra steps to ensure the segment registers contained the appropriate selectors to allow kernel-to-kernel copying after writing to this variable with our kernel write vulnerability. Interested readers can look at our slides and published code for the details. The end result of either of these steps is that we can now read from arbitrary kernel memory.
- The final step uses our newly-gained arbitrary read to follow a few pointers in memory. We already know where the thread_info struct is, sitting at the base of the process’ kernel stack. We then follow a pointer to find the task_struct, and another pointer to locate the cred struct, which contains credential information such as the uid, gid, etc. By writing into this structure with our kernel write vulnerability, we escalate privileges to root.
What does this say when something that is supposed to be a system hardening solution turns out to have an exploit like this one?
To be clear, we did not identify any vulnerabilities introduced by the use of grsecurity/PaX. We presented techniques that showed that given the presence of certain serious vulnerabilities, it’s possible to exploit even the most hardened kernels. It’s important to note that this exploitation process is many orders of magnitude more difficult when grsecurity/PaX is used.
You also suggest that there are defenses to protect against this exploit. How effective are they once installed?
Brad Spengler and the PaX team promptly responded to our presentation by implementing new features that protect against the exploit vectors we presented. In particular:
- Kernel stack entry point randomization was implemented for x86-64, making exploitation using Jon’s technique essentially impossible. Because the location of pointers on the stack now varies with each system call, it’s unlikely that an attacker would be able to find a consistent target.
- The thread_info structure was moved off the kernel stack entirely. This mitigates my technique, because I now have no idea where the addr_limit variable resides. They also implemented several other more general enhancements to make kernel exploitation more difficult.
Where do you see the greatest potential for further exploits to be found?
In general, the kernel is still a soft target. Exploit mitigation in the vanilla kernel is still in its infancy — I think it’s entirely too easy to get root on a vanilla Linux kernel given certain vulnerabilities, of which there are many. It’s hard to say where vulnerabilities will pop up next, but in general, code that’s been less thoroughly tested, newer code, and especially complex code are the obvious places to look. For more information on this subject, our code is available on Jon’s git repository at: https://github.com/jonoberheide/stackjacking The slides are available at: http://jon.oberheide.org/files/stackjacking-infiltrate11.pdf As for contact info, Jon’s website is: http://jon.oberheide.org/ Mine is: http://vulnfactory.org Our Twitter handles are @jonoberheide and @djrbliss.
