Linux Kernel Socket Buffer Memory Exhaustion DoS Vulnerability

iDefense Security Advisory 12.22.05
http://www.idefense.com/intelligence/vulnerabilities/display.php?id=362
December 22, 2005

I. BACKGROUND

Linux is a clone of the operating system Unix, written from scratch by
Linus Torvalds with assistance from a loosely-knit team of hackers
across the Net. It aims towards POSIX and Single UNIX Specification
compliance.

More information is available from the vendor website:

  http://www.kernel.org

II. DESCRIPTION

Local exploitation of a memory exhaustion vulnerability in Linux Kernel
versions 2.4 and 2.6 can allow attackers to cause a denial of service
condition.

The vulnerability specifically exists due to a lack of resource checking
during the buffering of data for transfer over a pair of sockets. An
attacker can create a situation that, depending on the amount of
available system resources, can cause the kernel to panic due to memory
resource exhaustion. The attack is conducted by opening up a number of
connected file descriptors or socketpairs and creating the largest
possible kernel buffer for the data transfer between the two sockets. By
causing the process to enter a zombie state or closing the file
descriptor while keeping a reference open, the data is kept in the
kernel until the transfer can complete. If done repeatedly, system
memory resources can be exhausted from the kernel.

III. ANALYSIS

Successful exploitation requires an attacker to have local access to an
affected Linux system and can result in complete system denial of
service. The system may not reboot after successful exploitation,
requiring human interaction to be restored to a working state. Depending
on available resources, systems with large amounts of physical memory
may not be affected.
     
IV. DETECTION

iDefense has confirmed that Linux 2.4.22 and Linux 2.6.12 are
vulnerable.

V. WORKAROUND

An effective workaround is not available for this vulnerability.

VI. VENDOR RESPONSE

The maintainer acknowledges that this issue is a design limitation in
the Linux kernel. The following advice has been offered for creating a
patch. It should be noted that this patch has not been fully tested.

The patch requires three steps:

1) Add a "struct user *" reference to the "struct file" file structure.

2) Whenever creating a new "struct file" add the following code:

        struct user *user = current->user;
        
        if (atomic_read(&user->files) > MAX_FILES_FOR_THIS_USER)
                return -EMFILE;
                
        file->user = user;
        if(user) {
            atomic_inc(&user->count);
            atomic_inc(&user->files);
        }

3) Whenever a "struct file" is released apply the following code:

        struct user *user = file->user;
        
        if (user) {
                atomic_dec(&user->files);
                free_uid(user);
        }

VII. CVE INFORMATION

The Common Vulnerabilities and Exposures (CVE) project has assigned the
name CAN-2005-3660 to this issue. This is a candidate for inclusion in
the CVE list (http://cve.mitre.org), which standardizes names for
security problems.

VIII. DISCLOSURE TIMELINE

11/17/2005 Initial vendor notification - Linux vendors
11/19/2005 Initial vendor responses
12/22/2005 Public disclosure

IX. CREDIT

The discoverer of this vulnerability wishes to remain anonymous.

Get paid for vulnerability research
http://www.idefense.com/methodology/vulnerability/vcp.php

Free tools, research and upcoming events
http://labs.idefense.com

X. LEGAL NOTICES

Copyright © 2005 iDefense, Inc.

Permission is granted for the redistribution of this alert
electronically. It may not be edited in any way without the express
written consent of iDefense. If you wish to reprint the whole or any
part of this alert in any other medium other than electronically, please
email customerservice@idefense.com for permission.

Disclaimer: The information in the advisory is believed to be accurate
at the time of publishing based on currently available information. Use
of the information constitutes acceptance for use in an AS IS condition.
There are no warranties with regard to this information. Neither the
author nor the publisher accepts any liability for any direct, indirect,
or consequential loss or damage arising from use of, or reliance on,
this information.