Microsoft DirectX Direct3D 9
 Microsoft DirectX RLE Compressed Targa Image File Heap Overflow

 Ruben Santamarta <ruben(at)reversemode(dot)com>
 07.18.2007

Affected products:

+ Microsoft DirectX Direct3D 9   runtime libraries.
  + D3dx9_28.dll – D3dx9d_28.dll and earlier

Microsoft DirectX is prone to a heap overflow vulnerability due to the
improper handling of targa files.

TGA FORMAT [Image Specification]
Image Type - Field 3 (1 byte). File offset : 00000003
The TGA File Format can be used to store Pseudo-Color, True-Color and
Direct-Color images of
various pixel depths. Truevision has currently defined seven image types:
Table 1 - Image Types
ImageType
  0 No Image Data Included
  1 Uncompressed, Color-mapped Image
  2 Uncompressed, True-color Image
  3 Uncompressed, Black-and-white Image
  9 Run-length encoded, Color-mapped Image
  10 Run-length encoded, True-color Image <<==RLE
  11 Run-length encoded, Black-and-white Image
Field 5 (10 bytes): File Offset: 00000008
  Field 5.1 (2 bytes) - X-origin of Image:
  Field 5.2 (2 bytes) - Y-origin of Image:
  Field 5.3 (2 bytes) - Image Width:
  Field 5.4 (2 bytes) - Image Height:


Allocating the vulnerable buffer...

Module: d3dx9d_28.dll

.text:00561EDD mov ecx, [ebp+var_C]     ; Pointer to the mapped TGA file.
.text:00561EE0 movzx edx, word ptr [ecx+0Ch]   ; image Width
.text:00561EE4 mov eax, [ebp+var_C]
.text:00561EE7 movzx ecx, word ptr [eax+0Eh]   ; image Height
.text:00561EEB imul edx, ecx         ; width * height
.text:00561EEE imul edx, [ebp+var_20]     ; (width * height) * bpp
[...]
.text:00561FBE mov eax, [ebp+var_18]
.text:00561FC1 push eax      Size ( width * height ) * bpp
.text:00561FC2 call sub_450010

{...}

.text:00450010   mov edi, edi
.text:00450012 push ebp
.text:00450013 mov ebp, esp
.text:00450015 sub esp, 0Ch
.text:00450018 mov eax, [ebp+arg_0]
.text:0045001B add eax, 20h
.text:0045001E mov [ebp+var_8], eax
.text:00450021 mov ecx, [ebp+var_8]
.text:00450024 cmp ecx, [ebp+arg_0]
.text:00450027 jnb short loc_450030
.text:00450029 xor eax, eax
.text:0045002B jmp loc_4500EB
.text:00450030 ;
---------------------------------------------------------------------------
.text:00450030
.text:00450030 loc_450030: ; CODE XREF: sub_450010+17#j
.text:00450030 mov edx, [ebp+var_8]
.text:00450033 push edx     ; Size( ( ( width*height ) * bpp )+20h )
.text:00450034 call ds:__imp_malloc

As we can see, DirectX relies on width and height fields within the TGA
texture file.

Overflowing the buffer...

Let´s see schematically  the execution  flow of
D3DXCreate***TextureFromFileEx -
D3DXLoad***SurfaceFromFileEx APIs

For example:

1. D3DXCreateCubeTextureFromFileEx:

1.1  CreateFile
1.2  CreateFileMapping
1.3  GetFileSize
.text:00795626 loc_795626: ; CODE XREF: sub_7954A0+182#j
   .text:00795626 push 0 ; lpFileSizeHigh
   .text:00795628 mov eax, [ebp+var_BC]
   .text:0079562E mov ecx, [eax]
   .text:00795630 push ecx ; hFile
   .text:00795631 call ds:GetFileSize
   .text:00795637 mov edx, [ebp+var_BC]
   .text:0079563D mov [edx+0Ch], eax   ; File->Size = GetFileSize
   1.4       MapViewOfFile
   .text:0079566C call ds:MapViewOfFile
   .text:00795672 mov ecx, [ebp+var_BC]
   .text:00795678 mov [ecx+8], eax   ; File->ptr = MapViewOfFile
  1.5       CreateCubeTextureFromFileInMemoryEx
   .text:0053C69A mov edx, [ebp+var_8]
   .text:0053C69D push edx
   .text:0053C69E mov eax, [ebp+arg_0]
   .text:0053C6A1 push eax
   .text:0053C6A2 call D3DXCreateCubeTextureFromFileInMemoryEx

Now the interesting part...

2. D3DXCreateCubeTextureFromFileInMemoryEx

  2.1 Parameter validation.
    Boring code.
  2.2 Texture type identification.
    Wake me up when it finishes...
  2.3 Copy the mapped file  into the allocated buffer.CrThe  flaw is coming

 .text:00562112 mov ecx, [ebp+arg_4]      ; File->Size ( GetFileSize )
    .text:00562115 cmp ecx, [ebp+var_20]  
    .text:00562118 jnb short loc_562124     ; oops!
    .text:0056211A mov eax, 80004005h
    .text:0056211F jmp loc_56227D
    .text:00562124 ;
-------------------------------------------------------------------------

    .text:00562124 loc_562124: ; CODE XREF: sub_561930+7E8#j
    .text:00562124 mov ecx, [ebp+var_20]
    .text:00562127 mov esi, [ebp+var_14] ; File->ptr (MapViewOfFile)
    .text:0056212A mov edi, [ebp+var_5C] ; buffer=( malloc( FakeSize ) )
    .text:0056212D mov edx, ecx
    .text:0056212F shr ecx, 2
    .text:00562132 rep movsd
    .text:00562134 mov ecx, edx
    .text:00562136 and ecx, 3
    .text:00562139 rep movsb          ;  overflow

Summing up:
Buffersize is based on the tga header, which can be specially crafted by
the attacker (width,height...).
Filesize (mapped file) is calculated by using the API GetFileSize.

If buffersize is less than Filesize we will be corrupting the heap with
controlled values. This vulnerability  can lead to remote code execution
within the context of the  application running vulnerable directx
runtime libraries.

      The reason why Microsoft has not released a security bulletin
is...[put your text here] :P

References:
http://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2006-4183
http://labs.idefense.com/intelligence/vulnerabilities/display.php?id=517
http://www.reversemode.com/index.php?option=com_remository&Itemid=2&func=fileinfo&id=52
(PDF)

----
Reversemode
Advanced Reverse Engineering Services
www.reversemode.com