VideoLAN is a project that develops software for playing video and other media formats across a local area network (LAN). It originally developed two programs for media streaming, VideoLAN Client (VLC) and VideoLAN Server (VLS), but most of the features of VLS have been incorporated into VLC, with the result renamed VLC media player. VLC media player is a free and open source media player and multimedia framework.

VLC media player can play many audio and video formats (MPEG, DivX, ogg, Wave etc.) as well as various streaming protocols. The Matroska Multimedia Container, an open standard, free container format, is one that can be played by VLC media player. The Matroska can hold an unlimited number of video, audio, picture or subtitle tracks inside a single file. It is intended to serve as a universal format for storing common multimedia content, like movies or TV shows. Matroska is similar in concept to other containers like AVI, MP4 or ASF, but is entirely open in specification, with many implementations in open source software. Matroska file types are .MKV for video (with subtitles and audio), .MKA for audio-only files and .MKS for subtitles only.

The Matroska file format is based on Extensible Binary Meta Language (EBML), a generalized file format similar to XML. The Matroska files only have two different top level elements, EBML and Segment. The Segment is the top level container for multimedia data. The Tracks element contains information about the tracks that are stored in the Segment, such as track type (audio, video, subtitles), the codec used, resolution and sample rate.

A heap buffer overflow vulnerability exists in VLC Media player. When handling subtitles, the application can overflow a heap buffer through lack of bounds checking in the StripTags() function while processing strings with an opening “<" without the terminating ">“. A remote attacker could exploit this vulnerability to overflow the heap buffer and inject arbitrary code. The injected code will be run under the security context of the logged in user.

SonicWALL IPS team has researched the vulnerability and created the following IPS signature to cover exploits related to it:

• 6286 VideoLAN VLC Media Player Subtitle Heap BO Exploit 1
• 6287 VideoLAN VLC Media Player Subtitle Heap BO Exploit 2

The vulnerability is referred by CVE as CVE-2011-0522