Video File Format Comparisons

by Paul Arnote

In my series of articles comparing various file formats (July, 2009: Graphic File Format Comparison, August, 2009: Audio File Format Comparison), I turn my attention to video file formats. Although we commonly call them video file formats, they are really containers for combining separate video and audio information into a synchronized data stream.

In my previous two articles, we talked about graphic and audio formats being either lossy or lossless in their compression. Because video files are so large, it is not common or practical to use uncompressed video. Thus, nearly all of the video formats use some type of compression, and for the most part, all of those compression techniques are considered lossy.

Before we can talk about the various video file formats, there are a few terms we need to become familiar with. Mind you that entire chapters of books have been written to explain these terms and formats. In this medium, I'm going to try to give the most simplified explanation as I can, mostly because we don't need to know all the intricacies involved to understand what they mean.

The first of these is bitrates. Bitrates refer to how many bits per second are used to represent the video portion of the file. These can vary from as low as 300 kbps (kilobits per second) for low quality video, to as high as 8,000 kbps for very high quality video. Just as with sound files, a higher bitrate translates to a better quality video.

The next thing we need to discuss is how the image appears. We have two terms to describe the video image: interlaced and progressive. Interlaced video is a way to make the best use of the limited bandwidth for transmitting video, especially in the older days of analog NTSC broadcasts. The receiver (your TV) "tricks" your eyes by drawing first the odd number lines on the screen 30 times per second (25 times per second in Europe), and alternating the drawing of the even numbered lines on the screen 30 times per second, in an odd — even — odd — even — odd — even pattern. Progressive video draws both the odd and even numbered lines at the same time, instead of alternating them. This makes the picture appear sharper and more vivid.

Next, is resolution. In the analog video days, video was commonly the equivalent of 352 x 240 pixels for NTSC video in North American, and 352 x 288 for PAL/SECAM video elsewhere. Today's digital televisions can display up to 1920 x 1200 pixel pictures. Computer monitors have much higher display resolution than conventional analog televisions. The move to digital televisions has made it feasible to multi-task the family television to double as a large computer monitor. A common resolution for DVD quality video is 720 x 480 for NTSC, and 720 x 576 for PAL/SECAM systems. The resolution for the new high definition television broadcasts are even higher, and provide Blu-Ray quality, able to display a pixel-for-pixel representation of the highest High Definition television specification at 1920 x 1200 pixels.

Formats, or Containers

Theora

We start with the Theora container, because it is open source. And, because this is a Linux publication, we are particularly fond of open source solutions. Developed by the Xiph.Org Foundation as part of the OGG project, it is derived from On2 Technologies' VP3 video codec. It's also named after Theora Jones, Edwin Carter's Controller on the old Max Headroom television show.

Because it is developed and maintained by the Xiph.Org Foundation, it commonly has Vorbis formatted audio streams, placed into an OGG container. Since On2 Technology released the VP3 codec and its source code to the public under an open source and free software license, and disclaimed any rights and patents on the technology in September 2001, it allowed anyone to use Theora and other VP3-derived codecs for any purpose. In 2002, On2 inked an agreement with the Xiph.Org Foundation to make VP3 the basis of a new and free Theora video codec, and declared Theora to be the successor to VP3.

The first stable v1.0 release of Theora was in November 2008. Since it is a relatively new format that has very little commercial support, Theora has struggled to gain acceptance from distributors, especially on the web. Yet, as the only mature and royalty free video codec, Theora is well established as a baseline video format in modern free software, and has become the format of choice for many other organizations (e.g., Wikipedia).

Theora does not make use of hardware acceleration, but because its algorithm is less complex than H.264, which is used by the competing MP4 format, the need for hardware acceleration is not considered to be as relevant.

There are several applications in the PCLinuxOS repository that can play, stream and edit Theora video files. For encoding, VLC and HandBrake are available. Cinelarra can edit Theora video files, while VLC can stream them.

MPEG-1

While development of the MPEG-1 standard began in 1988, it wasn't until 1992 that the final standard was approved and the first MPEG-1 decoder was made available. Compressing video 26:1 and audio 6:1, the MPEG-1 format is designed to compress VHS quality raw digital video and CD audio with a minimum of quality loss.

Today, it is the most widely compatible lossy compression format in the world. MPEG-1 standard is part of the same standard that gives us the MP3 audio format. Fortunately, the MPEG-1 video and Layer I/II audio can be used royalty free and without license fees, since the patents expired in 2003. Only the MPEG-1 Layer 3 Audio format (more commonly known as MP3) is still covered by patents and licensing restrictions.

The "standard" MPEG-1 video bitrate is 1,150 kbps, with audio at 224 kbps and 44.1 kHz sample rate. The video is recorded at 352 x 240 pixel screen size for NTSC, and 352 x 288 pixel screen size for PAL/SECAM systems. One distribution format that makes wide use of the MPEG-1 format is the Video Compact Disc, or VCD. Although it never caught on in the U.S., other parts of the world have seen wide adoption of the VCD, due to its relatively inexpensive production costs, coupled with the VCD's ability to withstand high humidity environments (as present in Southeast Asia) better than VHS tape.

MPEG-1 formatted video files are easily read by most of the video programs in the PCLinuxOS repository.

MPEG-2

The MPEG-2 format was born from the shortcomings of the MPEG-1 format. Specifically, the MPEG-1 format had less efficient audio compression, a lack of flexibility when it came to the packet types it accepted, and it does not support interlaced video. MPEG-2 is also the format of choice for digital televison broadcasts.

Work on the MPEG-2 format began in July, 1990 — before the first draft of MPEG-1 was ever written. It was intended to extend the MPEG-1 format to provide full broadcast quality video at high bitrates, between 3 — 5 Mbits/s, and added support for interlaced video.

MPEG-2 is the designated format for movies and other programs that are distributed on DVD, SuperVCD, and BluRay discs. As a result, the televisions in our living rooms, along with the TV broadcast stations and our DVD players are designed to the MPEG-2 standard.

Because MPEG-2 evolved from the MPEG-1 format, all standards compliant MPEG-2 decoders can also play MPEG-1 video streams. Thus, even though you may not realize it, you can play VCDs in most home DVD players.

Currently, over 640 patents, owned by more than 20 corporations and one university, and the patent pool managed by the MPEG Licensing Authority, burden the MPEG-2 format. In jurisdictions where software patents are upheld and enforced (currently, the U.S. is included in those jurisdictions), use of the MPEG-2 format requires payment of licensing fees to the patent owners. It"s best to check the laws of your jurisdiction, since they vary so widely. For example, software patents are not allowed in India and the Philippines, but they are in Japan and Korea, and to a limited extent, in Europe and Australia. Hence, this is one of the reasons that PCLinuxOS does not come with DVD playback software enabled by default; rather the end user must install the necessary libraries and dependencies themselves, based on the laws of their relevant jurisdiction.

MPEG-4

MPEG-4 represents a patented collection of methods to define compression of video and audio, designating a standard for a group of audio and video codecs. MPEG-4 encompasses many of the features of MPEG-1 and MPEG-2, while adding support for 3-D rendering, Digital Rights Management (DRM), and various types of interactivity.

MPEG-4 files are commonly encoded with codecs like DivX, Xvid, 3ivx, and x264 to name a few, and may be used by BluRay discs. Since most of the features of the MPEG-4 standard are left to the discretion of the individual developers to implement, they are free to choose which parts of the MPEG-4 format to implement. Thus, a "full implementation" of the MPEG-4 format does not exist, and most likely will never exist.

Introduced in late 1998, more than two dozen companies claim ownership of patents related to MPEG-4. The licenses for use of the MPEG-4 format (in jurisdictions where software patents are upheld and enforced) is handled by the MPEG Licensing Authority.

MPEG-4 files can usually be easily viewed on PCLinuxOS, using VLC or MPlayer.

QuickTime

Initially released in December, 1991, the QuickTime format is released under a proprietary license from Apple, beating Microsoft’s Video for Windows to the "market" by nearly a full year. Developed as a multimedia framework by Apple, Inc., it is capable of handling various formats of digital video, media clips, sound, text, animation, music, and interactive panoramic images. The most current stable release is QuckTime 7.6.2, and is available for Mac operating systems, and computers running Windows. Although no formal implementation exists for Linux, there are a number of programs in Linux which can handle (with varying degrees of success), QuickTime video playback. Possibly the best of the Linux programs that can handle playback of most QuickTime files are VLC and MPlayer, both of which are in the PCLinuxOS repository.

In February, 1998 the International Standards Organization approved the QuickTime file format as the basis of the MPEG-4 file format. The benefit is that MOV and MP4 files (containers) are interchangeable on a QuickTime-only environment (meaning running in an "official" QuickTime player, like QuickTime on the Mac OS X or QuickTime for Windows), since both use the same MPEG-4 codecs.

AVI

AVI, more formally known as Audio Video Interleave, was released in November, 1992 by Microsoft as a part of its Video for Windows technology. It is basically a file container the allows synchronized audio and video playback.

Since AVI files do not contain pixel aspect ration information, and many players render AVI files with square pixels, the frame (image) may appear stretched or squeezed horizontally when played back. However, VLC and MPlayer have solved most problems related to the playback of AVI files.

Although being "older" technology, there is a benefit to using AVI files. Because of it being around for so long, coupled with Microsoft’s market penetration, AVI files can frequently be played back on the widest variety of systems and software, second only to MPEG-1. It has gained widespread acceptance and adoption throughout the computer industry, and can be successfully played back, so long as the end user has the proper codec installed to decode the video properly. Additionally, the AVI format is well documented, not only from Microsoft, but also many, many third parties.

Matroska

Matroska files, commonly recognized by either their *.mkv file extension, represent a multimedia container that is an open standard free container format. It can hold an unlimited number of video, audio, picture, or subtitle tracks in a single file. Implementations of the Matroska format consists mostly of open source software.

The word, matroska, is an English word derived from the Russian word matroyshka, the name for the nesting doll, where one egg-shaped doll is found inside another, then another, and so on.

Matroska video files (*.mkv) can be played back on a wide variety of software in PCLinuxOS, including MPlayer, VLC, Avidemux, and MythTV. And, because Matroska is an open standards project, much like Theora, the source code of the libraries are developed under the GNU LGPL. It has gained popularity recently, especially among the warez crowd for ripping and distributing high definition content from HDTV and BluRay discs.

WMV

WMV, or Windows Media Video, is a compressed video file format, made with several different proprietary codecs, made by Microsoft. Originally intended to compete with RealVideo in 2003 as an alternative for streaming video, it has since gained adoption for use with BluRay discs.

The WMV files are often wrapped in the ASF, or Advanced Systems Format. WMV files, themselves, are not encoded. Rather, the ASF wrapper is often responsible for providing the support for digital rights management, or DRM. Based on Windows Media 9, WMV files can also be placed inside either an AVI or Matroska container. In that case, the WMV file claims either the AVI or MKV file extensions.

WMV files can be played on PCLinuxOS, using VLC, MPlayer, or most any other program that uses the FFmpeg implementation of the WMV codecs.

3GP

The 3GP format is actually two similar formats. The first, 3GPP, is designed as a container format for GSM phones (in the U.S., primary GSM wireless carriers are AT&T and T-Mobile). The second, 3GPP2, is designed as a container format for CDMA phones (in the U.S., primary CDMA wireless carriers are Verizon and Sprint). 3GPP files will often carry a 3GP file extension, while 3GPP2 files will often carry a 3G2 file extension.

3GP and 3G2 files store video streams using MPEG-4 Part 2, H.263, or AVC/H.264 codecs. Some cell phones will use the MP4 file extension to represent 3GP video. Both formats were designed to decrease storage and bandwidth requirements to accommodate mobile phones.

Software support under PCLinuxOS is, once again, achieved with VLC and MPlayer. Additionally, 3GP files (and most 3G2 files) can be encoded and decoded with FFmpeg.

FLV

FLV files, also known as Flash Video, are a file container format used primarily to deliver video over the Internet. In fact, it has become the defacto format of choice for such sites as YouTube, Google Video, Yahoo! Video, Metacafe, and many news outlets.

While the FLV format is an open format, the codecs used to produce FLV files are mostly patented. The most common codecs used are the Sorenson Spark (H.263 codec variant) and On2’s VP6. FLV files can also be encoded as H.264 in the more recent releases of Adobe Flash.

Most audio in an FLV file is recorded as MP3, although there are a few proprietary audio codecs used. Uncompressed ADPCM (Audio-Digital Pulse Code Modulation) can also be used to record audio in an FLV file. More recent versions (from Flash 9, upwards) also support the recording of audio in the AAC (Advanced Audio Coding) format.

Under PCLinuxOS, VLC and MPlayer support playback of FLV files. One can also encode FLV files with FFmpeg. Additionally, it’s usually a simple matter, using Avidemux, to convert from the FLV format to another, more universally read format — such as AVI or MPEG-1. You can also use Avidemux to edit FLV files.

And by the way — changing the file extension from FLV to AVI does not make it an AVI file. Rather, it’s just an FLV file with an AVI file extension. Simply changing the file extension does nothing to the file header in the video file, which is what determines they type of video file it really is. I point this out because of a post in the forum some time ago (perhaps a year ago?) that made this erroneous assertion. The poster achieved the results they did merely because the system knew how to handle a file with the AVI file extension, by playing it in VLC or MPlayer. In actuality, the programs are playing the FLV file, not an AVI file — something they already have the capability to do.

Summary

I hope this helps alleviate any confusion you might have had regarding video file formats. Plus, having a basic understanding of the various video file formats is helpful when trying to figure out how to convert from one format to another. While this isn’t meant as a comprehensive overview or discussion of the subject matter — entire books have been written, trying to cover the topic — it should be enough to get you at least started in the right direction. And, there are many more file formats out there that I didn’t talk about. Rather, I hand-picked some of the more common ones that you are likely to find or have a need to "work with."