|
(a) Single 1440 x 1080 RGB image from the
original sequence used for the comparative
quality analysis tests. This image was
resampled for this display to 500x281. |
|
NOTE: The quality
analysis on this page was performed and posted
in January 2005 using CineForm's 8-bit codec
version 1.2. The original analysis remains
on this page in its entirety.
Since
then, many visual quality enhancements have been
made. The current version of CineForm's
8-bit codec is v2.3.
Read the latest visual
quality analysis
that has been performed and posted in October
2005 on the 8-bit CineForm Intermediate codec
v2.3.
|
|
HDV Quality
Analysis
|
|
Visual Quality Comparison between CineForm
Intermediate™ and " Native"
MPEG2 Editing
(Click here for
the PDF version) |
|
Background
With the
recent launch of HDV, driven by new camcorders like Sony's
HDR-FX1, many have asked about the resulting visual quality when
editing using CineForm Intermediate™ compared to editing with
Native MPEG2-TS. There is no question these new HDV cameras
acquire a great picture using MPEG2 compression (see the above
source image (a)), but in this analysis we will show that
editing using the source (MPEG2) format usually does not result
in the highest quality final result. Most will agree that
adopting an intermediate format can greatly improve editing
workflow performance over that of MPEG2, yet what are the
quality impacts? This analysis reveals the visual quality
implications of both workflows, and demonstrates that CineForm
Intermediate maintains a quality advantage in addition to its
substantially higher editing performance.
In a separate analysis we have
compared CineForm's
10-bit CineForm
Intermediate compression technology with that of native DVCPRO-HD
editing for the professional HD and film editing markets.
|
|
Testing Methodology Introduction
The "quality level" used
for CineForm Intermediate compression in these tests was
"medium", which is the default offered by Adobe Premiere Pro's
HDV support. (NOTE: capturing at an even higher quality level is
available when using Aspect HD's "large" file size setting).
Also, the MPEG2-HDV settings used were the defaults for Adobe Premiere HDV.
(NOTE: All testing
was done using Adobe Premiere's (upcoming) v1.5.1 HDV features.) MPEG2 HDV
testing used the MPEG components included within
Premiere Pro. For multi-generation testing, uncompressed RGB
intermediates were created for both workflows.
A Note About 8bit vs 10bit: This describes the number of levels per color channel.
Most cameras use 8bit compression technology such as DV, HDV,
and DVCPRO-HD, for representing up to 256 levels in each of the
color primaries R,G,B or Y,U,V. Whereas 10bit formats are more
commonly used in high-end film and broadcast applications or
uncompressed material delivered across HD-SDI.
This analysis demonstrates the
perceptible loss
of video quality that occurs through multiple stages of
recompression. Even in the simplest of projects an image will
typically go through one generation of decompression followed by
recompression. Decompression is a necessary step to display, edit
and enhance (titles, color correction, etc) your source
material, and the recompression is necessary to output to your
preferred delivery format such as WMV-HD, MPEG2-TS for DVHS, or
back to original HDV format for export to the camera. Some
proponents of "native" MPEG editing claim HDV editing
in its native form can be lossless; this can only be achieved if you don't
change anything (including cuts -- which still require rendering
using
MPEG editing workflows). In practice, editing-session recompression must
occur for obtaining an output even in a cuts-only scenario. Below we show the image quality
difference though multiple stages of compression and
decompression using CineForm Intermediate and native MPEG2.
A Note About
the Images: All
images below have been converted into lossless PNG files for
viewing within your browser. In each case the PNG files
have been extracted and converted (losslessly) from the
respective MPEG files, CineForm Intermediate files, or
uncompressed files as appropriate. Some of
the images we have interpolated for an expanded view that is easier to see.
"Expanded" images have been interpolated up by 3X in each
dimension by using a bicubic interpolation.
Click
here to read a complete discussion of our test methodology. |
|
The First Generation
Image (b)
below represents a windowed region from the first capture of the
source image (a) with a simple title "1" added. Deeply colored
titles are one of the most difficult objects to reproduce within
most compressions systems -- this will challenge both MPEG and
CineForm Intermediate formats the most. Image (b) is the source
and has not yet undergone recompression. Image (c) shows the
image quality once the source (image b) is compressed back to
HDV MPEG2. This quality loss is clearly apparent in the red
title as the chroma detail in the title is greater than can be
stored in the interlaced 4:2:0 color space of HDV MPEG2. Image
(d) is the same title encoded using the CineForm Intermediate HD
format. Detail in the title is better preserved because CineForm
Intermediate uses a 4:2:2 color space and a higher-bitrate
Wavelet compression. But because CineForm Intermediate format is
only, well, intermediate, image (e) shows the final quality of
Image (d) after it is converted back to HDV MPEG2. |
|
First-Generation Comparison of
CineForm Intermediate vs MPEG2 Native
|
(b)
Windowed region from
uncompressed source
image (a),
with title added. |
(c)
Windowed region of
source image (a) compressed into
HDV MPEG2 representing final project output. |
(d) Windowed region
of source image (a)
compressed
into CineForm Intermediate.
|
(e) CineForm
Intermediate (image (d)) further compressed
into HDV
MPEG2 representing final project output. |
|
Clearly the
conversion into MPEG2 (images (c) and (e)) creates a severe quality
impact for sharp edges of the title, whether the source was raw (b) or
CineForm Intermediate (d). Much of the vertical blurring of MPEG images
is caused by its 4:2:0 color space. The 4:2:2 color space used by
CineForm Intermediate shows none of this vertical blurring. If a
CineForm Intermediate clip is sent back to MPEG2 (as in (e)) it also
will undergo the same MPEG losses. However, there is an important
observation to be made - that is, the final image quality of both
workflows look identical; i.e., (e) is visually the same as (c) - more
on this below. |
|
The
First Generation Difference Analysis
The images below
show the image difference between the original uncompressed source image
with title (b) compared to each of the compressed outputs. Differencing
is a simple way to display errors created during compression -- these
images represent only the errors. Differences were created by
taking the absolute value of one image subtracted from the other
(difference operator within GIMP), de-saturating, then adding 50% grey
(for easier viewing). Middle grey means no error; any lines or color
deviations from middle grey seen in these images identifies details lost
from the original. |
|
First-Generation Differences of
CineForm Intermediate vs MPEG2 Native
|
(f) Difference of
uncompressed image (b) with
itself, forming zero-loss reference. |
(g) Difference of
HDV MPEG2 output (image (c))
with uncompressed image (b). |
(h) Difference of
CineForm Intermediate output
(image (d))
with uncompressed image (b). |
(i) Difference of
HDV MPEG2 image (e) (derived from
CineForm
Intermediate source) with uncompressed image (b). |
|
(j) The difference
of the two differences (image (g)
and image (i)). Because this difference is zero,
individual image differences are about the same. |
|
The upper left image
(f) is the source image subtracted from itself, so the difference is of
course 0 - i.e., middle grey, which provides the reference. The
remainder of the images show the amount of loss inherent within each
workflow during editing. Image (g) clearly shows the loss in detail on
the edges of the title. Observing (h), the difference between the source
and CineForm Intermediate image, there is virtually no discernable loss.
In particular, during the editing process using CineForm Intermediate
you are accumulating essentially no losses. At the end of a project when
editing with CineForm Intermediate, if you intend to go back out to the
camera, you'll need to render back to MPEG. This is shown in (i), and
the MPEG losses again become visible. But an interesting observation
arises out of image (j), which represents the difference between image
(g) ("native" MPEG edited image) and image (i) (edited in CineForm
Intermediate and later converted to MPEG for output). Image (j) shows no
visible loss whatsoever, meaning that the final output errors for each
workflow are the same. This disproves the myth that only native editing
delivers the best quality. In further analysis below, we show that in
fact the opposite is true, and that you achieve a quality advantage
using a CineForm Intermediate workflow.
The
Second Generation
Second generation
quality hits can occur when you need to move video from one editing tool
to another or from one computer to another. Typically your timeline
needs to be "flattened" or rendered out to a file for importing into
another other tool or computer. This render will likely force a
generation loss in most editing scenarios. Either Professional or
collaborative workflows will often go though many of these stages. Even
when doing a simple project archive, the process of rendering out a
completed project for later manipulation will involve a generation hit.
Second generation hits are very common.
The images below
represent the mockups of our second generation experiment. In each of
these cases the project has been archived then re-imported to add the
second title layer "2" (a more realistic example may have been a color
correction pass, but the same observations apply.) The First image (k)
shows the clip archived to CineForm Intermediate, title added, then
exported again to CineForm Intermediate. The second image (l) is the
same example but uses native MPEG2 in both generations. The final image
(m) shows the CineForm project in (k) after it is exported to HDV. |
|
Second-Generation Comparison of
CineForm Intermediate vs MPEG2 Native
|
(k) Second
Generation CineForm Intermediate.
(Large version ~600KB) |
(l) Second
Generation MPEG2 Native (HDV).
(Large version ~600KB) |
(m) Second
generation CineForm Intermediate (image (k))
exported out as HDV MPEG2.
(Large version ~600KB) |
|
Beginning in this
second generation (and of course future generations) we can begin to see
that editing using CineForm Intermediate as the editing format, even if you need to export
back to MPEG2 at project completion, has a sustainable visual quality advantage over editing
in native MPEG2. If you observe the large versions of image (l) (which
has undergone two generations of HDV MPEG2 encoding) compared to image
(m) (which has undergone one generation of CineForm Intermediate
encoding followed by one generation of HDV MPEG2 encoding) you will
notice that the blocking artifacts are substantially fewer in image (m).
Multi-generational compression artifacts are cumulative
- MPEG2 loses more per generation, resulting in visible differences even
after two generations. Subtle blocking artifacts become visible in the
second generation of the native MPEG2 workflow (l), yet two generations
of CineForm later output as MPEG will have fewer artifacts because each
generation of CineForm Intermediate generates substantially fewer losses
than MPEG does. When both are exported back to HDV, the second
generation quality advantage goes to CineForm Intermediate. If your
production doesn't need HDV export (most don't) it is best to avoid the
MPEG quality hit in any stage. Remember, MPEG2 is a distribution format,
it wasn't designed for a visually-demanding post-production workflow -
that is why we see it break down so quickly.
The Ninth Generation
OK, the ninth
generation is overkill of most user work-flows, but it does serve to
demonstrate the robustness of any compression solution. Through this
many generations the very nature of any compression algorithm can often
be revealed. To make things difficult for both compression algorithms,
the selected frame has a lot of motion which decreases the effectiveness
of each codec. This frame has also been de-interlaced so the artifacts
for an individual field can be better observed. |
(o)
Single 1440 x1080 RGB image from the 9th-generation
uncompressed reference sequence
used for the comparative quality analysis tests. This image was
resampled for this display to 500x281. |
|
Ninth-Generation Comparison of
CineForm Intermediate vs MPEG2 Native
|
|
(p) Windowed
region from 9th-generation CineForm
Intermediate sequence.
(Large version
~600KB) |
(q) Windowed region
from 9th-generation HDV MPEG2
sequence.
(Large version
~600KB) |
|
Images (p) and (q)
clearly show the key artifacts after nine generations of compression and
decompression. The CineForm image on the left looks basically the same
as the uncompressed workflow albeit with some increase in noise. The
noise is difficult to see while the image is in motion. The noise
increase was not present in our 10-bit CineForm Intermediate
Quality Analysis even after 15 generations of YUV processing. (For those
considering this many generations, a 10-bit YUV solution, like
CineForm's Prospect HD, might be best.) The MPEG2 image on the right
exhibits many problems highlighting why MPEG2 is a poor choice for any
multi-generation editing work: 1) there is substantial detail loss in
the face of the parrot, 2) blocking artifacts are clearly visible
throughout the image, plus 3) strong ringing is apparent at the end of
the parrot's beak and the top of its head. The main problem with this
type of image distortion is that it is quite visible and often jarring
to the viewer; blocking artifacts "jump out" as they are so unnatural
looking, even while the scene is in motion.
The Ninth Generation Difference Analysis
Images (r) and (s) below show the differences
between the uncompressed 9th-generation result compared to the
respective 9th-generation compressed
output. |
|
Ninth Generation Differences of
CineForm Intermediate vs MPEG2 Native
|
(r) 9th-generation
CineForm Intermediate differenced
with 9th-generation
uncompressed reference.
|
(s) 9th-generation
HDV MPEG2 differenced with
9th-generation
uncompressed reference.
|
|
Final Word
A key separator in the
two methods are the resulting visual artifacts. The DCT block-based
nature of MPEG compression will be visible as little squares (the size
of its DCT transform size) when this compressor struggles with either
high detail or high motion. The Wavelet compression used within CineForm
Intermediate doesn't use a block-based structure so its generation
losses are far more natural looking even when pushed to nine
generations.
The quality of these
multi-generation images is also dependent on the bitrate of the
compression - HDV defines a fixed bitrate structure (25Mbps) independent
of scene complexity. This restriction doesn't match the needs in
post-production, here's why: If you capture a good looking image that
needs the full 25Mb/s of HDV, then you add information to the image
(like color corrections, transitions, filters, or titles) the image now
needs more than 25Mb/s. Clamping the image to the original 25Mb/s
forces data loss. Consequently, as scenes become more complex, artifacts
begin to appear when using HDV as an editing format.
CineForm Intermediate
uses a variable bitrate structure, often known as "Constant Quality"
that allows the bitrate to climb during demanding scenes or fall during
less demanding scenes. During multi-generation renders the bitrate may
also rise or fall as appropriate. There is no limit to a particular
bitrate with CineForm Intermediate - it rises or falls appropriately
based on scene complexity. During this particular test the CineForm
Intermediate encoded sequence increased in bit-rate 15% through the
ninth generation (1-3% per generation), to support the details added by
the titles. (By the way, not all "intermediate" formats that are
becoming available use a variable bitrate structure like in CineForm
Intermediate - they too may suffer from artifacts during complex
scenes.)
Given the visible
quality benefits and substantial performance increase, CineForm
Intermediate's "scalable" architecture is ideal for most forms of video
and film production. CineForm's scalable compression technology has now
been adopted by Adobe and Sony to enhance the HDV workflows within their
respective applications; it has also been used to online film for 35mm
print output, where the greatest quality concerns apply. The bottom line
is that the architecture designed for CineForm Intermediate delivers
optimal quality and performance for a high-definition post-production
workflow compared to other solutions.
Click here
to see details of the procedures and test methodology.
(Click
here for the PDF version) |
