August 2001
A Closer Look at Compact Image Formats
by Doug Henschen
TIFF Group IV, JPEG and PDF are all well-entrenched formats for document imaging, but developers keep looking for better alternatives. In a head-to-head test comparing them with two emerging "wavelet" compression formats, DjVu and Lura-Document, we created dramatically smaller files that nearly matched the quality of the standards.
Despite today's lower storage costs, file size continues to be a big concern in document imaging. Bandwidth restrictions are perhaps the biggest issue, as simultaneous downloads of large, multipage documents can bog down networks. The problem is even more challenging when Web access, emailing and wireless devices are involved.
DjVu and LuraDocument use similar combinations of image segmentation and "wavelet" techniques. Segmentation places high-contrast information such as text on a separate layer. Wavelet compression is applied to the remaining, soft-edged areas, discarding bits that aren't needed to preserve the image's perceptible quality. While both formats are lossy (meaning they lose some of the original image information), hard-edged areas such as text can be stored losslessly. Compression ratios with wavelet technologies can exceed 100 to 1, while JPEG compression can't exceed 30 to 1 without visible image degradation.
While similar in principle, DjVu and LuraDocument use different, proprietary compression algorithms and encoding methods. Each format requires its own plug-in to view images in a browser.
Originally developed by AT&T Labs, DjVu is now marketed by Lizard Tech, the Seattle-based company that also promotes the Mr. Sid graphical imaging compression format. Lizard Tech is promoting DjVu aggressively in the end-user community, with a product lineup that includes Solo, Workgroup and Enterprise capture and conversion packages selling for $299, $5,000 and $7,000, respectively. These tools offer integrated OCR technology that lets you create Image+Text-style documents with searchable recognition results hidden behind the image. The company also offers a software developers' kit (SDK) that sells for $30,000.
To support viewing, Lizard Tech offers a free 700K plug-in for download from the LizardTech site (www.lizardtech.com). The company says 25 million desktops are now DjVu capable. A free Active X control was recently added that allows DjVu files to be viewed from MS Office applications.
LuraDocument is a spin-off of the LuraWave graphical image format developed by Berlin-based LuraTech. The company's biggest success has been in Europe, where several document management companies have adopted the product. In North America, the technology is marketed primarily to developers through the company's Menlo Park, CA, office.
LuraTech's end-user products are limited to LuraDocument Capture, a single-user tool that can be downloaded for free (www.luratech.com) or upgraded to a $19.95 licensed version that supports multipage documents. This capture software does not offer OCR, and it is aimed primarily at color documents scanned at 300 dpi or even higher resolution levels. LuraTech also offers a Command Line tool for batch processing priced at $1,890 and an SDK that costs $2,500.
Our tests were aimed at conventional document imaging use; the goal was to capture readable (and in several cases, searchable) business records, not pristine Web downloads or marketing documents. We scanned a range of documents at 200 dpi and saved them in TIFF Group IV, PDF, JPEG, DjVu and LDF format.
As the file size chart on page 63 clearly shows, DjVu and Lura-Doc deliver on the promise of dramatically smaller files. Bitonal TIFF Group IV, PDF Image Only and PDF Image+Text images were about three times larger, on average, than DjVu images. While the LuraDoc format isn't really targeted for bitonal use, a few samples were about 50 percent smaller than comparable TIFF and PDF images.
Figure 1. DjVu and LuraDocument files were dramatically smaller, but some challenging
images weren't as readable as in conventional formats. Compared to a PDF image
(left), the handwritten text on the DjVu image (center) was somewhat fuzzy,
while the LuraDocument file (right) lost a bit of text detail and suffered
mottled artifacts in light, continuous-tone areas.
|
Results were most dramatic in color, with TIFF LZW files about 85 times larger, on average, than DjVu images. JPEG and PDF Searchable Images were about 20 times larger. LuraDoc images were also compact, but the DjVu files were about 15 percent smaller on average.
Adobe's PDF Normal format (PDF Formatted Text+Graphics in Acrobat Capture 3.0) is often discussed as the most compact alternative, but it is aimed at converting images to electronic documents. It is not suitable for images of record. The format did shine for a few samples suitable for conversion.
Given the dramatically smaller size of the wavelet files, you might expect compromises in image quality. The most challenging documents - carbon copies of a parking ticket and an insurance inspection form - were hard to read even as originals, and only the grayscale and color images were able to capture the handwritten information.
In these images, the wavelet formats weren't as readable as the TIFF, JPEG and PDF files. The DjVu images retained slightly more of the hard-edged information than the LuraDoc images, but the more lenient thres-holding gave faint handwriting a fuzzy appearance. The same text was crisper on the LuraDoc images, but more of the faint handwriting was lost. The LuraDoc files also suffered mottled artifacts on light, continuous-tone backgrounds, with subtle crinkles and folds taking on a shadowy appearance.
Unique with segmented formats such as DjVu, LuraDoc and PDF, you can view the hard-edged information layer and the softer-contrast layer separately to get a better view of details. Both LizardTech and LuraTech say their SDKs (as well as the DjVu Enterprise product) offer adjustable hard-edge and soft-edge thresholding for more difficult documents with faint text.
If your imaging needs involve decent-quality documents with clear, contrasty text, DjVu and LuraDoc should yield highly readable, compact images. If you encounter faint text, endorsements or light details on top of backgrounds, these formats may have difficulty deciding whether the information is part of the background.
DjVu and LuraDoc both support multipage documents, which is an advantage over JPEG, but one shortcoming is that neither offers the tagging capabilities available in TIFF and PDF. The DjVu and LuraDocument capture products also lack standard export features for common document management systems.
How big of an impediment are the proprietary plug-ins? If you have a small, closed community of users, they may present little problem. But if your application calls for broad distribution, can you count on your users to download and install the file?
The future holds the promise of a final JPEG 2000/JPM standard that will yield highly compact, wavelet-based images suitable for document imaging (see "Setting a New Standard," page 7). The standard will also have to be supported in the browser (notably by Microsoft) to succeed. Even if this happens within one year, it would be some time before the updated browsers and plug-ins would be widely distributed. Until that time, compact alternatives are available today for those who need them.
|
