January 1998

UnPBX

The UnPBX is the PC-based successor to the traditional PBX. It's cheaper, more flexible and more easily maintainable than the old clunker in your company's phone closet.

A couple of years ago, compression was limited to JPEG, GIF and Group 3 fax. Compression technologies have multiplied. Now there are also Group 4, PNG, wavelet and fractal compression. Compression is applied to still images, audio and video. Companies vie to make theirs the next compression standard.

There are three reasons to compress.

1. Save Storage Space. Many files, particularly audio and video files, take up a lot of hard drive space. Compressing files saves money by using existing storage more efficiently. This is a major concern when you back up your computer or network. Compressing backup files not only saves disk and tape space, it speeds up the backup process.

2. Conserve Bandwidth. Save time sending files across a company network or the Internet by compressing them first. Some network technologies like videoconferencing require compression. Without compression, they won't work. Compression saves transmission time and lets the network carry more traffic. More people can use your existing network. Save money by avoiding expensive network upgrades.

3. Speed Up Application Software. Disk drives are fast and getting faster all the time. A big file still takes time to load into memory. Compressed files take less time to load into memory and decompress than it takes to load an uncompressed file from disk. If your applications can work with compressed files directly, skip the decompression step. This saves time and reduces the resources needed by an application.

Compression Technologies

Compression technologies fall into two groups, lossless and lossy. Lossless compression preserves the original image down to the last pixel. The decompressed image is identical to the original. Lossy compression allows for degradation during the compression process. The decompressed image isn't the same as the original, although they may be visually indistinguishable.

Lossless compression is used on text files, computer and technical data and when maximum fidelity is required for pictures and sound. Lossless compression works best when the original image can be grouped into identical chunks.

One of the most popular lossless compression algorithms is Lempel-Zif-Welsh (LZW) compression owned by Unisys (Blue Bell, PA 215-986-4011). LZW compression is used in a number of file formats such as Tagged Image File Format (TIFF) and Graphic Interchange Format (GIF). GIF was popularized on the Web by Compuserve.

A popular form of GIF is GIF89. GIF89 allows multiple frames in a GIF image. In progressive GIF, an image is built frame by frame. The image starts out blurry. The image sharpens as each frame is added. Web users can view a picture before it's complete. If they don't want the picture the download process can be halted. This saves time.

GIF89 is also used for simple Web animations. The frames hold a succession of pictures giving the illusion of movement. They give life to otherwise static Web pages.

In GIF, repeated patterns of pixels are identified, assigned numbers and stored in a hash table. Large, repeated pixel patterns are compressed to a few bytes. GIF works best with non-photographic images such as text, line art and charts. GIF compresses 256 color images and stores custom palettes of colors. This makes low contrast images or images with custom spot colors look better.

Lossless compression algorithms are built into data compression programs like PKZIP from PKWare (Brown Deer, WI 414-354-8699) and Stuffit from Alladin (Watsonville, CA 408-761-6200). These programs are used with all files although their ability to compress will vary depending on the file content.

When compressing files on desktop computers bear in mind the limitations of disk file systems. Many file systems have a minimum file size determined by the amount of storage space divided by the number of entries available in the directory.

The file allocation table used by DOS and Windows only stores 65,518 entries. On a large hard disk, the minimum file size could be 8K, 16K or even 32K. Even if a file is made very small, it can still waste a lot of space. This is why most compression programs combine files into an archive.

Many lossless compression algorithms were created for specific applications. The most popular are Group 3 and Group 4 compression. They were originally designed for fax technology.

Group 3 is a one dimensional compression format. It reads a line across a page one pixel deep. Black and white pixels are grouped according to a standard table that assigns numbers to predefined patterns. This is called Huffman encoding after the developer, David Huffman.

Group 4 is similar to Group 3 compression extended to two dimensions. Instead of encoding one line at a time, Group 4 encodes several lines of pixels taking the relative positions of pixels on adjacent lines into account. Group 4 doesn't have as robust error correction as Group 3, but it compresses tighter.

Group 3 and Group 4 are international standards. Group 3 is built into every modern fax machine and modem. Because they are universal standards, and in the public domain, they have found use as compression standards for disk file formats. Most business scanning is black and white or grayscale. Group 3 and Group 4 are well suited for compressing these images. They are not the only compression schemes available.

JBIG, short for ýJoint Bi-level Image Experts Group,' does for bitonal images what JPEG has done for photo image compression. The JBIG algorithm is based on "Arithmetic Coding." JBIG predicts whether a pixel is likely to be white or black by looking at its surrounding pixels. If the pixel agrees with the prediction, it can be represented by a mathematical formula using less space. The data is then compressed again using a technique known as "interval subdivision."

JBIG supports progressive decompression. Image viewers can build JBIG images a frame at a time. The image starts out blurry and, as information is loaded, sharpens. Progressive JBIG doesn't do animations like GIF89, but you can see the image and decide if you want it before the complete download. JBIG is not exclusively bitonal. Images can be stored in grayscale. JBIG's grayscale sometimes gives better image quality than black and white.

The JBIG patent is held by a number of companies. The principal patent holder is IBM. JBIG compression can be licensed from them. JBIG's disadvantage is that there's no standard file format. There is no generally agreed method of storing JBIG compressed files on disk or for transmission across the Internet.

There are a number of application specific lossless compression algorithms. ABIC was created by IBM as a lossless bitonal and grayscale compression algorithm for images of financial documents like checks. ABIC compression can be used in a number of different file formats.

Digital Imaging and Communications in Medicine (DICOM) is a collection of file formats specifically designed for medical images. DICOM images have to be compressed losslessly to preserve their information content. They can also use lossy compression combined with tiling to transmit images to doctors across the Internet.

Lossy compression is used on analog data. Photographs, sound and video are all candidates for lossy compression. Lossy compression must be used with care as images degrade when compressed. Lossy compression works by sampling an image and creating a mathematical function to approximate the information between the samples.

In JPEG, the most common lossy compression used on the Web, every other pixel in a 16 x 16 block is sampled. The remaining information is approximated a Discrete Cosine Transformation. The missing information is replaced by a cosine curve. JPEG is a relatively simple compression algorithm and is limited to compression ratios of 30:1. Higher compression ratios result in visual degradation of the image.

Lossy compression works best on highly-random data. That is, data with a lot of variation that can be easily described by mathematical formulas. If you try using JPEG compression on line art you create mathematical formulas describing blank space. This is why you get a file many times the size of the original when you apply JPEG compression to charts or images of text.

Lossy compression is one of the hottest areas of development because of the need to send audio and video over the Internet. This has led to some cutting edge technologies.

Fractal compression looks for areas in an image that are random and self similar. A picture of clouds has a lot of random data but one patch of cloudy sky can look like another. In fractal compression a mathematical function is used to approximate the appearance of the clouds. Fractals recreate textures no matter how much an image is magnified. An ordinary image gets blocky and blurred when it is magnified.

Fractal compression can compress a file very tightly but it's more than that. It preserves textures on enlarged images regardless of magnification. This is a powerful feature in graphic arts work. But fractal compression needs a lot of processor power. It's impractical for slower computers. Nor is fractal compression a good choice for compressing sound and video.

Instead of dividing an image into chunks for processing, wavelet compression determines the averages and differences between all adjacent pixels in an image. The compression algorithm can be reapplied as many times as desired -- each time the image is compressed further. Wavelet compression is still lossy. Compressing an image too much results in distortion. Wavelet compression has advantages over other compression techniques.

Wavelet compression compresses the whole image all at once. JPEG compression divides an image into 16 x 16 blocks. Too much JPEG compression gives images a blocky appearance. Too much wavelet compression smears the image. Since there are no vertical or horizontal lines, wavelet's distortion is less obtrusive.

Wavelet compression feels faster to the end user. As an image is decompressed, users see the progression from the last transformation to the first with the image getting sharper and sharper.

Wavelet compression is used for compressing video. Compression is very tight. Ratios can be as high as 500:1 although 100:1 is more typical. Since wavelets work on an entire image all at once, they can be used to compress a video stream. This is good for video conferencing.

A final advantage for wavelet compression is that pixel locations are preserved. This lets handwriting and fingerprint recognition systems work on wavelet compressed images as well as on uncompressed bitmaps. Compared to low-resolution bitonal images, wavelet compressed images could theoretically hold more information while consuming less storage space and bandwidth.

File compression software is a key ingredient for imaging toolkits. Nearly all imaging toolkits include compression algorithms for bitonal compression like Group 3, Group 4 and JBIG. LZW compression is found in practically every software developer kit (SDK). However, Unisys owns LZW. It must be licensed separately. JPEG is also pretty universal and can be found in most SDKs. Vendors compete to see who can develop the most efficient JPEG compressor.

There are no standards for most lossy compression formats -- wavelet compression and various video compression formats. The lack of standards, even proprietary ones, is slowing down the acceptance of wavelet compression and other advanced compression techniques. This may not be a bad thing.

There are many advanced features that have yet to be added to the mix. Compression technologies have only just touched the surface about the limits of human perception. Compression algorithms treat all colors equally. If it's found that people can't see one color as well as the others, more compression could be applied without apparent loss of quality. The computer industry is conservative. It's better to have a fully realized image format than an incomplete one that has to be patched up later.

Image Compression Tools

As its name implies, AccuSoft's (Westborough, MA 508-898-2770) Image Gear ($3,000) has a complete set of imaging tools. All the major compression algorithms are supported including Group 3 and Group 4, JPEG and GIF. Image Gear supports ABIC compression for financial documents. A DICOM module toolkit is available separately.

Kofax's (Irvine, CA 714-727-1733) Adrenaline is a combination of hardware and software ($1,500-$3,600) geared towards business scanning. Group 3 and Group 4 compression is supported in hardware. This minimizes the amount of data that has to be passed through the scanning station.

Infinop (Denton, TX 940-484-1165) is working on wavelet compression for photo editing tools, image viewer utilities and software development toolkits. Their Lightning Strike Compressors ($100-$150) are available as standalone utilities for Windows and a Photoshop plug-in on the Macintosh. Infinop is working on bringing wavelet compression to the Internet as an image compression format and for streaming video.

Pegasus (Tampa, FL 813-875-7575) has number of compression products. Their ImagN' ($300-$400) and PICTools ($1,500) toolkits include proprietary JPEG compressors and ePIC compression. ePIC compression is based on Entropy Logarithmic-Scale coding. ELS compresses image files 10-30% tighter than JPEG. Pegasus is allowing free licensing of ELS compression.

TMSSequoia (Stillwater, OK 405-377-0880) has plenty of compression and decompression tools in their ViewDirector ($1,800) toolkit. Their browser plug-in, ViewDirector, can be downloaded free on the Web.

Image Compression

for the End User

Image compression tools are used to make image viewers. Image viewers are used by end users to look at images and to convert from one format to another. Graphics oriented programs like Adobe Photoshop and Corel Draw Suite can also be pressed into service as image compressors. Document scanning systems will compress images as part of the capture process.

When shopping for an image viewer, decide what type of images you will look at. Business documents are normally scanned in black and white and occasionally in grayscale. If you view scans of checks, invoices and other business documents, make sure your image viewer will decode bitonally compressed images. You need Group 3 and Group 4, JBIG, ABIC and GIF.

Some business applications use color. ID photos can be in grayscale or in color. For those types of images you need JPEG decompression. Most of these are public domain compression formats and are generally included in the image viewer.

Things get hairy if you're using an application specific format. Some of the more exotic file formats use different compression algorithms. Even if your image viewer can cope with them, you may not be able to post them to the Web. You'll need a utility that will convert the more exotic files to those easily digested by Web browsers.

InterTech (Atlanta, GA 770-804-8080) is known primarily for their DocuPact document management software ($150-$320). They make an image server that supports more than 350 file formats. Their server accepts images in any format and decompresses them creating thumbnails. When a user requests an image, they run a Java applet that downloads the thumbnails and only one full image. This is a common strategy used by many document management systems to save bandwidth.

Imagenation ($200) from Spicer (Kitchener, Ontario 519-748-2462) is designed for technical and engineering images. Many of the image formats Imagenation serves up are application specific. To deal with them, Imagenation has just about every decompression engine available.

Related Articles:

• Tiling Formats