Fast DNG decoding on CPU
Lossless JPEG encoding algorithm is widely used in many cameras shooting RAW. This is a must to increase the number of frames which could be stored in internal memory or flash card of the camera. Lossless encoding guarantees exact reconstruction of original RAW data, though compression ratio for such algorithm is quite moderate, usually it's around 1.5-2. For lossless JPEG, the standard permits any data precision between 2 and 16 bits per sample. Lossless encoding for RAW data is done in realtime inside camera and this is the case for majority of cameras.
Lossless JPEG compression algorithm
- As soon as we need to encode raw data, we have to bare in mind that original image is raw bayer CFA with pattern RGGB or alike. That's why we need to encrease image width in two times and to decrease image height in two times. It's good idea for better data correlation at encoding stage.
- Before lossless jpeg encoding, we have to choose prediction formula (one from 7 choices) to encode the difference between original and predicted values of each pixel. The most frequent choice is two-dimensional predictor from the formula No.6: Px = Rb + (Ra - Rc)/2. It means that for prediction we utilize values from upper pixel plus half of difference between left and upper-left pixels.
- Compression is done according to Huffman coding algorithm with fixed table. If an image has just one component, then one Huffman table in enough. Usually there are two components, so in that case we need one or two Huffman tables.
- From the very beginning we divide image into tiles to encode them independently. After compression we place all encoded tiles into RAW and add offset for each tile to the header.
Lossless JPEG decoding
- Performance of Lossless JPEG decoder is mostly limited by Huffman decoding. Actually, we need to read bitstream (bit after bit), to recover Huffman codes and data bits. Huffman decompression is essentially serial algorithm, so one can't implement it on GPU.
- Right after decoding we need to restore original pixel values according to prediction formula, and to restore image width and height.
- After decoding of all tiles, we compose original RAW image.
Many existing libraries for lossless jpeg decoding (dcraw, libraw, libjpeg, Adobe DNG SDK decoder, etc.) are not optimized for speed, which leads to slow RAW decoding. This is not actually a problem for a single image processing, but it could be not fast enough for workflow with high resolution images (up to 50-100 MPix and more), for batch processing or for Raw Video Player with smooth output in realtime.
PC for testing
- CPU Intel Core i7-6700 (Skylake, 4 cores, 3.4–4.0 GHz)
- GPU NVIDIA GeForce GTX 1080 (Pascal, 20 SMM, 2560 cores, 1.6–1.7 GHz)
- OS Windows 10 (x64)
Lossless JPEG decoders to compare
We will test the following lossless jpeg decoders:
- Lossless JPEG decoder from Adobe DNG SDK
- LJ92 decoder from MlRawViewer (liblj92 library)
- LJ Decoder from Fastvideo (lossless jpeg library on CPU)
In real use case, multithreaded decoder software is utilized, and each tile is decoded in a separate thread. We've done comparison for single thread applications for the same DNG image and at the same hardware. JPEG decoding is fully done on CPU, no image processing on GPU is implemented. Test image has resolution 16 MPix, 12-bit, two components (just one Huffman table), one tile, demosaic pattern is RGGB, timings include decoding computations only.
DNG decoding benchmarks for single thread applications
- Adobe DNG SDK - 503 ms (32 MPix/s)
- LJ92 (library liblj92) - 340 ms (47 MPix/s)
- Fastvideo LJ Decoder - 180 ms (89 MPix/s)
The fastest result of lossless jpeg decoding could be achieved with Fastvideo LJ Decoder due to highly-optimized decompression routines. We utilize that library to speed up DNG decoding at Fast CinemaDNG Processor software. This is very important issue to insure smooth video preview from CinemaDNG footages in realtime.
Soon we will publish more info about 12/14/16-bit DNG decoding. We will also consider how to accelerate decoding for CR2, NEF, ARW and other RAW formats which are utilizing lossless jpeg compression.