A new encryption-then-compression algorithm using the rate–distortion optimization

•We propose an encryption-then-compression scheme using the lifting scheme.•We obtain the optimum quantization step via the rate–distortion optimization.•The proposed scheme outperforms the pixel-domain permutation-based prior art.•It obtains the rate–distortion performance comparable to the conventional JPEG.•The developed scheme has a low computational cost.

Compressing the encrypted data remains a challenging task in many service-oriented scenarios like distributed processing and cloud computing. This paper presents a new encryption-then-compression scheme using the rate–distortion optimization. The original image is first decomposed via the lifting scheme (LS) into a wavelet pyramid, and the coarsest and detail subbands are then encrypted with the stream and permutation ciphers, respectively. The encrypted coarsest subband is compressed using a lossless technique by the third party like network or storage service providers, while the scrambled detail subbands are compressed through the successive quantization and adaptive arithmetic coding (AC), in which quantization steps are optimized with the rate–distortion theory. Upon receiving the encrypted and compressed sequence, the receiver conducts the sequential decompression, decryption, and inverse LS to reconstruct the original image. Security analysis shows that the proposed method is able to achieve a reasonable high-level security. Experimental simulations demonstrate that the proposed scheme has a low computational cost, and it outperforms the pixel-domain permutation-based prior art and obtains the rate–distortion performance comparable to the conventional JPEG with original, unencrypted images as input.