Abstract

In this paper, we propose a new class of high-efficient deep joint source-channel coding methods that can closely adapt to the source distribution under the nonlinear transform, it can be collected under the name nonlinear transform source-channel coding (NTSCC). In the considered model, the transmitter first learns a nonlinear analysis transform to map the source data into latent space, then transmits the latent representation to the receiver via deep joint source-channel coding. Our model incorporates the nonlinear transform as a strong prior to effectively extract the source semantic features and provide side information for source-channel coding. Unlike existing conventional deep joint source-channel coding methods, the proposed NTSCC essentially learns both the source latent representation and an entropy model as the prior on the latent representation. Accordingly, novel adaptive rate transmission and hyperprior-aided codec refinement mechanisms are developed to upgrade deep joint source-channel coding. The whole system design is formulated as an optimization problem whose goal is to minimize the end-to-end transmission rate-distortion performance under established perceptual quality metrics. Across test image sources with various resolutions, we find that the proposed NTSCC transmission method generally outperforms both the analog transmission using the standard deep joint source-channel coding and the classical separation-based digital transmission. Notably, the proposed NTSCC method can potentially support future semantic communications due to its content-aware ability and perceptual optimization goal.

Experimental Results

PSNR-CBR curves on Kodak and CLIC Professional Validation Set over the AWGN channel at SNR = 10dB.

PSNR-SNR curves on the maximum channel bandwidth ratio CBR = 1/16.

Resources

　　
• Paper: arXiv
　　
• Code: Github

Citation

@article{9791398,
    author={Dai, Jincheng and Wang, Sixian and Tan, Kailin and Si, Zhongwei and Qin, Xiaoqi and Niu, Kai and Zhang, Ping},
    journal={IEEE Journal on Selected Areas in Communications},
    title={Nonlinear Transform Source-Channel Coding for Semantic Communications},
    year={2022},
    volume={40},
    number={8},
    doi={10.1109/JSAC.2022.3180802}
}