Trustworthy image semantic communication with GenAI: Explainablity, controllability, and efficiency

Wang, Xijun; Ye, Doshan; Feng, Chenyuan; Yang, Howard H.; Chen, Xiang; Quek, Tony Q. S.
IEEE Wireless Communication (WCM), 31 July 2024

Image semantic communication (ISC) has garnered significant attention for its potential to achieve high efficiency in visual content transmission. However, existing ISC systems based on joint source-channel coding face challenges in interpretability, operability, and compatibility. To address these limitations, we propose a novel trustworthy ISC framework. This approach leverages text extraction and segmentation mapping techniques to convert images into explainable semantics, while employing Gen-erative Artificial Intelligence (GenAI) for multiple downstream inference tasks. We also introduce a multi-rate ISC transmission protocol that dynamically adapts to both the received explainable semantic content and specific task requirements at the receiver. Simulation results demonstrate that our framework achieves explainable learning, decoupled training, and compatible trans-mission in various application scenarios. Finally, some intriguing research directions and application scenarios are identified.


Type:
Journal
Date:
2024-07-31
Department:
Communication systems
Eurecom Ref:
7851
Copyright:
© 2024 IEEE. Personal use of this material is permitted. However, permission to reprint/republish this material for advertising or promotional purposes or for creating new collective works for resale or redistribution to servers or lists, or to reuse any copyrighted component of this work in other works must be obtained from the IEEE.
See also:

PERMALINK : https://www.eurecom.fr/publication/7851