Abstract

Abstract
Telecommunications and computer vision solutions have evolved significantly in recent years, allowing a huge advance in the functionalities and applications offered. However, these two fields have been making their way as separate areas, not exploring the potential benefits of merging the innovations brought from each of them. In challenging environments, for example, combining radio sensing and computer vision can strongly contribute to solving problems such as those introduced by obstructions or limited lighting. Machine learning algorithms, able to fuse heterogeneous and multi-modal data, are also a key element for understanding and inferring additional knowledge from raw and low-level data, able to create a new abstracting level that can significantly enhance many applications. This paper introduces the CONVERGE vision-radio concept, a new paradigm that explores the benefits of integrating two fields of knowledge towards the vision of View-to-Communicate, Communicate-to-View. The main concepts behind this vision, including supporting use cases and the proposed architecture, are presented. CONVERGE introduces a set of tools integrating wireless communications and computer vision to create a novel experimental infrastructure that will provide open datasets to the scientific community of both experimental and simulated data, enabling new research addressing various 6 G verticals, including telecommunications, automotive, manufacturing, media, and health.

Abstract
Audio-visual correlation learning aims at capturing and understanding natural phenomena between audio and visual data. The rapid growth of Deep Learning propelled the development of proposals that process audio-visual data and can be observed in the number of proposals in the past years. Thus encouraging the development of a comprehensive survey. Besides analyzing the models used in this context, we also discuss some tasks of definition and paradigm applied in AI multimedia. In addition, we investigate objective functions frequently used and discuss how audio-visual data is exploited in the optimization process, i.e., the different methodologies for representing knowledge in the audio-visual domain. In fact, we focus on how human-understandable mechanisms, i.e., structured knowledge that reflects comprehensible knowledge, can guide the learning process. Most importantly, we provide a summarization of the recent progress of Audio-Visual Correlation Learning (AVCL) and discuss the future research directions.

Abstract
There was an error in the original publication [1]. The statement in the Acknowledgments section is incorrect and should be removed because the official start of the project WATSON was after the paper’s publication date. The authors state that the scientific conclusions are unaffected. This correction was approved by the Academic Editor. The original publication has also been updated. © 2025 by the authors.

Abstract
This work introduces Dialogue-AV, a benchmarking dataset for Audio-Video-Language (AVL). We propose using dialogue to describe video content instead of single captions, capturing nuances and shared meanings between audio and visual elements. This approach contributes significantly to improving the diversity of video descriptions and enables comprehensive evaluation of AVL learning across different downstream tasks, such as Cross-Modal Retrieval, Visual Question-Answering, and Video Captioning. Our dataset comprises approximately 258k audiovisual samples accompanied by dialogue-based descriptions for benchmarking. Dialogue-AV builds upon existing State-of-the-Art (SOTA) datasets that feature human-generated descriptions, enhancing them with model-generated ones that describe all modalities. We also present zero-shot baseline results utilising SOTA Visual-Language Models (VLMs), demonstrating that Dialogue-AV is capable of benchmarking a variety of downstream tasks with diverse inputs. Our key contributions include: 1) Dialogue-AV, a benchmark dataset for dialogue-based AVL models; and 2) benchmarks that expose the limitations of current SOTA VLMs. The code and dataset are accessible at: github.com/lvilaca16/dialogue-av. © 2025 IEEE.