Cookies Policy
The website need some cookies and similar means to function. If you permit us, we will use those means to collect data on your visits for aggregated statistics to improve our service. Find out More
Accept Reject
  • Menu
Publications

Publications by CTM

2023

Instrument position in Immersive Audio: An empirical review of award-winning practices

Authors
Lopes, A; Barboza, JR; Bernardes, G;

Publication
2023 Immersive and 3D Audio: from Architecture to Automotive, I3DA 2023

Abstract
Immersive audio technologies have broadened postproduction strategies for spatial audio, gaining popularity among mainstream audiences. However, there is a lack of defined procedures and critical thinking regarding audio mixing guidelines for surround sound in popular music. In this context, we conducted an empirical study to identify trends concerning instrument position, trajectories, and dynamics from surround mixings. Furthermore, we assess the degree to which they differ from their stereo renderings. Seven award-winning songs in the Grammy category for Best Immersive Album were analyzed, including surround 5.1 and stereo versions. The study found consistent instrument positions in the songs, with rhythmic instruments and bass in the center, lead vocals spread across front channels, and harmonic instruments in wider positions. Solo instruments occupied left, right, and center channels, with dynamics emphasizing lead vocals and solos. Trajectories were rarely used, indicating channel-based thinking. Limited adoption of immersive audio dimensions and reliance on stereo techniques were observed, with no notable differences between the surround and stereo versions. Identified song outliers are discussed and offer avenues for exploration, highlighting the importance of diverse musical expressions in informing immersive audio mixing. © 2023 IEEE.

2023

Sound Design Impacts User Experience and Attention in Serious Game

Authors
Cao, Z; Magalhães, E; Bernardes, G;

Publication
Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

Abstract
We study the impact of sound design – soundscape, sound effects, and auditory notifications, namely earcons – on the player’s experience of serious games. Three sound design versions for the game Venci’s Adventures have been developed: 1) no sound; 2) standard sound design, including soundscapes and sound effects; and 3) standard sound design with auditory notification (namely, earcons). Perceptual experiments were conducted to evaluate the most suitable attention retention earcons from a diverse collection of timbres, pitch, and melodic patterns, as well as the user experience of the different sound design versions assessed in pairs (1 vs. 2 and 2 vs. 3). Our results show that participants (n= 23 ) perceive better user experience in terms of game playing competence, immersion, flow, challenge and affect, and enhanced attention retention when adopting standard sound design with the earcons. © 2023, The Author(s), under exclusive license to Springer Nature Switzerland AG.

2023

Computational Similarity of Portuguese Folk Melodies Using Hierarchical Reduction

Authors
Carvalho, N; Diogo, D; Bernardes, G;

Publication
THE 10TH INTERNATIONAL CONFERENCE ON DIGITAL LIBRARIES FOR MUSICOLOGY, DLFM 2023

Abstract
We propose a method for computing the similarity of symbolically-encoded Portuguese folk melodies. The main novelty of our method is the use of a preprocessing melodic reduction at multiple hierarchies to filter the surface of folk melodies according to 1) pitch stability, 2) interval salience, 3) beat strength, 4) durational accents, and 5) the linear combination of all former criteria. Based on the salience of each note event per criteria, we create three melodic reductions with three different levels of note retention. We assess the degree to which six folk music similarity measures at multiple reduction hierarchies comply with collected ground truth from experts in Portuguese folk music. The results show that SIAM combined with 75th quantile reduction using the combined or durational accents best models the similarity for a corpus of Portuguese folk melodies by capturing approximately 84-90% of the variance observed in ground truth annotations.

2023

En train d'oublier: Toward affective virtual environments

Authors
Forero, J; Mendes, M; Bernardes, G;

Publication
ACM International Conference Proceeding Series

Abstract
This study explores the development of intelligent affective virtual environments generated by bimodal emotion recognition techniques and multimodal feedback. A semantic and acoustic analysis predicts emotions conveyed by spoken language, fostering an expressive and transparent control structure. Textual contents and emotional predictions are mapped to virtual environments in real locations as audiovisual feedback. To demonstrate the application of this system, we developed a case study titled "En train d'oublier,"focusing on a train cemetery in Uyuni, Bolivia. The train cemetery holds historical significance as a site where abandoned trains symbolize the passage of time and the interaction between human activities and nature's reclamation. The space is transformed into an immersive and emotionally poetic experience through oral language and affective virtual environments that activate memories, as the system utilizes the transcribed text to synthesize images and modifies the musical output based on the predicted emotional states. The proposed bimodal emotion recognition techniques achieve 94% and 89% accuracy. The audiovisual mapping strategy allows for considering divergence in predictions generating an intended tension between the graphical and the musical representation. Using video and web art techniques, we experimented with the environments generated to create diverses poetic proposals. © 2023 ACM.

2023

TEC4SEA-Developing maritime technology for a sustainable blue economy

Authors
Monica, P; Cruz, N; Almeida, JM; Silva, A; Silva, E; Pinho, C; Almeida, C; Viegas, D; Pessoa, LM; Lima, AP; Martins, A; Zabel, F; Ferreira, BM; Dias, I; Campos, R; Araujo, J; Coelho, LC; Jorge, PS; Mendes, J;

Publication
OCEANS 2023 - LIMERICK

Abstract
One way to mitigate the high costs of doing science or business at sea is to create technological infrastructures possessing all the skills and resources needed for successful maritime operations, and make those capabilities and skills available to the external entities requiring them. By doing so, the individual economic and scientific agents can be spared the enormous effort of creating and maintaining their own, particular set of equivalent capabilities, thus drastically lowering their initial operating costs. In addition to cost savings, operating based on fully-fledged, shared infrastructures not only allows the use of more advanced scientific equipment and highly skilled personnel, but it also enables the business teams (be it industry or research) to focus on their goals, rather than on equipment, logistics, and support. This paper will describe the TEC4SEA infrastructure, created precisely to operate as described. This infrastructure has been under implementation in the last few years, and has now entered its operational phase. This paper will describe it, present its current portfolio of services, and discuss the most relevant assets and facilities that have been recently acquired, so that the research and industrial communities requiring the use of such assets can fully evaluate their adequacy for their own purposes and projects.

2023

DURIUS: A Multimodal Underwater Communications Approach for Higher Performance and Lower Energy Consumption

Authors
Loureiro, JP; Teixeira, FB; Campos, R;

Publication
2023 IEEE 9TH WORLD FORUM ON INTERNET OF THINGS, WF-IOT

Abstract
The exploration of the ocean has got an increasing interest, including activities such as offshore wind farms and deep-sea mining. However, the ocean environment and the high cost of operations, namely for manned missions, have led to the development of Autonomous Underwater Vehicles (AUVs) and other sensing platforms. AUVs play a vital role in these environments, relying on communications systems to operate and exchange sensor data. Yet, reliable and energy-efficient broad-band wireless communications underwater remain an unsolved challenge, despite the recent advances in the field. We present a novel multimodal approach, named DURIUS, that considers the movement of the AUV to convey the sensor data and selects the most suitable underwater wireless communications technology - acoustic, optical or radio - according to the underwater context, targeting maximum performance and minimum energy consumption. Our analytical results show that DURIUS increases data throughput and reduces energy consumption when compared with the state of the art approaches.

  • 45
  • 318