Cookies Policy
The website need some cookies and similar means to function. If you permit us, we will use those means to collect data on your visits for aggregated statistics to improve our service. Find out More
Accept Reject
  • Menu
Publications

Publications by CTM

2022

Emotional machines: Toward affective virtual environments

Authors
Forero, J; Bernardes, G; Mendes, M;

Publication
MM 2022 - Proceedings of the 30th ACM International Conference on Multimedia

Abstract
Emotional Machines is an interactive installation that builds affective virtual environments through spoken language. In response to the existing limitations of emotion recognition models incorporating computer vision and electrophysiological activity, whose sources are hindered by a head-mounted display, we propose the adoption of speech emotion recognition (from the audio signal) and semantic sentiment analysis. In detail, we use two machine learning models to predict three main emotional categories from high-level semantic and low-level speech features. Output emotions are mapped to audiovisual representation by an end-To-end process. We use a generative model of chord progressions to transfer speech emotion into music and a synthesized image from the text (transcribed from the user's speech). The generated image is used as the style source in the style-Transfer process onto an equirectangular projection image target selected for each emotional category. The installation is an immersive virtual space encapsulating emotions in spheres disposed into a 3D environment. Thus, users can create new affective representations or interact with other previous encoded instances using joysticks. © 2022 Owner/Author.

2022

Leveraging compatibility and diversity in computer-aided music mashup creation

Authors
Bernardo, G; Bernardes, G;

Publication
Personal and Ubiquitous Computing

Abstract
AbstractWe advance Mixmash-AIS, a multimodal optimization music mashup creation model for loop recombination at scale. Our motivation is to (1) tackle current scalability limitations in state-of-the-art (brute force) computational mashup models while enforcing the (2) compatibility of audio loops and (3) a pool of diverse mashups that can accommodate user preferences. To this end, we adopt the artificial immune system (AIS) opt-aiNet algorithm to efficiently compute a population of compatible and diverse music mashups from loop recombinations. Optimal mashups result from local minima in a feature space representing harmonic, rhythmic, and spectral musical audio compatibility. We objectively assess the compatibility, diversity, and computational performance of Mixmash-AIS generated mashups compared to a standard genetic algorithm (GA) and a brute force (BF) approach. Furthermore, we conducted a perceptual test to validate the objective evaluation function within Mixmash-AIS in capturing user enjoyment of the computer-generated loop mashups. Our results show that while the GA stands as the most efficient algorithm, the AIS opt-aiNet outperforms both the GA and BF approaches in terms of compatibility and diversity. Our listening test has shown that Mixmash-AIS objective evaluation function significantly captures the perceptual compatibility of loop mashups (p < .001).

2022

FluidHarmony: Defining an equal-tempered and hierarchical harmonic lexicon in the Fourier space

Authors
Bernardes, G; Carvalho, N; Pereira, S;

Publication
JOURNAL OF NEW MUSIC RESEARCH

Abstract
FluidHarmony is an algorithmic method for defining a hierarchical harmonic lexicon in equal temperaments. It utilizes an enharmonic weighted Fourier transform space to represent pitch class set (pcsets) relations. The method ranks pcsets based on user-defined constraints: the importance of interval classes (ICs) and a reference pcset. Evaluation of 5,184 Western musical pieces from the 16th to 20th centuries shows FluidHarmony captures 8% of the corpus's harmony in its top pcsets. This highlights the role of ICs and a reference pcset in regulating harmony in Western tonal music while enabling systematic approaches to define hierarchies and establish metrics beyond 12-TET.

2022

MID-LEVEL HARMONIC AUDIO FEATURES FOR MUSICAL STYLE CLASSIFICATION

Authors
Almeida, F; Bernardes, G; Weiû, C;

Publication
Proceedings of the 23rd International Society for Music Information Retrieval Conference, ISMIR 2022

Abstract
The extraction of harmonic information from musical audio is fundamental for several music information retrieval tasks. In this paper, we propose novel harmonic audio features based on the perceptually-inspired tonal interval vector space, computed as the Fourier transform of chroma vectors. Our contribution includes mid-level features for musical dissonance, chromaticity, dyadicity, triadicity, diminished quality, diatonicity, and whole-toneness. Moreover, we quantify the perceptual relationship between short- and long-term harmonic structures, tonal dispersion, harmonic changes, and complexity. Beyond the computation on fixed-size windows, we propose a context-sensitive harmonic segmentation approach. We assess the robustness of the new harmonic features in style classification tasks regarding classical music periods and composers. Our results align with, slightly outperforming, existing features and suggest that other musical properties than those in state-of-the-art literature are partially captured. We discuss the features regarding their musical interpretation and compare the different feature groups regarding their effectiveness for discriminating classical music periods and composers. © F. Almeida, G. Bernardes, and C. Weiû.

2022

Proof of Concept of a Low-Cost Beam-Steering Hybrid Reflectarray that Mixes Microstrip and Lens Elements Using Passive Demonstrators

Authors
Luo, Q; Gao, S; Hu, W; Liu, W; Pessoa, LM; Sobhy, M; Sun, YC;

Publication
IEEE COMMUNICATIONS MAGAZINE

Abstract
In this article, a proof-of-concept study on the use of a hybrid design technique to reduce the number of phase shifters of a beam-scanning reflectarray (RA) is presented. An extended hemispherical lens antenna with feeds inspired by the retrodirective array is developed as a reflecting element, and the hybrid design technique mixes the lenses with the microstrip patch elements to realize a reflecting surface. Compared to the conventional designs that only use microstrip antennas to realize a reflecting surface, given a fixed aperture size the presented design uses 25 percent fewer array elements while shows comparable beam-steering performance. As a result of using fewer elements, the number of required phase shifters or other equivalent components such as RF switches and tunable materials is reduced by 25 percent, which leads to the reduction of the overall antenna system's complexity, cost, and power consumption. To verify the design concept, two passive prototypes with a center frequency at 12.5 GHz were designed and fabricated. The reflecting surface was fabricated by using standard PCB manufacturing and the lenses were fabricated using 3D printing. Good agreement between the simulation and measurement results is obtained. The presented design concept can be extended to the design of RAs operating at different frequency bands including millimetre-wave frequencies with similar radiation performances. The presented design method is not limited to the microstrip patch reflecting elements and can also be applied to the design of the hybrid RAs with different types of reflecting elements.

2022

Editorial of the Special Issue from WorldCIST'20

Authors
Domingues, I; Sequeira, AF;

Publication
COMPUTATIONAL AND MATHEMATICAL ORGANIZATION THEORY

Abstract

  • 82
  • 368