2021
Autores
Barboza, JR; Magalhaes, E; Bernardes, G;
Publicação
2021 IMMERSIVE AND 3D AUDIO: FROM ARCHITECTURE TO AUTOMOTIVE (I3DA)
Abstract
Since the beginning of the XXI century, we have been witnessing a significant shift in the media landscape towards enhanced immersive audiovisual manifestations, from controlled research environments to gradual production market penetration. Virtual reality, augmented reality, mixed reality, extended reality, 360 degrees video, and digital games are representative examples of these immersive technologies. Spatial audio design and production are instrumental to the immersive experience. As Ambisonics techniques do potentially mean more expense - in memory, processing power, and production budget -, limited exploration in the development of new composition and production methodologies across popular music production has been considered beyond the traditional stereophonic format. Our work details a post-production case study using spatial audio, namely High Order Ambisonics. The case study is a Brazilian popular song, remixed using 3rd order Ambisonics from a multitrack recording session composed of monophonic and stereophonic audio tracks. The song encompasses a unique approach for audio spatialization guided by hierarchical audio content attributes across multiple structural time scales and musical contexts. The evaluation of our production process adopted iterative heuristic assessments comparing technical decisions and aesthetic intentions in fostering an augmented spatial audio song. A set of technical guidelines and good practices on how and why to positioning audio in space are abstracted from our case study evaluation, which critically advances the theory and practice of popular musical audio production in immersive technologies.
2021
Autores
Lu E.; Miller J.; Pereira N.; Rowe A.;
Publicação
Proceedings - 2021 IEEE International Symposium on Mixed and Augmented Reality, ISMAR 2021
Abstract
Public spaces like concert stadiums and sporting arenas are ideal venues for AR content delivery to crowds of mobile phone users. Unfortunately, these environments tend to be some of the most challenging in terms of lighting and dynamic staging for vision-based relocalization. In this paper, we introduce FLASH1, a system for delivering AR content within challenging lighting environments that uses active tags (i.e., blinking) with detectable features from passive tags (quads) for marking regions of interest and determining pose. This combination allows the tags to be detectable from long distances with significantly less computational overhead per frame, making it possible to embed tags in existing video displays like large jumbotrons. To aid in pose acquisition, we implement a gravity-assisted pose solver that removes the ambiguous solutions that are often encountered when trying to localize using standard passive tags. We show that our technique outperforms similarly sized passive tags in terms of range by 20-30% and is fast enough to run at 30 FPS even within a mobile web browser on a smartphone.
2021
Autores
Pereira N.; Rowe A.; Farb M.W.; Liang I.; Lu E.; Riebling E.;
Publicação
Proceedings - 2021 IEEE International Symposium on Mixed and Augmented Reality Adjunct, ISMAR-Adjunct 2021
Abstract
We propose supporting hybrid conference experiences using the Augmented Reality Edge Network Architecture (ARENA). ARENA is a platform based on web technologies that simplifies the creation of collaborative mixed reality for standard Web Browsers (Chrome, Firefox) in VR, Headset AR/VR Browsers (Magic Leap, Hololens, Oculus Quest 2), and mobile AR (WebXR Viewer for iOS, Chrome with experimental flags for Android, and our own custom WebXR fork for iOS). We use a 3D scan of the conference venue as the backdrop environment for remote users and a model to stage various AR interactions for in-person users. Remote participants can use VR in a browser or a VR headset to navigate the scene. In-person participants can use AR headsets or mobile AR through WebXR browsers to see and hear remote users. ARENA can scale up to hundreds of users in the same scene and provides audio and video with spatial sound that can more closely capture real-world interactions.
2021
Autores
Pereira, N; Rowe, A; Farb, MW; Liang, I; Lu, E; Riebling, E;
Publicação
2021 IEEE INTERNATIONAL SYMPOSIUM ON MIXED AND AUGMENTED REALITY (ISMAR 2021)
Abstract
Many have predicted the future of the Web to be the integration of Web content with the real-world through technologies such as Augmented Reality (AR). This has led to the rise of Extended Reality (XR) Web Browsers used to shorten the long AR application development and deployment cycle of native applications especially across different platforms. As XR Browsers mature, we face new challenges related to collaborative and multi-user applications that span users, devices, and machines. These collaborative XR applications require: (1) networking support for scaling to many users, (2) mechanisms for content access control and application isolation, and (3) the ability to host application logic near clients or data sources to reduce application latency. In this paper, we present the design and evaluation of the AR Edge Networking Architecture (ARENA) which is a platform that simplifies building and hosting collaborative XR applications on WebXR capable browsers. ARENA provides a number of critical components including: a hierarchical geospatial directory service that connects users to nearby servers and content, a token-based authentication system for controlling user access to content, and an application/service runtime supervisor that can dispatch programs across any network connected device. All of the content within ARENA exists as endpoints in a PubSub scene graph model that is synchronized across all users. We evaluate ARENA in terms of client performance as well as benchmark end-to-end response-time as load on the system scales. We show the ability to horizontally scale the system to Internet-scale with scenes containing hundreds of users and latencies on the order of tens of milliseconds. Finally, we highlight projects built using ARENA and showcase how our approach dramatically simplifies collaborative multi-user XR development compared to monolithic approaches.
2021
Autores
Ferreira, P; Nogueira, L; Pereira, N; Maia, C; Fernandes, M; Andrade, A; Faria, R; Goncalves, C;
Publicação
2021 WORLD ENGINEERING EDUCATION FORUM/GLOBAL ENGINEERING DEANS COUNCIL (WEEF/GEDC)
Abstract
Programming courses are needed for an increasing number of students in the Higher Education Institutions of today. Of all the programming languages covered in typical courses, the C and Assembly languages are among the most critical. As they are very low level languages, their knowledge helps the students to understand the inner workings of a computer. At the same time, their differences from other programming languages, demands from the learner a serious adjustment of the mental model. As the programming tools and environments are also different, there is the need of supporting the students in their learning, using a minimum of infrastructure, due to financial restrictions, and to support the maximum number of students, with the existing resources. The use of a Virtual Machine based on a Live Linux distribution, together with an enhanced set of software tests can provide students with an easy to install development platform, providing a good amount feedback, with very limited network usage. The methods described in this paper have been applied with good results, and can be used to support live or online classes.
2021
Autores
Sulun, S; Davies, MEP;
Publicação
IEEE JOURNAL OF SELECTED TOPICS IN SIGNAL PROCESSING
Abstract
In this paper, we address a subtopic of the broad domain of audio enhancement, namely musical audio bandwidth extension. We formulate the bandwidth extension problem using deep neural networks, where a band-limited signal is provided as input to the network, with the goal of reconstructing a full-bandwidth output. Our main contribution centers on the impact of the choice of low-pass filter when training and subsequently testing the network. For two different state-of-the-art deep architectures, ResNet and U-Net, we demonstrate that when the training and testing filters are matched, improvements in signal-to-noise ratio (SNR) of up to 7 dB can be obtained. However, when these filters differ, the improvement falls considerably and under some training conditions results in a lower SNR than the band-limited input. To circumvent this apparent overfitting to filter shape, we propose a data augmentation strategy which utilizes multiple low-pass filters during training and leads to improved generalization to unseen filtering conditions at test time.
The access to the final selection minute is only available to applicants.
Please check the confirmation e-mail of your application to obtain the access code.