Cookies Policy
The website need some cookies and similar means to function. If you permit us, we will use those means to collect data on your visits for aggregated statistics to improve our service. Find out More
Accept Reject
  • Menu
Publications

Publications by CTM

2021

Hybrid Conference Experiences in the ARENA

Authors
Pereira N.; Rowe A.; Farb M.W.; Liang I.; Lu E.; Riebling E.;

Publication
Proceedings - 2021 IEEE International Symposium on Mixed and Augmented Reality Adjunct, ISMAR-Adjunct 2021

Abstract
We propose supporting hybrid conference experiences using the Augmented Reality Edge Network Architecture (ARENA). ARENA is a platform based on web technologies that simplifies the creation of collaborative mixed reality for standard Web Browsers (Chrome, Firefox) in VR, Headset AR/VR Browsers (Magic Leap, Hololens, Oculus Quest 2), and mobile AR (WebXR Viewer for iOS, Chrome with experimental flags for Android, and our own custom WebXR fork for iOS). We use a 3D scan of the conference venue as the backdrop environment for remote users and a model to stage various AR interactions for in-person users. Remote participants can use VR in a browser or a VR headset to navigate the scene. In-person participants can use AR headsets or mobile AR through WebXR browsers to see and hear remote users. ARENA can scale up to hundreds of users in the same scene and provides audio and video with spatial sound that can more closely capture real-world interactions.

2021

ARENA: The Augmented Reality Edge Networking Architecture

Authors
Pereira, N; Rowe, A; Farb, MW; Liang, I; Lu, E; Riebling, E;

Publication
2021 IEEE INTERNATIONAL SYMPOSIUM ON MIXED AND AUGMENTED REALITY (ISMAR 2021)

Abstract
Many have predicted the future of the Web to be the integration of Web content with the real-world through technologies such as Augmented Reality (AR). This has led to the rise of Extended Reality (XR) Web Browsers used to shorten the long AR application development and deployment cycle of native applications especially across different platforms. As XR Browsers mature, we face new challenges related to collaborative and multi-user applications that span users, devices, and machines. These collaborative XR applications require: (1) networking support for scaling to many users, (2) mechanisms for content access control and application isolation, and (3) the ability to host application logic near clients or data sources to reduce application latency. In this paper, we present the design and evaluation of the AR Edge Networking Architecture (ARENA) which is a platform that simplifies building and hosting collaborative XR applications on WebXR capable browsers. ARENA provides a number of critical components including: a hierarchical geospatial directory service that connects users to nearby servers and content, a token-based authentication system for controlling user access to content, and an application/service runtime supervisor that can dispatch programs across any network connected device. All of the content within ARENA exists as endpoints in a PubSub scene graph model that is synchronized across all users. We evaluate ARENA in terms of client performance as well as benchmark end-to-end response-time as load on the system scales. We show the ability to horizontally scale the system to Internet-scale with scenes containing hundreds of users and latencies on the order of tens of milliseconds. Finally, we highlight projects built using ARENA and showcase how our approach dramatically simplifies collaborative multi-user XR development compared to monolithic approaches.

2021

Teaching Programming with a Limited Infrastructure

Authors
Ferreira, P; Nogueira, L; Pereira, N; Maia, C; Fernandes, M; Andrade, A; Faria, R; Goncalves, C;

Publication
2021 WORLD ENGINEERING EDUCATION FORUM/GLOBAL ENGINEERING DEANS COUNCIL (WEEF/GEDC)

Abstract
Programming courses are needed for an increasing number of students in the Higher Education Institutions of today. Of all the programming languages covered in typical courses, the C and Assembly languages are among the most critical. As they are very low level languages, their knowledge helps the students to understand the inner workings of a computer. At the same time, their differences from other programming languages, demands from the learner a serious adjustment of the mental model. As the programming tools and environments are also different, there is the need of supporting the students in their learning, using a minimum of infrastructure, due to financial restrictions, and to support the maximum number of students, with the existing resources. The use of a Virtual Machine based on a Live Linux distribution, together with an enhanced set of software tests can provide students with an easy to install development platform, providing a good amount feedback, with very limited network usage. The methods described in this paper have been applied with good results, and can be used to support live or online classes.

2021

On Filter Generalization for Music Bandwidth Extension Using Deep Neural Networks

Authors
Sulun, S; Davies, MEP;

Publication
IEEE JOURNAL OF SELECTED TOPICS IN SIGNAL PROCESSING

Abstract
In this paper, we address a subtopic of the broad domain of audio enhancement, namely musical audio bandwidth extension. We formulate the bandwidth extension problem using deep neural networks, where a band-limited signal is provided as input to the network, with the goal of reconstructing a full-bandwidth output. Our main contribution centers on the impact of the choice of low-pass filter when training and subsequently testing the network. For two different state-of-the-art deep architectures, ResNet and U-Net, we demonstrate that when the training and testing filters are matched, improvements in signal-to-noise ratio (SNR) of up to 7 dB can be obtained. However, when these filters differ, the improvement falls considerably and under some training conditions results in a lower SNR than the band-limited input. To circumvent this apparent overfitting to filter shape, we propose a data augmentation strategy which utilizes multiple low-pass filters during training and leads to improved generalization to unseen filtering conditions at test time.

2021

Explainability Metrics of Deep Convolutional Networks for Photoplethysmography Quality Assessment

Authors
Zhang, O; Ding, C; Pereira, T; Xiao, R; Gadhoumi, K; Meisel, K; Lee, RJ; Chen, YR; Hu, X;

Publication
IEEE ACCESS

Abstract
Photoplethysmography (PPG) is a noninvasive way to monitor various aspects of the circulatory system, and is becoming more and more widespread in biomedical processing. Recently, deep learning methods for analyzing PPG have also become prevalent, achieving state of the art results on heart rate estimation, atrial fibrillation detection, and motion artifact identification. Consequently, a need for interpretable deep learning has arisen within the field of biomedical signal processing. In this paper, we pioneer novel explanatory metrics which leverage domain-expert knowledge to validate a deep learning model. We visualize model attention over a whole testset using saliency methods and compare it to human expert annotations. Congruence, our first metric, measures the proportion of model attention within expert-annotated regions. Our second metric, Annotation Classification, measures how much of the expert annotations our deep learning model pays attention to. Finally, we apply our metrics to compare between a signal based model and an image based model for PPG signal quality classification. Both models are deep convolutional networks based on the ResNet architectures. We show that our signal-based one dimensional model acts in a more explainable manner than our image based model; on average 50.78% of the one dimensional model's attention are within expert annotations, whereas 36.03% of the two dimensional model's attention are within expert annotations. Similarly, when thresholding the one dimensional model attention, one can more accurately predict if each pixel of the PPG is annotated as artifactual by an expert. Through this testcase, we demonstrate how our metrics can provide a quantitative and dataset-wide analysis of how explainable the model is.

2021

Applying Machine Learning for Traffic Forecasting in Porto, Portugal

Authors
Maia, P; Morgado, J; Goncalves, T; Albuquerque, T;

Publication
MACHINE LEARNING AND PRINCIPLES AND PRACTICE OF KNOWLEDGE DISCOVERY IN DATABASES, PT II

Abstract
Pollutant emissions from passenger cars give rise to harmful effects on human health and the environment. Predicting traffic flow is a challenging problem, but essential to understand what factors influence car traffic and what measures should be taken to reduce carbon dioxide emissions. In this work, we developed a predictive model to forecast traffic flow in several locations in the city of Porto for 24 h later, i.e., the next day at the same time. We trained a XGBoost Regressor with multi-modal data from 2018 and 2019 obtained from traffic and weather sensors of the city of Porto and the geographic location of several points of interest. The proposed model achieved a mean absolute error, mean square error, Spearman's rank correlation coefficient, and Pearson correlation coefficient equal to 80.59, 65395, 0.9162, and 0.7816, respectively, when tested on the test set. The developed model makes it possible to analyse which areas of the city of Porto will have more traffic the next day and take measures to optimise this increasing flow of cars. One of the ideas present in the literature is to develop intelligent traffic lights that change their timers according to the expected traffic in the area. This system could help decrease the levels of carbon dioxide emitted and therefore decrease its harmful effects on the health of the population and the environment.

  • 139
  • 402