Cookies Policy
The website need some cookies and similar means to function. If you permit us, we will use those means to collect data on your visits for aggregated statistics to improve our service. Find out More
Accept Reject
  • Menu
Publications

Publications by CTM

2024

Design and Usability Assessment of Multimodal Augmented Reality System for Gait Training

Authors
Pinheiro, C; Figueiredo, J; Pereira, T; Santos, CP;

Publication
ROBOT 2023: SIXTH IBERIAN ROBOTICS CONFERENCE, VOL 2

Abstract
Biofeedback is a promising tool to complement conventional physical therapy by fostering active participation of neurologically impaired patients during treatment. This work aims at a user-centered design and usability assessment for different age groups of a novel wearable augmented reality application composed of a multimodal sensor network and corresponding control strategies for personalized biofeedback during gait training. The proposed solution includes wearable AR glasses that deliver visual cues controlled in real-time according to mediolateral center of mass position, sagittal ankle angle, or tibialis anterior muscle activity from inertial and EMG sensors. Control strategies include positive and negative reinforcement conditions and are based on the user's performance by comparing real-time sensor data with an automatically user-personalized threshold. The proposed solution allows ambulatory practice on daily scenarios, physiotherapists' involvement through a laptop screen, and contributes to further benchmark biofeedback regarding the type of sensor. Although old healthy adults with low academic degrees have a preference for guidance from an expert person, excellent usability scores (SUS scores: 81.25-96.87) were achieved with young and middle-aged healthy adults and one neurologically impaired patient.

2024

A Semantic-oriented Approach for Underwater Wireless Communications using Generative AI

Authors
Loureiro, JP; Mateus, A; Teixeira, FB; Campos, R;

Publication
2024 15TH IFIP WIRELESS AND MOBILE NETWORKING CONFERENCE, WMNC

Abstract
Underwater wireless communications are crucial for supporting multiple maritime activities, such as environmental monitoring and offshore wind farms. However, the challenging underwater environment continues to pose obstacles to the development of long-range, broadband underwater wireless communication systems. State of the art solutions are limited to long range, narrowband acoustics and short range, broadband radio or optical communications. This precludes real-time wireless transmission of imagery over long distances. In this paper, we propose SAGE, a semantic-oriented underwater communications approach to enable real-time wireless imagery transmission over noisy and narrowband channels. SAGE extracts semantically relevant information from images at the sender located underwater and generates a text description that is transmitted to the receiver at the surface, which in turn generates an image from the received text description. SAGE is evaluated using BLIP for image-to-text and Stable Diffusion for text-to-image, showing promising image similarity between the original and the generated images, and a significant reduction in latency up to a hundred-fold, encouraging further research in this area.

2024

<i>DeViL</i>: Decoding Vision features into Language

Authors
Dani, M; Rio Torto, I; Alaniz, S; Akata, Z;

Publication
PATTERN RECOGNITION, DAGM GCPR 2023

Abstract
Post-hoc explanation methods have often been criticised for abstracting away the decision-making process of deep neural networks. In this work, we would like to provide natural language descriptions for what different layers of a vision backbone have learned. Our DeViL method generates textual descriptions of visual features at different layers of the network as well as highlights the attribution locations of learned concepts. We train a transformer network to translate individual image features of any vision layer into a prompt that a separate off-the-shelf language model decodes into natural language. By employing dropout both per-layer and per-spatial-location, our model can generalize training on image-text pairs to generate localized explanations. As it uses a pre-trained language model, our approach is fast to train and can be applied to any vision backbone. Moreover, DeViL can create open-vocabulary attribution maps corresponding to words or phrases even outside the training scope of the vision model. We demonstrate that DeViL generates textual descriptions relevant to the image content on CC3M, surpassing previous lightweight captioning models and attribution maps, uncovering the learned concepts of the vision backbone. Further, we analyze fine-grained descriptions of layers as well as specific spatial locations and show that DeViL outperforms the current state-of-the-art on the neuron-wise descriptions of the MILANNOTATIONS dataset.

2024

An interpretable machine learning system for colorectal cancer diagnosis from pathology slides

Authors
Neto, PC; Montezuma, D; Oliveira, SP; Oliveira, D; Fraga, J; Monteiro, A; Monteiro, J; Ribeiro, L; Gonçalves, S; Reinhard, S; Zlobec, I; Pinto, IM; Cardoso, JS;

Publication
NPJ PRECISION ONCOLOGY

Abstract
Considering the profound transformation affecting pathology practice, we aimed to develop a scalable artificial intelligence (AI) system to diagnose colorectal cancer from whole-slide images (WSI). For this, we propose a deep learning (DL) system that learns from weak labels, a sampling strategy that reduces the number of training samples by a factor of six without compromising performance, an approach to leverage a small subset of fully annotated samples, and a prototype with explainable predictions, active learning features and parallelisation. Noting some problems in the literature, this study is conducted with one of the largest WSI colorectal samples dataset with approximately 10,500 WSIs. Of these samples, 900 are testing samples. Furthermore, the robustness of the proposed method is assessed with two additional external datasets (TCGA and PAIP) and a dataset of samples collected directly from the proposed prototype. Our proposed method predicts, for the patch-based tiles, a class based on the severity of the dysplasia and uses that information to classify the whole slide. It is trained with an interpretable mixed-supervision scheme to leverage the domain knowledge introduced by pathologists through spatial annotations. The mixed-supervision scheme allowed for an intelligent sampling strategy effectively evaluated in several different scenarios without compromising the performance. On the internal dataset, the method shows an accuracy of 93.44% and a sensitivity between positive (low-grade and high-grade dysplasia) and non-neoplastic samples of 0.996. On the external test samples varied with TCGA being the most challenging dataset with an overall accuracy of 84.91% and a sensitivity of 0.996.

2024

Performance update of the combined GNAO plus GIRMOS imaging system based on the newly derived adaptive optics bench

Authors
Lamb, M; Sivo, G; Sivanandam, S; Tschimmel, M; Scharwachter, J; McConnachie, A; Muzzin, A; Jouve, P; Correia, C;

Publication
ADAPTIVE OPTICS SYSTEMS IX

Abstract
The GNAO facility is an upcoming adaptive optics (AO) system for the Gemini North Telescope. It will deliver both wide and narrow field AO capabilities to its first light instrument GIRMOS. GIRMOS is a multi-object AO (MOAO) instrument that houses four near infrared (NIR) IFU spectrographs and a NIR imager similar to GSAOI at Gemini South. The required sensitivity of the combined system is largely driven by rapid transient followup AO-corrected Imaging and the required sensitivity is in part driven by the performance of the AO system. Up until recently, the estimated AO performance feeding the combined GNAO+GIRMOS imaging system was derived from models using limited information on what the actual parameters will eventually be. However, the AO system (currently called the AO Bench, or AOB) recently underwent a competitive bidding process to derive an AO design that met or exceeded our AO requirements. This work summarizes the update to the combined GNAO+GIRMOS imaging system performance based on the newly designed AOB parameters. We discuss the impact due to the changes in performance, specifically with respect to key science cases of the GNAO+GIRMOS imaging system compared to the previous models of the AO system. We also discuss the largest hurdles in terms of parameters that affect performance, such as telescope vibrations and detector quantum efficiency and our plans for mitigation.

2024

How to make METIS SCAO work

Authors
Bertram, T; Absil, O; Bizenberger, P; Brandi, B; Brandner, W; Briegel, F; Vazquez, MCC; Coppejans, H; Correira, C; Feldt, M; Häberle, M; Huber, A; Kulas, M; Laun, W; Mohr, L; Mortimer, D; Naranjo, V; Obereder, A; de Xivry, GO; Rohloff, RR; Scheithauer, S; Steuer, H; van Boekel, R;

Publication
ADAPTIVE OPTICS SYSTEMS IX

Abstract
METIS, the Mid-infrared ELT Imager and Spectrograph, will be one of the first instruments to be used at ESO's 39m Extremely Large Telescope (ELT), that is currently under construction. With that, a number of firsts are to be addressed in the development of METIS' single-conjugate Adaptive Optics (SCAO) system: the size of the telescope and the associated complexity of the wavefront control tasks, the unique scientific capabilities of METIS, including high contrast imaging, the interaction with the newly established, integrated wavefront control infrastructure of the ELT, the integration of the near-infrared Pyramid Wavefront Sensor and other key Adaptive Optics (AO) hardware embedded within a large, fully cryogenic instrument. METIS and it's AO system have passed the final design review and are now in the manufacturing, assembly, integration and testing phase. The firsts are approached through a compact hard- and software design and an extensive test program to mature METIS SCAO before it is deployed at the telescope. This program includes significant investments in test setups that allow to mimic conditions at the ELT. A dedicated cryo-test facility allows for subsystem testing independent of the METIS infrastructure. A telescope simulator is being set up for end-to-end laboratory tests of the AO control system together with the final SCAO hardware. Specific control algorithm prototypes will be tested on sky. In this contribution, we present the progress of METIS SCAO with an emphasis on the preparation for the test activities foreseen to enable a successful future deployment of METIS SCAO at the ELT.

  • 36
  • 377