Cookies
O website necessita de alguns cookies e outros recursos semelhantes para funcionar. Caso o permita, o INESC TEC irá utilizar cookies para recolher dados sobre as suas visitas, contribuindo, assim, para estatísticas agregadas que permitem melhorar o nosso serviço. Ver mais
Aceitar Rejeitar
  • Menu
Sobre

Sobre

Armando Jorge Miranda de Sousa received his PhD in 2004, in Electrical and Computer Engineering (ECE) at University of Porto - Faculty of Engineering (FEUP), Portugal. His thesis work was in the subarea of Robotics and Automation.

He is currently an Associate Professor in the ECE department of FEUP and an integrated senior researcher at Centre for Intelligent and Industrial Systems (CRIIS) at the INESC TEC interface institute. He earned in 2014 the international pedagogical certification "ING.PAED.IGIP" from the International Society for Engineering Pedagogy and is currently an active member for the European Society for Engineering Education (SEFI).

His main research areas include Higher Education and Robotics, but most recently focusing on Robot Learning and Learning for Cyber Physical Systems. Application areas include not only intelligent robots for agriculture and forest but also robotic manipulation of flexible objects. As a frequent participant in robotic contests, some of which used AI in real world robotics, he has earned several national and international merits (examples: vice champion of RoboCup Robotic Soccer in 2006, winner of Autonomous Driving of Portuguese Robotics Open of 2022).

He has also earned educational awards such as the University of Porto (UP) excellence award in 2015 and 10 best at ECEL 2015 excellence e-learning awards. He has published over 80 indexed peer reviewed articles both in pedagogical issues and more technical areas. Also, he has a patent entitled "Device and method for identifying a cork stopper and respective kit". He is also involved in educational and technical funded projects such as "IntelWheels 2" and "blockchain.pt".

He currently (co-)supervises 7 PhD students.

More details in https://www.cienciavitae.pt/en/1C17-7D93-4CF3 and https://fe.up.pt/asousa.

Tópicos
de interesse
Detalhes

Detalhes

  • Nome

    Armando Sousa
  • Cargo

    Investigador Sénior
  • Desde

    01 junho 2009
009
Publicações

2025

JEMA-SINDYc: End-to-end Control using Joint Embedding Multimodal Alignment in Directed Energy Deposition

Autores
Sousa, J; Brandau, B; Hemschik, R; Darabi, R; Sousa, A; Reis, LP; Brueckner, F; Reis, A;

Publicação
ADDITIVE MANUFACTURING

Abstract
Bringing AI models from digital to real-world applications presents significant challenges due to the complexity and variability of physical environments, often leading to unexpected model behaviors. We propose a framework that learns to translate images into control actions by modeling multimodal real-time data and system dynamics. This end-to-end controller offers enhanced explainability and robustness, making it well suited for complex manufacturing processes. This end-to-end framework differs from traditional approaches that rely on manually engineered features by learning complex relationships directly from raw data. Labels are only required during training to define the observable feature to be optimized. This adaptability significantly reduces development time and enhances scalability across varying conditions. This approach was tested in the Directed Energy Deposition (L-DED) process, a laser-based metal additive manufacturing technique that produces near-net-shape parts with exceptional energy efficiency and flexibility in both geometry and material selection. L-DED is inherently complex, involving multiphysics interactions, multiscale phenomena, and dynamic behaviors, which make modeling and optimization difficult. Effective control is crucial to ensure part quality in this dynamic environment. To address these challenges, we introduce Joint Embedding Multimodal Alignment with Sparse Identification of Nonlinear Dynamics for control (JEMA-SINDYc). It combines an image-based JEMA monitoring model, which predicts the melt pool size using only the on-axis sensor with labels provided by the off-axis camera, and dynamic modeling using SINDYc, which acts as a World Model by capturing system dynamics within the embedding space. Together, these components enable the development of an advanced controller trained via Behavioral Cloning. This approach improves part quality by minimizing porosity and reducing deformation. Thin-walled cylindrical parts were produced to validate and compare this approach with other control strategies, including both open-loop and JEMA-PID. This framework improves the reliability of AI-driven manufacturing and enhances control of complex industrial processes, potentially enabling wider adoption of the process.

2025

Using interdisciplinarity to promote the interconnection between ethics, sustainability and electrical engineering through electrical installations

Autores
Monteiro, F; Sousa, A;

Publicação
EUROPEAN JOURNAL OF ENGINEERING EDUCATION

Abstract
Engineering is considered important in solving unsustainability. However, it is a complex problem that must be viewed, analysed and studied from various perspectives and taking with the contribution of various areas of knowledge. This work studied the use of interdisciplinarity as a contribution to interconnect ethics and sustainability with technical-scientific contents of electrical engineering. The research intended to use interdisciplinarity to help engineering students recognise that engineering is not ethically neutral, and that, therefore, the problems (and solutions) must also be analysed from an ethical and sustainability perspective. A framework was developed, and a pedagogical activity using interdisciplinarity was applied. Results show that, after the activity, students recognise that ethical values influence calculations in the area of electrical installations; and move from a single view to identify different alternatives, perspectives, motivations and multiple objectives. This leads to studying more alternatives and hopefully better overall technical solutions.

2025

Exploring the Potential of LLM-based Chatbots for Task Scheduling in Robot Operations

Autores
Rema, C; Sousa, A; Sobreira, H; Costa, P; Silva, MF;

Publicação
2025 IEEE INTERNATIONAL CONFERENCE ON AUTONOMOUS ROBOT SYSTEMS AND COMPETITIONS, ICARSC

Abstract
The rise of Industry 4.0 has revolutionized manufacturing by integrating real-time data analysis, artificial intelligence (AI), automation, and interconnected systems, enabling adaptive and resilient smart factories. Autonomous Mobile Robots (AMRs), with their advanced mobility and navigation capabilities, are a pillar of this transformation. However, their deployment in job shop environments adds complexity to the already challenging Job Shop Scheduling Problem (JSSP), expanding it to include task allocation, robot scheduling, and travel time optimization, creating a multi-faceted, non-deterministic polynomial-time hardness (NP-hard) problem. Traditional approaches such as heuristics, meta-heuristics, and mixed integer linear programming (MILP) are commonly used. Recent AI advancements, particularly large language models (LLM), have shown potential in addressing these scheduling challenges due to significant improvements in reasoning and decision-making from textual data. This paper examines the application of LLM to tackle scheduling complexities in smart job shops with mobile robots. Guided by tailored prompts inserted manually, LLM are employed to generate scheduling solutions, being these compared to an heuristic-based method. The results indicate that LLM currently have limitations in solving complex combinatorial problems, such as task scheduling with mobile robots. Due to issues with consistency and repeatability, they are not yet reliable enough for practical implementation in industrial environments. However, they offer a promising foundation for augmenting traditional approaches in the future.

2025

AR/VR Digital Twin for simulation and data collection of robotic environments

Autores
Martins, JG; Nutonen, K; Costa, P; Kuts, V; Otto, T; Sousa, A; Petry, MR;

Publicação
2025 IEEE INTERNATIONAL CONFERENCE ON AUTONOMOUS ROBOT SYSTEMS AND COMPETITIONS, ICARSC

Abstract
Digital twins enable real-time modeling, simulation, and monitoring of complex systems, driving advancements in automation, robotics, and industrial applications. This study presents a large-scale digital twin-testing facility for evaluating mobile robots and pilot robotic systems in a research laboratory environment. The platform integrates high-fidelity physical and environmental models, providing a controlled yet dynamic setting for analyzing robotic behavior. A key feature of the system is its comprehensive data collection framework, capturing critical parameters such as position, orientation, and velocity, which can be leveraged for machine learning, performance optimization, and decision-making. The facility also supports the simulation of discrete operational systems, using predictive modeling to bridge informational gaps when real-time data updates are unavailable. The digital twin was validated through a matrix manufacturing system simulation, with an Augmented Reality (AR) interface on the HoloLens 2 to overlay digital information onto mobile platform controllers, enhancing situational awareness. The main contributions include a digital twin framework for deploying data-driven robotic systems and three key AR/VR integration optimization methods. Demonstrated in a laboratory setting, the system is a versatile tool for research and industrial applications, fostering insights into robotic automation and digital twin scalability while reducing costs and risks associated with real-world testing.

2025

Gen-JEMA: enhanced explainability using generative joint embedding multimodal alignment for monitoring directed energy deposition

Autores
Ferreira, J; Darabi, R; Sousa, A; Brueckner, F; Reis, LP; Reis, A; Tavares, JMRS; Sousa, J;

Publicação
JOURNAL OF INTELLIGENT MANUFACTURING

Abstract
This work introduces Gen-JEMA, a generative approach based on joint embedding with multimodal alignment (JEMA), to enhance feature extraction in the embedding space and improve the explainability of its predictions. Gen-JEMA addresses these challenges by leveraging multimodal data, including multi-view images and metadata such as process parameters, to learn transferable semantic representations. Gen-JEMA enables more explainable and enriched predictions by learning a decoder from the embedding. This novel co-learning framework, tailored for directed energy deposition (DED), integrates multiple data sources to learn a unified data representation and predict melt pool images from the primary sensor. The proposed approach enables real-time process monitoring using only the primary modality, simplifying hardware requirements and reducing computational overhead. The effectiveness of Gen-JEMA for DED process monitoring was evaluated, focusing on its generalization to downstream tasks such as melt pool geometry prediction and the generation of external melt pool representations using off-axis sensor data. To generate these external representations, autoencoder (AE) and variational autoencoder (VAE) architectures were optimized using Bayesian optimization. The AE outperformed other approaches achieving a 38% improvement in melt pool geometry prediction compared to the baseline and 88% in data generation compared with the VAE. The proposed framework establishes the foundation for integrating multisensor data with metadata through a generative approach, enabling various downstream tasks within the DED domain and achieving a small embedding, allowing efficient process control based on model predictions and embeddings.