Cookies
O website necessita de alguns cookies e outros recursos semelhantes para funcionar. Caso o permita, o INESC TEC irá utilizar cookies para recolher dados sobre as suas visitas, contribuindo, assim, para estatísticas agregadas que permitem melhorar o nosso serviço. Ver mais
Aceitar Rejeitar
  • Menu
Publicações

Publicações por CRIIS

2025

Dual-Arm Manipulation of a T-Shirt from a Hanger for Feeding a Hem Sewing Machine

Autores
Almeida, F; Leão, G; Costa, CM; Rocha, CD; Sousa, A; da Silva, LG; Rocha, LF; Veiga, G;

Publicação
ICINCO (1)

Abstract
The textile industry is experiencing rapid advancement, reflected in the adoption of innovative and efficient manufacturing techniques. The automation of clothing sewing systems has the potential to reduce the allocation of repetitive tasks to operators, freeing them for more value-added operations. There are several machines on the market that automatically sew the bottom hem of T-shirts, a key component of the garment that fulfills both functional and aesthetic purposes. However, most of them require the fabric to be positioned manually by an operator. To address this issue, this work presents a solution to automate the process of feeding a T-shirt into a SiRUBA sewing machine using a YuMi dual-arm robot. In this scenario, the T-shirt arrives at the workstation with the main front and back pieces of cloth sewn together, seams facing out, and with no sleeves yet. This setup starts by turning the garment inside out with the aid of an automated hanger, ensuring that the seams are facing inward (as the machine requires), and then using the dual-arm robot to feed the garment into the sewing machine. With our approach, the feeding and hemming process took less than 35 seconds, with a feeding success rate of 98%. Therefore, this work can serve as a steppingstone towards more efficient automated sewing systems within the garment production industry.

2025

JEMA-SINDYc: End-to-end Control using Joint Embedding Multimodal Alignment in Directed Energy Deposition

Autores
Sousa, J; Brandau, B; Hemschik, R; Darabi, R; Sousa, A; Reis, LP; Brueckner, F; Reis, A;

Publicação
ADDITIVE MANUFACTURING

Abstract
Bringing AI models from digital to real-world applications presents significant challenges due to the complexity and variability of physical environments, often leading to unexpected model behaviors. We propose a framework that learns to translate images into control actions by modeling multimodal real-time data and system dynamics. This end-to-end controller offers enhanced explainability and robustness, making it well suited for complex manufacturing processes. This end-to-end framework differs from traditional approaches that rely on manually engineered features by learning complex relationships directly from raw data. Labels are only required during training to define the observable feature to be optimized. This adaptability significantly reduces development time and enhances scalability across varying conditions. This approach was tested in the Directed Energy Deposition (L-DED) process, a laser-based metal additive manufacturing technique that produces near-net-shape parts with exceptional energy efficiency and flexibility in both geometry and material selection. L-DED is inherently complex, involving multiphysics interactions, multiscale phenomena, and dynamic behaviors, which make modeling and optimization difficult. Effective control is crucial to ensure part quality in this dynamic environment. To address these challenges, we introduce Joint Embedding Multimodal Alignment with Sparse Identification of Nonlinear Dynamics for control (JEMA-SINDYc). It combines an image-based JEMA monitoring model, which predicts the melt pool size using only the on-axis sensor with labels provided by the off-axis camera, and dynamic modeling using SINDYc, which acts as a World Model by capturing system dynamics within the embedding space. Together, these components enable the development of an advanced controller trained via Behavioral Cloning. This approach improves part quality by minimizing porosity and reducing deformation. Thin-walled cylindrical parts were produced to validate and compare this approach with other control strategies, including both open-loop and JEMA-PID. This framework improves the reliability of AI-driven manufacturing and enhances control of complex industrial processes, potentially enabling wider adoption of the process.

2025

Exploring the Potential of LLM-based Chatbots for Task Scheduling in Robot Operations

Autores
Rema, C; Sousa, A; Sobreira, H; Costa, P; Silva, MF;

Publicação
2025 IEEE INTERNATIONAL CONFERENCE ON AUTONOMOUS ROBOT SYSTEMS AND COMPETITIONS, ICARSC

Abstract
The rise of Industry 4.0 has revolutionized manufacturing by integrating real-time data analysis, artificial intelligence (AI), automation, and interconnected systems, enabling adaptive and resilient smart factories. Autonomous Mobile Robots (AMRs), with their advanced mobility and navigation capabilities, are a pillar of this transformation. However, their deployment in job shop environments adds complexity to the already challenging Job Shop Scheduling Problem (JSSP), expanding it to include task allocation, robot scheduling, and travel time optimization, creating a multi-faceted, non-deterministic polynomial-time hardness (NP-hard) problem. Traditional approaches such as heuristics, meta-heuristics, and mixed integer linear programming (MILP) are commonly used. Recent AI advancements, particularly large language models (LLM), have shown potential in addressing these scheduling challenges due to significant improvements in reasoning and decision-making from textual data. This paper examines the application of LLM to tackle scheduling complexities in smart job shops with mobile robots. Guided by tailored prompts inserted manually, LLM are employed to generate scheduling solutions, being these compared to an heuristic-based method. The results indicate that LLM currently have limitations in solving complex combinatorial problems, such as task scheduling with mobile robots. Due to issues with consistency and repeatability, they are not yet reliable enough for practical implementation in industrial environments. However, they offer a promising foundation for augmenting traditional approaches in the future.

2025

AR/VR Digital Twin for simulation and data collection of robotic environments

Autores
Martins, JG; Nutonen, K; Costa, P; Kuts, V; Otto, T; Sousa, A; Petry, MR;

Publicação
2025 IEEE INTERNATIONAL CONFERENCE ON AUTONOMOUS ROBOT SYSTEMS AND COMPETITIONS, ICARSC

Abstract
Digital twins enable real-time modeling, simulation, and monitoring of complex systems, driving advancements in automation, robotics, and industrial applications. This study presents a large-scale digital twin-testing facility for evaluating mobile robots and pilot robotic systems in a research laboratory environment. The platform integrates high-fidelity physical and environmental models, providing a controlled yet dynamic setting for analyzing robotic behavior. A key feature of the system is its comprehensive data collection framework, capturing critical parameters such as position, orientation, and velocity, which can be leveraged for machine learning, performance optimization, and decision-making. The facility also supports the simulation of discrete operational systems, using predictive modeling to bridge informational gaps when real-time data updates are unavailable. The digital twin was validated through a matrix manufacturing system simulation, with an Augmented Reality (AR) interface on the HoloLens 2 to overlay digital information onto mobile platform controllers, enhancing situational awareness. The main contributions include a digital twin framework for deploying data-driven robotic systems and three key AR/VR integration optimization methods. Demonstrated in a laboratory setting, the system is a versatile tool for research and industrial applications, fostering insights into robotic automation and digital twin scalability while reducing costs and risks associated with real-world testing.

2025

Artificial Intelligence for Control in Laser-Based Additive Manufacturing: A Systematic Review

Autores
Sousa, J; Brandau, B; Darabi, R; Sousa, A; Brueckner, F; Reis, A; Reis, LP;

Publicação
IEEE ACCESS

Abstract
Laser-based additive manufacturing (LAM) offers the ability to produce near-net-shape metal parts with unparalleled energy efficiency and flexibility in both geometry and material selection. Despite advantages, these processes are inherently, as they are characterized by multiphysics interactions, multiscale phenomena, and highly dynamic behaviors, making their modeling and optimization particularly challenging. Artificial intelligence (AI) has emerged as a promising tool for enhancing the monitoring and control of additive manufacturing. This paper presents a systematic review of AI applications for real-time control of laser-based manufacturing processes, analyzing 16 relevant articles sourced from Scopus, IEEE Xplore, and Web of Science databases. The primary objective of this work is to contribute to the advancement of autonomous manufacturing systems capable of self-monitoring and self-correction, ensuring optimal part quality, enhanced efficiency, and reduced human intervention. Our findings indicate that 62.5 % of the 16 analyzed studies have deployed AI-driven controllers in real-world scenarios, with over 56 % using AI for control strategies, such as Reinforcement Learning. Furthermore, 62.5 % of the studies employed AI for process modeling or monitoring, which was integral to the development or data pipelines of the controllers. By defining a groundwork for future developments, this review not only highlights current advancements but also hints future innovations that will likely include AI-based controllers.

2025

Spray Quality Assessment on Water-Sensitive Paper Comparing AI and Classical Computer Vision Methods

Autores
Simoes, I; Sousa, AJ; Baltazar, A; Santos, F;

Publicação
AGRICULTURE-BASEL

Abstract
Precision agriculture seeks to optimize crop yields while minimizing resource use. A key challenge is achieving uniform pesticide spraying to prevent crop damage and environmental contamination. Water-sensitive paper (WSP) is a common tool used for assessing spray quality, as it visually registers droplet impacts through color change. This work introduces a smartphone-based solution for capturing WSP images within vegetation, offering a tool for farmers to assess spray quality in real-world conditions. To achieve this, two approaches were explored: classical computer vision techniques and machine learning (ML) models (YOLOv8, Mask-RCNN, and Cellpose). Addressing the challenges of limited real-world data and the complexity of manual annotation, a programmatically generated synthetic dataset was employed to enable sim-to-real transfer learning. For the task of WSP segmentation within vegetation, YOLOv8 achieved an average Intersection over Union of 97.76%. In the droplet detection task, which involves identifying individual droplets on WSP, Cellpose achieved the highest precision of 96.18%, in the presence of overlapping droplets. While classical computer vision techniques provided a reliable baseline, they struggled with complex cases. Additionally, ML models, particularly Cellpose, demonstrated accurate droplet detection even without fine-tuning.

  • 10
  • 399