Gonçalo Leão

Abstract
Robot Learning is one of the most important areas in Robotics and its relevance has only been increasing. The Robot Operating System (ROS) has been one of the most used architectures in Robotics but learning it is not a simple task. Additionally, ROS 1 is reaching its end-of-life and a lot of users are yet to make the transition to ROS 2. Reinforcement Learning (RL) and Robotics are rarely taught together, creating greater demand for tools to teach all these components. This paper aims to develop a learning kit that can be used to teach Robot Learning to students with different levels of expertise in Robotics. This kit works with the Flatland simulator using open-source free software, namely the OpenAI Gym and Stable-Baselines3 packages, and contains tutorials that introduce the user to the simulation environment as well as how to use RL to train the robot to perform different tasks. User tests were conducted to better understand how the kit performs, showing very positive feedback, with most participants agreeing that the kit provided a productive learning experience.

Abstract
This paper explores the use of Robotics and decentralized Multi-Agent Reinforcement Learning (MARL) for side-by-side navigation in Intelligent Wheelchairs (IW). Evolving from a previous work approach using traditional single-agent methodologies, it adopts a Multi-Agent Deep Deterministic Policy Gradient (MADDPG) algorithm to provide control input and enable a pair of IW to be deployed as decentralized computing agents in real-world environments, discarding the need to rely on communication between each other. In this study, the Flatland 2D simulator, in conjunction with the Robot Operating System (ROS), is used as a realistic environment to train and test the navigation algorithm. An overhaul of the reward function is introduced, which now provides individual rewards for each agent and revised reward incentives. Additionally, the logic for identifying side-by-side navigation was improved, to encourage dynamic alignment control. The preliminary results outline a promising research direction, with the IWs learning to navigate in various realistic hallways testing scenarios. The outcome also suggests that while the MADDPG approach holds potential over single-agent techniques for the decentralized IW robotics application, further investigation are needed for real-world deployment.

Abstract
Artificial I ntelligence ( AI) a nd M achine Learning are frequently used to develop player skills in robotic soccer scenarios. Despite the potential of deep reinforcement learning, its computational demands pose challenges when learning complex behaviors. This work explores less demanding methods, namely Evolution Strategies (ES) and Hierarchical Reinforcement Learning (HRL), for enhancing coordination and cooperation between two agents from the FC Portugal 3D Simulation Soccer Team, in RoboCup. The goal is for two robots to learn a high-level skill that enables a robot to pass the ball to its teammate as quickly as possible. Results show that the trained models under-performed in a traditional robotic soccer two-agent task and scored perfectly in a much simpler one. Therefore, this work highlights that while these alternative methods can learn trivial cooperative behavior, more complex tasks are difficult t o learn.

Abstract
Effective navigation in mobile robotics relies on precise environmental mapping, including the detection of complex objects as geometric primitives. This work introduces a deep learning model that determines the pose, type, and dimensions of 2D primitives using a mobile robot equipped with a noisy LiDAR sensor. Simulated experiments conducted in Webots involved randomly placed primitives, with the robot capturing point clouds which were used to progressively build a map of the environment. Two mapping techniques were considered, a deterministic and probabilistic (Bayesian) mapping, and different levels of noise for the LiDAR were compared. The maps were used as input to a YOLOv5 network that detected the position and type of the primitives. A cropped image of each primitive was then fed to a Convolutional Neural Network (CNN) that determined the dimensions and orientation of a given primitive. Results show that the primitive classification achieved an accuracy of 95% in low noise, dropping to 85% under higher noise conditions, while the prediction of the shapes' dimensions had error rates from 5% to 12%, as the noise increased. The probabilistic mapping approach improved accuracy by 10-15% compared to deterministic methods, showcasing robustness to noise levels up to 0.1. Therefore, these findings highlight the effectiveness of probabilistic mapping in enhancing detection accuracy for mobile robot perception in noisy environments.

Abstract
The manipulation of deformable objects poses a significant challenge for the automotive industry. In particular, the assembly of flexible cables and wire-harnesses in vehicles is still performed manually as there is yet to be a reliable and general solution for this problem. This paper presents a simple yet efficient motion planning algorithm to mount a flexible wire in an assembly jig, where the wire must traverse a set of forks in order. The algorithm uses a heuristic based on a set of control points to guide the wire's movement. Various controlled assembly scenarios are built in simulation using MuJoCo, a physics engine that can emulate the dynamics of Deformable Linear Objects (DLO). Experimental results in simulation demonstrated that the amount and orientation of the forks has a large impact in the solution's performance and highlighted several key ideas and challenges moving forward. Thus, this work serves as a stepping stone towards the development of more complete solutions, capable of assembling flexible items in vehicles.