Cookies Policy
The website need some cookies and similar means to function. If you permit us, we will use those means to collect data on your visits for aggregated statistics to improve our service. Find out More
Accept Reject
  • Menu
About

About

Electrical engineering formation with an emphasis on electronic systems and a master's in electrical engineering from the Federal University of Juiz de Fora, his training allowed him to have competence in the area of digital and analog electronics. However, the central area of expertise is robotics in which he is since 2012 as a researcher and promoted his master's degree. He is currently R&D and develops research in the robotics area at the Institute of Systems and Computer Engineering, Technology and Science - INESC TEC. He has experience in several R&D projects that he developed in Brazil and Portugal. Expertise areas, experience, and skills include the creation of firmware, middleware and software, embedded systems, control techniques, and artificial and computational intelligence. Development of robots, autonomous unmanned aerial vehicles, autonomous systems, robotic manipulators, industrial robotics, and sensing (computer vision and 3D perception).

Interest
Topics
Details

Details

  • Name

    João Pedro Souza
  • Role

    Assistant Researcher
  • Since

    21st May 2018
008
Publications

2023

Bin Picking for Ship-Building Logistics Using Perception and Grasping Systems

Authors
Cordeiro, A; Souza, JP; Costa, CM; Filipe, V; Rocha, LF; Silva, MF;

Publication
ROBOTICS

Abstract
Bin picking is a challenging task involving many research domains within the perception and grasping fields, for which there are no perfect and reliable solutions available that are applicable to a wide range of unstructured and cluttered environments present in industrial factories and logistics centers. This paper contributes with research on the topic of object segmentation in cluttered scenarios, independent of previous object shape knowledge, for textured and textureless objects. In addition, it addresses the demand for extended datasets in deep learning tasks with realistic data. We propose a solution using a Mask R-CNN for 2D object segmentation, trained with real data acquired from a RGB-D sensor and synthetic data generated in Blender, combined with 3D point-cloud segmentation to extract a segmented point cloud belonging to a single object from the bin. Next, it is employed a re-configurable pipeline for 6-DoF object pose estimation, followed by a grasp planner to select a feasible grasp pose. The experimental results show that the object segmentation approach is efficient and accurate in cluttered scenarios with several occlusions. The neural network model was trained with both real and simulated data, enhancing the success rate from the previous classical segmentation, displaying an overall grasping success rate of 87.5%.

2022

Industrial robot programming by demonstration using stereoscopic vision and inertial sensing

Authors
de Souza, JPC; Amorim, AM; Rocha, LF; Pinto, VH; Moreira, AP;

Publication
INDUSTRIAL ROBOT-THE INTERNATIONAL JOURNAL OF ROBOTICS RESEARCH AND APPLICATION

Abstract
Purpose The purpose of this paper is to present a programming by demonstration (PbD) system based on 3D stereoscopic vision and inertial sensing that provides a cost-effective pose tracking system, even during error-prone situations, such as camera occlusions. Design/methodology/approach The proposed PbD system is based on the 6D Mimic innovative solution, whose six degrees of freedom marker hardware had to be revised and restructured to accommodate an IMU sensor. Additionally, a new software pipeline was designed to include this new sensing device, seeking the improvement of the overall system's robustness in stereoscopic vision occlusion situations. Findings The IMU component and the new software pipeline allow the 6D Mimic system to successfully maintain the pose tracking when the main tracking tool, i.e. the stereoscopic vision, fails. Therefore, the system improves in terms of reliability, robustness, and accuracy which were verified by real experiments. Practical implications Based on this proposal, the 6D Mimic system reaches a reliable and low-cost PbD methodology. Therefore, the robot can accurately replicate, on an industrial scale, the artisan level performance of highly skilled shop-floor operators. Originality/value To the best of the authors' knowledge, the sensor fusion between stereoscopic images and IMU applied to robot PbD is a novel approach. The system is entirely designed aiming to reduce costs and taking advantage of an offline processing step for data analysis, filtering and fusion, enhancing the reliability of the PbD system.

2021

Reconfigurable Grasp Planning Pipeline with Grasp Synthesis and Selection Applied to Picking Operations in Aerospace Factories

Authors
de Souza, JPC; Costa, CM; Rocha, LF; Arrais, R; Moreira, AP; Pires, EJS; Boaventura Cunha, J;

Publication
ROBOTICS AND COMPUTER-INTEGRATED MANUFACTURING

Abstract
Several approaches with interesting results have been proposed over the years for robot grasp planning. However, the industry suffers from the lack of an intuitive and reliable system able to automatically estimate grasp poses while also allowing the integration of grasp information from the accumulated knowledge of the end user. In the presented paper it is proposed a non-object-agnostic grasping pipeline motivated by picking use cases from the aerospace industry. The planning system extends the functionality of the simulated annealing optimization algorithm for allowing its application within an industrial use case. Therefore, this paper addresses the first step of the design of a reconfigurable and modular grasping pipeline. The key idea is the creation of an intuitive and functional grasping framework for being used by factory floor operators according to the task demands. This software pipeline is capable of generating grasp solutions in an offline phase, and later on, in the robot operation phase, can choose the best grasp pose by taking into consideration a set of heuristics that try to achieve a successful grasp while also requiring the least effort for the robotic arm. The results are presented in a simulated and a real factory environment, relying on a mobile platform developed for intralogistic tasks. With this architecture, new state-of-art methodologies can be integrated in the future for growing the grasping pipeline and make it more robust and applicable to a wider range of use cases.

2021

Robotic grasping: from wrench space heuristics to deep learning policies

Authors
de Souza, JPC; Rocha, LF; Oliveira, PM; Moreira, AP; Boaventura Cunha, J;

Publication
ROBOTICS AND COMPUTER-INTEGRATED MANUFACTURING

Abstract
The robotic grasping task persists as a modern industry problem that seeks autonomous, fast implementation, and efficient techniques. Domestic robots are also a reality demanding a delicate and accurate human-machine interaction, with precise robotic grasping and handling. From decades ago, with analytical heuristics, to recent days, with the new deep learning policies, grasping in complex scenarios is still the aim of several works' that propose distinctive approaches. In this context, this paper aims to cover recent methodologies' development and discuss them, showing state-of-the-art challenges and the gap to industrial applications deployment. Given the complexity of the related issue associated with the elaborated proposed methods, this paper formulates some fair and transparent definitions for results' assessment to provide researchers with a clear and standardised idea of the comparison between the new proposals.

2021

Low-Cost and Reduced-Size 3D-Cameras Metrological Evaluation Applied to Industrial Robotic Welding Operations

Authors
de Souza, JPC; Rocha, LF; Filipe, VM; Boaventura Cunha, J; Moreira, AP;

Publication
2021 IEEE INTERNATIONAL CONFERENCE ON AUTONOMOUS ROBOT SYSTEMS AND COMPETITIONS (ICARSC)

Abstract
Nowadays, the robotic welding joint estimation, or weld seam tracking, has improved according to the new developments on computer vision technologies. Typically, the advances are focused on solving inaccurate procedures that advent from the manual positioning of the metal parts in welding workstations, especially in SMEs. Robotic arms, endowed with the appropriate perception capabilities, are a viable solution in this context, aiming for enhancing the production system agility whilst not increasing the production set-up time and costs. In this regard, this paper proposes a local perception pipeline to estimate joint welding points using small-sized/low-cost 3D cameras, following an eyes-on-hand approach. A metrological 3D camera comparison between Intel Realsene D435, D415, and ZED Mini is also discussed, proving that the proposed pipeline associated with standard commercial 3D cameras is viable for welding operations in an industrial environment.