Cookies Policy
The website need some cookies and similar means to function. If you permit us, we will use those means to collect data on your visits for aggregated statistics to improve our service. Find out More
Accept Reject
  • Menu
Publications

Publications by CRIIS

2022

Benchmark of Deep Learning and a Proposed HSV Colour Space Models for the Detection and Classification of Greenhouse Tomato

Authors
Moreira, G; Magalhaes, SA; Pinho, T; dos Santos, FN; Cunha, M;

Publication
AGRONOMY-BASEL

Abstract
The harvesting operation is a recurring task in the production of any crop, thus making it an excellent candidate for automation. In protected horticulture, one of the crops with high added value is tomatoes. However, its robotic harvesting is still far from maturity. That said, the development of an accurate fruit detection system is a crucial step towards achieving fully automated robotic harvesting. Deep Learning (DL) and detection frameworks like Single Shot MultiBox Detector (SSD) or You Only Look Once (YOLO) are more robust and accurate alternatives with better response to highly complex scenarios. The use of DL can be easily used to detect tomatoes, but when their classification is intended, the task becomes harsh, demanding a huge amount of data. Therefore, this paper proposes the use of DL models (SSD MobileNet v2 and YOLOv4) to efficiently detect the tomatoes and compare those systems with a proposed histogram-based HSV colour space model to classify each tomato and determine its ripening stage, through two image datasets acquired. Regarding detection, both models obtained promising results, with the YOLOv4 model standing out with an F1-Score of 85.81%. For classification task the YOLOv4 was again the best model with an Macro F1-Score of 74.16%. The HSV colour space model outperformed the SSD MobileNet v2 model, obtaining results similar to the YOLOv4 model, with a Balanced Accuracy of 68.10%.

2022

SCARA Self Posture Recognition Using a Monocular Camera

Authors
Tinoco, V; Silva, MF; Santos, FN; Morais, R; Filipe, V;

Publication
IEEE ACCESS

Abstract
Robotic manipulators rely on feedback obtained from rotary encoders for control purposes. This article introduces a vision-based feedback system that can be used in an agricultural context, where the shapes and sizes of fruits are uncertain. We aim to mimic a human, using vision and touch as manipulator control feedback. This work explores the use of a fish-eye lens camera to track a SCARA manipulator with coloured markers on its joints for the position estimation with the goal to reduce costs and increase reliability. The Kalman Filter and the Particle Filter are compared and evaluated in terms of accuracy and tracking abilities of the marker's positions. The estimated image coordinates of the markers are converted to world coordinates using planar homography, as the SCARA manipulator has co-planar joints and the coloured markers share the same plane. Three laboratory experiments were conducted to evaluate the system's performance in joint angle estimation of a manipulator. The obtained results are promising, for future cost effective agricultural robotic arms developments. Besides, this work presents solutions and future directions to increase the joint position estimation accuracy.

2022

VineInspector: The Vineyard Assistant

Authors
Mendes, J; Peres, E; dos Santos, FN; Silva, N; Silva, R; Sousa, JJ; Cortez, I; Morais, R;

Publication
AGRICULTURE-BASEL

Abstract
Proximity sensing approaches with a wide array of sensors available for use in precision viticulture contexts can nowadays be considered both well-know and mature technologies. Still, several in-field practices performed throughout different crops rely on direct visual observation supported on gained experience to assess aspects of plants' phenological development, as well as indicators relating to the onset of common plagues and diseases. Aiming to mimic in-field direct observation, this paper presents VineInspector: a low-cost, self-contained and easy-to-install system, which is able to measure microclimatic parameters, and also to acquire images using multiple cameras. It is built upon a stake structure, rendering it suitable for deployment across a vineyard. The approach through which distinguishable attributes are detected, classified and tallied in the periodically acquired images, makes use of artificial intelligence approaches. Furthermore, it is made available through an IoT cloud-based support system. VineInspector was field-tested under real operating conditions to assess not only the robustness and the operating functionality of the hardware solution, but also the AI approaches' accuracy. Two applications were developed to evaluate Vinelnspector's consistency while a viticulturist' assistant in everyday practices. One was intended to determine the size of the very first grapevines' shoots, one of the required parameters of the well known 3-10 rule to predict primary downy mildew infection. The other was developed to tally grapevine moth males captured in sex traps. Results show that VineInspector is a logical step in smart proximity monitoring by mimicking direct visual observation from experienced viticulturists. While the latter traditionally are responsible for a set of everyday practices in the field, these are time and resource consuming. VineInspector was proven to be effective in two of these practices, performing them automatically. Therefore, it enables both the continuous monitoring and assessment of a vineyard's phenological development in a more efficient manner, making way to more assertive and timely practices against pests and diseases.

2022

End-Effectors for Harvesting Manipulators - State Of The Art Review

Authors
Oliveira, F; Tinoco, V; Magalhaes, S; Santos, FN; Silva, MF;

Publication
2022 IEEE INTERNATIONAL CONFERENCE ON AUTONOMOUS ROBOT SYSTEMS AND COMPETITIONS (ICARSC)

Abstract
There has been an increase in the variety of harvesting manipulators. However, sometimes the lack of efficiency of these manipulators makes it difficult to compete with harvesting tasks performed by humans. One of the key components of these manipulators is the end-effector, responsible for picking the fruits from the plant. This paper studies different types of end-effectors used by some harvesting manipulators and compares them. The objective is to analyse their advantages and limitations to better understand the requirements to design an end-effector to improve the performance of a custom Selective Compliance Assembly Robot Arm (SCARA) on the harvest of different types of fruits.

2022

ATOM: A general calibration framework for multi-modal, multi-sensor systems

Authors
Oliveira, M; Pedrosa, E; de Aguiar, AP; Rato, DFPD; dos Santos, FN; Dias, P; Santos, V;

Publication
EXPERT SYSTEMS WITH APPLICATIONS

Abstract
The fusion of data from different sensors often requires that an accurate geometric transformation between the sensors is known. The procedure by which these transformations are estimated is known as sensor calibration. The vast majority of calibration approaches focus on specific pairwise combinations of sensor modalities, unsuitable to calibrate robotic systems containing multiple sensors of varied modalities. This paper presents a novel calibration methodology which is applicable to multi-sensor, multi-modal robotic systems. The approach formulates the calibration as an extended optimization problem, in which the poses of the calibration patterns are also estimated. It makes use of a topological representation of the coordinate frames in the system, in order to recalculate the poses of the sensors throughout the optimization. Sensor poses are retrieved from the combination of geometric transformations which are atomic, in the sense that they are indivisible. As such, we refer to this approach as ATOM - Atomic Transformations Optimization Method. This makes the approach applicable to different calibration problems, such as sensor to sensor, sensor in motion, or sensor to coordinate frame. Additionally, the proposed approach provides advanced functionalities, integrated into ROS, designed to support the several stages of a complete calibration procedure. Results covering several robotic platforms and a large spectrum of calibration problems show that the methodology is in fact general, and achieves calibrations which are as accurate as the ones provided by state of the art methods designed to operate only for specific combinations of pairwise modalities.

2022

Kiwi Plant Canker Diagnosis Using Hyperspectral Signal Processing and Machine Learning: Detecting Symptoms Caused by Pseudomonas syringae pv. actinidiae

Authors
Reis Pereira, M; Tosin, R; Martins, R; dos Santos, FN; Tavares, F; Cunha, M;

Publication
PLANTS-BASEL

Abstract
Pseudomonas syringae pv. actinidiae (Psa) has been responsible for numerous epidemics of bacterial canker of kiwi (BCK), resulting in high losses in kiwi production worldwide. Current diagnostic approaches for this disease usually depend on visible signs of the infection (disease symptoms) to be present. Since these symptoms frequently manifest themselves in the middle to late stages of the infection process, the effectiveness of phytosanitary measures can be compromised. Hyperspectral spectroscopy has the potential to be an effective, non-invasive, rapid, cost-effective, high-throughput approach for improving BCK diagnostics. This study aimed to investigate the potential of hyperspectral UV-VIS reflectance for in-situ, non-destructive discrimination of bacterial canker on kiwi leaves. Spectral reflectance (325-1075 nm) of twenty plants were obtained with a handheld spectroradiometer in two commercial kiwi orchards located in Portugal, for 15 weeks, totaling 504 spectral measurements. Several modeling approaches based on continuous hyperspectral data or specific wavelengths, chosen by different feature selection algorithms, were tested to discriminate BCK on leaves. Spectral separability of asymptomatic and symptomatic leaves was observed in all multi-variate and machine learning models, including the FDA, GLM, PLS, and SVM methods. The combination of a stepwise forward variable selection approach using a support vector machine algorithm with a radial kernel and class weights was selected as the final model. Its overall accuracy was 85%, with a 0.70 kappa score and 0.84 F-measure. These results were coherent with leaves classified as asymptomatic or symptomatic by visual inspection. Overall, the findings herein reported support the implementation of spectral point measurements acquired in situ for crop disease diagnosis.

  • 76
  • 366