Abstract
The introduction of Cyber-Physical Systems (CPS) in the industry through the digitalization of equipment, also known as Digital Twins, allows for a more customized production. Due to high market fluctuation, the implementation of a CPS should guarantee a high flexibility in both hardware and software levels to achieve a high responsiveness of the system. The software reconfiguration, specifically, introduces a question: “With heterogeneous equipment with different capabilities - namely processing and memory capabilities - where a certain software module should execute?”; that question fits on the task/resource allocation area applied to CPS software reconfiguration. Although in task allocation issue several approaches address such a problem, only a few of them focus on CPS resources optimization. Given that, an approach based on the Dutch Auction algorithm is proposed, implemented at the CPS level enables the software reconfiguration of the CPS according to the existing equipment resources. This approach, besides the optimization of the CPS resources and the energy consumption, transforms the CPS in more reliable and fault-tolerant systems. As shown by the results, despite the demonstration of its suitability in task/resource allocation problems in decentralized architectures, the proposed approach also as a major advantage of quickly finding a near-optimal solution. © 2022, The Author(s), under exclusive license to Springer Nature Switzerland AG.

Abstract

Abstract
In this work, we propose a Dynamic Difficulty Adjustment methodology to achieve automatic video game balance. The balance task is modeled as a meta game, a game where actions change the rules of another base game. Based on the model of Reinforcement Learning (RL), an agent assumes the role of a game master and learns its optimal policy by playing the meta game. In this new methodology we extend traditional RL by adding the existence of a meta environment whose state transition depends on the evolution of a base environment. In addition, we propose a Multi Agent System training model for the game master agent, where it plays against multiple agent opponents, each with a distinct behavior and proficiency level while playing the base game. Our experiment is conducted on an adaptive grid-world environment in singleplayer and multiplayer scenarios. Our results are expressed in twofold: (i) the resulting decision making by the game master through gameplay, which must comply in accordance to an established balance objective by the game designer; (ii) the initial conception of a framework for automatic game balance, where the balance task design is reduced to the modulation of a reward function (balance reward), an action space (balance strategies) and the definition of a balance space state. © 2020, Springer Nature B.V.

Abstract
Robotic soccer simulation is a challenging area, where the development of new techniques is paramount to remain competitive. Robotic skill evolution has accelerated with recent developments in deep learning algorithms, leading to improvements in behavior number and complexity. Shooting a ball towards a defined target is one of the most basic yet indispensable skills in soccer. However, fast and accurate kicks pose several challenges. In order to reach that target, the skill is highly dependent on the ability of the agent to self-locate and self-orient, in order to better position itself before the kick. To tackle these issues, a 6D localization technique was devised. To optimize the kick behavior, two scenarios were proposed. In the first, the robot walks to the ball, stops, and then kicks. In the second, it kicks the ball while moving. We used state-of-the-art algorithms - Proximal Policy Optimization and Soft Actor Critic - to solve these complex problems and show their applicability in the context of RoboCup. Obtained results have shown very significant improvements over previously used behaviors by FC Portugal 3D team. The new kick in motion executes 5 times faster than the previous kick, and the new 6D pose estimator has an average error of just 6.3mm, a reduction of more than 97%.

Abstract
Data-driven decision support systems rely on increasing amounts of information that needs to be converted into actionable knowledge in business intelligence processes. The latter have been applied to diverse business areas, including governmental organizations, where they can be used effectively. The Portuguese Food and Economic Safety Authority (ASAE) is one example of such organizations. Over its years of operation, a rich dataset has been collected which can be used to improve their activity regarding prevention in the areas of food safety and economic enforcement. ASAE needs to inspect Economic Operators all over the country, and the efficient and effective generation of optimized and flexible inspection routes is a major concern. The focus of this paper is, thus, the generation of optimized inspection routes, which can then be flexibly adapted towards their operational accomplishment. Each Economic Operator is assigned an inspection utility – an indication of the risk it poses to public health and food safety, to business practices and intellectual property as well as to security and environment. Optimal inspection routes are then generated typically by seeking to maximize the utility gained from inspecting the chosen Economic Operators. The need of incorporating constraints such as Economic Operators’ opening hours and multiple departure/arrival spots has led to model the problem as a Multi-Depot Periodic Vehicle Routing Problem with Time Windows. Exact and meta-heuristic methods were implemented to solve the problem and the Genetic Algorithm showed a high performance with realistic solutions to be used by ASAE inspectors. The hybrid approach that combined the Genetic Algorithm with the Hill Climbing also showed to be a good manner of enhancing the solution quality. © 2021, Springer Nature Switzerland AG.