Abstract
There are many open issues and challenges in the multi-agent reward-based learning field. Theoretical convergence guarantees are lost, and the complexity of the action-space is also exponential to the amount of agents calculating their optimal joint-action. Function approximators, such as deep neural networks, have successfully been used in singleagent environments with high dimensional state-spaces. We propose the Multi-agent Double Deep Q-Networks algorithm, an extension of Deep Q-Networks to the multi-agent paradigm. Two common techniques of multi-agent Q-learning are used to formally describe our proposal, and are tested in a Foraging Task and a Pursuit Game. We also demonstrate how they can generalize to similar tasks and to larger teams, due to the strength of deep-learning techniques, and their viability for transfer learning approaches. With only a small fraction of the initial task's training, we adapt to longer tasks, and we accelerate the task completion by increasing the team size, thus empirically demonstrating a solution to the complexity issues of the multi-agent field.

Abstract

Abstract
This paper provides a bare-bone introduction to evolutionary and bio-inspired metaheuristic in the context of environmental greenhouse control. Besides presenting general evolutionary algorithm principles, specific details are provided regarding the genetic algorithm, particle swarm optimization and differential evolution techniques. A review of these algorithms within greenhouse control applications is presented, both for single and multiple objectives, as well as current trends.

Abstract
The smart city seeks a highly interconnected, monitored and globally optimized environment to profit from the synergies among systems such as energy, transports or waste management. From an energy perspective, transport systems and facilities are among the bigger energy consumers inside cities. However, despite the research available on such systems, few works focus on their interactions and potential synergies to increase their efficiencies. This paper address this problem by assessing the benefits of the interconnection and joint management of different energy systems in a smart city context. This is done using a linear programming problem, modelling a district with residential loads, distributed energy resources (DER) and electric vehicles (EV), which are also connected to an electrical metro substation. This connection allows to store the metro regenerative braking energy into EVs' batteries to be used later for other trains or for the EVs themselves. The objective of the linear programming model is to find the optimal planning and operation of all the considered systems, achieving minimum energy costs. Therefore, the main contributions of this paper are the assessment of synergies of the interconnection of these systems and the detailed analysis of the impact of different EV penetration levels. Results show important economic benefits for the overall system (up to 30%) when the investments and its operation are globally optimized, especially reducing the metro energy costs. Also, analysing the energy transfers between metro-EV, it is evident that the metro takes advantages of the cheaper energy coming from the district (through the EVs), showing the existence of “opportunistic” synergies. Lastly, EV saturation points (where extra EVs represent more load but do not provide additional useful storage to the system) can be relatively small (200–300 EVs) when the energy transfer to the metro electrical substation is restricted, but it is also reduced by the presence of DER systems. © 2017 Elsevier Ltd

Abstract
Agent-based modeling (ABM) is a bottom-up modeling approach, where each entity of the system being modeled is uniquely represented as an independent decision-making agent. Large scale emergent behavior in ABMs is population sensitive. As such, the number of agents in a simulation should be able to reflect the reality of the system being modeled, which can be in the order of millions or billions of individuals in certain domains. A natural solution to reach acceptable scalability in commodity multi-core processors consists of decomposing models such that each component can be independently processed by a different thread in a concurrent manner. In this paper we present a multithreaded Java implementation of the PPHPC ABM, with two goals in mind: (1) compare the performance of this implementation with an existing NetLogo implementation; and, (2) study how different parallelization strategies impact simulation performance on a shared memory architecture. Results show that: (1) model parallelization can yield considerable performance gains; (2) distinct parallelization strategies offer specific trade-offs in terms of performance and simulation reproducibility; and, (3) PPHPC is a valid reference model for comparing distinct implementations or parallelization strategies, from both performance and statistical accuracy perspectives.

Abstract