Abstract

Abstract
The current energy scenario requires actions towards the reduction of energy consumption and the use of renewable resources. In this context, a microgrid is a self-sustained network that can operate connected to the smart grid or in isolation. The long-term scheduling of on/off cycles of devices is a critical problem that has been commonly addressed by centralized approaches. In this work, we propose a novel agent-based method to solve the long-term scheduling problem as a distributed constraint optimization problem (DCOP) by modelling future system configurations rather than reacting to changes. Moreover, with respect to approaches based on decentralised reinforcement learning, we can directly encode system-wide hard constraints (such as for example the Kirchhoff law) which are not easy to represent in a factored representation of the problem. We compare different multi-agent DCOP algorithms showing that the proposed method can find optimal/near-optimal solutions for a specific case study.

Abstract
Optimizing the participation of a large number of prosumers in the electricity markets is a challenging problem, especially for portfolios with thousands or millions of flexible resources. To address this problem, this paper proposes a cluster-based optimization approach to support an aggregator in the definition of demand and supply bids for the day-ahead energy market. This approach consists of two steps. In the first step, the aggregated flexibility of the entire portfolio is computed by a centroid-based clustering algorithm. In the second step, the supply and demand bids are defined by an optimization model that can assume the form of a deterministic or a two-stage stochastic problem. A case study of 10,000 prosumers from the Iberian market is used to evaluate and compare the performance of the bidding optimization models with and without pre-clustering. The numerical results show that the optimized bidding strategies outperform an inflexible strategy by more than 20% of cost savings. The centroid-based clustering algorithm reduces effectively the execution times of the bidding optimization problems, without affecting the quality of the energy bids.

Abstract
Typical simultaneous lotsizing and scheduling models consider the limited capacity of the production system by respecting a maximum time the respective machines or production lines can be available. Further limitations of the production quantities can arise by the scarce availability of, e.g., setup tools, setup operators or raw materials which thus cannot be neglected in optimization models. In the literature on simultaneous lotsizing and scheduling, these production factors are called secondary resources. This paper provides a structured overview of the literature on simultaneous lotsizing and scheduling involving secondary resources. The proposed classification yields for the first time a unified view of scarce production factors. The insights about different types of secondary resources help to develop a new model formulation generalizing and extending the currently used approaches that are specific for some settings. Some illustrative examples demonstrate the functional principle and flexibility of this new formulation which can thus be used for a wide range of applications.

Abstract
This paper investigates the optimal allocation of Spinning Reserve (SR) for power systems in the presence of Renewable Energy Sources (RES) and Electrical Energy Storage (EES) devices. This is done in order to reduce the system's dependency on thermal generation units and the decrease total daily operational cost. A Security Constrained Unit Commitment (SCUC) model for a typical power system was used, which includes thermal and renewable generation units and EES devices in the form of batteries. In the proposed model, the hourly operation strategy is determined by adopting a predetermined level of SR. In order to optimize SR requirements, the Independent System Operator (ISO) runs the SCUC problem and determines the minimum SR that should be provided by generation units and EES devices. The simulation results illustrate that by optimizing the operation of batteries, the ISO can effectively reduce the required capacity of thermal units. Therefore, optimal SR allocation under RES uncertainty is determined in this study.

Abstract
When compared with their single-agent counterpart, multi-agent systems have an additional set of challenges for reinforcement learning algorithms, including increased complexity, non-stationary environments, credit assignment, partial observability, and achieving coordination. Deep reinforcement learning has been shown to achieve successful policies through implicit coordination, but does not handle partial-observability. This paper describes a deep reinforcement learning algorithm, based on multi-agent actor-critic, that simultaneously learns action policies for each agent, and communication protocols that compensate for partial-observability and help enforce coordination. We also research the effects of noisy communication, where messages can be late, lost, noisy, or jumbled, and how that affects the learned policies. We show how agents are able to learn both high-level policies and complex communication protocols for several different partially-observable environments. We also show how our proposal outperforms other state-of-the-art algorithms that don't take advantage of communication, even with noisy communication channels.