Yassine Baghoussi

Abstract
Federated Learning (FL) enables decentralized model training while preserving data privacy but remains susceptible to poisoning attacks. Malicious clients can manipulate local data or model updates, threatening FL's reliability, especially in privacy-sensitive domains like healthcare and finance. While client-side optimization algorithms play a crucial role in training local models, their resilience to such attacks is underexplored. This study empirically evaluates the robustness of three widely used optimization algorithms: SGD, Adam, and RMSProp-against label-flipping attacks (LFAs) in image classification tasks using Convolutional Neural Networks (CNNs). Through 900 individual runs in both federated and centralized learning (CL) settings, we analyze their performance under Independent and Identically Distributed (IID) and Non-IID data distributions. Results reveal that SGD is the most resilient, achieving the highest accuracy in 87% of cases, while Adam performs best in 13%. Additionally, centralized models outperform FL on CIFAR-10, whereas FL excels on Fashion-MNIST, highlighting the impact of dataset characteristics on adversarial robustness.

Abstract
Traditional recurrent neural networks operate as passive observers of data, unable to modify the information they learn from despite errors that may arise from suboptimal input representations. We introduce Read & Write LSTM (read-write LSTM), a new variant within the family of read & write machine learning (RW-ML) architectures that address this fundamental limitation by integrating input modification directly into the backpropagation process. Read-write LSTM establishes a dynamic feedback loop where input representations evolve alongside model weights through gradient transformation mechanisms. Our approach introduces a principled gradient scaling framework with an adaptive correction rate that carefully controls the extent of data modification, preserving data integrity while enhancing representational power. We comprehensively evaluate read-write LSTM against traditional LSTMs and state-of-the-art transformer models on the M4 competition and Numenta Anomaly Benchmark datasets, demonstrating significant improvements in forecasting accuracy. Notably, read-write LSTM consistently out-performs standard LSTM models in over 70% of time series with complex patterns and achieves superior performance on 55% of anomaly-rich datasets. Through extensive experimentation and analysis, we establish both the theoretical foundations and practical benefits of integrating data modification with neural computation, paving the way for a new generation of adaptive learning systems that actively reshape their inputs rather than merely adapting to them.

Abstract

Abstract
Traditional recurrent neural networks (RNNs) are essential for processing time-series data. However, they function as read-only models, lacking the ability to directly modify the data they learn from. In this study, we introduce the corrector long short-term memory (cLSTM), a Read & Write LSTM architecture that not only learns from the data but also dynamically adjusts it when necessary. The cLSTM model leverages two key components: (a) predicting LSTM’s cell states using Seasonal Autoregressive Integrated Moving Average (SARIMA) and (b) refining the training data based on discrepancies between actual and forecasted cell states. Our empirical validation demonstrates that cLSTM surpasses read-only LSTM models in forecasting accuracy across the Numenta Anomaly Benchmark (NAB) and M4 Competition datasets. Additionally, cLSTM exhibits superior performance in anomaly detection compared to hierarchical temporal memory (HTM) models.

Abstract
Forecasting methods are affected by data quality issues in two ways: 1. they are hard to predict, and 2. they may affect the model negatively when it is updated with new data. The latter issue is usually addressed by pre-processing the data to remove those issues. An alternative approach has recently been proposed, Corrector LSTM (cLSTM), which is a Read & Write Machine Learning (RW-ML) algorithm that changes the data while learning to improve its predictions. Despite promising results being reported, cLSTM is computationally expensive, as it uses a meta-learner to monitor the hidden states of the LSTM. We propose a new RW-ML algorithm, Kernel Corrector LSTM (KcLSTM), that replaces the meta-learner of cLSTM with a simpler method: Kernel Smoothing. We empirically evaluate the forecasting accuracy and the training time of the new algorithm and compare it with cLSTM and LSTM. Results indicate that it is able to decrease the training time while maintaining a competitive forecasting accuracy.