IEEE CLOUD 2023 – The 16th International Conference on Cloud Computing (CLOUD) was held from July 2nd to 8th, 2023 in Chicago, USA as a hybrid event. In the context of the H2020 PUZZLE Project, Dr. Ioannis Konstantinou, on behalf of our consortium partner ICCS/CSLAB presented his work entitled “Deep Reinforcement Learning in Cloud Elasticity through Offline Learning and Return Based Scaling” in front of more than 50 attendees. The work is available at this link.
ABSTRACT:
Elastic resource allocation is a desirable feature of cloud environments, and one of the main reasons for them widespread adoption. Resource elasticity allows for adaptive and real-time infrastructure scaling that can follow workload fluctuations in a cost-effective manner without sacrificing performance. Promising Machine Learning (ML) approaches employ Reinforcement Learning (RL) where an agent interacts with the cloud environment by employing actions, observing the reward of their outcome and modifying its strategy accordingly.
Nevertheless, one of the main problems of RL in this setup is that to acquire a sufficient initial knowledge of the environment, the agent needs to perform numerous time-consuming and performance-degrading interactions with the cloud. In this work we design and implement RBS-CQL, and Deep-RL Kubernetes agent system to monitor and automatically scale the containers of a NoSQL application according to incoming workload. We combine training optimization techniques from contemporary literature as well as offline RL algorithms to reduce the training with that. We provide empirical results that show that RBS-CQL achieves systematic improvement of more than 10% compared to its online equivalent for a given number of experiences and that it is able to extract improved decision-making policies even from data of lower quality.
The authors of this work are Miltiadis Chrysopoulos, CSLAB NTUA, Athens, Greece; Ioannis Konstantinou, University of Thessaly Dept. of Informatics & Telecommunications Lamia, Greece and Nectarios Koziris CSLAB NTUA Athens, Greece.
Regarding IEEE CLOUD 2023
The IEEE International Conference on Cloud Computing (CLOUD) acts as an important international forum for researchers and industry practitioners to exchange the most recent fundamental developments in the state of the art and practice of cloud computing, identify emerging research topics, and determine the cloud computing industry’s future. Cloud computing-related subjects are all consistent with the CLOUD theme. In 2023, the event strives to advance the largest international professional forum on cloud computing, Cloud 2023 Program.
IEEE CLOUD is the flagship conference focusing on innovative cloud computing across all “as a service” categories (XaaS). This includes modeling, developing, publishing, monitoring, managing, delivering, and scaling XaaS offerings from the core to the edge. The conference is a prime international forum for researchers, academics, businesses, industry, and standard bodies to exchange the latest fundamental advances in the state of the art and practice of cloud computing, identify emerging research topics, and define the future of cloud computing.
IEEE CLOUD 2023 #ieeecloud2023 invited original papers addressing all aspects of cloud computing infrastructure, applications, business innovations, and much more.
