Intelligent Replication Management for HDFS Using Reinforcement Learning

08/19/2020
by   Hyunsung Lee, et al.
0

Storage systems for cloud computing merge a large number of commodity computers into a single large storage pool. It provides high-performance storage over an unreliable, and dynamic network at a lower cost than purchasing and maintaining large mainframe. In this paper, we examine whether it is feasible to apply Reinforcement Learning(RL) to system domain problems. Our experiments show that the RL model is comparable, even outperform other heuristics for block management problem. However, our experiments are limited in terms of scalability and fidelity. Even though our formulation is not very practical,applying Reinforcement Learning to system domain could offer good alternatives to existing heuristics.

READ FULL TEXT
research
01/12/2022

Efficient Hierarchical Storage Management Framework Empowered by Reinforcement Learning

With the rapid development of big data and cloud computing, data managem...
research
11/09/2020

Challenges of Applying Deep Reinforcement Learning in Dynamic Dispatching

Dynamic dispatching aims to smartly allocate the right resources to the ...
research
08/23/2018

LIFT: Reinforcement Learning in Computer Systems by Learning From Demonstrations

Reinforcement learning approaches have long appealed to the data managem...
research
09/04/2019

Inductive Bias-driven Reinforcement Learning For Efficient Schedules in Heterogeneous Clusters

The problem of scheduling of workloads onto heterogeneous processors (e....
research
09/14/2022

Analysis of Reinforcement Learning for determining task replication in workflows

Executing workflows on volunteer computing resources where individual ta...
research
05/18/2021

OpenGraphGym-MG: Using Reinforcement Learning to Solve Large Graph Optimization Problems on MultiGPU Systems

Large scale graph optimization problems arise in many fields. This paper...
research
12/15/2020

Gegelati: Lightweight Artificial Intelligence through Generic and Evolvable Tangled Program Graphs

Tangled Program Graph (TPG) is a reinforcement learning technique based ...

Please sign up or login with your details

Forgot password? Click here to reset