Deploying in-network caches in support of distributed scientific data sharing

03/14/2022
by   Alex Sim, et al.
0

The importance of intelligent data placement, management, and analysis has become apparent as scientific data volumes across the network continue to increase. To that end, we describe the use of in-network caching service deployments as a means to improve application performance and preserve available network bandwidth in a high energy physics data distribution environment. Details of the software and hardware deployments, performance considerations, and cache usage analysis will be described. We include thoughts on possible future deployment models involving caching node installations at the edge along with methods to scale our approach.

READ FULL TEXT

page 2

page 3

research
05/11/2022

Access Trends of In-network Cache for Scientific Data

Scientific collaborations are increasingly relying on large volumes of d...
research
08/02/2022

Smart caching in a Data Lake for High Energy Physics analysis

The continuous growth of data production in almost all scientific areas ...
research
02/26/2019

Rucio - Scientific Data Management

Rucio is an open source software framework that provides scientific coll...
research
05/03/2021

Analyzing scientific data sharing patterns for in-network data caching

The volume of data moving through a network increases with new scientifi...
research
05/01/2023

Analyzing Transatlantic Network Traffic over Scientific Data Caches

Large scientific collaborations often share huge volumes of data around ...
research
04/07/2020

DataFed: Towards Reproducible Research via Federated Data Management

The increasingly collaborative, globalized nature of scientific research...
research
10/31/2020

A Qualitative Study of Application-level Caching

Latency and cost of Internet-based services are encouraging the use of a...

Please sign up or login with your details

Forgot password? Click here to reset