Pufferscale: Rescaling HPC Data Services for High Energy Physics Applications - Université de Rennes Accéder directement au contenu
Communication Dans Un Congrès Année : 2020

Pufferscale: Rescaling HPC Data Services for High Energy Physics Applications

Résumé

User-space HPC data services are emerging as an appealing alternative to traditional parallel file systems, because of their ability to be tailored to application needs while eliminating unnecessary overheads incurred by POSIX compliance. The High Energy Physics (HEP) community is progressively turning towards such services to enable high-throughput accesses under heavy concurrency to billions of event data produced by instruments and consumed by subsequent analysis workflows. Such services would benefit from the possibility to be rescaled up and down to adapt to changing workloads, as experimental campaigns progress , in order to optimize resource usage. This paper formalizes rescaling a distributed storage system as a multi objective optimization problem considering three criteria: load balance, data balance, and duration of the rescaling operation. We propose a heuristic for rapidly finding a good approximate solution, while allowing users to weight the criteria as needed. The heuristic is evaluated with Pufferscale, a new rescaling manager for microservice-based distributed storage systems. To validate our approach in a real-world ecosystem, we showcase the use of Pufferscale as a means to enable storage malleability in the HEPnOS storage system for HEP applications.
Fichier principal
Vignette du fichier
CCGrid_2020_paper_140.pdf (298.92 Ko) Télécharger le fichier
Origine : Fichiers produits par l'(les) auteur(s)
Loading...

Dates et versions

hal-02961907 , version 1 (08-10-2020)

Identifiants

Citer

Nathanael Cheriere, Matthieu Dorier, Gabriel Antoniu, Stefan M. Wild, Sven Leyffer, et al.. Pufferscale: Rescaling HPC Data Services for High Energy Physics Applications. CCGRID 2020 - 20th IEEE/ACM International Symposium on Cluster, Cloud and Internet Computing, May 2020, Melbourne, Australia. pp.182-191, ⟨10.1109/CCGrid49817.2020.00-75⟩. ⟨hal-02961907⟩
95 Consultations
197 Téléchargements

Altmetric

Partager

Gmail Facebook X LinkedIn More