DATA MIGRATION FOR LARGE SCIENTIFIC DATASETS IN CLOUDS
Journal Title: Azerbaijan Journal of High Performance Computing - Year 2018, Vol 1, Issue 1
Abstract
Transferring large data files between various storages including cloud storages is an important task both for academic and commercial users. This should be done in an efficient and secure way. The paper describes Data Avenue that fulfills all these conditions. Data Avenue can efficiently transfer large files even in the range of TerraBytes among storages having very different access protocols (Amazon S3, OpenStack Swift, SFTP, SRM, iRODS, etc.). It can be used in personal, organizational and public deployment with all the security mechanisms required for these usage configurations. Data Avenue can be used by a GUI as well as by a REST API. The paper describes in detail all these features and usage modes of Data Avenue and also provides performance measurement results proving the efficiency of the tool that can be accessed and used via several public web pages.
Authors and Affiliations
Akos Hajnal, Eniko Nagy, Peter Kacsuk, Istvan Marton
Keywords
Related Articles
- EP ID EP525573
- DOI 10.32010/26166127.2018.1.1.66.86
- Views 66
- Downloads 0
THE INFLUENCE OF EXASCALE ON RESOURCE DISCOVERY AND DEFINING AN INDICATOR
Resource discovery in distributed Exascale computing systems, in addition to managing events resulting in failures due to not finding the resource, as well as failure to perform resource discovery activities at the accep...
A SURVEY OF RESOURCE MANAGEMENT CHALLENGES IN MULTI-CLOUD ENVIRONMENT: TAXONOMY AND EMPIRICAL ANALYSIS
HOME ABOUT SUBMISSION SEARCH CURRENT ARCHIVES A SURVEY OF RESOURCE MANAGEMENT CHALLENGES IN MULTI-CLOUD ENVIRONMENT: TAXONOMY AND EMPIRICAL ANALYSIS Hits: 159 PDF Volume 1 (1), July 2018, Pages 51-65 Bandar Aldawsari1,...
LOAD BALANCING IN DISTRIBUTED EXASCALE COMPUTING BASED ON PROCESS REQUIREMENTS
In distributed Exascale systems, the occurrence of a dynamic and interactive nature changes the workload of the system’s computing elements. Because of this, the load balancer needs to collect information on the system s...
EXAMIG MATRIX: PROCESS MIGRATION BASED ON MATRIX DEFINITION OF SELECTING DESTINATION IN DISTRIBUTED EXASCALE ENVIRONMENTS
In traditional computing system, load balancer, interim selecting the process, determine the destination computing node based on describing Indicators process status. In distributed Exascale computing system, due to the...
CHALLENGES OF RESOURCE DISCOVERY TO SUPPORT DISTRIBUTED EXASCALE COMPUTING ENVIRONMENT
The resource discovery management unit (RD) in distributed Exascale systems needs to be able to manage the occurrence of dynamic and interactive events in the requesting process when running activities related to RD. The...