英语(全英文)architecture-design-of-global-distributed-storage-system-for-data-grid.doc
《英语(全英文)architecture-design-of-global-distributed-storage-system-for-data-grid.doc》由会员分享,可在线阅读,更多相关《英语(全英文)architecture-design-of-global-distributed-storage-system-for-data-grid.doc(21页珍藏版)》请在咨信网上搜索。
1、英语毕业论文Architecture Design of Global Distributed Storage System for Data GridLongbo Ran, Hai Jin, Zhiping Wang, Chen Huang, Yong Chen, and Yongjie JiaInternet and Cluster Computing CenterHuazhong University of Science and Technology, Wuhan 430074, ChinaEmail: hjinAbstractData grids are becoming incre
2、asingly important for sharing large data collections, archiving and disseminating. In this paper we describe architecture of global distributed storage system for data grid. We focus on the management and the capability for the maximum users and maximum resources on the Internet, as well as performa
3、nce and other issues.Keywords: Data grids, Match tree, Metadata, Name space1. IntroductionData-intensive, high-performance computing applications require the efficient management and transfer of terabytes or petabytes of information in wide-area, distributed computing environments 128. Examples of d
4、ata-intensive applications include experimental analyses and simulations in several scientific disciplines, such as high-energy physics, climate modeling, earthquake engineering and astronomy 23. These applications share several requirements. Massive data sets must be shared by a large community of
5、hundreds or thousands of users distributed around the world. Data grids are becoming increasingly important for sharing large data collections, archiving and disseminating.Researches on massive storage system have gained significant achievements. There are already a number of storage systems used by
6、 the grid community, each of which was designed to satisfy specific needs and requirements for storing, transferring and accessing large datasets. These include Distributed Parallel Storage System (DPSS) and High Performance Storage System (HPSS), which provide high performance access to data and ut
7、ilize parallel data transfer and/or striping across multiple servers to improve performance 428. Distributed File System (DFS) supports high-volume usage, dataset replication and local caching. OceanStore is a global persistent data store designed to scale to billions of users. It provides a consist
8、ent, high available, and durable storage utility atop an infrastructure comprised of untrusted servers 5. GridFTP is a high-performance, secure, reliable data transfer protocol optimized for high-bandwidth wide-area networks 68. Storage Resource Broker (SRB) connects heterogeneous data collections,
9、provides a uniform client interface to storage repositories, and provides a metadata catalog for describing and locating data within the storage system 4. Other systems allow clients to access structured data from a variety of underlying storage systems.In this paper, we present a novel architecture
10、 of a global distributed storage system built atop SAN, NAS, or any other storage systems, called Global Storage Provider (GSP). We provide a data management service in the data grid environment. Our purpose is to construct a distribute storage system with high scalability, high security, high effic
11、iency, which offers a high quality storage service to millions of users over Internet. The scalability and efficiency of global name space and the meta-data service is discussed in detail in order to provide easy and efficient access or share of files to the wide area storage system.We give a user a
12、nd group-based multi-namespace architecture, and develop a new approach to solve the bottleneck problem of metadata server. A new component, called Storage Service Provider (SSP), is introduced to supply storage service to users, and plays as a user agent to the storage system. The data sharing and
13、access control among different users and groups are completed by the combination of user-based access control methods and role-based access control methods. To supply different QoS to different users, files can be replicated, clipped, and stored in different storage pools. A prototype has been devel
14、oped to provide extended ftp service to end users and some simple file APIs.The rest of the paper is organized as follows. Section 2 describes the design principles of GSP. Section 3 details the architecture of GSP. Section 4 discusses the relate works closely to our project. Section 5 ends with the
15、 current state of our project and the future work.2. Design Principles of Global Storage ProviderGSP is a middleware to unify heterogeneous storage resources to provide huge available storage resources for enormous users. The system provides high availability, high expansibility and high speed.2.1 G
16、SP InterfacesIn the global data grid environment there are many kinds of storage resources existing on different platforms. Uniform interface for the users access different resources transparently must be provided.In order to meet the requirements of different applications, three kinds of interfaces
17、 are needed. First, standard FTP interface is needed, because most general methods for access storage resources through network is still by using FTP. This is why GridFTP chooses FTP protocol as the basic protocol 7. Second, interface like file system is needed. This is mainly because for many speci
18、al circumstances, file system interface makes it easy and convenient to develop the applications. Third, parallel file interface is also needed, as many high performance computing applications need parallel file interface.2.2 Metadata Server of GSPFor a global storage system the huge storage resourc
19、e must be managed effectively 913. In our system directory-based metadata server (MS) is adopted to store the metadata. MS contains many important metadata such as file logical view, data location, file slices, file copies, file content abstract. The search efficiency affects the whole efficiency of
20、 the system, and MS easily becomes the bottleneck of the whole system on expansibility and availability. With the resources increasing in the system, the information of the files and the directories become enormous, a good approach to store and search the metadata efficiently is needed.In many syste
21、ms, such as SRB 7 or GridFTP 6, hierarchical directory structure is adopted. Generally when the metadata becomes enormous, the system employs several metadata servers. There are several limitations for directory servers. One is that it must keep the logic tree among the directory servers; the other
22、is that the directory servers must cooperate and the result will return from the root node that adds the overload to the root server; the third is that when the root server is out of service the whole meta server will out of service too, so it is difficult to provide high availability; and the last
23、is that it is difficult to expand the scale of meta servers.SoftSysNet3comMusicCn enOldJackSwitchpopRoot DS1 DS2 DS3DS4 Readme Figure 1 Example of Metadata Server Logical StructureIn this paper, we bring out an algorithm called match tree. Figure 1 is a logical structure of a MS stored on 4 director
24、y servers, showing the file structure of a user or a group. For example, if we want to access a file rootsoftsysnet3comswitchreadme, the process will trace from DS1 to DS3, and then reach DS4, finally return the metadata from DS1 to user. Match tree is kept in the memory of the scheduler. Figure 2 s
- 配套讲稿:
如PPT文件的首页显示word图标,表示该PPT已包含配套word讲稿。双击word图标可打开word文档。
- 特殊限制:
部分文档作品中含有的国旗、国徽等图片,仅作为作品整体效果示例展示,禁止商用。设计者仅对作品中独创性部分享有著作权。
- 关 键 词:
- 英语 英文 architecture design of global distributed storage system for data grid
1、咨信平台为文档C2C交易模式,即用户上传的文档直接被用户下载,收益归上传人(含作者)所有;本站仅是提供信息存储空间和展示预览,仅对用户上传内容的表现方式做保护处理,对上载内容不做任何修改或编辑。所展示的作品文档包括内容和图片全部来源于网络用户和作者上传投稿,我们不确定上传用户享有完全著作权,根据《信息网络传播权保护条例》,如果侵犯了您的版权、权益或隐私,请联系我们,核实后会尽快下架及时删除,并可随时和客服了解处理情况,尊重保护知识产权我们共同努力。
2、文档的总页数、文档格式和文档大小以系统显示为准(内容中显示的页数不一定正确),网站客服只以系统显示的页数、文件格式、文档大小作为仲裁依据,平台无法对文档的真实性、完整性、权威性、准确性、专业性及其观点立场做任何保证或承诺,下载前须认真查看,确认无误后再购买,务必慎重购买;若有违法违纪将进行移交司法处理,若涉侵权平台将进行基本处罚并下架。
3、本站所有内容均由用户上传,付费前请自行鉴别,如您付费,意味着您已接受本站规则且自行承担风险,本站不进行额外附加服务,虚拟产品一经售出概不退款(未进行购买下载可退充值款),文档一经付费(服务费)、不意味着购买了该文档的版权,仅供个人/单位学习、研究之用,不得用于商业用途,未经授权,严禁复制、发行、汇编、翻译或者网络传播等,侵权必究。
4、如你看到网页展示的文档有www.zixin.com.cn水印,是因预览和防盗链等技术需要对页面进行转换压缩成图而已,我们并不对上传的文档进行任何编辑或修改,文档下载后都不会有水印标识(原文档上传前个别存留的除外),下载后原文更清晰;试题试卷类文档,如果标题没有明确说明有答案则都视为没有答案,请知晓;PPT和DOC文档可被视为“模板”,允许上传人保留章节、目录结构的情况下删减部份的内容;PDF文档不管是原文档转换或图片扫描而得,本站不作要求视为允许,下载前自行私信或留言给上传者【精****】。
5、本文档所展示的图片、画像、字体、音乐的版权可能需版权方额外授权,请谨慎使用;网站提供的党政主题相关内容(国旗、国徽、党徽--等)目的在于配合国家政策宣传,仅限个人学习分享使用,禁止用于任何广告和商用目的。
6、文档遇到问题,请及时私信或留言给本站上传会员【精****】,需本站解决可联系【 微信客服】、【 QQ客服】,若有其他问题请点击或扫码反馈【 服务填表】;文档侵犯商业秘密、侵犯著作权、侵犯人身权等,请点击“【 版权申诉】”(推荐),意见反馈和侵权处理邮箱:1219186828@qq.com;也可以拔打客服电话:4008-655-100;投诉/维权电话:4009-655-100。
链接地址:https://www.zixin.com.cn/doc/2554836.html