热门搜索: 论文 发表 社科期刊 北大核心 南大核心 cssci 科技期刊 教育

当前位置:主页 > 科技论文 > 计算机论文 >

元数据服务器集群负载均衡技术的研究

发布时间:2018-08-11 11:30  文章来源:笔耕文化传播
【摘要】:随着信息技术的不断发展,全球信息存储量正在以57%的复合年增长率膨胀。大量数据密集型应用对现有的网络存储架构提出了更高的要求。对象存储技术同时具备传统SAN快速访问的特点和NAS跨平台数据共享和强安全性的优势。同时对象存储设备本身具备的智能管理功能。因此,对象存储系统具有高性能、可扩展、高可用性和强安全性等特点,是未来网络存储的发展方向。元数据服务器(MetaData Server, MDS)是对象存储系统的重要组成部分,在对象存储系统中的主要功能包括:管理文件系统名字空间、文件/目录和对象物理存储位置之间的映射、用户访问控制等。因此,MDS集群的处理能力是整个对象存储系统高效运作的重要一环。 本文分析了当前主流网络存储架构中元数据服务器集群的负载均衡方案。针对元数据服务器性能异构造成的集群负载不均衡问题,本文提出了一种基于权重的元数据服务器集群负载均衡策略(Weight-based Metadata Server Cluster Load Balancing Strategy, WLBS),并对其在Ceph框架中进行实现。本文的主要研究和开发内容包括以下四个方面: (1)在分析对象存储与传统网络存储技术的基础上,总结了MDS负载均衡技术所涉及的包括元数据分区和服务器热点评价两个关键问题; (2)提出了元数据服务器热点评测模型,主要解决准确获取服务负载状况的问题。该模型在传统计数法的基础上增加了对MDS性能值的考虑,提出了基于操作时延的MDS性能模型。该性能模型避免了传统“白盒”性能模型量化资源影响力的难题,准确评估服务器的当前性能值; (3)提出了一种基于权重负载均衡策略,主要解决MDS性能异构情况下的负载均衡问题。该策略根据(2)中热点评测模型获取节点当前的负载状况,并计算出集群的平均负载水平。为了实现负载与性能相对应的目标,高于平均负载水平的MDS会主动以目录子树迁移方式进行负载迁移,将一部分目录子树迁移到低于平均负载水平的MDS,最终实现异构MDS集群的负载均衡; (4)搭建实验平台,并对策略进行实验评测。对WLBS是否具备对异构MDS集群负载进行调整而使负载均衡化和保持MDS集群处理能力进行验证。 大量实验证明本文提出的负载均衡策略可以在异构环境下使元数据服务器集群保持负载均衡,提高了系统的元数据处理能力。
[Abstract]:With the development of information technology, global information storage is expanding at a compound annual growth rate of 57%. A large number of data-intensive applications put forward higher requirements for the existing network storage architecture. Object storage technology has the characteristics of traditional SAN fast access and NAS cross-platform data sharing and strong security advantages. At the same time, the object storage device itself has the intelligent management function. Therefore, object storage system has the characteristics of high performance, scalability, high availability and strong security, which is the development direction of network storage in the future. Metadata server (MetaData Server, MDS) is an important part of object storage system. Its main functions in object storage system include managing file system namespace, mapping file / directory and object physical storage location. User access control, etc. Therefore, the processing capability of MDS cluster is an important part of the efficient operation of the whole object storage system. This paper analyzes the load balancing scheme of metadata server cluster in current mainstream network storage architecture. Aiming at the problem of cluster load imbalance caused by heterogeneous metadata server performance, this paper proposes a load balancing strategy (Weight-based Metadata Server Cluster Load Balancing Strategy, WLBS),) for metadata server cluster based on weight and implements it in the framework of Ceph. The main research and development contents of this paper include the following four aspects: (1) based on the analysis of object storage and traditional network storage technology, This paper summarizes two key issues involved in MDS load balancing technology, including metadata partition and server hot spot evaluation. (2) A metadata server hot spot evaluation model is proposed, which mainly solves the problem of accurately obtaining service load status. Based on the traditional counting method, the MDS performance value is considered in this model, and the MDS performance model based on operation delay is proposed. The performance model avoids the problem of quantifying the resource influence of the traditional "white box" performance model and accurately evaluates the current performance value of the server. (3) A load balancing strategy based on weight is proposed. It mainly solves the load balancing problem in the case of heterogeneous MDS performance. According to the hot spot evaluation model in (2), the current load status of the node is obtained, and the average load level of the cluster is calculated. In order to achieve the goal of corresponding load and performance, MDS with higher than average load level will actively migrate the load in directory subtree mode. Part of the directory subtrees are migrated to MDSs which are below average load level, finally the load balance of heterogeneous MDS cluster is realized. (4) the experimental platform is built, and the policy is evaluated experimentally. This paper verifies whether WLBS has the ability to adjust the load of heterogeneous MDS cluster to balance the load and maintain the processing ability of MDS cluster. A large number of experiments show that the proposed load balancing strategy can keep the load balance of metadata server cluster in heterogeneous environment and improve the metadata processing ability of the system.
【学位授予单位】:杭州电子科技大学
【学位级别】:硕士
【学位授予年份】:2012
【分类号】:TP368.5

【参考文献】

相关期刊论文 前9条

1 郭全生,舒继武,毛希平,温冬蝉,郑纬民;基于LVS系统的负载动态平衡设计与实现[J];计算机研究与发展;2004年06期

2 杨德志,黄华,张建刚,许鲁;大容量、高性能、高扩展能力的蓝鲸分布式文件系统[J];计算机研究与发展;2005年06期

3 喻莉;石冰心;;基于Web服务器的性能模型与参数分析[J];计算机科学;1999年07期

4 唐丹,金海,张永坤;集群动态负载平衡系统的性能评价[J];计算机学报;2004年06期

5 郭成城,晏蒲柳;一种异构Web服务器集群动态负载均衡算法[J];计算机学报;2005年02期

6 曾国荪,263.net,陆鑫达;异构计算中的负载共享[J];软件学报;2000年04期

7 单志广,戴琼海,林闯,杨扬;Web请求分配和选择的综合方案与性能分析[J];软件学报;2001年03期

8 于磊,林宗楷,郭玉钗,林守勋;多服务器系统中的负载平衡与容错[J];系统仿真学报;2001年03期

9 王娟;冯丹;王芳;廖振松;;一种元数据服务器集群的负载均衡算法[J];小型微型计算机系统;2009年04期



本文编号:2176871


论文下载
论文发表
教材专著
专利申请


    下载步骤:1.微信扫码 2.备注编号 2176871. 3.下载文档
    注:1.必须备注编号,否则无法下载;2.扫码后10分钟即可下载,如有问题,点击微信联系客服。

    扩展阅读
    1. 元数据服务器集群负载均衡技术的研究 - 豆丁网
    2. 元数据服务器集群负载均衡技术的研究
    3. 对象文件系统中元数据服务器集群的负载均衡研究
    4. 基于LDAP的元数据服务器集群的研究与实现-手机知网
    5. 基于访问热点的pNFS元数据服务器集群优化技术研究-手机知网
    6. 基于HDFS的云存储关键技术研究—硕士毕业论文下载
    7. 基于开源软件构建高性能集群NAS系统,包括负载均衡(刘爱贵..._博客园
    8. 一种元数据服务器集群的负载均衡算法


    本文连接:http://www.bigengculture.com/kejilunwen/jisuanjikexuelunwen/2176871.html

    ×