GRAPES全球格点模式的并行计算负载平衡策略

Parallelism of the GRAPSE global model and its loading balance strategy

中文关键词:  GRAPES模式， 并行计算，消息传递组通讯，负载平衡，IBM cluster1600

英文关键词:GRAPES model, Parallel computing, Group message passing, Load balance, IBM cluster1600

中文摘要:

      随着高性能计算机技术的发展和应用，并行计算已成为保证数值天气预报模式业务运行时效的关键技术之一。目前高性能计算机计算能力已达到每秒千万亿次浮点计算，系统中处理器数量也早已达十万甚至更多，如此巨大的计算资源对应用软件系统的设计也提出了挑战。数值天气预报软件系统要充分利用高性能计算机提供的计算资源，必须依靠并行计算方法，这包括适合计算问题的可扩展并行算法的设计、合适的数据分配方案以及良好的任务负载平衡方案。作为中国新一代数值天气预报格点模式，GRAPES(Global and Regional Assimilation and PrEdiction System)设计的最终目标是一个科研/业务通用，区域/全球通用模式。作为一个格点模式，GRAPES的并行计算具有与欧洲中期数值预报研究中心谱模式并行计算不同的特点，GRAPES的并行计算采用了经典的水平网格数据划分。但对于全球的GRAPES模式，由于采用拉格朗日差分方案，模式极地及附近区域格点与格点之间距离的减小，使得模式并行计算在采用简单的经纬网格划分方式实现时,必须考虑极地区域并行计算跨越多个处理器时导致的频繁通讯解决途径。本研究提出了利用消息传递组通讯实现全球格点模式并行计算的一种方法，其核心思想是将极点附近一定区域内的处理器按纬向划归不同的处理器组。文中还给出了该实现方法的任务分配算法，提出了改进的任务分配负载平衡方案。在中国气象局高性能计算机IBM cluster1600上的测试表明，算法具有较好的可扩展性，其负载平衡方案改善了计算的绝对墙钟时间，使并行计算效率提高10%以上。模式的准业务运行结果表明计算墙钟时间基本可以满足数值预报业务的实时性要求。

英文摘要:

      For the past 10 years, the peak floating point operation execution rate (floating point operations per second, or FLOPS ) in parallel computers has increased by several orders of magnitude. Their large-scale applications in science rely on larger configuration of the supercomputers which comprises more than a hundred thousand of processors. Development of parallel software has been thought of as time and effort intensive. But the parallel algorithm design involves more than just using multiple processors, it also focuses on its efficiency and scalability. Setting load balance tasks' assigning can commonly speedup the running. The GRAPES (Global and Regional Assimilation and PrEdiction System) global model is a semi implicit semi Lagrangian numerical prediction model formulated in spherical coordinates. For the parallelism of the GRAPES software system, there are two problems must to be solved, one is how to gather the data when computing the up stream points near the south and north poles along Lagrangian trajectory, and another is how to set the values of periodic boundary latitudinally and of symmetry boundary longitudinally. Due to the convergence of the meridians, the longitudinal grid size decreases toward zero as the poles are approached. So the parallelism near the poles is a tough issue. How to get a better running time at the high performance computer system is a key issue for the GRAPES software development. In this paper, a new method is proposed, which is based on group message passing of MPI. A more efficient load balance solution of assignment is also discussed. Experiments on the IBM cluster 1600 of Chinese Meteorology Administration (CMA) showed that the parallel algorithm is scalable, efficient and portable. Its computing time cost can be acceptable for real time weather forecast.