Effect of reordering internal messages in MPI broadcast according to the load imbalance

Takesi Soga, Takeshi Nanri, Motoyoshi Kurokawa, Kazuaki Murakami

    研究成果: Chapter in Book/Report/Conference proceedingConference contribution

    抄録

    To achieve higher scalability of parallel programs on large scale parallel computers, reducing the time spent for collective communications is one of the most important issue. In this paper, a dynamic optimization method to adjust the implementation of Broadcast operation, one of the most popular collective communications, is introduced. Though there have been many attempts to speed up this operation, they assume that each rank starts this operation at the same time. However, in real execution, the time can be different because of load-imbalance among ranks. This paper first claims that this difference can cause increase of the cost for this operation. Then, as a method to avoid this problem, an optimization method that adjusts the order of point-to-point messages in Broadcast operations is introduced. This method uses the wait time of each rank at the operation to determine the status of load-imbalance. From the results of experiments, it is shown that this optimization method can reduced the time for the operation. In addition to that, it is also shown that the effect of the optimization depends on the size of data to be broadcasted and the amount of load-imbalance.

    本文言語英語
    ホスト出版物のタイトルInnovative Architecture for Future Generation High-Performance Processors and Systems, IWIA 2008
    ページ11-16
    ページ数6
    DOI
    出版ステータス出版済み - 12 1 2008
    イベント2008 International Workshop on Innovative Architecture for Future Generation High Performance Processors and Systems, IWIA'08 - Hilo, HI, 米国
    継続期間: 1 21 20081 23 2008

    出版物シリーズ

    名前Proceedings of the Innovative Architecture for Future Generation High-Performance Processors and Systems

    その他

    その他2008 International Workshop on Innovative Architecture for Future Generation High Performance Processors and Systems, IWIA'08
    Country米国
    CityHilo, HI
    Period1/21/081/23/08

    All Science Journal Classification (ASJC) codes

    • Hardware and Architecture

    フィンガープリント 「Effect of reordering internal messages in MPI broadcast according to the load imbalance」の研究トピックを掘り下げます。これらがまとまってユニークなフィンガープリントを構成します。

    引用スタイル