Code

update broadcast algo#447

Code

Draft

Binyang2014 wants to merge 27 commits intomainmicrosoft/mscclpp:mainfrom binyli/amd-testmicrosoft/mscclpp:binyli/amd-testCopy head branch name to clipboard

+116-92

Commits

Commits on Nov 9, 2024

• Lazily create the context stream

  chhwang

  committed
  76ede02

  Copy full SHA for 76ede02

• Use the default stream for CudaIpcConnection

  chhwang

  committed
  68db835

  Copy full SHA for 68db835

Commits on Dec 7, 2024

• Broadcast implementation underprogress for AMD GPUs.

  SreevatsaAnantharamu

  committed
  eccb783

  Copy full SHA for eccb783

• In-place bcast works fine!

  SreevatsaAnantharamu

  committed
  203f61c

  Copy full SHA for 203f61c

• Works fine when we run only one single message size

  SreevatsaAnantharamu

  committed
  8992d17

  Copy full SHA for 8992d17

• Removed channelOutOffset from buff.

  SreevatsaAnantharamu

  committed
  5b88be0

  Copy full SHA for 5b88be0

Commits on Dec 11, 2024

• Moved smChans to shared memory.

  SreevatsaAnantharamu

  committed
  ffffc94

  Copy full SHA for ffffc94

• Slightly improved performance by using more blocks.

  SreevatsaAnantharamu

  committed
  e446fb1

  Copy full SHA for e446fb1

Commits on Dec 12, 2024

• Expose nccl bcast api, uses broadcast (#409)

  pash-msft

  authored
  6090022

  Copy full SHA for 6090022

Commits on Dec 17, 2024

• Scratch buffer copy-based implementation of ncclBcast / ncclBroadcast (#417)

  Show description for b98b977
  SreevatsaAnantharamu

  authored
  b98b977

  Copy full SHA for b98b977

Commits on Jan 7, 2025

• Fused copy of the one-to-all algorithm

  SreevatsaAnantharamu

  committed
  e93add9

  Copy full SHA for e93add9

• Basic version that gives correct results!

  SreevatsaAnantharamu

  committed
  909cab5

  Copy full SHA for 909cab5

• Remove root rank from sync. Need to enhance further.

  SreevatsaAnantharamu

  committed
  ce7d823

  Copy full SHA for ce7d823

• Heavy bcast sync. Still performs better than RCCL for 114688 bytes.

  SreevatsaAnantharamu

  committed
  3a83816

  Copy full SHA for 3a83816

Commits on Jan 8, 2025

• Initial version.

  SreevatsaAnantharamu

  committed
  d5a17e1

  Copy full SHA for d5a17e1

• Initial version.

  SreevatsaAnantharamu

  committed
  9919c7f

  Copy full SHA for 9919c7f

• Setting unitBytesPerThread to 32 works but not 64.

  SreevatsaAnantharamu

  committed
  98cd5fd

  Copy full SHA for 98cd5fd

• Fixed the correctness issue

  SreevatsaAnantharamu

  committed
  1574891

  Copy full SHA for 1574891

• Fixed a bug for messages > 140M

  SreevatsaAnantharamu

  committed
  753d5e3

  Copy full SHA for 753d5e3

• WIP

  Binyang2014

  committed
  23880ce

  Copy full SHA for 23880ce

Commits on Jan 9, 2025

• update

  Binyang2014

  committed
  34b0a18

  Copy full SHA for 34b0a18

• WIP

  Binyang2014

  committed
  230f1af

  Copy full SHA for 230f1af

• WIP

  Binyang2014

  committed
  738bb57

  Copy full SHA for 738bb57

• WIP

  Binyang2014

  committed
  986f4ae

  Copy full SHA for 986f4ae

• WIP

  Binyang2014

  committed
  c8e71f6

  Copy full SHA for c8e71f6

• update

  Binyang2014

  committed
  5cc2b38

  Copy full SHA for 5cc2b38

• merge main

  Binyang2014

  committed
  5bb4568

  Copy full SHA for 5bb4568