Code

Rewrite to torch distributed#2

Code

Merged

sehoffmann merged 21 commits intodevelopsehoffmann/dmlcloud:developfrom feature/torch-distributedsehoffmann/dmlcloud:feature/torch-distributedCopy head branch name to clipboardMar 13, 2024

+1,816-1,547

Commits

Commits on Oct 25, 2023

• first batch

  sehoffmann

  committed
  cf4e1b3

  Copy full SHA for cf4e1b3

Commits on Nov 6, 2023

• initializa torch.distributed via mpi

  sehoffmann

  committed
  e914643

  Copy full SHA for e914643

Commits on Feb 15, 2024

• feat: reworked metric system

  sehoffmann

  committed
  77d116e

  Copy full SHA for 77d116e

Commits on Feb 16, 2024

• chore: cleanup

  sehoffmann

  committed
  0b2567c

  Copy full SHA for 0b2567c

• chore: cleanup2

  sehoffmann

  committed
  2b1d50b

  Copy full SHA for 2b1d50b

Commits on Feb 19, 2024

• feat: rework, first running training

  sehoffmann

  committed
  f7a0d67

  Copy full SHA for f7a0d67

Commits on Feb 20, 2024

• fix: allreduce metrics

  sehoffmann

  committed
  cb9bb0d

  Copy full SHA for cb9bb0d

• feat: track metrics

  sehoffmann

  committed
  3dfe656

  Copy full SHA for 3dfe656

• feat: distributed training

  sehoffmann

  committed
  6f9de0f

  Copy full SHA for 6f9de0f

• feat: streamlined training + diagnostics

  sehoffmann

  committed
  45f1c47

  Copy full SHA for 45f1c47

Commits on Mar 11, 2024

• feat: checkpoint directories

  sehoffmann

  committed
  2dd0d09

  Copy full SHA for 2dd0d09

Commits on Mar 12, 2024

• feat: wandb support

  sehoffmann

  committed
  250e9b3

  Copy full SHA for 250e9b3

• chore: formatting

  sehoffmann

  committed
  59716ca

  Copy full SHA for 59716ca

Commits on Mar 13, 2024

• fix: tests

  sehoffmann

  committed
  6399ae8

  Copy full SHA for 6399ae8

• chore: bump version to 0.3.0

  sehoffmann

  committed
  f05f9b8

  Copy full SHA for f05f9b8

• \Merge branch 'develop' into feature/torch-distributed

  sehoffmann

  committed
  2a0bd7d

  Copy full SHA for 2a0bd7d

• fix: remove horovod as dependency

  sehoffmann

  committed
  62fad57

  Copy full SHA for 62fad57

• chore: README badges

  sehoffmann

  committed
  ca7c596

  Copy full SHA for ca7c596

• fix: dependencies

  sehoffmann

  committed
  f0ed11a

  Copy full SHA for f0ed11a

• fix: init_process_group_dummy() on machines without gpu

  sehoffmann

  committed
  3e69580

  Copy full SHA for 3e69580

• fix: fixated progress_table version

  sehoffmann

  committed
  2df0e9b

  Copy full SHA for 2df0e9b