Code

Add changes to support FSDP#598

Code

Merged

regisss merged 20 commits intohuggingface:mainhuggingface/optimum-habana:mainfrom vivekgoe:fsdpvivekgoe/optimum-habana:fsdpCopy head branch name to clipboardJan 23, 2024

+526-29

Commits

Commits on Jan 23, 2024

• Commit 1

  vivekgoe

  committed
  bc19cc5

  Copy full SHA for bc19cc5

• Commit 2

  vivekgoe

  committed
  4159224

  Copy full SHA for 4159224

• Commit 3

  vivekgoe

  committed
  126d70d

  Copy full SHA for 126d70d

• Commit 4

  vivekgoe

  committed
  067b261

  Copy full SHA for 067b261

• Commit 5

  vivekgoe

  committed
  40d90b4

  Copy full SHA for 40d90b4

• Commit 6

  vivekgoe

  committed
  5244420

  Copy full SHA for 5244420

• Commit 7

  vivekgoe

  committed
  553d11e

  Copy full SHA for 553d11e

• Commit 8

  vivekgoe

  committed
  aa56dfc

  Copy full SHA for aa56dfc

• Add FSDP tests

  vivekgoe

  committed
  b762912

  Copy full SHA for b762912

• update fsdp test

  vivekgoe

  committed
  7d5990b

  Copy full SHA for 7d5990b

• Add peft linear layer monkey patch for hpu

  vivekgoe

  committed
  b69d99d

  Copy full SHA for b69d99d

• Fix issues with activation_checkpointing code

  vivekgoe

  committed
  55322c8

  Copy full SHA for 55322c8

• Add FSDP configuration file for Llama2 FT

  vivekgoe

  committed
  8a5c83b

  Copy full SHA for 8a5c83b

• Update as per review comments

  vivekgoe

  committed
  ad5a04a

  Copy full SHA for ad5a04a

• Run formatter

  vivekgoe

  committed
  0d0ed0d

  Copy full SHA for 0d0ed0d

• Rebase to commit 979c132

  vivekgoe

  committed
  760a7e7

  Copy full SHA for 760a7e7

• Additional wrap policy for reducing device memory usage with PEFT

  vivekgoe

  committed
  79ec05a

  Copy full SHA for 79ec05a

• Update save checkpoint code to fix issue with LoRA + FSDP checkpoints

  vivekgoe

  committed
  938411e

  Copy full SHA for 938411e

• Delete question-answering/gaudi_config.json

  vivekgoe

  committed
  0df0f76

  Copy full SHA for 0df0f76

• Update device id usage in dataclasses.py

  vivekgoe

  committed
  31083c1

  Copy full SHA for 31083c1