Code

Relative position encoding#956

Code

Open

b-albar wants to merge 12 commits intoDao-AILab:mainDao-AILab/flash-attention:mainfrom catie-aq:rpe-biascatie-aq/flash-attention:rpe-biasCopy head branch name to clipboard

+827-99

Commits

Commits on May 14, 2024

• Add relative position encoding

  b-albar

  committed
  ed7d7ed

  Copy full SHA for ed7d7ed

Commits on May 16, 2024

• Transpose the bucket tensor and simplify the rpe loading

  b-albar

  committed
  9e3045f

  Copy full SHA for 9e3045f

• Fix _split_kv

  b-albar

  committed
  c71682d

  Copy full SHA for c71682d

Commits on May 17, 2024

• Fix interface for fwd

  b-albar

  committed
  61a9a90

  Copy full SHA for 61a9a90

Commits on May 21, 2024

• Add backward

  b-albar

  committed
  d0ea84a

  Copy full SHA for d0ea84a

• Fix bwd

  b-albar

  committed
  f032988

  Copy full SHA for f032988

Commits on May 30, 2024

• Fix mistake col/row in bwd

  b-albar

  committed
  708465c

  Copy full SHA for 708465c

Commits on Jul 16, 2024

• Merge branch 'main' into rpe-bias

  b-albar

  committed
  e74f014

  Copy full SHA for e74f014

Commits on Sep 11, 2024

• Optimize backward pass

  b-albar

  committed
  3b3fe82

  Copy full SHA for 3b3fe82

• Merge branch 'main' into rpe-bias

  b-albar

  authored
  abcf72a

  Copy full SHA for abcf72a

• Fix for bwd

  b-albar

  committed
  da1a8ee

  Copy full SHA for da1a8ee

Commits on Oct 2, 2024

• Merge branch 'main' into rpe-bias

  b-albar

  committed
  40f74d8

  Copy full SHA for 40f74d8