Add Context Parallelism support to cudnn Flash Attention#1133

Open

kocchop wants to merge 2 commits intoAI-Hypercomputer:mainfrom kocchop:faysal/add-cp-to-cudnn-flash-te

+45-8

Commits on Dec 31, 2024

added context parallelism support
kocchop
committed

Commits on Jan 1, 2025

added test config for context parallelism on GPUs
kocchop
committed