Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

flashattnvarlen support tree attention #1188

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

efsotr
Copy link

@efsotr efsotr commented Aug 30, 2024

flashattnvarlen support tree attention (doesn't support tree attention + alibi / sliding window)

varlen
varlen_tree

@efsotr efsotr changed the title flashattnvarlen support tree attention(does not support tree attentio… flashattnvarlen support tree attention Aug 30, 2024
@v-lmn
Copy link

v-lmn commented Jan 14, 2025

Thank you very much, I want to understand some questions.Can you explain some of your test case,I don't understand what's the meaning of "~create_mask(max_seqlen, half_prompts_lens)" and "mask" and "start_pos_id" and "end_pos_id". Can you give me your personal email?

@v-lmn
Copy link

v-lmn commented Jan 15, 2025

start_pos_id = create_pos_id(lens, [0, 1, 2])
end_pos_id = create_pos_id(lens, [2, 1, 2])
why [0, 1, 2] and [2, 1, 2]

@v-lmn
Copy link

v-lmn commented Jan 23, 2025

hi

@v-lmn
Copy link

v-lmn commented Jan 23, 2025

can you help

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants