Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Vllm integration #14

Merged
merged 37 commits into from
Jun 24, 2024
Merged

Vllm integration #14

merged 37 commits into from
Jun 24, 2024

Conversation

lyutyuh
Copy link
Collaborator

@lyutyuh lyutyuh commented Jun 20, 2024

No description provided.

lyutyuh and others added 24 commits June 19, 2024 12:34
@lyutyuh
vllm inference
e4dda7e
@lyutyuh
vllm particle
e5ff3de
@lyutyuh
vllm base
4e059a0
@benlipkin @lyutyuh
more makefile cleanup
d98994b
@timvieira @lyutyuh
Fix: Use the draw argument in proposal distributions and LM samplin…
0eafb43 
…g methods.
@postylem @lyutyuh
make smc-steer print out particle info when verbosity>0
c92cec8
@postylem @lyutyuh
make smc-steer print out particle info when verbosity>0
5e71fed
@timvieira @lyutyuh
tidying up
b283e08
@timvieira @lyutyuh
misc cleanups
ee1ff33
@timvieira @lyutyuh
fix: hf_tokenizers should use genparse.tokenization methods.
5bda0d8
@timvieira @lyutyuh
tidy
e0a8467
@leo-du @lyutyuh
refactor prompt formatter to be compatible with hf and vllm
95d07db
@leo-du @lyutyuh
add script to evaluate spider on vllm
873b62f 
related to issue #10
@leo-du @lyutyuh
add vLLM setup instructions
2855191
@timvieira @lyutyuh
tidy
0e294df
@timvieira @lyutyuh
whoops; forget to add file
db7849b
@timvieira @lyutyuh
()
9c62b33
@timvieira @lyutyuh
new utils to simplify simplify instantiation of HFPPLSampler, loadi…
385c9cf 
…ng models, and creating CFG guides (Issue #11)
@lyutyuh
vllm inference
b5e115f
@lyutyuh
vllm base
988af54
@lyutyuh
fix params
01939d0
@lyutyuh
ignore
505ff9f
@lyutyuh
Merge branch 'main' into vllm
7dd732f
@lyutyuh
fixing p_next for vllm
b1cddb3
@lyutyuh lyutyuh requested a review from timvieira June 20, 2024 13:31
@lyutyuh
Copy link
Collaborator Author

lyutyuh commented Jun 20, 2024

#10

@benlebrun benlebrun self-requested a review June 20, 2024 16:24
@benlebrun
Copy link
Collaborator

hey @lyutyuh, this looks great, though I think you need to add vllm to the dependencies, since all of the tests are failing. You can run make test to confirm that all the tests pass. I can handle the merge conflicts with the proposals.

@lyutyuh
Copy link
Collaborator Author

lyutyuh commented Jun 23, 2024

steering methods rewritten for vllm: vllm_steer.py

@timvieira timvieira merged commit 2024f78 into main Jun 24, 2024
4 checks passed
@timvieira timvieira deleted the vllm branch June 29, 2024 01:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@timvieira timvieira Awaiting requested review from timvieira

@benlebrun benlebrun Awaiting requested review from benlebrun

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
6 participants
@lyutyuh @benlebrun @timvieira @benlipkin @postylem @leo-du