Skip to content
This repository has been archived by the owner on Dec 1, 2024. It is now read-only.

Killed Issue with flexgen when running python script #136

Open
foreverpiano opened this issue Apr 15, 2024 · 2 comments
Open

Killed Issue with flexgen when running python script #136

foreverpiano opened this issue Apr 15, 2024 · 2 comments

Comments

@foreverpiano
Copy link

python3 -m flexgen.flex_opt --model facebook/opt-30b --percent 0 100 100 0 100 0 --offload-dir /scratch/bcjw/ding3/flexgen_offload_dir  --path /scratch/bcjw/ding3/opt_weights
<run_flexgen>: args.model: facebook/opt-30b
model size: 55.803 GB, cache size: 2.789 GB, hidden size (prefill): 0.029 GB
init weight...
Killed

I use this script. it shows killed and has no output.

@foreverpiano
Copy link
Author

foreverpiano commented Apr 15, 2024

run on 2xA100 node, pytorch 2.2.1
@Ying1123 @BinhangYuan

@foreverpiano
Copy link
Author

it seems stuck at self.init_weight(j) where j = 68 for opt-30b

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant