How to get the hiddens for adj.py

[WenzhengZhang]

Hi,
Thanks for your great work! I have a question about the --res_path argument in https://github.com/thunlp/MoEfication/blob/48971265506cbad00d6bbed8e643034a9acbaeae/moefication/adj.py#L12 It should be the hidden states of each T5 layer before ffn if my understanding is correct. How do you get those hidden states?

[zzy14]

Hi, sorry for the late response.
Yes, it should be the hidden states of each T5 layer before FFNs. You can use python examples/t5-sst2-inf.py to get them.