关于微调后的LoRA模型大小
#305
Replies: 2 comments
-
|
取决于你训练了什么东西。如果只训练lora（rank大小决定其参数量），不训练embedding、lm-head的话也没什么奇怪的。 |
Beta Was this translation helpful? Give feedback.
0 replies
-
|
我们提供的lora里包含了embed_tokens和lm_head，这部分参数量比较大 |
Beta Was this translation helpful? Give feedback.
0 replies
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
-
为什么我自己训练用数据微调出来的LoRA模型只有几十MB的大小，而这里提供的LoRA模型却是1个GB，是什么决定了LoRA的大小。是lora_r这个参数决定的嘛？还是其他
Beta Was this translation helpful? Give feedback.
All reactions