New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

中文预训练模型的精度有评测吗？ #48

Open

puppet101 opened this issue May 11, 2023 · 2 comments

puppet101 commented May 11, 2023

@s-JoL 非常感谢分享这么好的工作，中文预训练模型真的好稀缺。我想了解一下，目前的预训练模型，有在一些评测集上测试过指标吗？

Owner

s-JoL commented May 11, 2023 •

edited

Loading

目前在进行三种测评，分别是

EleutherAI/lm-evaluation-harness，是一个英文领域比较常用的测评。
Measuring Massive Multitask Chinese Understanding 一个近期出的中文测评集，包含的范围挺广的，数据量也不少。
参考Vicuna使用GPT4作为评判，将英文问题翻译为中文。

本周应该可以补充部分结果。

Author

puppet101 commented May 11, 2023

感谢回复！请问多久会出更大参数量的预训练模型啊？ 7B还是有点太小了~

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment